Develop Reference

Checking Elasticsearch Heap Size

How can I check the heap size that is assigned to Elasticsearch engine, is there a way to check it using URL API ? And can I use NEST to check it?
Thanks

In a clustered environment, heap settings can be queried as :
curl -sS "localhost:9200/_cat/nodes?h=heap*&v"
Eg:
curl -sS "localhost:9200/_cat/nodes?h=heap*&v"
heap.current heap.percent heap.max
321.1mb 32 989.8mb
424.1mb 42 989.8mb
280.3mb 28 989.8mb
This can also be checked from the ps results, though it will only shed light on the min-max values.
~#ps aux | grep --color=auto -i Xms
elastic 6020 0.0 0.0 12788 936 pts/4 S+ 04:24 0:00 grep --color=auto -i Xms elastic+ 7218
0.6 9.5 5001220 1565112 ? Ssl Jun24 5:14 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch.WoiU4NhH -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Xms1g -Xmx1g -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=deb -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
where -Xms is the minheap and -Xmx is maxheap configured.

use GET _nodes/stats
Then look at the following in the return /jvm/mem/heap_committed_in_bytes

This is a new account so I couldn't comment on jhilden's answer. His answer is correct but to answer your comment I use Postman which is a chrome app. It's very easy to use and has been ideal for me whilst working with Elasticsearch.
Putting the following in postman and hitting send does the job!
http://localhost:9200/_nodes/stats
review the results and look for the "jvm" block. Here is what is returned when I run it on a default elasticsearch setup:
"jvm": {
"timestamp": 1477474766408,
"uptime_in_millis": 1309586,
"mem": {
**"heap_used_in_bytes": 87134248**,
"heap_used_percent": 8,
"heap_committed_in_bytes": 259522560,
"heap_max_in_bytes": 1038876672,
"non_heap_used_in_bytes": 56166992,
"non_heap_committed_in_bytes": 57348096,

As per the online documentation, the default heap size is 1 Gb unless you explicitly mention in the Environment Variable [ES_HEAP_SIZE]
The default installation of Elasticsearch is configured with a 1 GB
heap. For just about every deployment, this number is far too small.
If you are using the default heap values, your cluster is probably
configured incorrectly.
There are two ways to change the heap size in Elasticsearch. The
easiest is to set an environment variable called ES_HEAP_SIZE. When
the server process starts, it will read this environment variable and
set the heap accordingly. As an example, you can set it via the
command line as follows:
export ES_HEAP_SIZE=10g Alternatively, you can pass in the heap size
via a command-line argument when starting the process, if that is
easier for your setup:
./bin/elasticsearch -Xmx10g -Xms10g
Ensure that the min (Xms) and max (Xmx) sizes are the same to prevent
the heap from resizing at runtime, a very costly process.
Generally, setting the ES_HEAP_SIZE environment variable is preferred
over setting explicit -Xmx and -Xms values.

The update from my AIF Webservice for SalesAgreementHeader [AX 2012] fails

Hello I've created a query with only the SalesAgreementHeader as table. Then I used the wizard to create a document aif webservice.
The webservice works fine, but the update always reports an "Invalid xml document." error.
This error is thrown by the method moveToNextElement in the class AxdBaseCreate.
The reason for the exception is, that
reader.fieldName() = AgreementClassification
and
_expectedElementName = AgreementRelationType.
How can I fix this? Or is there any good way to debug this?
Before I created the query with only the SalesAgreementHeader I had an more complex query and had the same error. Then I created a new one and reduced it to only the SalesAgreementHeader.
Update:
A haven't solved the problem yet, but I think the problem could be because SalesAgreementHeader extends the AgreementHeader and the field RelationType is not visible in the AOS, but if you look directly in the DB.
Update 2:
This is the call stack:
[s] \Classes\AxdBaseCreate\moveToNextElement 9
[s] \Classes\AxdBaseCreate\readSurrogateForeignKeyValue 37
[s] \Classes\AxdBaseCreate\readProperty 14
[s] \Classes\AxdBaseCreate\readProperties 122
[s] \Classes\AxdBaseUpdate\deserializeTopEntity 25
[s] \Classes\AxdBaseUpdate\deserializeDocument 131
[s] \Classes\AxdBaseUpdate\updateDocumentList 42
[s] \Classes\AxdBase\updateList 64
[s] \Classes\AifDocumentService\updateList 34
[s] \Classes\SalesAgreementService\update 4

Could you please provide code which you're using?
I have an assumption (based on the name of AgreementRelationType it looks like this is Enim ) that you're trying to update field based on enum. If this is the case, then I hope I can help you.
When you need to update enum use following approach:
//here is an example how to change Status on SalesTable
//as you may see when you change the value of enum you have to set boolean autogenerated field to "true".
//Name of such fields ends with suffix "Specified"
salesTable.SalesStatus = AxdEnum_SalesStatus.Invoiced;
salesTable.SalesStatusSpecified = true;
In advance I'd like to suggest you (in case you didn't do that) use following try..catch block. This will make your life easier.
try
{
//your code here
}
catch (System.ServiceModel.FaultException<SalesOrderDelete.SalesOrderDeleteTcpNet.AifFault> aifFaults) // This code catches error messages even when "Logging mode = Logging is disabled" on Inbound port
{
SalesOrderDelete.SalesOrderDeleteTcpNet.InfologMessage[] infologMessageList = aifFaults.Detail.InfologMessageList;
foreach (SalesOrderDelete.SalesOrderDeleteTcpNet.InfologMessage infologMessage in infologMessageList)
{
Console.WriteLine("Exception: " + infologMessage.Message + "\n");
}
Console.WriteLine("\nPress any key to quit.\n");
Console.ReadKey();
cl.Abort();
}

HtmlAgilityPack Save Process Not Letting Go Of File

I am saving some of the rendered html of a web site by overriding the Render method and using HtmlAgilityPack. Here is the code:
protected override void Render(HtmlTextWriter writer)
{
using (HtmlTextWriter htmlwriter = new HtmlTextWriter(new StringWriter()))
{
base.Render(htmlwriter);
string output= htmlwriter.InnerWriter.ToString();
var doc = new HtmlDocument();
doc.LoadHtml(output);
doc.Save(currDir + "\\" + reportDir + "\\dashboardTable.html");
}
}
However, some process does not let go of the saved file and I am unable to delete it from the server. Does anyone know of an HtmlAgilityPack issue that would cause this?
Any advice is appreciated.
Regards.
EDIT:
I have tried both of the methods suggested. I can't tell if they are the solution yet because my app is frozen on the server due to the files I can't delete. However, when I use these solutions on my own machine, the rendered HTML does not save as an HTML table anymore but rather like this:
INCIDENT MANAGEMENT
Jul '12 F'12
Trend F'12 2011
(avg)
Severe Incidents (Sev1/2): 3 2.1 4.16
Severe Avoidable Incidents (Sev1/2): 1 1.3 1.91
Incidents (Sev3): 669 482 460.92
Incidents (Sev4) - No business Impact: 1012 808 793
Proactive Tickets Opened: 15 19.3 14
Proactive Tickets Resolved/Closed: 14 17.3 11
CHANGE MANAGEMENT
Total Planned Changes: 531 560 583.58
Change Success Rate (%): 99.5 99.4 99
Non-Remedial Urgent Changes: 6 11 47.08
PROBLEM MANAGEMENT
New PIRs: 2 1.4 2
Closed PIRs: 0 2 3
Overdue Action items: 2 3.2 0
COMPLIANCE MEASUREMENTS
Jul Trend Jun
Total Number of Perimeter Devices: 250 258
Perimeter Devices - Non Compliant: 36 31
Total Number of Internal Devices: 6676 6632
Internal Devices - Non Compliant: 173 160
Unauthorized Perimeter Changes: 0 0
Unauthorized Internal Changes 0 0
LEGEND
ISP LINKS
July June Trend
SOC CPO DRP SOC CPO DRP
40% 34% 74% 39% 35% 74%
BELL MPLS HEAD ENDS
July June Trend
SOC CPO SOC CPO
8% 5% 7% 10% 8% 5.5% 7% 10%
ENTERPRISE NETWORK (# of issues called out)
July June Trend
CORE FW/DMZ CORE FW/DMZ
1 0 1 0
US & INTL (# of issues called out)
July June Trend
US Intl US Intl
2 2 2 3
LINE OF BUSINESS BELL WAN MPLS
<> 50%-65% >65% <> 50%-65% >65% Trend
Retail: 2272 0 1 2269 4 0
Business Banking: 59 1 0 60 0 0
Wealth: 122 2 0 121 2 1
Corporate: 51 0 0 49 2 0
Remote ATM: 280 0 0 280 0 0
TOOLS
Version Currency Vulnerability Status Health Status
Key Messages:
where only the text data has been saved and all of the HTML and CSS is missing. If I just use doc.Save() I get an exact representation of the table as it displays on the website.

Try this instead. Maybe the Save method isn't closing the underlying stream.
using( FileStream stream = File.OpenWrite( currDir + "\\" + reportDir + "\\dashboardTable.html" ) ){
doc.Save(stream);
stream.Close();
}

Edit
Per #L.B's comments it appears that HtmlAgilityPack does use a using block as in my example so it will be ensuring that the stream gets closed.
Thus as I suggested at the end of my original answer this must be a server environment problem
Original Answer
This may be some sort of bug with HtmlAgilityPack - you may want to report it to the developers.
However to eliminate that possibility you may want to consider explicitly controlling the creation of the StreamWriter for the file so you are explicitly closing it yourself. Replace this line:
doc.Save(currDir + "\\" + reportDir + "\\dashboardTable.html");
With the following:
using (StreamWriter fileWriter = new StreamWriter(currDir + "\\" + reportDir + "\\dashboardTable.html"))
{
doc.Save(fileWriter);
fileWriter.Close();
}
If the issue still persists even with this change then that would suggest an issue with your server environment rather than an issue with HtmlAgilityPack. Btw to test if this change makes a difference you should start from a clean server environment rather than one where you are already having issues deleting the file in question.

Saving A Specific Node As HTML Using HtmlAgilityPack

I have a web site that displays a table along with some buttons, navigation tabs, etc. Using HtmlAgilityPack I would like to save only the node containing the table (a div) to a .html file. I have been experimenting with this code:
string dashboard = doc.DocumentNode.SelectSingleNode("//div[#id='Dashboard']").InnerHtml;
doc.LoadHtml(dashboard);
doc.Save(currDir + "\\dashboardOutputFile.html");
But this only preserves the text components of the table and none of its layout/display information. I have tried adding a Doctype declaration as well as enclosing html and body tags to the dashboard string but it has no effect.
Any advice is appreciated.
Regards.
EDIT:
I should be more explicit. I am capturing the html of my page by overriding the Render method:
protected override void Render(HtmlTextWriter writer)
{
using (HtmlTextWriter htmlwriter = new HtmlTextWriter(new StringWriter()))
{
base.Render(htmlwriter);
string renderedContent = htmlwriter.InnerWriter.ToString();
var doc = new HtmlDocument();
doc.LoadHtml(renderedContent);
try
{
string dashboard = doc.DocumentNode.SelectSingleNode("//div[#id='Dashboard']").InnerHtml;
dashboard = "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\"><br><html xmlns=\"http://www.w3.org/1999/xhtml\"><head></head><body><div><br>" + dashboard + "<br></div></body></html>";
doc.LoadHtml(dashboard);
doc.Save(currDir + "\\dashboardOutputFile.html");
}
catch (NullReferenceException ex)
{
// Do nothing.
}
writer.Write(renderedContent);
}
}
To the best of my knowledge the string "dashboard" should contain a complete HTML page after this. However the resulting .html file displays like this:
INCIDENT MANAGEMENT
Jun '12 F'12
Trend F'12 2011
(avg)
Severe Incidents (Sev1/2): 2 2 4.16
Severe Avoidable Incidents (Sev1/2): 1 1.3 1.91
Incidents (Sev3): 600 459 460.92
Incidents (Sev4) - No business Impact: 869 782 793
Proactive Tickets Opened: 24 20 14
Proactive Tickets Resolved/Closed: 22 17.8 11
CHANGE MANAGEMENT
Total Planned Changes: 623 564 583.58
Change Success Rate (%): 99.9 99.4 99
Non-Remedial Urgent Changes: 5 11.4 47.08
PROBLEM MANAGEMENT
New PIRs: 1 1.4 2
Closed PIRs: 1 2.3 3
Overdue Action items: 2 3.4 0
COMPLIANCE MEASUREMENTS
Jun Trend May
Total Number of Perimeter Devices: 258 242
Perimeter Devices - Non Compliant: 31 7
Total Number of Internal Devices: 6632 6521
Internal Devices - Non Compliant: 160 38
Unauthorized Perimeter Changes: 0 0
Unauthorized Internal Changes 0 0
LEGEND
ISP LINKS
June May Trend
SOC CPO DRP SOC CPO DRP
39% 35% 74% 38% 35% 73%
BELL MPLS HEAD ENDS
June May Trend
SOC CPO SOC CPO
8% 5.5% 7% 10% 7.5% 5% 6% 10%
ENTERPRISE NETWORK (# of issues called out)
June May Trend
CORE FW/DMZ CORE FW/DMZ
1 0 1 0
US & INTL (# of issues called out)
June May Trend
US Intl US Intl
2 3 2 3
LINE OF BUSINESS BELL WAN MPLS
<> 50%-65% >65% <> 50%-65% >65% Trend
Retail: 2269 4 0 2271 2 0
Business Banking: 60 0 0 51 1 0
Wealth: 121 2 1 111 3 2
Corporate: 49 2 0 51 1 1
Remote ATM: 280 0 0 279 0 0
TOOLS
Version Currency Vulnerability Status Health Status
Key Messages:
Only the text of the table cells is being displayed. There is no table layout, no background colours, etc. Even when I copy the file into JSFiddle it displays like this. What am I missing to make this a complete HTML page?

There's probably a css file linked into the original html not being read in your downloaded page.

snmp oid for scanner detail(HP LaserJet 3055)

How can i access scanner details (eg. HP LaserJet 3055 ) in windows service (.net framework 4.0).by using mib tree ?
Anyone please tell me the OID for getting detail such as scanner_ADF_PageCount,TotalPagesJammed,etc for scanner.
I compare each oid with "1.3.6.1.4.1.11.2.3.9.4.2.1.2.2". But still it is not giving above detail.

1) scanner-accessory-adf-sheet-count - .1.3.6.1.4.1.11.2.3.9.4.2.1.2.2.1.20.0
2) hrprinterdetectederrorstate - .1.3.6.1.2.1.25.3.5.1.2 . It returns an octet string and you need to interpret it as
Condition Bit # hrDeviceStatus
lowPaper 0 warning(3)
noPaper 1 down(5)
lowToner 2 warning(3)
noToner 3 down(5)
doorOpen 4 down(5)
jammed 5 down(5)
offline 6 down(5)
serviceRequested 7 warning(3)
I got this information from HP-LASERJET-COMMON-MIB and you can download it from here. Browse through it for more details.