Develop Reference

ASP.NET Core application exits unexpectedly during execution. The program '[] XYZ.exe' has exited with code 3 (0x3)

Application exists unexpectedly during iterating over IAsyncEnumerable and inserting records to database using EF context. No error or exception is thrown. In the console window it shows only The program '[5372] XYZ.exe' has exited with code 3 (0x3).
The code is somthing like:
public async Task LoadDataAsync()
{
try
{
_context.ChangeTracker.AutoDetectChangesEnabled = false;
int bufferSize = 50;
var myEntityBuffer = new List<MyEntity>(bufferSize);
//GetDataAsync returns IAsyncEnumerable
await foreach (var element in loader.GetDataAsync(cancellationToken).WithCancellation(cancellationToken))
{
myEntityBuffer.Add(new MyEntity(element));
if (addressPointBuffer.Count == bufferSize)
{
_context.MyEntities.AddRange(myEntityBuffer);
await _context.SaveChangesAsync(cancellationToken);
myEntityBuffer = new List<MyEntity>(bufferSize);
_context.ChangeTracker.Clear();
}
}
await _context.SaveChangesAsync(cancellationToken);
}
catch (Exception ex)
{
_logger.LogError(ex, ex.Message);
}
finally
{
_context.ChangeTracker.AutoDetectChangesEnabled = true;
}
}
The same pattern is used by other commands and they work fine, I could not spot the difference except that the element structure is different, different entity. The number of records is large, but the application exists after inserting approx. 80 000 record (the number differs from run to run)
I cannot trace the source of the problem and what makes the application exit. I run this on Development environment.
I appreciate if you could suggest how to trace the issue
So far I have tried the following:
Checking the application Logs - the running code is wrapped with try-catch, however no error were thrown and logged
placing a breakpoint in catch and finally blocks (these lines are not reached)
Wrapping Main method code in try-catch (no execption was catched)
Adding IHostApplicationLifetime Stopped and OnStopping events (these events are not raised upon exit)
increasing logging level for Microsoft.EntityFramework to Debug (I can see only SQL commands but no issues)
If I comment out lines doing database operations (AddRange, SaveChangesAsync) the method completes without issues and the application is still running.
I use the following stack
.NET Runtime 5.0.11
ASP.NET Core Runtime 5.0.11
.NET Desktop Runtime 5.0.11
EntityFramework Core + NetTopologySuite
SQLite + Spatialite
UPDATE
I have stopped using SQLite. I had plans to move to postgresql anyway. On postgresql it does not happen. Still would be nice to know how to diagnose such issues...

uglify crashing after running child_process.execFile

EDIT 2
I "solved" the problem, but I don't want to post it as an answer b/c it doesn't explain what actually happened. In the code for the .NET resourceReader.exe I use
Console.OutputEncoding = System.Text.Encoding.UTF8;
to output the internationalized resources to stdout in unicode. If I reset the encoding at the end of my program with
Console.OutputEncoding = System.Text.Encoding.Default;
then I don't get any errors in Node. If I don't reset it, I get the error described in the original question. It seems that .NET is somehow messing up some of the output encoding settings on the cmd.exe and causing the subsequent node run to fail!
EDIT
I narrowed down the error to being caused by resourceReader.exe. It's a.NET program which reads some resource streams out of the .NET assembly and prints them to the stdout using Console.WriteLine. I added Console.OutputEncoding = System.Text.Encoding.UTF8 to resourceReader.exe because some of the resources are in non ASCII letters and that's whats causing the crash in grunt!
If I take that line out, the task doesn't crash, but the resources show up in non printable ASCII characters! Also, the crash only happens if I actually print non-ASCII to sdtout. If I don't print them, it doesn't error.
ORIGINAL
I added a step to my Gruntfile which uses child_process.execFile to run an read some data from an external program and uses it in the build. Now whenever I run my build, it runs fine the first time, but crashes the second time!
Here's the output from the crash (this is during the uglify task):
File build/Scripts/NDB.contacts.min.js created: 16.85 kBevents.js:85
throw er; // Unhandled 'error' event
^
Error: This socket is closed.
at WriteStream.Socket._writeGeneric (net.js:656:19)
at WriteStream.Socket._write (net.js:709:8)
at doWrite (_stream_writable.js:301:12)
at writeOrBuffer (_stream_writable.js:288:5)
at WriteStream.Writable.write (_stream_writable.js:217:11)
at WriteStream.Socket.write (net.js:634:40)
at Log._write (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\index.js:161:26)
at Log.wrapper [as _write] (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\node_modules\lodash\index.js:3095:19)
at Log._writeln (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\index.js:166:8)
at Log.wrapper [as _writeln] (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\node_modules\lodash\index.js:3095:19)
at Log.writeln (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\index.js:177:8)
at Log.wrapper (C:\...\node_modules\grunt\node_modules\grunt-legacy-log\node_modules\lodash\index.js:3095:19)
at writeln (C:\...\node_modules\grunt\lib\grunt\fail.js:30:13)
at Object.fail.fatal (C:\...\node_modules\grunt\lib\grunt\fail.js:46:3)
at process.uncaughtHandler (C:\...\node_modules\grunt\lib\grunt.js:121:10)
at process.emit (events.js:129:20)
at process._fatalException (node.js:236:26)
at Task.runTaskFn (C:\...\node_modules\grunt\lib\util\task.js:250:7)
at Task.<anonymous> (C:\...\node_modules\grunt\lib\util\task.js:293:12)
at C:\...\node_modules\grunt\lib\util\task.js:220:11
at process._tickCallback (node.js:355:11)
Here's the code for the task which uses child_process.
function readAllCultures() {
var readDeferred = q.defer();
childProc.execFile("../tools/resourceReader.exe", function (err, stdout, stderr) {
if (err) throw new Error(err);
var cultures = JSON.parse(stdout);
readDeferred.resolve(cultures);
});
return readDeferred.promise;
}
Here's some things I discovered debugging that might be helpful
If I redirect the output of grunt (using either > filename or | process) it runs fine
When I redirect output, I never see the message from uglify that it created the main output only that it created the source map.
If I close and reopen my command prompt (cmd.exe) it works fine
I added a listener to the exit and close events of the child process using rdr.on("close", function() { console.log("close"); }); and the same for exit. Both events fire as expected in the first run.
Using Process Explorer, I can see node.exe open under the command prompt and close again when my command finishes running. There are no processes visibly "left open" under the command prompt.

The fact that the stack trace is ultimately showing a socket.write error is interesting. Nothing in the code you've provided suggests you are trying to write to a socket.
If we follow the stack down, we can see that it's actually grunt that is trying to write to a socket, as it's attempting to log an uncaught exception.
So, first your grunt task throws a currently unknown error, and then grunt itself errors because it can't tell you about it.
I would first try and log the error that occurs in your child process. Currently if there is an error present, you simply throw it without finding out what it is. It's likely that this is what grunt is trying and failing to log.
Replace
if (err) throw new Error(err);
With
if (err) console.error(err);
This will hopefully avoid the socket.write problem and give you specific information about what error is happening on the child process. What do you see?
Second, I would try using child_process.exec instead of child_process.execFile. This will spawn a shell and run resourceReader.exe inside of it (instead of running it directly). This might help avoid any problems you are encountering with the command still running/failing in the background, which could be the cause of the socket.write error.

Access Violation Exception in SqlCeConnection dispose

Application/Code description:
My application is based on c# and uses SQL Server CE and iv'e got this exception only twice at the same code location. the crash with this exception was not introduced till this version. the only change in this version was changing the .net framework to 4.5.2.
I'm getting access violation exception on the dispose of an SqlCeConnection with the following error:
Attempted to read or write protected memory. This is often an
indication that other memory is corrupt.
This exception is not intercepted by the try catch clause of .net- it causes a crash.
In my code I use the following to run
try
{
var connectionString = string.Format("{0}{1}{2}", "Data Source=", _localDB, ";File Mode=Read Write;Max Database Size=4000;Persist Security Info=False;");
using (var sqlCeConnection = new SqlCeConnection(connectionString))
{
using (var sqlCeCommand = new SqlCeCommand())
{
sqlCeCommand.Connection = sqlCeConnection;
sqlCeCommand.CommandText = "SELECT * FROM Application";
sqlCeConnection.Open();
var result = (string)sqlCeCommand.ExecuteScalar();
isValid = !IsValid(result);
}
}
}
catch (Exception ex)
{
_log.Error("exception", ex);
}
call stack for the first crash:
ntdll!ZwWaitForMultipleObjects+a
KERNELBASE!WaitForMultipleObjectsEx+e8
kernel32!WaitForMultipleObjectsExImplementation+b3
kernel32!WerpReportFaultInternal+215
kernel32!WerpReportFault+77
kernel32!BasepReportFault+1f
kernel32!UnhandledExceptionFilter+1fc
ntdll! ?? ::FNODOBFM::`string'+2365
ntdll!_C_specific_handler+8c
ntdll!RtlpExecuteHandlerForException+d
ntdll!RtlDispatchException+45a
ntdll!KiUserExceptionDispatcher+2e
sqlcese35!__SafeRelease+c
sqlcese35!Column::`vector deleting destructor'+5c
sqlcese35!Object::DeleteObjects+39
sqlcese35!Table::`vector deleting destructor'+45
sqlcese35!Table::Release+27
sqlcese35!HashTable::~HashTable+2a
sqlcese35!Store::~Store+12b
sqlcese35!Store::Release+2a
sqlceme35!ME_SafeRelease+17
DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr ByRef)+78
[[InlinedCallFrame] (System.Data.SqlServerCe.NativeMethods.SafeRelease)] System.Data.SqlServerCe.NativeMethods.SafeRelease(IntPtrByRef)
System.Data.SqlServerCe.SqlCeConnection.ReleaseNativeInterfaces()+147
System.Data.SqlServerCe.SqlCeConnection.Dispose(Boolean)+f1
System_ni!System.ComponentModel.Component.Dispose()+18
call stack for the second crash:
ntdll!NtWaitForMultipleObjects+a
KERNELBASE!WaitForMultipleObjectsEx+e8
kernel32!WaitForMultipleObjectsExImplementation+b3
kernel32!WerpReportFaultInternal+215
kernel32!WerpReportFault+77
kernel32!BasepReportFault+1f
kernel32!UnhandledExceptionFilter+1fc
ntdll! ?? ::FNODOBFM::`string'+2335
ntdll!_C_specific_handler+8c
ntdll!RtlpExecuteHandlerForException+d
ntdll!RtlDispatchException+45a
ntdll!KiUserExceptionDispatcher+2e
<Unloaded_sqlcese35.dll>+7c88c
<Unloaded_sqlceqp35.dll>+102790
0x06ccc898
0x06f9efc8
0x1eca8018
0x1f207400
<Unloaded_sqlcese35.dll>+228dc
0x00000004
0x2edff008
0x00000002
0x00000003
0x00000004
<Unloaded_sqlcese35.dll>+3fbd9
0x06ccc898
DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr ByRef)+78
[[InlinedCallFrame] (System.Data.SqlServerCe.NativeMethods.SafeRelease)] System.Data.SqlServerCe.NativeMethods.SafeRelease(IntPtrByRef)
System.Data.SqlServerCe.SqlCeConnection.ReleaseNativeInterfaces()+147
System.Data.SqlServerCe.SqlCeConnection.Dispose(Boolean)+f1
System_ni!System.ComponentModel.Component.Dispose()+1b
I found some references on the internet that suggests some solutions:
Probable solution: check multithreading issue on the same connection (attempted to read write protected memory. this is often an indication that other memory is corrupt)
Rejection:
a. the connection is created in the using brackets and doesn't get reused.
b. the calling method is called every 5 minutes and verified via the dump file that it was not called simultaneously.
Probable solution: sql ce version mismatch (http://blogs.msdn.com/b/sqlservercompact/archive/2009/05/06/troubleshooting-access-violation-exception-while-using-sql-server-compact-database-with-ado-net-provider.aspx)
Probable Rejection: I can see in the version installed is 3.5 SP2 (3.5.8080.0) and from the modules located in the dump I can see the sqlceme35.dll, System.Data.SqlServerCe.dll DLL's are with version 3.05.8080.0
Probable solution which is in question is the following:
https://stackoverflow.com/a/20492181/1447518
Probable Rejection: it doesn't sound right from a statistical perspective- the code crashed twice in the same place although there another place in the application code which writes and read to a different DB and the application didn't crash there.
The last thing I was thinking about, may suggest a unload problem of DLLs (take a look at the second call stack). My guess is that the dll's are unloaded from the application while the application needed them in order to do a dispose, but it seams a bit blurry and a 'long shot'
My question is: what may cause the problem, and what is a probable solution?

Although this solution is not yet verified, the solution is as follows:
From the second call stack i can see there is unload of native DLL's, my guess was that the dispose method of the SQL connection was using one of the methods it currently disposed.
I verified thorough the Process dump that all the SqlCeConnection types were in the process of dispose.
Seeing ErikEj comment made me realize it's will be better if i will take a look in the code differences that was made between SQL-CE 3.5 to 4.0 (System.Data.SqlServerCe.dll).
after viewing the code, i could see that the method of the release was moved to a later position inside the dispose method.
In addition i could see that before calling SafeRelease there was another check that checks if the native DLLs that were needed for safe release was released already- and throw an exception.
bottom line, SQL-CE 4.0 has 2 solutions for the same issue.
my guess is that this issue was caused because of this.
the solution for now was to keep a connection during all the application life-cycle (which has no connection string), this cause the pointer pool to keep the native Dlls in the memory for all the application life-cycle.
the better solution is to move to SQL-CE 4.0 .

How do I allocate more of my physical memory?

I am getting this error when loading large files into memory. What I don't understand is that my memory (as monitored by Task Manager still says only 7G used on a machine that has 32G). Is this memory exception referring to a constrained part of this memory? And, if so, how do I allocate more. The code producing the error is below.
System.OutOfMemoryException occurred
HResult=-2147024882
Message=Exception of type 'System.OutOfMemoryException' was thrown.
Source=mscorlib
StackTrace:
at System.IO.File.InternalReadAllBytes(String path, Boolean checkHost)
InnerException:
Active (x86) and 64bit Windows 7
Code:
public void LoadAllBinaries(string aKey)
{
if (msDatas != null)
return;
msDatas = new SortedDictionary<string, byte[]>();
var dataFiles = File.ReadAllLines(G.ConfigDir + #"\dates_out.txt");
foreach (var df in dataFiles)
{
try
{
string fn = G.DataDir + "\\n" + aKey + df + ".dft";
byte[] ba = File.ReadAllBytes(fn);
msDatas.Add(fn,ba);
ba = null;
}
catch (Exception e)
{
Console.WriteLine("OpenSLoadAllBinariestreams ERROR: " + e.Message);
}
}
}

You've either got a memory leak somewhere, or a handle leak. Both will give you the Out Of Memory error.
You can see if you've got a handle leak by opening TaskManager, go to View, Select Columns..., and turn on Handles. Start your process and let it run through a few iterations. Jot down the numbers you get. Then come back in a few hours and compare the numbers. If the handles numbers are considerably higher, you're not Disposing an object than you should be.
Also ensure that you dispose of your objects as well. You can also try use System.GC.Collect for garbage collection. Hope this helps.

Not exactly sure but your exception stacktrace says
at System.IO.File.InternalReadAllBytes(String path, Boolean checkHost)
To me it looks like the below line of code is the culprit and it's throwing OOM exception cause most most probably the size of dates_out.txt is high enough.
File.ReadAllLines(G.ConfigDir + #"\dates_out.txt")
What's the size of the file dates_out.txt you are reading? if it's in size of GB then try reading it line by line rather reading the entire file at once (as suggested by others over comment).
EDIT:
If you want to see current memory consumption of a program’s object, then you can get it using GC.GetTotalMemory method. You can also use windbg.exe tool or Microsoft’s CLR Profiler
long memoryUsed = GC.GetTotalMemory (true);

In visual studio go to Project Properties and Build screen. Check that where Any CPU is selected that the platform target checkbox for Prefer 32 bit is not checked.
You can also explicitly select x64 there.
This resolved the same error for me.

How could the existence of pseudo debug strings cause a difference in functionality?

If I have these sprinkled throughout my code:
MessageBox.Show("See a format exception yet? #1");//todo: remove
(there are 7 of these, numbered from 1..7, most of which display (1,2,5,6,7))
I end up with one err msg ("Exception: Cannot find table 0 Location: frmFunction.SetPlatypus")
If I comment out all of those, I end up with a different err msg ("Exception: FormatException Location frmFunction.getDuckbillRecord")
How could this be? Shouldn't the existence/display of such an information msg have no effect on the way the code executes/the path it takes, etc.?
Note: getDuckbillRecord() is where all of the MessageBoxes are.
UPDATE
Using RT's suggestions as motivation, I came up with this:
public static StringBuilder LogMsgs = new StringBuilder();
public static void ExceptionHandler(Exception ex, string location)
{
try
{
LogMsgs.Append(string.Format("{0}\r\n", ex.Message)); //TODO: Comment out before deploying?
DateTime dt = DateTime.Now;
string timeAsStr = string.Format("{0}_{1}_{2}_{3}.txt", dt.Hour, dt.Minute, dt.Second, dt.Millisecond);
using (StreamWriter file = new StreamWriter(timeAsStr))
{
file.WriteLine(LogMsgs.ToString());
}
. . .
//in the problematic code, replacing the MessageBox.Show() calls:
TTBT.LogMsgs.Append("Made it to location 1 in frmOverFunction.GetDuckbillRecord()\r\n");
...and it did help - the exception is reached right after the first of 7 "code waypoints," so there's something rotten in that particular Denmark, apparently.
UPDATE 2
After the rare experience of being able to run the app without crashing somewhere, I realized I also need that file-writing code in the main form's OnClosing() event - being in the global exception handler had worked for awhile - I wasn't expecting a clean termination of the app to ever occur again, I guess.

I'd strongly encourage you NOT to use MessageBox.Show(...) to log/trace/debug errors, because popping-up a dialog can introduce many subtle bugs and changes in behavior, focus, etc., especially if your UI flow/state is important.
Instead, use System.Diagnostics.Trace for writing tracing code (which you can optionally compile into your production/release code) or System.Diagnostics.Debug to write debug messages which get removed when you build your release version.
For example:
public void DoSomething(int a, int b)
{
Trace.TraceInformation("Starting to do something: a = {0}, b = {1}",
a, b);
try
{
int c = a / b;
}
catch (DivideByZeroException e)
{
Debug.WriteLine("OH NO ... 'b' WAS ZERO!!!! RUN AWAY!!!!");
throw e;
}
Trace.TraceInformation("Done doing something");
}
In the example above, the debug output will be available in debug builds, but will be removed in release builds. Trace output will be available in debug and retail bugs (unless you turn off the TRACE build parameter) but introduces no measurable perf impact unless you hook-up a listener and start writing trace output to a slow medium like spinning disks, ticker-tape printers, etc. ;)
When you run code with Trace and/or Debug builds with Debug messages in Visual Studio, the messages are written to the Output pane so you can watch what's going on inside your app as it runs.
If you want to, you can also write a stand-alone trace listener that listens for Trace and/or Debug messages and log them to a file/database or display them in a console window (or GUI if you want to get fancy).
You can also use tools like log4net which enables you to write log/trace messages to a variety of file formats, databases, etc.