I have a console application written in C# .Net 4.5 and running on Mono 3.2.8 on Linux (Ubuntu 14.04 LTS).
The console app is started as a service with upstart and I am logging output using log4net v2.0.5 and a console appender.
Upstart redirects all the output in /var/log/upstart/{appname}.log
Randomly, after few hours up to a couple of days, the application hangs and I see nothing in the logs.
What I know:
Disk is not full
Memory is not full and there is no swap
Command ps shows the process still running
The application is not sending anymore data to external server. When working properly the external server receives data every few seconds.
Log file is not populated anymore
I used the command strace to see the last syscall and this is what I get:
$ strace -p 5602
Process 5602 attached
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}], 2, -1) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
--- SIGPWR {si_signo=SIGPWR, si_code=SI_TKILL, si_pid=5602, si_uid=1001} ---
rt_sigprocmask(SIG_BLOCK, [XCPU], NULL, 8) = 0
futex(0x26b18c, FUTEX_WAKE_PRIVATE, 1) = 1
rt_sigsuspend(~[XCPU RTMIN RT_1]) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGXCPU {si_signo=SIGXCPU, si_code=SI_TKILL, si_pid=5602, si_uid=1001} ---
rt_sigreturn() = -1 EINTR (Interrupted system call)
rt_sigprocmask(SIG_UNBLOCK, [XCPU], NULL, 8) = 0
futex(0x26b18c, FUTEX_WAKE_PRIVATE, 1) = 1
rt_sigreturn() = -1 EINTR (Interrupted system call)
The 2 file descriptors 6 and 8 are pipes:
$ file /proc/5602/fd/6
/proc/5602/fd/6: broken symbolic link to pipe:[6562495]'
$ file /proc/5602/fd/8
/proc/5602/fd/8: broken symbolic link to pipe:[6562496]'
I do not understand the root cause of the issue and I do not know what to try to fix it.
EDIT:
As suggested by #sushihangover I ran the app in a terminal with --debug option. Eventually the app hanged again. No exception, no error.
The code is acquiring frames from MJPEG streams and doing some work on a background worker to detect object in the frame.
The work only happens when the background worker is not busy. The last log trace I get is:
[Background worker] nothing to do
I suspect the app is hanging while trying to get the next frame from the camera. I am using AForge.NET to read the MJPEG stream. AForge raise an events every time a new frame arrives.
Here is some code:
private static void Camera_NewFrame(object sender, NewImageEventArgs e)
{
var bmp = (Bitmap)e.Frame;
log.DebugFormat("got image " + DateTime.Now.Ticks + " {0} x {1}", bmp.Width, bmp.Height);
if (!bWorker.IsBusy)
{
// Run the background operation to check image and update cloud
log.Debug("Starting background work");
bWorker.RunWorkerAsync();
}
else { } // skip frame
}
private static void BWorker_DoWork(object sender, DoWorkEventArgs e)
{
log.Debug("[Background worker] enter");
if (registeredCar)
{
log.Debug("[Background worker] opening the gate");
OpenGate();
}
else
{
log.Debug("[Background worker] nothing to do");
}
}
And the backtrace given by gdb attached to the hanging process:
(gdb) bt
0 0xb6e19fc0 in poll () at ../sysdeps/unix/syscall-template.S:81
1 0xb4b264be in Mono_Unix_UnixSignal_WaitAny () from /usr/lib/libMonoPosixHelper.so
2 0xb4b6f740 in ?? ()
Related
I have a C# .NET 4.8 project that will cause my antivirus software to prevent it from running the executable file for the project. Here is the error it throws in Visual Studio:
And here is the notification my antivirus software appears to give me:
I have narrowed down the code that is causing this... In summary, I am using a timer that will run every minute, and on its tick event (when the timer is up) it will do a check to look for a file name (I call this file "Terminate"), if there is this file it will close the program after 3 minutes.
The end users are all on a virtual environment on the same network and are all running this program from the same directory. Thus, if there is any maintenance that needs to be performed on these virtual environments, I can manually change the file name from "xTerminate" to "Terminate" and the programs will close in all of those environments without having to get users to do it manually.
public Timer poisonPill = new Timer();
poisonPill.Interval = (1 * 60 * 1000); // every 1 minutes
poisonPill.Tick += new EventHandler(poisonPill_Tick);
poisonPill.Start();
When this 1-minute timer reaches the interval time-length, it runs this event:
private void poisonPill_Tick(object sender, EventArgs e)
{
if (File.Exists("Terminate.txt"))
{
poisonPill.Enabled = false;
string message;
try
{ // Open the text file using a stream reader.
using (StreamReader sr = new StreamReader("Terminate.txt"))
{
// Read the stream to a string
String line = sr.ReadToEnd();
message = line;
}
}
catch (Exception ex)
{
message = "Application closing in 3 minutes for maintenance by administrator";
}
Timer terminator = new Timer();
terminator.Interval = (3 * 60 * 1000);
terminator.Tick += terminator_Tick;
terminator.Start();
MessageBox.Show(message + "\n\nNotification Time: " + DateTime.Now.ToString("h:mm tt") +
"\n\nApplication closes 3 minutes after Notification Time.", "3-Minute Application Closing Alert");
}
}
And then once that 3-minute timer finishes...:
private void terminator_Tick(object sender, EventArgs e)
{
Application.Exit();
}
Anti-Virus software does that often because it's a new executable with weird behavior that it's not used to. It doesn't recognize it like an official app like Steam or Discord. It does it with pirated games, too.
But it's nothing to worry about, you made the app, you know what it was made to do.
The antivirus doesn't know, and it's just defending.
I usually just turn off my antivirus temporarily, or add an exception so it won't turn my app off when I'm testing.
You can add an exception in the antivirus settings, depending on what antivirus you use, there must be a place where you can select the exe location, and the antivirus will ignore it.
Since updating to Windows 10 1809, my application appears to be leaking memory on a WinForms C++/CLI TextBox.Text set call.
The textbox is a log updated from a separate device thread logging Bluetooth communications. After a random period of time (sometimes 10 mins, sometimes a few hours) the GUI event handler will lock on the TextBox.Text set call and WorkingSet memory will increase linearly (~ 10MB/s) for around 30s to a minute before recovering. However the allocated commit memory is retained and then built upon the next time it fails. This builds until the application eventually crashes with a stack overflow or out of memory exception.
This is the bones of the Comms event handler:
System::Void MainForm::OnCommsMessageLog(System::Object^ sender, CustomEventArgs::LogMessageEventArgs^ plogMessageEventArgs)
{
if (InvokeRequired)
{
Invoke(gcnew commsMessageDelegate(this, &MainForm::OnCommsMessageLog), sender, plogMessageEventArgs);
}
else
{
msclr::lock OnCommsMessageLogLock(mpOnCommsLogLock);
String^ logMessage = plogMessageEventArgs->LogMessage;
// Pipe out to file to check for log contents
StreamWriter^ pWriter = gcnew StreamWriter(mLogFilePath + pBthDevice->Name + ".txt", true);
pWriter->Write(logMessage);
pWriter->Close();
mpDeviceLog += logMessage; // Add new log line to String member variable
if(mpDeviceLog->Length > MAX_LOG_LENGTH) // MAX_LOG_LENGTH = 100000
{
mpDeviceLog = mpDeviceLog->Substring(MAX_LOG_LENGTH / 5);
logTextBox->Text = mpDeviceLog; // <-- LOCKS HERE
}
else
{
logTextBox->AppendText(logMessage);
//logTextBox->Text += logMessage; // <-- ALTERNATIVELY LOCKS HERE IF THIS METHOD IS USED
}
}
}
This cannot be replicated on any non-1809 machine, which could be coincidental, but seems unlikely.
The machines in question are relatively low spec, running 2GB RAM with a Celeron 1.6GHz processor, but the application is pretty thin, and under full load typically only uses 50MB of WorkingSet memory.
The rate at which it leaks memory appears to be dependant on the contents of the text box at the time of the lockup.
Update
Appears to be an issue with TextBox.MultiLine on any Windows 1809 machine. Can be replicated by creating a small application with a MultiLine text box then running the following code:
logTextBox.Text = "a\r\n";
logTextBox.Text = "a\r"; // Will lock here for 30 to 60s
The issue can be rectified by adding in the following between the Text set operations:
logTextBox.Clear();
logTextBox.Multiline = false;
logTextBox.Multiline = true;
Issue has also been raised on MSDN awaiting a reply.
Even though there are command-line commands to start (most of?) the various Control Panel screens in Windows 10, a specific scenario seems to fail:
If the machine starts with Bluetooth turned off (not disabled), running the command which should open the Bluetooth settings screen, simply does nothing. The command could be either ms-settings:bluetooth, bthprops.cpl or ms-settings:Bluetooth.
I've also tried to directly launch the Bluetooth Devices screen (using the command %windir%\explorer.exe shell:::{28803F59-3A75-4058-995F-4EE5503B023C} as described here), but clicking on the "Bluetooth settings" in this window does nothing as well.
The only way to get directly to the Bluetooth settings screen without going through the main Control Panel window and without turning on Bluetooth first, is by right clicking on the relevant tile in Windows Action Center:
Although this seems like a bug on the operating system level, I was wondering if there's any way to know when the launch fails from within C# code. So I've tried using the following code:
try
{
var process = new Process();
process.StartInfo.FileName = "control";
process.StartInfo.Arguments = "bthprops.cpl";
process.Exited += (s, e) =>
{
if (process.ExitCode != 0)
{
TurnOnBt();
}
};
var res = process.Start();
if (!res)
{
TurnOnBt();
}
}
catch (System.Exception ex)
{
int test = 6; // just for breakpoint
}
Problem is, no exception was ever thrown, and most of the time the Process.Exit event was never called.
Further more, calling Windows.Devices.Radios.Radio.GetRadiosAsync() returns an empty list!
Currently the only solution I've found is to manually turn on Bluetooth - it wouldn't change the Process.Start/Exit behavior, but it does allow to successfully lunch the command to directly open Bluetooth Settings window, and to get the list of the machine's Bluetooth/Radio devices. Still, when turning off Bluetooth and restarting the machine, same problem would happen all over again.
Any ideas for a code-based workaround?
note - all this based only on my debugging research, nothing from this is documented
i look how BT-settings window is open via Action Center (win8.1, win 10):
the IApplicationActivationManager interface created and called ActivateApplication method with:
appUserModelId = L"windows.immersivecontrolpanel_cw5n1h2txyewy!microsoft.windows.immersivecontrolpanel"
arguments = L"page=SettingsPagePCSystemBluetooth"
code on c++ can be look like:
if (0 <= CoInitialize(0))
{
IApplicationActivationManager* pAppAct;
if (0 <= CoCreateInstance(__uuidof(ApplicationActivationManager), 0, CLSCTX_ALL, IID_PPV_ARGS(&pAppAct)))
{
ULONG dwProcessId;
pAppAct->ActivateApplication(
L"windows.immersivecontrolpanel_cw5n1h2txyewy!microsoft.windows.immersivecontrolpanel",
L"page=SettingsPagePCSystemBluetooth",
AO_NONE ,
&dwProcessId);
pAppAct->Release();
}
CoUninitialize();
}
the processId (if all ok) reference to "X:\Windows\ImmersiveControlPanel\SystemSettings.exe" -ServerName:microsoft.windows.immersivecontrolpanel
for c# - look IApplicationActivationManager::ActivateApplication in C#?
the "windows.immersivecontrolpanel_cw5n1h2txyewy!microsoft.windows.immersivecontrolpanel" and "page=SettingsPagePCSystemBluetooth" strings not documented anywhere and I not sure are it is "persist" - but currently it used to open manage Bluetooth page in system settings. as is.
in case we run control.exe bthprops.cpl - the process ( control.exe ) launched without any error - as result you and not got any errors when you call this code.
then control.exe bthprops.cpl exec new process rundll32.exe Shell32.dll,Control_RunDLL bthprops.cpl, and exit
the rundll32.dll call Control_RunDLLW(HWND_DESKTOP, (HINSTANCE)&__ImageBase, L"bthprops.cpl", SW_SHOWDEFAULT);
we can and direct call void WINAPI Control_RunDLLW(HWND hwndParent, HINSTANCE hInst, PCWSTR cplName, int nCmdShow ); this api exported from shell32.dll
internally Control_RunDLLW load "bthprops.cpl" (3-rd argument - cplName), find CPlApplet entry point, and send CPL_INIT message. the bthprops.cpl on this message check are bthserv is running via OpenService(L"BTHSERV", ) + QueryServiceStatus (in function BthpIsbluetoothServiceRunning) and if "BTHSERV" not running - it return zero (fail code)
I am unable to successfully start multiple processes from a console application.
I have a visual studio solution with a console application A, which must start multiple instances of another console application (B).
B runs in a loop catching incoming network traffic on a port specified in the process arguments.
To simplify the problem, I have removed any network logic and made it as basic as possible, yet I still have the same problem.
Code for A, where Test.exe is console application B:
static void Main(string[] args)
{
try
{
for (int i = 0; i < 5; i++)
{
System.Diagnostics.Process.Start(new System.Diagnostics.ProcessStartInfo { FileName = "Test.exe" });
}
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex.Message);
}
}
Code for B:
static void Main(string[] args)
{
int counter = 0;
while (Console.KeyAvailable == false || Console.ReadKey().Key != ConsoleKey.Escape)
{
Console.WriteLine("" + counter);
System.Threading.Thread.Sleep(1000);
counter++;
}
Console.ReadLine();
}
Once I run console application A in Visual Studio debug, A pops up as intended, but only one window of B pops up starting to count as specified. However, if I look in my task manager I can see that in fact two Test.exe is running, where one is using 116K memory and the other is using 180,000K memory.
Once the counter reaches 15, it closes and opens two new windows which both starts counting from 0. This behavior continues, opening new instances of B every time counter reaches 15 until 5 instances is running as specified in A.
Obviously this behaviour is not what I intended, I want A to launch B 5 times (in this example) immediately without waiting for the proccesses to exit.
After re-installing .net and removing certain features I no longer needed, it is finally working as intended. Starting 50 instances is no problem and the weird memory usage is no longer evident.
The culprit was either Windows Phone SDK or Windows Azure for VS.
I have a .net app that does a variety of file operations. It has been scheduled via task manager and runs without issue. We are moving the job to be controlled by autosys and have the job configured. When it kicks off I see all the files move as expected and I get a log file indicating that everything ran as expected. The app is working. Autosysy, however, reports that it failed.
Status/[Event] Time Ntry ES ProcessTime Machine
-------------- --------------------- -- -- --------------------- ----------------------------------------
RUNNING 09/26/2013 15:30:21 1 PD 09/26/2013 15:31:12
FAILURE 09/26/2013 15:31:59 1 PD 09/26/2013 15:32:17
[*** ALARM ***]
JOBFAILURE 09/26/2013 15:32:16 1 PD 09/26/2013 15:32:17
[STARTJOB] 09/26/2013 16:00:00 0 UP
The application is a winform app - here's the meat of the code:
static int Main(string[] args)
{
Console.WriteLine("Starting processing...");
Console.WriteLine(DateTime.Now.ToString(CultureInfo.InvariantCulture));
if (args.Length > 0) //if we call the app with args we do some stuff, otherwise we show the UI to let the user choose what to do
{
//stuff happens here that works, other method calls, etc.
Console.WriteLine(DateTime.Now.ToString(CultureInfo.InvariantCulture));
Console.WriteLine("Process complete.");
return 0;
}
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new FileLoader());
return 0;
}
The job IS working, it's doing everything that it is supposed to do without logging any exceptions BUT autosys still reports failure. What am I doing wrong?
Autosys will mark the job as successfully completed when the process ends. In your case you've told that it is an Win-Form application. So what may happen here is the autosys is starting the Application and also the application is working fine and it is doing what it suppose to do. But the process will not end unless untill someone manually close the window application or if you have some techinque to close itself. And so Autosys is assuming that the process not ended thus marking the job as failed.
The solution for this is make your application as console application or else I remember there is one property when setting up the Autosys job not to consider the Process ending so the job will just start the program and mark the job as completed.