Develop Reference

WriteXml not saving a file

I have a task that is saving my DataTable in to a file every 10 seconds:
void WriteTask(DataTable dt, bool final)
{
if (final)
{
exiting = false;
}
while (!exiting)
{
lock (users)
{
try
{
dt.WriteXml(filePath+ "users.xml_be");
File.Copy(#$"{filePath}users.xml_be", #$"{filePath}users.xml_betemp");
File.Replace(#$"{filePath}users.xml_betemp", #$"{filePath}users.xml", #$"{filePath}users.xml_be2");
if (final)
exiting = true;
}
catch (Exception ex)
{
try
{
File.AppendAllText($"{filePath}log.txt", $"{DateTime.Now.ToString()} {ex.Message}" + Environment.NewLine);
File.AppendAllText($"{filePath}log.txt", $"{ex.StackTrace}" + Environment.NewLine);
File.AppendAllText($"{filePath}log.txt", "////////////////////////////////////////////////////////////////" + Environment.NewLine);
}
catch (Exception)
{
client.SendMessage(tChannel, ErrorMes1);
client.SendMessage(tChannel, $"{ex.Message}|{ex.StackTrace}");
}
}
}
Thread.Sleep(10000);
}
}
And recently I noticed that file are not getting changed. Weirdiest think is that when i open file in notepad++, its saying that the file has been chaged and it wants to reload it, but afte I reload the fle there is not chnages inside. Could it be because file is getting pretty big? It have around 1mil of lines.
Update:
So, I created async tasks for all things that have to do with datatables:
Task that is in a unending loop, saving every 30 seconds
async void WriteTask(DataTable dt, bool final)
{
if (final)
{
exiting = false;
}
while (!exiting)
{
Task UvalClockTaksTask = new Task(() => UvalClockTaksAsync());
Task dbSavingTask = new Task(() => WriteTaskAsync(dt, final));
Console.WriteLine("start saving");
UvalClockTaksTask.Start();
await UvalClockTaksTask;
dbSavingTask.Start();
await dbSavingTask;
Console.WriteLine("saved");
Thread.Sleep(30000);
}
}
Here I'm updating timestamp for every user in dt
void UvalClockTaksAsync()
{
foreach (DataRow row in users.Rows)
{
if (Convert.ToDouble(row[uvaltimePost]) < TimeCurrent)
row[uvaltimePost] = TimeCurrent;
}
}
And here I do the saving
void WriteTaskAsync(DataTable dt, bool final)
{
try
{
dt.WriteXml(filePath + "users.xml_be");
File.Replace(#$"{filePath}users.xml_be", #$"{filePath}users.xml", #$"{filePath}users.xml_be2");
if (final)
exiting = true;
}
catch (Exception ex)
{
try
{
File.AppendAllText($"{filePath}log.txt", $"{DateTime.Now.ToString()} {ex.Message}" + Environment.NewLine);
File.AppendAllText($"{filePath}log.txt", $"{ex.StackTrace}" + Environment.NewLine);
File.AppendAllText($"{filePath}log.txt", "////////////////////////////////////////////////////////////////" + Environment.NewLine);
}
catch (Exception)
{
client.SendMessage(tChannel, ErrorMes1);
client.SendMessage(tChannel, $"{ex.Message}|{ex.StackTrace}");
}
}
}
And it still not saving every time and the weirdest thing is that files users.xml and users.xml_be are not the same! Users timestamps are always different by 10s seconds. I mean wtf

If the file has a large volume of data, it could take time to write in a physical path. So can you increase the timeline (Thread.Sleep(10000);)

parallel read/process/write stop working without any exception

I have to read a file and after processing it's data write result to another file. This process takes too much time so I tried to do read/process/write in a parallel way. Code works well. But there is a problem, when I was testing it for processing very huge file (100GB) after about 40GB program stops working. It do not throw any exception (except 'TimeoutException'). I spend several days and tried to change many things. I know it is not about method or memory. But I am really confused about reason and way to make it works well.
Note: I wanted to post this on Code Review, but Code Review rules says do not post for trouble shooting and else so I posted it here.
The way I use code (It is on backgroundworker)
BackgroundWorker worker = (BackgroundWorker)sender;
ReaderWriterMultiThread readerWriterMultiThread = null;
int bufferSize = 2 * 1024 * 1024;
int readerWriterMultiThreadPartsNumber = 10;
int sizeToReadInThisIteration = 0;
int oldprecentage = 0;
long fileDid = 0;
using (FileStream streamReader = new FileStream(fromAddress, FileMode.Open))
using (BinaryReader binaryReader = new BinaryReader(streamReader))
using (FileStream streamWriter = new FileStream(toAddress, FileMode.Open))
using (BinaryWriter binaryWriter = new BinaryWriter(streamWriter))
{
sizeToReadInThisIteration = bufferSize * readerWriterMultiThreadPartsNumber;
streamWriter.Seek(0, SeekOrigin.Begin);
while (streamWriter.Position < length)
{
if (worker.CancellationPending)
{
e.Cancel = true;
return;
}
//change sizeToReadInThisIteration if needs
if (streamWriter.Position + sizeToReadInThisIteration > length)
{ sizeToReadInThisIteration = Convert.ToInt32(length - streamWriter.Position); }
//new it
readerWriterMultiThread = new ReaderWriterMultiThread();
//read/do/write
readerWriterMultiThread.Start(binaryReader, binaryWriter, bufferSize, sizeToReadInThisIteration,
(ref byte[] bytes) => DoNothing(ref bytes));
//report process if needs
fileDid += sizeToReadInThisIteration;
if (((int)(fileDid * 100 / length)) > oldprecentage)
{
oldprecentage = (int)(fileDid * 100 / length);
worker.ReportProgress(oldprecentage);
}
}//while
}//using
DoNothing method is:
public void DoNothing(ref byte[] bufferToCode)
{ }
and ReaderWriterMultiThread class is:( Originally code used threads but I changed it to use tasks.)
public class ReaderWriterMultiThread
{
#region variables
//buffer(contain several part)
List<byte[]> buffer = new List<byte[]>();
//lock objects
private object bufferLockForRead = new object();
private object bufferLockForWrite = new object();
//indexes
int readIndex = 0;
int doReadIndex = 0;
int doWriteIndex = 0;
int writeIndex = 0;
//complete vars
int lastIndex = int.MaxValue;
bool readCompleted = false;
//waiting properties
private bool doIsWaiting = false;
private bool writerIsWaiting = false;
//error properties
private bool anyErrorHappend = false;
private string errorsMessage = string.Empty;
//proc delegate
public delegate void DelegateMethod(ref byte[] bytes);
//proc delegate instance
DelegateMethod delegateM;
//
#endregion variables
//==============================
#region methods
//
public void Start(BinaryReader binaryReader, BinaryWriter binaryWriter, int bufferPartsSize, int size, DelegateMethod delegateMethod)
{
//new delegate
delegateM = new DelegateMethod(delegateMethod);
//for wait all
Task[] tasks = new Task[3];
//run
var parentTask = Task.Factory.StartNew(() =>
{
tasks[0] = Task.Factory.StartNew(() =>
{
Writer(binaryWriter);
});
tasks[1] = Task.Factory.StartNew(() =>
{
Do();
});
tasks[2] = Task.Factory.StartNew(() =>
{
Reader(binaryReader, bufferPartsSize, size);
});
});
//wait
parentTask.Wait();
if (!Task.WaitAll(tasks, 10000))
{ throw new TimeoutException(); }
if (anyErrorHappend)
{ throw new Exception(errorsMessage); }
}
private void AddByReader(byte[] newBytes, bool completed)
{
try
{
lock (bufferLockForRead)
{
//add data to buffer
buffer.Add(newBytes);
//updare readIndex
readIndex++;
//if completed show it
if (completed)
{
readCompleted = true;
lastIndex = buffer.Count;//it uses as <lastIndex (so lastIndex = buffer.Count is ok)
}
//manage happend error
if (anyErrorHappend)
{
readCompleted = true;
lastIndex = doReadIndex + 1;
}
//if do is waiting pulse it
if (doIsWaiting)
{ Monitor.Pulse(bufferLockForRead); }
}
}
catch (Exception ex)
{ Debug.Assert(false, ex.ToString()); }
}
private byte[] GetByDo()
{
try
{
lock (bufferLockForRead)
{
//if data did not read already wait
if (doReadIndex == readIndex)
{
doIsWaiting = true;
Monitor.Wait(bufferLockForRead);
}
//do is not waiting now
doIsWaiting = false;
//in case of emergency
if (doReadIndex > readIndex)
{ return new byte[0]; }
//return
return buffer[doReadIndex++];
}
}
catch (Exception ex)
{
Debug.Assert(false, ex.ToString());
return new byte[0];
}
}
private void AddByDo(byte[] newBytes, string errorMessageFromDO)
{
try
{
lock (bufferLockForWrite)
{
//add data
buffer[doWriteIndex] = newBytes;
//update doWriteIndex
doWriteIndex++;
//error happend in Do
if (errorMessageFromDO.Length > 0)
{
anyErrorHappend = true;
errorsMessage += errorMessageFromDO;
lastIndex = -1;
Monitor.Pulse(bufferLockForWrite);
}
//if reader completed and writer is in wait state pulse it
if (readCompleted && writerIsWaiting)
{
Monitor.Pulse(bufferLockForWrite);
}
}
}
catch (Exception ex)
{ Debug.Assert(false, ex.ToString()); }
}
private byte[] GetByWriter()
{
try
{
lock (bufferLockForWrite)
{
//if data did not proccessed wait
if (writeIndex == doWriteIndex)
{
writerIsWaiting = true;
Monitor.Wait(bufferLockForWrite);
}
//writer is not waithing
writerIsWaiting = false;
//return
return buffer[writeIndex++];
}
}
catch (Exception ex)
{
Debug.Assert(false, ex.ToString());
return new byte[0];
}
}
private void Reader(BinaryReader binaryReader, int bufferPartSize, int sizeToRead)
{
try
{
//vars
bool completed = false;
int readedSize = 0;
byte[] readedBytes = new byte[0];
while (readedSize < sizeToRead && !anyErrorHappend)
{
//change bufferPartSize & completed if needs
if (readedSize + bufferPartSize >= sizeToRead)
{
bufferPartSize = sizeToRead - readedSize;
completed = true;
}
try
{
//read
readedBytes = binaryReader.ReadBytes(bufferPartSize);
}
catch (Exception ex)
{
Debug.Assert(false, ex.ToString());
//error happend
anyErrorHappend = true;
errorsMessage += ex.Message;
//for pulse Do() if it is waiting
byte[] amptyBytesArray = new byte[0];
AddByReader(amptyBytesArray, true);//it is better to do it instead change lastIndex here
break;
}
//add to buffer
AddByReader(readedBytes, completed);
//update readedSize
readedSize += bufferPartSize;
}
}
catch (Exception ex)
{ Debug.Assert(false, ex.ToString()); }
}
private void Writer(BinaryWriter binaryWriter)
{
try
{
//vars
byte[] bytesToWrite = new byte[0];//for put getted data in
for (int i = 0; i < lastIndex; i++)
{
//get data from buffer
bytesToWrite = GetByWriter();
try
{
//write
binaryWriter.Write(bytesToWrite);
}
catch (Exception ex)
{
Debug.Assert(false, ex.ToString());
lastIndex = -1;
anyErrorHappend = true;
errorsMessage = ex.Message;
break;
}
}
}
catch (Exception ex)
{ Debug.Assert(false, ex.ToString()); }
}
private void Do()
{
try
{
//vars
byte[] bytes = new byte[0];//for put readed data/result in
for (int i = 0; i < lastIndex; i++)
{
//get data from buffer
bytes = GetByDo();
try
{
//do
delegateM(ref bytes);
}
catch (Exception ex)
{
Debug.Assert(false, ex.ToString());
//add
AddByDo(new byte[0], "error: " + ex.Message);
break;
}
//add data to buffer
AddByDo(bytes, string.Empty);
}
}
catch (Exception ex)
{ Debug.Assert(false, ex.ToString()); }
}
//
#endregion methods
}

You code is throwing an exception here:
if (!Task.WaitAll(tasks, 10000))
{ throw new TimeoutException(); }
Which means that one of your tasks is taking more than 10 seconds to complete.
This could be because it needs more time. It could also be because one of your tasks is getting stuck waiting to enter a lock. It could also because the Monitor.Wait is getting stuck waiting for a lock.
You can add logging to see where it is getting hung. Additionally if one of your assert statements fail it will prevent a task from completing in time.
As a side note, Disk IO is time consuming and in general trying to parallelize IO operations like this typically aren't going to help because the threads end up stomping all over each other contending for disk access time. The disk ends up needing to spin to numerous positions back and forth to read and write data and you end up with a net effect of actually slowing things down. You might be able to speed things up if you are using RAID or you are reading from one disk and adding the reads to a queue and your write thread is reading from that queue and writing that data to a different disk.

Exception Handiling in Independent functions, without stoping the C# Console Application

I am working on C# console application.
Want to impliment "Exception Handling".
Below are the code of ConsoleApp.
static public void Main(string[] args)
{
try
{
for (int i = 0; i < 10; i++)
{
ONE_IndependentProcess(i);
}
TWO_IndependentProcess();
THR_IndependentProcess();
}
catch (System.TimeoutException ex)
{
Console.WriteLine("System.TimeoutException");
}
catch (System.Exception ex)
{
Console.WriteLine("System.Exception");
}
finally
{
Console.WriteLine("End.");
}
}
If an error occurs in TWO_IndependentProcess... application stops, without executing the THR_IndependentProcess
#1. In this case i want THR_IndependentProcess to be execute, and write a Exception Error Information of TWO_IndependentProcess in the Console.
If an error occurs in the ONE_IndependentProcess(3) (in 3rd iteration) application stops, without executing the: remaining iteration(4 to 9), TWO_IndependentProcess & THR_IndependentProcess
#2. In this case also i want the app to execute all, and just write a Exception Error Information of ONE_IndependentProcess(3) in the Console.
Note: I have four Catch block with Detail Exception Information and i want to apply all these four catch block in all three IndependentProcess function.

I would recommend you keep just one copy of your error handling logic and make use of lambda functions to wrap the calls with it:
static private void WithCatch(Action f)
{
try
{
f();
}
catch (System.TimeoutException ex)
{
Console.WriteLine("System.TimeoutException");
}
catch (System.Exception ex)
{
Console.WriteLine("System.Exception");
}
}
static public void Main(string[] args)
{
WithCatch(() => {
for (int i = 0; i < 10; i++)
ONE_IndependentProcess(i);
});
// You could also do this inside the for loop for each one if you want
// to attempt all 10 even if one fails:
//for (int i = 0; i < 10; i++)
// WithCatch(() => {ONE_IndependentProcess(i);});
WithCatch(() => {TWO_IndependentProcess();});
WithCatch(() => {THR_IndependentProcess();});
}

You need to move each independent process into a separate try/catch block.
e.g.
static public void Main(string[] args)
{
try
{
for (int i = 0; i < 10; i++)
{
ONE_IndependentProcess(i);
}
}
catch (System.TimeoutException ex)
{
Console.WriteLine("System.TimeoutException");
}
catch (System.Exception ex)
{
Console.WriteLine("System.Exception");
}
finally
{
Console.WriteLine("End.");
}
try
{
TWO_IndependentProcess();
}
catch (System.TimeoutException ex)
{
Console.WriteLine("System.TimeoutException");
}
catch (System.Exception ex)
{
Console.WriteLine("System.Exception");
}
finally
{
Console.WriteLine("End.");
}
try
{
THR_IndependentProcess();
}
catch (System.TimeoutException ex)
{
Console.WriteLine("System.TimeoutException");
}
catch (System.Exception ex)
{
Console.WriteLine("System.Exception");
}
finally
{
Console.WriteLine("End.");
}
}

You will have to use catch blocks within independent functions/process and deal with the exception within that catch block. This way you can allow program execution to continue without terminating.
try using generic functions to record exceptions - that way you can reuse this function within different catch blocks.

Printing error code on the console

I have this C# windows application. Exceptions occur at times. When an exception occurs, I want to drill down on the Error Code for that exception and then fetch and print it.
Posting sample code
public void CreateSkill(int row)
{
try
{
Excel1.MWMClient.MWMServiceProxy.Skill skill = new Excel1.MWMClient.MWMServiceProxy.Skill();
skill.name = "perl";
// skill.description = "bowler";
skill.expertiseLevel = "3";
skill.ID = 1;
Console.WriteLine(skillClient.CreateSkill(skill));
Console.WriteLine(skill.ID);
ExcelRecorder(null, row);
}
catch (Exception ex)
{
ExcelRecorder(ex.Message, row);
}
finally
{
System.GC.Collect();
}
}
ExcelRecorder() is a method which prints the exception message in a cell in an excel sheet.
I want to print the exception as well as the error code in my excel sheet. How do I fetch this error code through my code?
Posting an image

Try this one:
try {
}
catch(Exception e) {
int code = e.HResult;
}
or even:
try {
}
catch (Exception e){
var Wex = e as Win32Exception;
int code = Wex.ErrorCode;
};

System.NullReferenceException Not Being Caught in First Try/Catch Block

I have a C# Console app that starts in 'static int Main(string[] args)', creates an instance of 'EventRecievedProcessor' class and then calls a method on the instance:
static int Main(string[] args)
{
try
{
EventRecievedProcessor proc = new EventRecievedProcessor
if (!proc.Processs())
{
Console.Write(Type + " processing failed. Please check logs for more information.");
Log.Error("Failed to process s");
return (int)RETURNCODES.INTERNALAPPERROR;
}
}
catch (Exception ex)
{
// This is where the System.NullReferenceException from GetLatestEventInfo is currently being caught
Console.WriteLine("Exception message: " + ex.Message);
Console.WriteLine("Exception stack trace: " + ex.StackTrace);
Log.Fatal("An exception has been thrown. Message: " + ex.Message, ex);
return (int)RETURNCODES.INTERNALAPPERROR;
}
}
The instance of 'EventRecievedProcessor' grabs a collection of records and does a foreach over it. It calls a static method (GetLatestEventInfo) on the 'Event' class for each record in the collection:
public class EventRecievedProcessor
{
public bool Processs()
{
List<Event> events = DAL.GetEvents;
foreach (Event e in events)
{
try
{
EventInfo lastEvent = Eventhistory.GetLatestEventInfo(e);
}
catch (Exception ex)
{
// Log exception and continue processing in foreach loop
// This is where I want to catch the NullReferenceException from GetLatestEventInfo
Log.DebugFormat("Error with eventid " + e.EventID);
Log.Error(ex);
}
}
return true;
}
}
When the follwoing method is called, a System.NullReferenceException is thrown:
public class EventHistory
{
public static EventInfo GetLatestEventInfo(int customerCode, string premiscode)
{
EventInfo info = new EventInfo();
// Do some work here...
// This is where the NullReferenceException is being generated.
return info;
}
}
When the NullReferenceException is thrown here, I would expect the catch block in the foreach loop to catch it, log it, and then return control to the foreach loop to continue processing. Instead, the exception is being caught in the top level 'Main' method, which means the app aborts and the remaining records are not processed.
I'm at loss as to how/why the exception bypasses the first catch block. Any ideas on what I'm doing wrong here?
Adding the stack trace:
System.NullReferenceException: Object reference not set to an instance of an object.
at EventProcessor.EventHistory.GetLatestEventInfo(Event e) in C:\Dev\release6.01.100\Events\EventProcessor\EventProcessor\EventHistory.cs:line 65
at EventProcessor.Processors.EventProcessor.Process() in C:\Dev\release6.01.100\Events\EventProcessor\EventProcessor\Processors\EventProcessor.cs:line 32
at EventProcessor.Program.Main(String[] args) in C:\Dev\release6.01.100\Events\EventProcessor\EventProcessor\Program.cs:line 132
Sorry if I've munched some of the names. This is work code, so I tried to change it up a little to avoid any privacy conflicts.

It doesn't bypass anything. Look closely at your stack trace.
Try using Console.WriteLine(ex.ToString());.
You'll see that the exception is not being thrown from where you thought it was.

This simply isn't the case and here's the proof:
class Program
{
static void Main()
{
// no need of try/catch here as exceptions won't propagate to here
Looper();
}
static void Looper()
{
int processedRecords = 0;
for (int i = 0; i < 10; i++)
{
try
{
Thrower(i);
processedRecords++;
}
catch (NullReferenceException ex)
{ }
}
// prints 5 as expected
Console.WriteLine("processed {0} records", processedRecords);
}
static void Thrower(int i)
{
if (i % 2 == 0)
{
throw new NullReferenceException();
}
}
}