Develop Reference

System.Core Pipe is broken

I currently just inherited some complex code that unfortunately I do not fully understand. It handles a large number of inventory records inputting/outputting to a database. The solution is extremely large/advanced where I am still on the newer side of c#. The issue I am encountering is that periodically the program will throw an IO Exception. It doesn't actually throw a failure code, but it messes up our output data.
the try/catch block is as follows:
private static void ReadRecords(OleDbRecordReader recordReader, long maxRows, int executionTimeout, BlockingCollection<List<ProcessRecord>> processingBuffer, CancellationTokenSource cts, Job theStack, string threadName) {
ProcessRecord rec = null;
try {
Thread.CurrentThread.Name = threadName;
if(null == cts)
throw new InvalidOperationException("Passed CancellationToken was null.");
if(cts.IsCancellationRequested)
throw new InvalidOperationException("Passed CancellationToken is already been cancelled.");
long reportingFrequency = (maxRows <250000)?10000:100000;
theStack.FireStatusEvent("Opening "+ threadName);
recordReader.Open(maxRows, executionTimeout);
theStack.FireStatusEvent(threadName + " Opened");
theStack.FireInitializationComplete();
List<ProcessRecord> inRecs = new List<PIRecord>(500);
ProcessRecord priorRec = rec = recordReader.Read();
while(null != priorRec) { //-- note that this is priorRec, not Rec. We process one row in arrears.
if(cts.IsCancellationRequested)
theStack.FireStatusEvent(threadName + " cancelling due to request or error.");
cts.Token.ThrowIfCancellationRequested();
if(rec != null) //-- We only want to count the loop when there actually is a record.
theStack.RecordCountRead++;
if(theStack.RecordCountRead % reportingFrequency == 0)
theStack.FireProgressEvent();
if((rec != null) && (priorRec.SKU == rec.SKU) && (priorRec.Store == rec.Store) && (priorRec.BatchId == rec.BatchId))
inRecs.Add(rec); //-- just store it and keep going
else { //-- otherwise, we need to process it
processingBuffer.Add(inRecs.ToList(),cts.Token); //-- note that we don't enqueue the original LIST! That could be very bad.
inRecs.Clear();
if(rec != null) //-- Again, we need this check here to ensure that we don't try to enqueue the EOF null record.
inRecs.Add(rec); //-- Now, enqueue the record that fired this condition and start the loop again
}
priorRec = rec;
rec = recordReader.Read();
} //-- end While
}
catch(OperationCanceledException) {
theStack.FireStatusEvent(threadName +" Canceled.");
}
catch(Exception ex) {
theStack.FireExceptionEvent(ex);
theStack.FireStatusEvent("Error in RecordReader. Requesting cancellation of other threads.");
cts.Cancel(); // If an exception occurs, notify all other pipeline stages, then rethrow
// throw; //-- This will also propagate Cancellation, but that's OK
}
In the log of our job we see the output loader stopping and the exception is
System.Core: Pipe is broken.
Does any one have any ideas as to what may cause this? More importantly, the individual who made this large-scale application is no longer here. When I debug all of my applications, I am able to add break points in the solution and do the standard VS stepping through everything to find the issue. However, this application is huge and has a GUI that pops up when I debug the application. I believe the GUI was made for testing purposes, but it hinders me from actually being able to step through everything. However when the .exe is run from our actual job stream, there is no GUI it just executes the way it's supposed to.
The help I am asking for is 2 things:
just suggestions as to what may cause this. Could an OleDB driver be the cause? Reason I ask is because I have this running on 2 different servers. One test and one not. The one with a new OleDB driver version does not fail (7.0 i believe whereas the other where it fails is 6.0).
Is there any code that I could add that may give me a better indication as to what may be causing the broken pipe? The error only happens periodically. If I run the job again right after, it may not happen. I'd say it's 30-40% of the time it throws the exception.
If you have any additional questions about the structure just let me know.

Checking if an object exists after calling Activator.GetObject

I'm developing a project with passive replication where servers exchange messages among themselves. The locations of each server are well-known by every other server.
So, it may happen that when a server comes up, it will check the other servers, that may haven't come up yet. When I call Activator.GetObject, is it the only way to find out that other servers are down by invoking a method on the object, and expect an IOException (such as the example below)?
try
{
MyType replica = (MyType)Activator.GetObject(
typeof(IMyType),
"tcp://localhost:" + location + "/Server");
replica.ping();
}
catch (IOEXception){} // server is down
I do this and it works most of the times (even though is slow), but sometimes it blocks on a method called NegotiateStream.ProcessRead during the process, and I can't understand why...

When a server is down, the timeout has always been slow for me (using a TcpChannel, which doesn't let you set the timeout properly in .NET Remoting). Below is a workaround for how I use my Ping function (it's likely a bit complex for your needs, so I'll explain the parts that matter for you):
[System.Diagnostics.DebuggerHidden] // ignore the annoying breaks when get exceptions here.
internal static bool Ping<T>(T svr)
{
// Check type T for a defined Ping function
if (svr == null) return false;
System.Reflection.MethodInfo PingFunc = typeof(T).GetMethod("Ping");
if (PingFunc == null) return false;
// Create a new thread to call ping, and create a timeout of 5 secons
TimeSpan timeout = TimeSpan.FromSeconds(5);
Exception pingexception = null;
System.Threading.Thread ping = new System.Threading.Thread(
delegate()
{
try
{
// just call the ping function
// use svr.Ping() in most cases
// PingFunc.Invoke is used in my case because I use
// reflection to determine if the Ping function is
// defined in type T
PingFunc.Invoke(svr, null);
}
catch (Exception ex)
{
pingexception = ex;
}
}
);
ping.Start(); // start the ping thread.
if (ping.Join(timeout)) // wait for thread to return for the time specified by timeout
{
// if the ping thread returned and no exception was thrown, we know the connection is available
if (pingexception == null)
return true;
}
// if the ping thread times out... return false
return false;
}
Hopefully the comments explain what I do here, but I'll give you a breakdown of the whole function. If you're not interested, just skip down to where I explain the ping thread.
DebuggerHidden Attribute
I set the DebuggerHidder attribute because when debugging, exceptions can be thrown here constantly in the ping thread, and they are expected. It is easy enough to comment this out should debugging this function become necessary.
Why I use reflection and a generic type
The 'svr' parameter is expected to be a type with a Ping function. In my case, I have a few different remotable interfaces implemented on the server with a common Ping function. In this way, I can just call Ping(svr) without having to cast or specify a type (unless the remote object is instantiated as an 'object' locally). Basically, this is just for syntactical convenience.
The Ping Thread
You can use whatever logic you want to determine an acceptable timeout, in my case, 5 seconds is good. I create a TimeSpan 'timeout' with a value of 5 seconds, an Exception pingexception, and create a new thread that tries to call 'svr.Ping()', and sets 'pingexception' to whatever exception is thrown when calling 'svr.Ping()'.
Once I call 'ping.Start()', I immediately use the boolean method ping.Join(TimeSpan) to wait for the thread to return successfully, or move on if the thread doesn't return within the specified amount of time. However, if the thread finished executing but an exception was thrown, we still don't want Ping to return true because there was a problem communicating with the remote object. This is why I use the 'pingexception' to make sure that no exceptions occurred when calling svr.Ping(). If 'pingexception' is null at the end, then I know I'm safe to return true.
Oh and to answer the question you originally asked (....sometimes it blocks on a method called NegotiateStream.ProcessRead during the process, and I can't understand why...), I have never been able to figure out the timeout issues with .NET Remoting, so this method is what I've baked and cleaned up for our .NET Remoting needs.

I've used an improved version of this with generics:
internal static TResult GetRemoteProperty<T, TResult>(string Url, System.Linq.Expressions.Expression<Func<T, TResult>> expr)
{
T remoteObject = GetRemoteObject<T>(Url);
System.Exception remoteException = null;
TimeSpan timeout = TimeSpan.FromSeconds(5);
System.Threading.Tasks.Task<TResult> t = new System.Threading.Tasks.Task<TResult>(() =>
{
try
{
if (expr.Body is System.Linq.Expressions.MemberExpression)
{
System.Reflection.MemberInfo memberInfo = ((System.Linq.Expressions.MemberExpression)expr.Body).Member;
System.Reflection.PropertyInfo propInfo = memberInfo as System.Reflection.PropertyInfo;
if (propInfo != null)
return (TResult)propInfo.GetValue(remoteObject, null);
}
}
catch (Exception ex)
{
remoteException = ex;
}
return default(TResult);
});
t.Start();
if (t.Wait(timeout))
return t.Result;
throw new NotSupportedException();
}
internal static T GetRemoteObject<T>(string Url)
{
return (T)Activator.GetObject(typeof(T), Url);
}

How to in case of timeout to execute method again and again until it completes successfully?

I have asp.net application. All business logic in business layer.
Here is the example of the method
public void DoSomething()
{
PersonClass pc = new PersonClass();
pc.CreatePerson();
pc.AssignBasicTask();
pc.ChangePersonsStatus();
pc.CreateDefaultSettings();
}
what happens once in a while, one of the sub method can timeout, so as a result the process can be incompleted.
what I think in this case to make sure all steps completed properly is
public void DoSomething()
{
PersonClass pc = new PersonClass();
var error = null;
error = pc.CreatePerson();
if(error != timeout exception)
error = pc.AssignBasicTask();
else
return to step above
if(error != timeout exception)
error = pc.ChangePersonsStatus();
else
return to step above
if(error != timeout exception)
error = pc.CreateDefaultSettings();
else
return to step above
}
but it's just an idea, more then sure it's a proper way how to handle this.

Of course, this can be done more or less elegantly, with different options for timing out or giving up - but an easy way to achieve what you want, would be to define a retry method which keeps retrying an action until it succeeds:
public static class RetryUtility
{
public T RetryUntilSuccess<T>(Func<T> action)
{
while(true)
{
try
{
return action();
}
catch
{
// Swallowing exceptions is BAD, BAD, BAD. You should AT LEAST log it.
}
}
}
public void RetryUntilSuccess(Action action)
{
// Trick to allow a void method being passed in without duplicating the implementation.
RetryUntilSuccess(() => { action(); return true; });
}
}
Then do
RetryUtility.RetryUntilSuccess(() => pc.CreatePerson());
RetryUtility.RetryUntilSuccess(() => pc.AssignBasicTask());
RetryUtility.RetryUntilSuccess(() => pc.ChangePersonsStatus());
RetryUtility.RetryUntilSuccess(() => pc.CreateDefaultSettings());
I must urge you to think about what to do if the method keeps failing, you could be creating an infinite loop - perhaps it should give up after N retries or back off with exponentially raising retry time - you will need to define that, since we cannot know enough about your problem domain to decide that.

You have it pretty close to correct in your psuedo-code, and there a lot of ways to do this, but here is how I would do it:
PersonClass pc = new PersonClass();
while(true)
if(pc.CreatePerson())
break;
while(true)
if(pc.AssignBasicTask())
break;
This assumes that your methods return true to indicate success, false to indicate a timeoiut failure (and probably an exception for any other kind of failure). And while I didn't do it here, I would strongly recommend some sort of try counting to make sure it doesn't just loop forever and ever.

Use a TransactionScope for to make sure everything is executed as a unit. More info here: Implementing an Implicit Transaction using Transaction Scope
You should never retry a timed out operation infinitely, you may end up hanging the server or with an infinite loop or both. There should always be a threshold of how many retries is acceptable to attempt before quitting.
Sample:
using(TransactionScope scope = new TransactionScope())
{
try
{
// Your code here
// If no errors were thrown commit your transaction
scope.Complete();
}
catch
{
// Some error handling
}
}

Best way to handle a WCF timeout

I have a real time app that tracks assets around a number of sites across the country. As part of this solution I have 8 client apps that update a central server.
My question is that sometimes the apps lose connection to the central server and I am wondering what is the best way to deal with this ? I know I could just increase the max send/receive times to deal with the timeout BUT I also want a graceful solution to deal with if the connection to the server is down:
For example I'm calling my services like this :
using (var statusRepository = new StatusRepositoryClient.StatusRepositoryClient())
{
statusId = statusRepository.GetIdByName(licencePlateSeen.CameraId.ToString());
}
I was thinking of adding a try/catch so...
using (var statusRepository = new StatusRepositoryClient.StatusRepositoryClient())
{
try
{
statusId = statusRepository.GetIdByName(licencePlateSeen.CameraId.ToString());
}
catch (TimeoutException timeout)
{
LogMessage(timeout);
}
catch (CommunicationException comm)
{
LogMessage(comm);
}
}
Dealing it this way doesn't allow me to rerun the code without having a ton of code repeat. Any one got any suggestions ?
EDIT: Looking into Sixto Saez and user24601 answers having an overall solution is better than dealing with timeouts on an individual exception level BUT... I'm was thinking that the below would solve my problem (but it would add a TON of extra code error handling):
void Method(int statusId)
{
var statusRepository = new StatusRepositoryClient.StatusRepositoryClient()
try
{
IsServerUp();
statusId = statusRepository.GetIdByName(licencePlateSeen.CameraId.ToString());
statusRepository.Close();
}
catch (Exception ex)
{
statusRepository.Abort();
if (ex is TimeoutException || ex is CommunicationException)
{
LogMessage(timeout);
Method(statusId);
}
else
{
throw new Exception(ex.Message + ex.InnerException);
}
}
}
}
bool IsServerUp()
{
var x = new Ping();
var reply = x.Send(IPAddress.Parse("127.0.0.1"));
if (reply == null)
{
IsServerUp();
}
else
{
if (reply.Status != IPStatus.Success)
{
IsServerUp();
}
}
return true;
}

For starters I think your Wcf error handling is wrong. It should look like this
var statusRepository = new StatusRepositoryClient.StatusRepositoryClient();
try
{
statusId = statusRepository.GetIdByName(licencePlateSeen.CameraId.ToString());
statusRepository.Close()
}
catch(Exception e)
{
statusRepository.Abort();
LogMessage(e);
throw; //I would do this to let user know.
}
I would also re-throw the error to let the user know about the problem.

Prior to designing your exception handling, one important decision to make is whether you want guaranteed delivery of each message the client sends or is it OK for the service to "lose" some. For guaranteed delivery, the best built-in solution is the netMsmqBinding assuming the client can be configured to support it. Otherwise, there is a lightweight reliable messaging capability built into WCF. You'll be going down a rabbit hole if you try to handle message delivery purely through exception handling... :)

I have a two-pronged approach to verifying the server is up:
1) I have set up a 'PING' to the server every 5 seconds. The server responds with a 'PONG' and a load rating (low, medium, high, so the client can adjust its load on the server). If the client EVER doesn't receive a pong it assumes the server is down (since this is very low stress on the server - just listen and respond).
2) Random timeouts like the one you are catching are logged in a ConnectionMonitor class along with all successful connections. A single one of these calls timing out is not enough to consider the server down since some may be very processor heavy, or may just take a very long time. However, a high enough percentage of these will cause the application to go into server timeout.
I also didn't want to throw up a message for every single connection timeout, because it was happening too frequently to people who use poorer servers (or just some computer lying in their lab as a server). Most of my calls can be missed once or twice, but missing 5 or 6 calls are clearly going to cause instrusion.
When a server-timeout state happens, I throw up a little dialog box explaining what's happening to the user.

Hi Please see my solution below. Also please note that the below code has not been compliled so may have some logic and typing errors.
bool IsServerUp()
{
var x = new Ping();
var reply = x.Send(IPAddress.Parse("127.0.0.1"));
if (reply == null) return false;
return reply.Status == IPStatus.Success ? true : false;
}
int? GetStatusId()
{
try
{
using (var statusRepository = new StatusRepositoryClient.StatusRepositoryClient())
{
return statusRepository.GetIdByName(licencePlateSeen.CameraId.ToString());
}
}catch(TimeoutException te)
{
//Log TimeOutException occured
return null;
}
}
void GetStatus()
{
try
{
TimeSpan sleepTime = new TimeSpan(0,0,5);
int maxRetries = 10;
while(!IsServerUp())
{
System.Threading.Thead.Sleep(sleepTime);
}
int? statusId = null;
int retryCount = 0;
while (!statusId.HasValue)
{
statusId = GetStatusId();
retryCount++;
if (retryCount > maxRetries)
throw new ApplicationException(String.Format("{0} Maximum Retries reached in order to get StatusId", maxRetries));
System.Threading.Thead.Sleep(sleepTime);
}
}catch(Exception ex)
{
//Log Exception Occured
}
}

Example of "using exceptions to control flow" [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
What would a piece of code which "uses exceptions to control flow" look like? I've tried to find a direct C# example, but cannot. Why is it bad?
Thanks

By definition, an exception is an occurrence which happens outside the normal flow of your software. A quick example off the top of my head is using a FileNotFoundException to see if a file exists or not.
try
{
File.Open(#"c:\some nonexistent file.not here");
}
catch(FileNotFoundException)
{
// do whatever logic is needed to create the file.
...
}
// proceed with the rest of your program.
In this case, you haven't used the File.Exists() method which achieves the same result but without the overhead of the exception.
Aside from the bad usage, there is overhead associated with an exception, populating the properties, creating the stack trace, etc.

It's roughly equivalent to a goto, except worse in terms of the word Exception, and with more overhead. You're telling the code to jump to the catch block:
bool worked;
try
{
foreach (Item someItem in SomeItems)
{
if (someItem.SomeTestFailed()) throw new TestFailedException();
}
worked = true;
}
catch(TestFailedException testFailedEx)
{
worked = false;
}
if (worked) // ... logic continues
As you can see, it's running some (made-up) tests; if they fail, an exception is thrown, and worked will be set to false.
Much easier to just update the bool worked directly, of course!
Hope that helps!

Bad
The below code catches an exception that could easily be avoided altogether. This makes the code more difficult to follow and typically incurs a performance cost as well.
int input1 = GetInput1();
int input2 = GetInput2();
try
{
int result = input1 / input2;
Output("{0} / {1} = {2}", input1, input2, result);
}
catch (OverflowException)
{
Output("There was an overflow exception. Make sure input2 is not zero.");
}
Better
This code checks for a condition that would throw an exception, and corrects the situation before the error occurs. This way there is no exception at all. The code is more readable, and the performance is very likely to be better.
int input1 = GetInput1();
int input2 = GetInput2();
while (input2 == 0)
{
Output("input2 must not be zero. Enter a new value.");
input2 = GetInput2();
}
int result = input1 / input2;
Output("{0} / {1} = {2}", input1, input2, result);

Here's a common one:
public bool TryParseEnum<T>(string value, out T result)
{
result = default(T);
try
{
result = (T)Enum.Parse(typeof(T), value, true);
return true;
}
catch
{
return false;
}
}

Probably the grossest violation I've ever seen:
// I haz an array...
public int ArrayCount(object[] array)
{
int count = 0;
try
{
while (true)
{
var temp = array[count];
count++;
}
}
catch (IndexOutOfRangeException)
{
return count;
}
}

I'm currently working with a 3rd party program that does this. They have a "cursor" interface (basically an IEnumerable alternative), where the only way to tell the program you're finished is to raise an exception. The code basically looks like:
// Just showing the relevant section
bool finished = false;
public bool IsFinished()
{
return finished;
}
// Using something like:
// int index = 0;
// int count = 42;
public void NextRecord()
{
if (finished)
return;
if (index >= count)
throw new APIProgramSpecificException("End of cursor", WEIRD_CONSTANT);
else
++index;
}
// Other methods to retrieve the current value
Needless to say, I hate the API - but its a good example of exceptions for flow control (and an insane way of working).

I'm not fond of C# but you can see some similarities between try-catch-finally statements and normal control flow statements if-then-else.
Think about that whenever you throw an exception you force your control to be passed to the catch clause. So if you have
if (doSomething() == BAD)
{
//recover or whatever
}
You can easily think of it in terms of try-catch:
try
{
doSomething();
}
catch (Exception e)
{
//recover or do whatever
}
The powerful thing about exception is that you don't have to be in the same body to alter the flow of the program, you can throw an exception whenever you want with the guarantee that control flow will suddently diverge and reach the catch clause. This is powerful but dangerous at the same time since you could have done actions that need some backup at the end, that's why the finally statement exists.
In addition you can model also a while statement without effectively using the condition of it:
while (!finished)
{
//do whatever
}
can become
try
{
while (true)
{
doSomethingThatEventuallyWillThrowAnException();
}
}
catch (Exception e)
{
//loop finished
}

A module developed by a partner caused our application to take a very long time to load. On closer examination, the module was looking for a config file at app startup. This by itself was not too objectionable, but the way in which it was doing it was outrageously bad:
For every file in the app directory, it opened the file and tried to parse it as XML. If a file threw an exception (because it wasn't XML), it caught the exception, squelched it, and tried the next file!
When the partner tested this module, they only had 3 files in the app directory. The bonehead config file search didn't have a noticeable effect on the test app startup. When we added it to our application, there were 100's of files in the app directory, and the app froze for nearly a minute at startup.
To add salt to the wound, the name of the config file the module was searching for was predetermined and constant. There was no need for a file search of any kind.
Genius has its limits. Stupidity is unbounded.

One example would be using exceptions to return a result from a recursive method:
public void Search(Node node, object data)
{
if(node.Data.Equals(data))
{
throw new ResultException(node);
}
else
{
Search(node.LeftChild, data);
Search(node.RightChild, data);
}
}
Doing something like this is a problem for several reasons.
It's completely counter-intuitive. Exceptions are designed for exceptional cases. Something working as intended should (we hope) never be an exceptional scenario.
You can't always rely on an exception being thrown and propagated to you. For example, if the exception-throwing code runs in a separate thread, you'll need some extra code to capture it.
It is a potential performance problem. There is an overhead associated with exceptions and if you throw a lot of them, you might see a performance drop in your application.
There are a few more examples and some interesting discussion on this subject here.
Disclaimer: The code above is adapted from the first sample on that wiki page to turn it into C#.