How can I get the full stack of an exception that's happening in an otherwise functioning Web application during SignalR connection setup?
Background
I'm part of a team maintaining a Web application with C# clients that uses an extremely basic SignalR setup (version 2.2) to effectively deliver push notifications about progress during long-running server processes. Like, out-of-the-box,
app.MapSignalR();
HubConnection connection = new HubConnection(_applicationConfiguration.ApiBaseUri + "/signalr");
await connection.Start();
basic. Some of our clients run on remoting services and periodically run into an issue where the other functions of the Web application work fine, but the client code that calls connection.Start() returns a 500 internal server error with no further information. They can address it by refreshing the remote connection but this is less than ideal, so I'm trying to get some information about where in the connection setup process this error is happening.
Problem
Following the information about setting up error handling for SignalR on MSDN, I've tried to simulate the problem by inserting the following pipeline module into the GlobalHost.HubPipeline:
public class HubErrorHandlingModule : HubPipelineModule
{
public override Func<IHub, Task> BuildConnect(Func<IHub, Task> connect)
{
throw new InvalidOperationException("Testing Connection Exceptions");
}
protected override void OnIncomingError(ExceptionContext exceptionContext,
IHubIncomingInvokerContext invokerContext)
{
// some logging happens here
base.OnIncomingError(exceptionContext, invokerContext);
}
}
and it kind of works, in that I can see the exception get thrown in the pipeline code, and my test C# client is also seeing a 500 internal server error with no further information.
But it also kind of doesn't work, in that I've dropped in breakpoints and the OnIncomingError code is never hit. That sort of makes sense, since it's not code in any Hub method that's causing the exception, but I don't know where this exception is happening; it could be anywhere during the client call to connection.Start.
I've also tried passing in an alternate HubConfiguration with EnableDetailedErrors = true but that doesn't seem to improve anything.
It doesn't really matter where I get the full stack trace, since I control both the server and the client code, but in order to understand their problem I need to see the full trace somewhere.
What I've Tried And Why It Doesn't Work
app.MapSignalR(new HubConfiguration { EnableDetailedErrors = true });
I think this is meant to show detailed errors from Hub processing, not connection handshaking? Supposedly it's meant to send a message tagged as an error that might be traced by the connection even if it's never bubbled up to any consumer. Unfortunately...
var writer = new StreamWriter("C:\\Logs\\ClientLog.txt");
writer.AutoFlush = true;
connection.TraceLevel = TraceLevels.All;
connection.TraceWriter = writer;
This does trace successful communication to the SignalR backend, once I remove the deliberate pipeline error. But when I set it back up, all I see is a failed attempt to establish the connection and a 500 internal server error. No trace.
<system.diagnostics>
<sharedListeners ... >
<switches ...>
<sources ...>
<trace autoflush="true" />
</system.diagnostics>
Set up both after the MSDN trace details and this commentary on GitHub. Neither set of details works. As I play around by moving the pipeline exception to different pipeline events, I can sometimes see a stack trace show up in the SignalR.HubDispatcher source mentioned only in the GitHub details, but it happens when I throw the exception after the connection's been established and what arrives at the client side is a different error than just a 500 internal server error, so that's probably happening too late to be whatever's going wrong at the client installation.
In my case I have to put the SignalR.cs in my root path.
Then in the view I include the script:
<script src="~/signalr/hubs"></script>
This is what my SignalR.cs looks like:
public class NotificationHub : Hub
{
public void SendUpdateNotification(string message)
{
// message = "show" / "hide"
if (message.Equals("show"))
Config._MaintenanceMode = true;
else
Config._MaintenanceMode = false;
// Call the broadcastUpdate method to update clients.
Clients.All.broadcastUpdate(message);
}
}
To handle errors that SignalR raises, you can add a handler for the error event on the connection object
connection.Error += ex => Console.WriteLine("SignalR error: {0}", ex.StackTrace);
To handle errors from method invocations, wrap the code in a try-catch block.
HubConnection connection = new HubConnection(_applicationConfiguration.ApiBaseUri + "/signalr");
try
{
await connection.Start();
}
catch (Exception ex)
{
Console.WriteLine("Error " + ex);
}
To enable detailed error messages for troubleshooting purposes,
var hubConfiguration = new HubConfiguration();
hubConfiguration.EnableDetailedErrors = true;
App.MapSignalR(hubConfiguration);
In your code the hub pipeline module I do not see you are logging/printing the error
Console.WriteLine("Exception " + exceptionContext.Error.Message);
base.OnIncomingError(exceptionContext, invokerContext);
and now hook up the custom HubPipelineModule we've created, this is achieved in the startup class
public partial class Startup
{
public void Configuration(IAppBuilder app)
{
GlobalHost.HubPipeline.AddModule(new HubErrorHandlingModule());
app.MapSignalR();
}
}
References:
SignalR Notify user about disconnections
SingalR How to Handle Errors
SignalR 500 Internal Server Error
Related
I had a troubling issue dealing with SignalR (v 2.4.1) hubs last week, and despite doing enough by the documents, I could not broadcast messages without hacking my way through it. For some extra context, this is a self hosted (Owin) hub attached to a windows service.
MSDN documents, and runtime errors (Using a Hub instance not created by the HubPipeline is unsupported), suggest that we are supposed to get the hub context by calling GlobalHost.ConnectionManager.GetHubContext<ThisTypeOfHub>(), and at that point we can make calls to the client.
(https://learn.microsoft.com/en-us/aspnet/signalr/overview/guide-to-the-api/hubs-api-guide-server#callfromoutsidehub)
When I originally set up calls to the clients, I was doing it this way:
public void OnDisplayMessage(string message)
{
var hub = GlobalHost.ConnectionManager.GetHubContext<ThisTypeOfHub>();
hub.Clients.All.BroadcastToClient(message);
}
The browsers that contained the method and had connected to signalR never got called, however. The clients could call server methods, and even the server methods would send out a callback which the clients responded to, but the hub context could never call out to the client methods when called from outside the hub. In the end, I directly brought back the hub reference in the IoC container, and called out to clients using that, as shown below.
public class LogicWithUI : Logic
{
Hub hub;
public LogicWithUI(IDependencyInjectionContainer container)
{
this.hub = container.Resolve<ThisTypeOfHub>(); // ThisTypeOfHub inherits from Hub
}
public class OnDisplayMessage(string message)
{
try
{
this.hub.Clients.All.DisplayMessage(string);
}
catch (Exception)
{
//do nothing, no webpage has connected yet
}
}
}
This way it is finally working fine (you can see the catch placed there for when an error occurs -- only when no webpages have connected to this yet), but it doesn't make sense.
What would cause the HubContext called from outside the hub to not actually broadcast the method?
Why would the unsupported error get thrown only when no clients are connected?
Are there any obvious mistakes I'm overlooking here?
The primary goal here is to have a functioning product, but I also want to do it the right/documented way. It's a little confusing when that way isn't working.
I also understand I might have left out some important details regarding the SignalR configuration, I can answer any follow up questions, but wanted to start with the basic explanation.
I would like to implement an ASP.NET Core API, which is not responding to HTTP requests, but upon startup starts listening to Google Cloud Pub/Sub messages, and it keeps listening indefinitely throughout its lifetime.
What is the preferred way to implement this with the official Pub/Sub SDK?
I can think of two ways:
Approach 1: Just use a SimpleSubscriber, and in the Startup.Configure start listening to messages:
public void Configure(IApplicationBuilder app)
{
var simpleSubscriber = await SimpleSubscriber.CreateAsync(subscriptionName);
var receivedMessages = new List<PubsubMessage>();
simpleSubscriber.StartAsync((msg, cancellationToken) =>
{
// Process the message here.
return Task.FromResult(SimpleSubscriber.Reply.Ack);
});
...
}
Approach 2: Use a library specifically created to periodically run a job, for example Quartz, Hangfire or FluentScheduler, and every time the job is triggered, pull the new messages with a SubscriberClient.
Which one is the preferred approach? The first one seems simpler, but I'm not sure if it's really reliable.
The first approach is definitely how this is intended to be used.
However, see the docs for StartAsync:
Starts receiving messages. The returned Task completes when either
StopAsync(CancellationToken) is called or if an unrecoverable
fault occurs. This method cannot be called more than once per
SubscriberClient instance.
So you do need to handle unexpected StartAsync shutdown on unrecoverable error. The simplest thing to do would be be use an outer loop, although given these errors are considered unrecoverable it is likely something about the call needs to be changed before it can succeed.
The code might look like this:
while (true)
{
// Each SubscriberClientinstance must only be used once.
var subscriberClient = await SubscriberClient.CreateAsync(subscriptionName);
try
{
await subscriberClient.StartAsync((msg, cancellationToken) =>
{
// Process the message here.
return Task.FromResult(SimpleSubscriber.Reply.Ack);
});
}
catch (Exception e)
{
// Handle the unrecoverable error somehow...
}
}
If this doesn't work as expected, please let us know.
Edit: SimpleSubscriber was renamed to SubscriberClient in the library so the answer has been edited accordingly.
The setup I'm trying to accomplish is this: I have a signalR server that supports 2 types of clients - one type that supplies data and another that receives data. When a "receiver" type client chooses one of the "sender" type clients to monitor, the SignalR server makes a call to the "sender" client to open a connection. The "sender" client is an ASP.NET Core instance running as a Windows Service. The POST request calls a service that starts a timer and tries to open a SignalR connection as a client. The POST request is going through as expected and it calls the service properly. When the service tries to open the SignalR connection, though, I get: System.TypeLoadException in SignalR.Client.dll (Inheritance security rules violated by type: 'System.Net.Http.WebRequestHandler'. Derived types must either match the security accessibility of the base type or be less accessible).
The action method in the controller looks like this:
[HttpPost]
public IActionResult StartUpdate()
{
_updateDriver.StartTiming();
return Ok("Timing has started");
}
And the method in the service looks like this:
public void StartTiming()
{
if (!_isTiming)
{
try
{
_hubProxy = _hubConnection.CreateHubProxy("RemoteDataServerHub");
_hubConnection.Start().Wait();
_hubProxy.Invoke("JoinGroup", "Sender").Wait();
_timer.Start();
_isTiming = true;
}
catch (Exception ex)
{
Debug.Print(ex.Message);
_isTiming = false;
}
}
}
The exception is thrown on the _hubConnection.Start().Wait() line. Does anyone have any idea what's going on here? I'm not very experienced in web applications or ASP.NET Core or SignalR. I may be biting off a bit more than I can chew here. But if anyone can point me in the right direction, it would be much appreciated!
Thanks!
For the past day or so I've been trying to implement Transient Fault Handling on an Azure SQL database. Although I have a working connection to the DB, I'm not convinced that it's handling the transient faults as expected.
So far my approach has involved
public static void SetRetryStratPol()
{
const string defaultRetryStrategyName = "default";
var strategy = new Incremental(defaultRetryStrategyName, 3, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(2));
var strategies = new List<RetryStrategy> { strategy };
var manager = new RetryManager(strategies, defaultRetryStrategyName);
RetryManager.SetDefault(manager);
retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(strategy);
retryPolicy.Retrying += (obj, eventArgs) =>
{
var msg = String.Format("Retrying, CurrentRetryCount = {0} , Delay = {1}, Exception = {2}", eventArgs.CurrentRetryCount, eventArgs.Delay, eventArgs.LastException.Message);
System.Diagnostics.Debug.WriteLine(msg);
};
}
I call that method from the Global.asax's, Application_Start(). [retryPolicy is a global static variable on a static class which also includes this next method.]
I then have a method
public static ReliableSqlConnection GetReliableConnection()
{
var conn = new ReliableSqlConnection("Server=...,1433;Database=...;User ID=...;Password=...;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;", retryPolicy);
conn.Open();
return conn;
}
I then use this method
using (var conn = GetReliableConnection())
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT COUNT(*) FROM ReliabilityTest";
result = (int) cmd.ExecuteScalarWithRetry();
return View(result);
}
So far, this works. Then, in order to test the retry policy, I've tried using a wrong username (a suggestion from here).
But when I step through that code the cursor immediately jumps to my catch statement with
Login failed for user '[my username]'.
I would have expected that this exception only be caught after several seconds, but no delay is incurred at all.
Furthermore, I've also tried with the Entity Framework, following exactly this post, but get the same result.
What have I missed? Is there a configuration step or am I incorrectly inducing a transient fault?
Transient Fault Handling block is for handling transient errors. Failed login because of incorrect username/password is certainly not one of them. From this web page: http://msdn.microsoft.com/en-us/library/dn440719%28v=pandp.60%29.aspx:
What Are Transient Faults?
When an application uses a service, errors can occur because of
temporary conditions such as intermittent service,
infrastructure-level faults, network issues, or explicit throttling by
the service; these types of error occur more frequently with
cloud-based services, but can also occur in on-premises solutions. If
you retry the operation a short time later (maybe only a few
milliseconds later) the operation may succeed. These types of error
conditions are referred to as transient faults. Transient faults
typically occur very infrequently, and in most cases, only a few
retries are necessary for the operation to succeed.
You may want to check the source code for this application block (http://topaz.codeplex.com/) and see what error codes returned from SQL databases are considered transient errors and are thus retried.
You can always extend the functionality and include failed login as one of the transient error to test your code.
UPDATE
Do take a look at the source code here: http://topaz.codeplex.com/SourceControl/latest#source/Source/TransientFaultHandling.Data/SqlDatabaseTransientErrorDetectionStrategy.cs. This is where the retry magic happens. What you could do is create a class (let's call it CustomSqlDatabaseTransientErrorDetectionStrategy) and copy the entire code from the link to this class). Then for testing purpose, you can add login failed scenario as one of the transient error and use this class in your application instead of SqlDatabaseTransientErrorDetectionStrategy.
I have a program that calls an external web service, and I want to present the user with a friendly dialog if e.g. the server is down, someone cut the cable etc. Assuming the following code
try {
client.MyWebService()
}
catch(? ex)
{
// display friendly dialog explaining what went wrong
}
what exception(s) should I put in place of the question mark in the code? It is kind of hard to actually test situations like this when everything is working smoothly and I have no control over the external part, so some insight would be appreciated.
Thanks!
The first thing to do is take advantage of the .Faulted event on your proxy, which you can wire up like this:
((ICommunicationObject)client).Faulted += new EventHandler(client_Faulted);
In your client_Faulted event handler you can then try re-connecting, or shifting to a backup server, or disabling the UI, logging the error, or displaying a message there.
It's obviously still good practice to wrap each call in a try-catch as well, but the .Faulted event can let you deal with most channel problems even earlier.
As for the exception itself, you can have your service throw a FaultException that gets passed back to the client with the details you provide. See an example of its use at this blog posting.
You won't get a FaultException if the channel itself fails (FaultException is a way for the server to communicate its own internal faults to the client).
For channel faults, you may get a CommunicationException or TimeoutException.
Finally, take a look at this project on Codeplex for generating Exception Handling WCF proxies. It may give you a more flexible way of handing faults.
It's not really the client's job to provide as much detail as possible. The maximum amount you really have to provide at the client side is as much as you get back in your exception.
var userName = "bob";
try
{
client.MyWebService(userName);
}
catch(Exception ex)
{
//Maybe we know WellKnownExceptions and can provide Foo advice:
if (ex is WellKnownException)
{
Console.WriteLine("WellKnownException encountered, do Foo to fix Bar.");
}
//otherwise, this is the best you can do:
Console.WriteLine(string.Format(
"MyWebService call failed for {0}. Details: {1}", userName, ex));
}
I was asking the same question, as I have to implement some exception handling on web services calls at my client application, so I ended up here. Although it's an old question, I'd like to give my two cents, updating it a little bit.
The answer given by C. Lawrence Wenham was already very good and points to some interesting information, although the blog link is broken and Codeplex is now archived.
I found those articles very valuables:
Sending and Receiving Faults
https://learn.microsoft.com/en-us/dotnet/framework/wcf/sending-and-receiving-faults
Expected Exceptions
https://learn.microsoft.com/en-us/dotnet/framework/wcf/samples/expected-exceptions
And this article from Michèle Leroux Bustamante (apparently the creator of the Exception Handling WCF Proxy Generator CodePlex project) is very insighful also:
An Elegant Exception-Handling Proxy Solution
http://www.itprotoday.com/microsoft-visual-studio/elegant-exception-handling-proxy-solution
I'm still studying the subject but I guess I'll use a lot of ideias from Michèle. I'm just a little bit concerned about using reflection to call the web service's methods, but I wonder if this would have any impact in such kind of operation, that is inherently slow already.
Just to answer here explicitly what was asked originally, which are the exceptions that could be tested against a web service call:
string errorMessage = null;
// A class derived from System.ServiceModel.ClientBase.
MyWebService wcfClient = new MyWebService();
try
{
wcfClient.Open();
wcfClient.MyWebServiceMethod();
}
catch (TimeoutException timeEx)
{
// The service operation timed out.
errorMessage = timeEx.Message;
}
catch (FaultException<ExceptionDetail> declaredFaultEx)
{
// An error on the service, transmitted via declared SOAP
// fault (specified in the contract for an operation).
errorMessage = declaredFaultEx.Detail.Message;
}
catch (FaultException unknownFaultEx)
{
// An error on the service, transmitted via undeclared SOAP
// fault (not specified in the contract for an operation).
errorMessage = unknownFaultEx.Message;
}
catch (CommunicationException commEx)
{
// A communication error in either the service or client application.
errorMessage = commEx.Message;
}
finally
{
if (wcfClient.State == CommunicationState.Faulted)
wcfClient.Abort();
else
wcfClient.Close();
}
As stated by the articles, the order the exceptions are catched is important, since FaultException<TDetail> derives from FaultException, and FaultException derives from CommunicationException.