How to troubleshoot "CancellationTokenSource has been disposed" error? - c#

I am seeing a number of these errors in the logs for a .NET Core 2.2.x web app:
Connection ID ""13546832108852449106"", Request ID ""80018f57-0407-bc00-b63f-84710c7967bb"": An unhandled exception was thrown by the application.
System.ObjectDisposedException: The CancellationTokenSource has been disposed.
at System.Threading.CancellationTokenSource.ThrowObjectDisposedException()
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.<>c__DisplayClass314_0.<AbortIO>b__0(Object t)
As you can see, the error occurs in the .net core framework rather than my code.
I looked up the specific Request Id mentioned in the error and it seems to have processed just fine. The error appears in the log about a second after the request has processed and returned data. I also checked the Event Viewer and there wasn't anything suspicious around the time frame of the error.
How can I troubleshoot this issue?
The Windows 2016 server has .NET Core 2.2.6 runtime installed.

Related

Azure - App crashed because of Stack Buffer Overrun Exception + HTTP 502 errors

We are stuck with an issue while hosting our application on the Azure app service. Can someone help us to find the root cause of the issue? Also please post if more information is required.
We are getting this issue while trying to establish a database connection. Our database also exists on Azure as SQL Managed Instance and we are using Active Directory Managed Identity to create the connection.
Your app crashed because of Stack Buffer Overrun Exception and aborted the requests it was processing when the overflow occurred. As a result, your app’s users may have experienced HTTP 502 errors
This call stack caused the exception:
InlinedCallFrame
InlinedCallFrame
DomainBoundILStubClass.IL_STUB_PInvoke
Microsoft.Data.SqlClient.SNILoadHandle..ctor
Microsoft.Data.SqlClient.SNILoadHandle..ctor
Microsoft.Data.SqlClient.SNILoadHandle..cctor
GCFrame
HelperMethodFrame
Microsoft.Data.SqlClient.TdsParser..cctor
GCFrame
HelperMethodFrame
Microsoft.Data.SqlClient.TdsParser..ctor
Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover
Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist
Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor
Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection
Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection
Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject
Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest
Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection
Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection
Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection
Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal
Microsoft.Data.SqlClient.SqlConnection.TryOpenInner
Microsoft.Data.SqlClient.SqlConnection.TryOpen
Microsoft.Data.SqlClient.SqlConnection.Open
mmd.DBConnection.establishConnection
NewAuthentication.OnPageLoad
System.Web.UI.Control.OnLoad
System.Web.UI.Control.LoadRecursive
System.Web.UI.Page.ProcessRequestMain
System.Web.UI.Page.ProcessRequest
System.Web.UI.Page.ProcessRequest
System.Web.UI.Page.ProcessRequest
ASP.classic_common_newauthentication_aspx.ProcessRequest
System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute
System.Web.HttpApplication+<>c__DisplayClass285_0.b__0
System.Web.HttpApplication.ExecuteStepImpl
System.Web.HttpApplication.ExecuteStep
System.Web.HttpApplication+PipelineStepManager.ResumeSteps
System.Web.HttpApplication.BeginProcessRequestNotification
System.Web.HttpRuntime.ProcessRequestNotificationPrivate
System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper
System.Web.Hosting.PipelineRuntime.ProcessRequestNotification
DomainNeutralILStubClass.IL_STUB_ReversePInvoke
InlinedCallFrame
InlinedCallFrame
DomainNeutralILStubClass.IL_STUB_PInvoke
System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper
System.Web.Hosting.PipelineRuntime.ProcessRequestNotification
DomainNeutralILStubClass.IL_STUB_ReversePInvoke
ContextTransitionFrame
Thanks Everyone,
The issue was with the latest version of Microsoft.Data.SqlClient. As per the article below we downgraded the package and our issue was fixed.
https://weblog.west-wind.com/posts/2021/Dec/07/Connection-Failures-with-MicrosoftDataSqlClient-4-and-later

"Resource temporarily unavailable" error from System.Net.Sockets

I am working on a .NET API that runs inside of a docker container. At some point it makes a call to a Python Flask API that is also running in a container.
var response = await httpClient.GetAsync("http://service-name:8000/actual/url")
which then produces the following error:
System.Net.Http.HttpRequestException: Resource temporarily unavailable
---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)
Has anyone had experience with this before and potentially knows a solution? I cant find much on the web about it at all. I have some seen some mentions of the issue potentially being related to the Flask API not using async methods but that doesnt make sense to me.
The Flask API produces the appropriate responses when accessed through a web browser or Postman using localhost:8000/actual/url and the container logs these responses. I have tried using the localhost URL in the .NET API but that does not work either.
If anymore information is needed please leave a comment and I will do my best to update the post quickly.
-- Christie
TLDR
A reason for the "Resource temporarily unavailable" error is when during name resolution the DNS Server responds with RCODE 2 (Server failure).
Long answer
I noticed the same behavior in a dotnet application running in a dotnet runtime alpine docker container. Here are the results of my investigation:
The error message "Resource temporarily unavailable" corresponds to the EAGAIN error code which gets returned by various functions from the C standard library. At first I suspected the connect() function because the C# stack trace indicates the error happening during the ConnectAsync() call of the c# socket. And indeed the EAGAIN error code appears in the man page of connect() with this description: "No more free local ports or insufficient entries in the routing cache".
I simulated a system with depleted local ports and noticed that a different exception gets thrown in that case, which rules out local port availability as a root cause for the original exception. Regarding the other mentioned cause in the man page it turns out that the routing cache was removed from Linux in 2012. commit
I started to look around for EAGAIN in the source of the musl C lib which is used in the dotnet runtime alpine docker container. After a while I finally noticed the gethostbyname2_r function which is used for resolving a domain name to an ip address via DNS. During System.Net.Sockets.Socket.ConnectAsync() the hostname is still a string and the name resolving happens in native code using the gethostbyname2_r function (or one of its variations).
The final question is: When does gethostbyname2_r return the EAGAIN error code? It's when the RCODE field in the header of the DNS Response has the value 2, which stands for "Server failure". source line 166
To verify this result I ran a simple mock DNS server which always returns the RCODE 2 in the DNS response. The resulting c# exception along with the stack trace matched the original exception exactly.

the service instance will remain suspended until administratively resumed or terminated following error exception type initializer

I am getting the above error, exception type initializer.
Could please give what type of take an action to resolved this issue.
I got following error when am trying to drop a file in receive location and I did resumed in biztalk admin console it shows same error in event application logs.
You are getting this error due to your logging component, your logging component is trying to load itself via a static constructor most likely and failing there. It looks like it depends on Microsoft enterprise logging block, make sure you have all dependencies installed in GAC properly or check your logging component configuration to see why it's failing. It's not related to BizTalk issue.

Google.Apis.Admin.Email_Migration_v2 HTTP 410 status code returned for a 503 error

Essentially, while using the .NET version of the Email Migration v2 Google API our application is sending up too many requests per second to a single Google Apps mailbox/user; greater than 1 Request per second. A GoogleApiException is being returned which is fine, and expected, however the body of the error message states a service unavailable (503) error has occurred, but yet the "HttpStatusCode" property of that same GoogleApiException instance is equal to an Http status code of Gone (410), I will include a code snippet and some log output below. At this point, see the questions section at the bottom or read on for better detail.
What steps will reproduce the problem?
Create a process/application that does the following:
Create a Google.Apis.Admin.email_migration_v2.AdminService object, properly initialize it using your OAuth2.0 credentials.
For each message that needs to be sent to Google Apps Create a Google.Apis.Admin.email_migration_v2.MailResource.InsertMediaUpload instance using the AdminService object from above through using AdminService.Mail.Insert() providing proper parameters.
Call MailResource.InsertMediaUpload.UploadAsync while catching any errors that occur.
Do the following:
Begin sending messages at an exponential rate by spooling off hundreds of instances of this process/application all pointing to the same user, using the same OAuth2.0 credentials.
Sit back sip your mountain dew and wait for the [503] errors to roll in...
Once errors start rolling in close down your applications.. no need to hammer the poor Google servers other than for testing the applications exception handling..
What is the expected output? What do you see instead?
Using an instance of the following type: Google.GoogleApiException
One would expect to see the instance's HttpStatusCode property be equivalent to System.Net.HttpStatusCode.ServiceUnavailable instead of System.Net.HttpStatusCode.Gone
What version of the product are you using?
Google.Apis.Admin.Email_Migration_v2 (1.8.1.20)
What is your operating system?
Windows Server 2008 R2 Enterprise (SP1)
What is your IDE?
Visual Studio 2013 Premium
What is the .NET framework version?
4.0.30319
Please provide any additional information below.
Here is a code snippet of the method being called for uploading purposes:
UploadStatus TryUpload(MailResource.InsertMediaUpload insertMediaUpload)
{
try
{
IUploadProgress uploadProgress = insertMediaUpload.UploadAsync(_cancellationToken).Result; // Task.Result locks this thread until completed.
if (uploadProgress != null && uploadProgress.Exception != null)
{
// Display additional information on any of the various exceptions that can be returned by the upload call.
HandleUploadProgressException(uploadProgress);
}
return uploadProgress != null ? uploadProgress.Status : UploadStatus.Failed;
Here is a code snippet from the method that is displaying the output of the exception.
void HandleUploadProgressException(IUploadProgress uploadProgress)
{
if (uploadProgress.Exception is GoogleApiException)
{
GoogleApiException gApiEx = uploadProgress.Exception as GoogleApiException;
throw new vsEventException(vsMapEvent.MapHttpError(gApiEx.HttpStatusCode, vsEventMessages.Id.errGmailUnidentifiableGoogleApiException),
String.Format("GoogleApiException handled in GmailMessenger.HandleUploadProgressException. HttpStatusCode: {0}", gApiEx.HttpStatusCode),
gApiEx);
}
Here is paraphrased output of the GoogleApiException handled by the HandleUploadProgressException method: (note using a custom logging class;
outputting to DebugView)
** Context Info **
Error attempting to write item to Gmail...
** Event Details **
VS-EventID: 30003(errGmailTryUpload) GoogleApiException handled in
GmailMessenger.HandleUploadProgressException. HttpStatusCode: Gone
** Inner Exception Details **
The service admin has thrown an exception: Google.GoogleApiException:
Google.Apis.Requests.RequestError
Service unavailable. Please try again [503]
Errors [ Message[Service unavailable. Please try again] Location[ - ]
Reason[backendError] Domain[global] ]
at
Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task)
at
Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task
task)
at
Google.Apis.Upload.ResumableUpload`1.d__e.MoveNext()
Questions:
Can anyone shed some light on if this is expected behavior or a bug?
If this is the expected result how should the application handle a 410 based error? I know that when most if not all 400 errors are encountered processing of that particalar item should stop, but this does not seem to be a local issue, more a server side issue.
I appreciate any responses returned, I know it can be hard for questions as specific as this.

IIS8 HTTP Error 500.0 - Internal Server Error

I'm trying to move my website to another host, a shared hosting, and I keep getting this message:
Detailed Error Information: Module
AspNetInitializationExceptionModule Notification BeginRequest
Handler ExtensionlessUrlHandler-Integrated-4.0 Error Code
0x00000000 Requested URL http://1.3144.co.il:80/ Physical Path
h:\root\home\picsize-001\www\3144clubmember Logon Method Not yet
determined Logon User Not yet determined
I've tried many things but nothing seems to work for me.
It looks like either your host doesn't support .net 4.5 or you don't have it enabled.
Things to check in your hosts control panel:
You have enabled .net 4.5
It's running in integrated mode and not classic pipeline

Categories

Resources