I have a console app that calls a WCF service to authorize scheduled credit card payments for a given store ID. Last night it called this service over 100 times and one of those calls, it got this error:
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to [service url] This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
What brought this to our attention was that this store had it's first 100 transactions authorized but 43 were left not authorized (not attempted). The service method simply grabs all 143 and does a foreach loop on them, authorizing them 1 at a time. The method returns a boolean result wrapped in a custom object that can also return error info if necessary, but that object should be small and does not change size in respect to the amount of transactions processed. I mention that because I read one case where this exception occurred in responses over a certain size. But it my case, it seems that something happened before it even tried to return. The service stopped processing in the middle of looping at which point the client received this exception. Any ideas?
Related
As Suggested by #John Wu's Answer on Stackoverflow. I am using the following code to make concurrent HTTP Get Requests on the same host for sending SMS
public async Task SendBulk(List<string> recipentURLs)
{
using (var client = new HttpClient())
{
//Start requests for all of them
var requests = recipentURLs.Select(url => client.GetAsync(url)).ToList();
//Wait for all the requests to finish
await Task.WhenAll(requests);
}
}
Now I am having the exception saying
System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond..
---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
The API I am sending requests to doesn't require a response (as I am only using it to send sms).
I am using this in a backgroud job by using HangFire. So I am okay if it takes a long time.
What I am currently looking is an efficient way to send multiple HTTP Get Requests at the Same host.
How can I safely handle all these concurrent requests and retry if any of it fails?
Also I am currently researching that due HTTP Throttling I cannot use 2 connection to the same host?
Closed HttpClient instances leave sockets open in the TIME_WAIT state for a short period of time. If a code path that creates and disposes of HttpClient objects is frequently used, the app may exhaust available sockets. You should use HttpClientFactory to avoid that.
See the post Make HTTP requests using IHttpClientFactory in ASP.NET Core.
I have .NET program that takes attachments from a particular mailbox id and processes the attachment file.
The problem is we have enables TLS 1.2 and disabled all old versions on windows server. Since then we are facing the following error while trying to communicate with exchange server through my .NET program.
ErrorMessage:Exchange error: The request failed. The underlying connection was closed: An unexpected error occurred on a receive.
InnerExceptionSystem.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.ComponentModel.Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm
below are files I have in my program:
Microsoft.Exchange.WebServices.Auth.dll
Microsoft.Exchange.WebServices.Auth.pdb
Microsoft.Exchange.WebServices.Auth.xml
Microsoft.Exchange.WebServices.dll
Microsoft.Exchange.WebServices.pdb
Microsoft.Exchange.WebServices.xml
Please let me know how I can change my code to resolve this issue.
I have a database in azure which have standart s2 edition.In logs of my application I always see many exceptions such formats:
1.
System.Data.SqlClient.SqlException: The client was unable to establish a connection because of an error during connection initialization process before login. Possible causes include the following: the client tried to connect to an unsupported version of SQL Server; the server was too busy to accept new connections; or there was a resource limitation (insufficient memory or maximum allowed connections) on the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.) ---> System.ComponentModel.Win32Exception: An existing connection was forcibly closed by the remote host
2.
System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached
3
System.Data.SqlClient.SqlException (0x80131904): The client was unable to establish a connection because of an error during connection initialization process before login. Possible causes include the following: the client tried to connect to an unsupported version of SQL Server; the server was too busy to accept new connections; or there was a resource limitation (insufficient memory or maximum allowed connections) on the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.) ---> System.ComponentModel.Win32Exception (0x80004005): An existing connection was forcibly closed by the remote host.
I use SqlAzureExecutionStrategy so this exceptions are thrown after some number of retries.
I see different performance metrics on azure portla,but it seem they are ok.
How can I identify the problem?
I think that your database is under too heavy load, or you have some queries which are still running or not letting go of the connection.
I use this query to see what is running:
SELECT (SELECT TOP 1 SUBSTRING(s2.text,statement_start_offset / 2+1 ,
( (CASE WHEN statement_end_offset = -1
THEN (LEN(CONVERT(nvarchar(max),s2.text)) * 2)
ELSE statement_end_offset END) - statement_start_offset) / 2+1)) AS sql_statement,
s1.* FROM sys.dm_exec_requests s1
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS s2
ORDER BY 1
See if you have queries still running here or keep an eye on the CPU usage in the Azure portal.
The S2 databases aren't particularly good and it will throttle your requests so if you are doing lots of them (even small ones), it might be rejecting them.
Your retry strategy could also be making the problem worse, but throwing more requests at it when it has already been filled. You could try using an exponential back-off if this is the case.
All three could be explained by your connection pooling design. Are you re-using your connections, or is every call to the database opening it's own connection? Are you closing connections at the end of each DBContext? Are you implementing any kind of caching layer to reduce the number of round trips to your database to a minimum?
Here's a way to see if it is an issue with your pooling. From the portal go to the database in question, look at the Resource utilization graph, then hit edit.
Then add Sessions percentage and workers percentage from the select list, and hit OK.
If your pooling is an issue, you'll find that your sessions and workers percentages are high, and may be pegged at 100% for periods. If you hit 100%, you can be denied new connections for up to 5 minutes until the current sessions and workers either finish, or get killed off.
We have a WCF service published in IIS on our local development machine. The service exposes several methods, one of them accepting an array of bytes as a parameter. When invoked we are getting an error when sending over 3MBs of data.
When invoked for the first time we get the following error: “An existing connection was forcibly closed by the remote host - WCF.”
If we invoke it again we then get this error: “An error occurred while receiving the HTTP response to http://[server]/service/method. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.”
We know how to solve the error by increasing the buffersize and maxreceivedmessagesize in the binding and the maxrequestlength in the system.httpruntime.
The problem we find is that the error is causing the service host to close down and stops accepting further requests.
We’d like to know why the error is causing the servicehost to close down.
I was under the impression that any exceptions caused by a request would return a faultexception but wouldn’t necessarily close the host. Any even if the host is closed on the next request it would open it again.
Could somebody shed some light on this?
Check the maxRequestEntityAllowed limit.
See:
http://www.iis.net/configreference/system.webserver/asp/limits
http://msdn.microsoft.com/en-us/library/ms524953%28v=vs.90%29.aspx
I've written a c#/.net program that accesses a Web Reference WSDL made in PHP NuSoap, that accesses a soap method through this call:
public kiosk_wsdl.MEMBER_DATA_RECORD record;
using (kiosk_wsdl.kiosk webService = new kiosk_wsdl.kiosk())
{
kiosk_wsdl.USER user = new kiosk_wsdl.KIOSK_WSDL_USER();
record = webService.GetMemberDetails("000000000001", user);
}
It works well in my localhost and our development servers, but when I transferred the NuSoap server to the production server, this exception occurs when I make the call:
The underlying connection was closed: An unexpected error occurred on a receive.
with the inner exception:
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
While in this same production server, the method works well when accessed via a PHP NuSoap Client, so I'm guessing the problem is in my .NET client.
I've googled this problem and some say that it is possible that the server might have a default maximum message size configured to be too small, but how can I modify that in our production server? (and why does it work in our development machines where we have the same NuSoap configuration?) I've tried different solutions but to no avail.