Intermittent problems EF6 on Azure connecting to remote MySQL DB - c#

I have a site on the Azure free subscription, which is C# on EF6 connecting to a MySQL database on a remote host. The application will run fine for some time, but then for an authenticated user, it will error out on an ObjectDataSource select event with the following:
Inner exception information (level 1):
Exception type: System.Data.Entity.Core.EntityException
Exception message: The underlying provider failed on Open.
Inner exception information (level 2):
Exception type: MySql.Data.MySqlClient.MySqlException
Exception message: Unable to connect to any of the specified MySQL hosts.
Request information:
Request URL: https://xxx.azurewebsites.net:443/default.aspx
Request path: /default.aspx
User host address: 111.111.111.111
User: xxx
Is authenticated: True
Authentication Type: ApplicationCookie
Thread account name: IIS APPPOOL\xxx
Thread information:
Thread ID: 24
Thread account name: IIS APPPOOL\xxx
Is impersonating: False
Stack trace: at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Web.UI.WebControls.ObjectDataSourceView.InvokeMethod(ObjectDataSourceMethod method, Boolean disposeInstance, Object& instance)
at System.Web.UI.WebControls.ObjectDataSourceView.ExecuteSelect(DataSourceSelectArguments arguments)
at System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e)
at System.Web.UI.WebControls.ListControl.PerformSelect()
at System.Web.UI.WebControls.BaseDataBoundControl.DataBind()
at System.Web.UI.WebControls.BaseDataBoundControl.EnsureDataBound()
at System.Web.UI.WebControls.DropDownList.LoadPostData(String postDataKey, NameValueCollection postCollection)
at System.Web.UI.WebControls.DropDownList.System.Web.UI.IPostBackDataHandler.LoadPostData(String postDataKey, NameValueCollection postCollection)
at System.Web.UI.Page.ProcessPostData(NameValueCollection postData, Boolean fBeforeLoad)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
Other times, for an unauthenticated user, it just won't load in the first place, with:
Inner exception information (level 1):
Exception type: System.Data.Entity.Core.ProviderIncompatibleException
Exception message: An error occurred accessing the database. This usually means that the connection to the database failed. Check that the connection string is correct and that the appropriate DbContext constructor is being used to specify it or find it in the application's config file. See the inner exception for details of the failure.
Inner exception information (level 2):
Exception type: System.Data.Entity.Core.ProviderIncompatibleException
Exception message: The provider did not return a ProviderManifestToken string.
Request information:
Request URL: https://xxx.azurewebsites.net:443/
Request path: /
User host address: 111.111.111.111
User:
Is authenticated: False
Authentication Type:
Thread account name: IIS APPPOOL\xxx
Thread information:
Thread ID: 64
Thread account name: IIS APPPOOL\xxx
Is impersonating: False
Stack trace: at System.Web.HttpApplicationFactory.EnsureAppStartCalledForIntegratedMode(HttpContext context, HttpApplication app)
at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)
at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)
at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)
at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)
or
Thread information:
Thread ID: 27
Thread account name: IIS APPPOOL\xxx
Is impersonating: False
Stack trace: at System.Data.Entity.Core.EntityClient.EntityConnection.<OpenAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Core.Objects.ObjectContext.<EnsureConnectionAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Core.Objects.ObjectContext.<ExecuteInTransactionAsync>d__3d`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Utilities.TaskExtensions.CultureAwaiter`1.GetResult()
at System.Data.Entity.Core.Objects.ObjectQuery`1.<GetResultsAsync>d__e.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Utilities.TaskExtensions.CultureAwaiter`1.GetResult()
at System.Data.Entity.Internal.LazyAsyncEnumerator`1.<FirstMoveNextAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Infrastructure.IDbAsyncEnumerableExtensions.<FirstOrDefaultAsync>d__25`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Identity.EntityFramework.UserStore`1.<FindAsync>d__d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Identity.AsyncHelper.RunSync[TResult](Func`1 func)
at Microsoft.AspNet.Identity.UserManagerExtensions.Find[TUser,TKey](UserManager`2 manager, UserLoginInfo login)
at TDT.Account.RegisterExternalLogin.Page_Load()
at System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e)
at System.Web.UI.Control.OnLoad(EventArgs e)
at System.Web.UI.Control.<LoadRecursiveAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Util.WithinCancellableCallbackTaskAwaitable.WithinCancellableCallbackTaskAwaiter.GetResult()
at System.Web.UI.Page.<ProcessRequestMainAsync>d__14.MoveNext()
My connections strings are like this: the 1st one is used for the ApplicationDbContext for Identity stuff, and the 2nd one is used by the EF Model:
<add name="IdentityConnection" connectionString="server=xxx.xxx.xx.xxx;user id=xxx;database=xxx;password=Password;persistsecurityinfo=True" providerName="MySql.Data.MySqlClient" />
<add name="context" connectionString="metadata=res://*/MyModel.csdl|res://*/MyModel.ssdl|res://*/MyM‌​odel.msl;provider=MySql.Data.MySqlClient;provider connection string="server=111.111.11.11;user id=xxx;database=MyDB;password=password;persistsecurityinfo=True"" providerName="System.Data.EntityClient" />
Since it is on the free Azure, and you can have 10 sites for free, I have uploaded an identical version of the site to another Azure subdomain, so when the first site errors, I check the second site, and that site connects to the DB perfectly well and the application loads normally. Anyone any ideas what is going on? Is it a restriction on the free Azure subscription for multiple users or a limited number of users/connections at one time? I read that there was a limit of 4 connections to a ClearDB MySQL, but this MySQL is completely separate and shouldn't have those restrictions. Although the 2nd site does load when the 1st errors, the 2nd site also produces errors if used at length.
thanks

I had a similar issue with one of my Java services that connected to an external MySQL server. I finally resolved the issue with help from a Azure technician.
If the issue is related to mine - it is caused by a SNAT table used for routing on Azure, here is a post that describes the issue in detail.
I ended up sending keep-alive packets for all our TCP/IP socket connections. The other solution is to assign a PIP address, but I'm not sure if you can do that with an Azure Website.
I observed that the time needed to reconnect to our MySQL varied a lot, so the conclusion was that while your remote server thought that the connection was still open it wouldn't allow traffic from the same IP (the VIP of your website in Azure lingo) on the TCP level.
You may try setting a keepalive time to something below 4 minutes (our app is set to 60 seconds) for your DB connection:
server=xxx.xxx.xx.xxx;user id=xxx;database=xxx;password=Password;persistsecurityinfo=True;Keepalive=60;
I have almost no experience in C#, so check if the KeepAlive option is valid for your connector. It seems that it is according to this site.

Related

Deployed azure function get bad request from GraphApi Rest call but the action is a success

I developed an azure function inside Visual Studio, which access some graph api endpoints and writes into azure storage some information. When i tested the function on my pc, everything works perfect.
When i deployed the function into azure, everything works but not the azure storage part. What am I doing wrong?
The variables are in DEV-OPS pipeline and I check in azure portal and there are present as well.
Edit 1: As I started to try to log information I was able to see that my functions is just running the first call and not even finishing it. I used before and after each call.
log.Info()
This is the GraphAPI Call, I know it is in preview but this shouldn't be a problem as this code is running fine on my pc in visual studio.
public static async Task<ChatMessage> CreateNewMessageThread(Employee employee)
{
var chatMessage = ChatThreadMainMessage(employee);
var result = await GraphServiceClient
.Teams[ApplicationConfigurationFields.TeamsId]
.Channels[ApplicationConfigurationFields.ChannelId]
.Messages
.Request()
.AddAsync(chatMessage);
return result;
}
Edit 2: I am getting a BadRequest from the call which throws an exception. although i am getting a bad request, the message is still created... Even if I find the problem I would like to understand why is that happening.
This is the stack trace, don't know if it will help someone.
Status Code: BadRequest
Microsoft.Graph.ServiceException: Code: BadRequest
Message: Bad Request
Inner error:
AdditionalData:
request-id: 220c1508-5bbd-xxxxx-xxxxx
date: 2020-04-27T16:02:01
ClientRequestId: 220c1508-5bbd-xxxxxx-xxxxx
at Microsoft.Graph.HttpProvider.<SendAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Graph.BaseRequest.<SendRequestAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Graph.BaseRequest.<SendAsync>d__31`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MsTeamsNotifications.MsTeamsNotifications.<CreateNewMessageThread>d__19.MoveNext()
Thank you!

Connecting to gmail with Mailkit ImapClient using TLS not SSL

I'm trying to poll a gmail account in C# code.
I am using the Mailkit libraries (https://github.com/jstedfast/MailKit).
I can connect successfully when I tell the client to use SSL:
using (var client = new ImapClient ())
{
client.Connect ("imap.friends.com", 993, true);
client.Authenticate ("joey", "password");
client.Disconnect (true);
}
But it's my understanding (possibly wrong) that SSL is insecure and we shouldn't be using it. So I'm trying to force a TLS connection:
using (var client = new ImapClient ())
{
client.Connect ("imap.friends.com", 993, SecureSocketOptions.StartTls);
client.Authenticate ("joey", "password");
client.Disconnect (true);
}
But this errors on the client.connect line:
Message: The IMAP Server has unexpectedly disconnected
Stack Trace:
at MailKit.Net.Imap.ImapStream.<ReadAheadAsync>d__54.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MailKit.Net.Imap.ImapStream.<ReadTokenAsync>d__69.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MailKit.Net.Imap.ImapEngine.<ConnectAsync>d__140.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at MailKit.Net.Imap.ImapClient.<ConnectAsync>d__81.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MailKit.Net.Imap.ImapClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken)
I'm running with the protocol logger, but that's not telling me much, it holds only 1 line:
Connected to imap://imap.gmail.com:993/?starttls=always
So I guess my questions are:
1) Should I be worried about using insecure SSL 3.0 to access gmail? I find it hard to believe that they are forcing me to use a deprecated security protocol.
2) If so, how can I force a TLS connection, so I can keep SSL3.0 turned off for clients on my application server?
MailKit has 2 different ways of doing SSL/TLS:
Use SSL/TLS immediately upon connecting to the remote server
Use the STARTTLS command to toggle into SSL/TLS mode after connecting and reading the greeting to check if the server supports it
You are trying to use the second mode but you are connecting to a port (993) which requires the first mode.
Which version of SSL vs TLS gets used with either of these modes is entirely dependent upon what the server supports (actually, technically, MailKit doesn't support any version of SSL by default, it only supports TLSv1.0, TLSv1.1, and TLSv1.2 - I removed SSLv3 by default a few years ago).
The way that you can change the supported SSL and/or TLS versions that you'd like to limit MailKit to can be done by setting the client.SslProtocols property.

ASP.NET OWIN Identity Error AAD Auth Failure when SSL is Offloaded From Server

I am using Azure Active Directory login via ASP.NET OWIN Identity for authentication.
I have a load balancer which is doing SSL offloading, so my server itself is running http protocol and not https protocol, but the load balancer is SSL.
I am getting this error:
[WebException: The remote server returned an error: (400) Bad Request.]
System.Net.HttpWebRequest.GetResponse() +1686
Microsoft.IdentityModel.Clients.ActiveDirectory.<GetResponseSyncOrAsync>d__2.MoveNext() +392
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.IdentityModel.Clients.ActiveDirectory.<SendPostRequestAndDeserializeJsonResponseAsync>d__0`1.MoveNext() +512
[AdalServiceException: AADSTS70002: Error validating credentials. AADSTS50011: The reply address 'http://intx.mydomain.com/default' does not match the reply address 'https://intx.mydomain.com/default' provided when requesting Authorization code.
Trace ID: d5861c62-9d42-4f2b-886a-06cc63b01f00
Correlation ID: f77d0d3e-9dd2-4af8-a38f-65f7838bc680
Timestamp: 2018-10-28 05:13:16Z]
Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.RunAsyncTask(Task`1 task) +133
CloudmersiveAdminApp.Startup.<ConfigureAuth>b__10_0(AuthorizationCodeReceivedNotification context) in C:\Src\APIs\Account3\CloudmersiveAccount\CloudmersiveAdminApp\App_Start\StartupAuth.cs:109
Microsoft.Owin.Security.OpenIdConnect.<AuthenticateCoreAsync>d__1a.MoveNext() +5428
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
Microsoft.Owin.Security.OpenIdConnect.<AuthenticateCoreAsync>d__1a.MoveNext() +5937
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.Owin.Security.Infrastructure.<BaseInitializeAsync>d__0.MoveNext() +817
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.Owin.Security.Infrastructure.<Invoke>d__0.MoveNext() +329
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.Owin.Security.Infrastructure.<Invoke>d__0.MoveNext() +768
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.<RunApp>d__5.MoveNext() +196
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.<DoFinalWork>d__2.MoveNext() +184
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.StageAsyncResult.End(IAsyncResult ar) +117
System.Web.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +367
System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) +195
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +128
Note how it is complaining about the protocol being http and not https. The thing is - in the browser, it is https! It is just that the https part is being offloaded by the load balancer.
How can I configure ASP.NET to not throw this error? Any ideas would be greatly appreciated.
Looking at the exception, your problem might be related to reply URL specified in code/configuration.
Make sure that any request to Azure AD still specifies the redirect URL/reply URL to be https://intx.mydomain.com/default.
Usual suspects would be -
a. StartupAuth.cs (around line 109 as per stack trace). Check in that file wherever you have specified RedirectUri
b. Any other place in code where you try to acquire token and specify redirect URL explicitly.
If it still doesn't work out, try adding some more code details to your question about Startup configuration and Token acquire logic.

Google NLP authentication/call issue

I am working on an MVC web application that uses Google Natural Language Processing API to parse different input from users.
I have successfully consumed and implemented the API operations and everything works fine as long as I run the application on my local machine. But as soon as I publish a version and upload it on a server I receive the following error on calling the API methods (e.g. AnalyzeSentiment):
Status(StatusCode=Unauthenticated, Detail="Getting metadata from plugin failed with error: Exception occured in metadata credentials plugin.")
With the help of the answers from post: Google Datastore authentication issue - C# I was able to further get details on the error (using gRCP):
An error occurred while sending the request.
Stacktrace: at Google.Apis.Http.ConfigurableMessageHandler.<SendAsync>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.<ExecuteAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Auth.OAuth2.ServiceAccountCredential.<RequestAccessTokenAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Auth.OAuth2.ServiceCredential.<GetAccessTokenForRequestAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Auth.OAuth2.ServiceAccountCredential.<GetAccessTokenForRequestAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass2_0.<<FromCredential>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.<GetMetadataAsync>d__11.MoveNext()
This seemed like an authentication issue so I double checked the jsonKey file which is fine. Please note, I have used code to set the credentials in Environment variables:
Environment.SetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS", jsonPath);
and verified it using:
Environment.GetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS")
The call is made as follows:
private AnalyzeSentimentResponse AnalyzeSentiment(string statement)
{
GrpcEnvironment.SetLogger(new MyLogger());
var client = LanguageServiceClient.Create();
var response = client.AnalyzeSentiment(new Document()
{
Content = statement,
Type = Document.Types.Type.PlainText
});
return response;
}
Cannot figure out why it works fine when I run it on my local machine and fails when it is deployed on the server. There is also no restrictions of any kind on the said server.
The result for:
GoogleCredential.GetApplicationDefaultAsync().Result.Underly‌​ingCredential.GetTyp‌​e()
is:
Google.Apis.Auth.OAuth2.ServiceAccountCredential
Note: The server is our own (Windows Server 2012R2)
With the suggestion given by #JonSkeet, I copied the code into a console application and executed the call. Unfortunately, the issue persisted. What I did next was to move the console application onto another server, it worked there.
So, it was indeed an issue with the server where there maybe some features missing (the firewall is disabled). Network dept is checking it out whereas I have deployed my web application on another server.
Update: There was an issue on the server where some required framework features were not installed. The issue has been resolved by moving the deployment to another server.

OperationCanceledException on stress test

I just made an WebApi (C#, .net 4.5.2) and published it to the web. In order to make sure it working good, I started a to test it.
The REST web-service failed the "stress" test. I sent the service 30+- http requests, each second, and got back this typical error message:
System.OperationCanceledException: The operation was canceled.
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at System.Net.Http.HttpContentExtensions.<ReadAsAsyncCore>d__0`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.ModelBinding.FormatterParameterBinding.<ExecuteBindingAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.HttpActionBinding.<ExecuteBindingAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()
Same error occurred many times in the log.
When I entered the specific machine, I saw that the CPU is on 100% and the RAM is on 80%. Which mean, the machine gives her best to handle the load.
Questions
According to the StackTrace I provided above, I cannot understand
where exactly was the problem in my code?
My code is scale out automatically (hosted in the cloud). But, new machine created only after 5 minutes (depends on the average CPU > 50%). How should I handle sudden mass of HTTP request? Maybe the server should say: "Hey, try agian in 30 seconds" or something like that? What is the right solution?
System.OperationCanceledException: The operation was canceled.
This usually indicates that the client connecting to the service closed the connection before the service could send a response. In terms of api layer this usually means that you have specified a timeout while establishing the http connection, or the library that you are using for making http calls has a default timeout.
You may want to check the iis request logs to see what is going wrong. Since this is happening on a stress test, my guess is your service is running short of threads (Read this for more details) . You may want to consider async model for your request handlers to improve the scalability and utilize threads better. (Read this)

Categories

Resources