Develop Reference

How to resolve exception with Selenium & ChromeDriver

I'm running ChromeDriver (v89) & Selenium 4.1 on MacOS and am getting an exception when instantiating ChromeDriver by passing an instance of ChromeDriverService and these options:
{
"browserName": "chrome",
"goog:chromeOptions": {
"args": [
"no-sandbox"
],
"debuggerAddress": "127.0.0.1:8080"
}
}
Details of the exception:
OpenQA.Selenium.WebDriverException: The HTTP request to the remote WebDriver server for URL http://localhost:49385/session timed out after 60 seconds.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (89): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at OpenQA.Selenium.Remote.HttpCommandExecutor.MakeHttpRequest(HttpRequestInfo requestInfo) at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute) --- End of inner exception stack trace --- at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute) at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute) at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary2 parameters)
at OpenQA.Selenium.WebDriver.StartSession(ICapabilities desiredCapabilities)
at OpenQA.Selenium.WebDriver..ctor(ICommandExecutor executor, ICapabilities capabilities)
at OpenQA.Selenium.Chromium.ChromiumDriver..ctor(ChromiumDriverService service, ChromiumOptions options, TimeSpan commandTimeout)
at OpenQA.Selenium.Chrome.ChromeDriver..ctor(ChromeDriverService service, ChromeOptions options, TimeSpan commandTimeout)
at OpenQA.Selenium.Chrome.ChromeDriver..ctor(ChromeDriverService service, ChromeOptions options)
The same code worked fine on Windows, so I dare to say that it's been tested.
Libraries I'm using:
OpenQA,/Users/mbaas/.nuget/packages/Selenium.WebDriver/4.1.0/lib/netstandard2.1/WebDriver.dll
OpenQA.Selenium,/Users/mbaas/.nuget/packages/Selenium.WebDriver/4.1.0/lib/netstandard2.1/WebDriver.dll
OpenQA.Selenium.Chrome,/Users/mbaas/.nuget/packages/Selenium.WebDriver/4.1.0/lib/netstandard2.1/WebDriver.dll
Newtonsoft.Json,/Users/mbaas/.nuget/packages/Newtonsoft.Json/12.0.3/lib/netstandard2.0/Newtonsoft.Json.dll
OpenQA.Selenium.Support,/Users/mbaas/.nuget/packages/Selenium.Support/4.1.0/lib/netstandard2.1/WebDriver.Support.dll
OpenQA.Selenium.Support.UI,/Users/mbaas/.nuget/packages/Selenium.Support/4.1.0/lib/netstandard2.1/WebDriver.Support.dll

I think I found the answer: the plan was to launch a separate process beforehand that runs CEF and the site - but that step failed, so there is no browser it could talk to.
My bad - I should have checked that before posting here.

Timeout with HTTPClient C# on VPS Linux, ping and curl OK, no firewall

I want to retrieve the content of a page with httpclient. I did a simple method, which works well on visual studio.
private async Task<string> Connection(string Url)
{
using HttpClient client = new();
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36 OPR/77.0.4054.277");
client.DefaultRequestHeaders.Add("Cache-Control", "no-cache, no-store, must-revalidate");
client.DefaultRequestHeaders.Add("Accept-Language", "fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7");
client.DefaultRequestHeaders.Add("Pragma", "no-cache");
double timestamp = GlobalMethod.Timestamp();
string json = await client.GetStringAsync(Url);
return json;
}
But when I run my application on a linux VPS (Ubuntu, host OVH, .NET 5), Httpclient returns an error timeout.
On my VPS, the ping to the url I want to retrieve, and CURL function, works fine. But not the Httpclient.
Same settings of my Postman request, and no problem with it.
Do you have an idea of ​​the problem ?
It looks a lot like this topic, without me understanding how to solve it: httpClient call in C# goes to timeout, while cUrl is working
API link : https://api.store.nvidia.com/partner/v1/feinventory?skus=FR~NVGFT070~NVGFT080~NVGFT090~NVLKR30S~NSHRMT01~NVGFT060T~187&locale=FR
Thanks !
Error with timeout:
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at Castiel_Bot_Discord.Modules.NvidiaCommand.Connection(String Url) in C:\Users\gaeta\source\repos\Castiel_Bot_Discord\Castiel_Bot_Discord\Modules\NvidiaCommand.cs:line 114
at Castiel_Bot_Discord.Modules.NvidiaCommand.SearchGpu(String args) in C:\Users\gaeta\source\repos\Castiel_Bot_Discord\Castiel_Bot_Discord\Modules\NvidiaCommand.cs:line 42

Same issue on my Debian machine. Suddenly timeout exception occured for https requests, but I did not performed any configuration steps before at all. The app just stopped to work!
What I've found out to solve:
https://github.com/dotnet/runtime/issues/47267
According to that you need to disable ipv6. For my host I've changed sysctl.conf as described here: https://wiki.it-kb.ru/unix-linux/debian/how-to-turn-off-ipv6-in-debian-linux
Now the timeout issue solved for me.

.net Core Grpc Client unable to call Greeter Service

I have been looking at this for a long time, looking through all the different posts on the internet to see if I could find someone with the same issue but somehow I seem to be the only have having this issue.
I have an ASP.net Core server running the basic greeter service of Grpc.
Calling this server using the program "BloomRPG" work great, and has excellent response times.
However when I try to do the same from my own written (very basic) client I get the following errors:
On the Client:
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Starting gRPC call. Method type: 'Unary', URI: 'https://www.MYSERVER.com:50005/greet.Greeter/SayHello'.
dbug: Grpc.Net.Client.Internal.GrpcCall[18]
Sending message.
trce: Grpc.Net.Client.Internal.GrpcCall[21]
Serialized 'TestService1.HelloRequest' to 15 byte message.
trce: Grpc.Net.Client.Internal.GrpcCall[19]
Message sent.
fail: Grpc.Net.Client.Internal.GrpcCall[6]
Error starting gRPC call.
System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Grpc.Net.Client.Internal.GrpcCall2.RunCall(HttpRequestMessage request, Nullable1 timeout) info: Grpc.Net.Client.Internal.GrpcCall[3] Call failed with gRPC error status. Status code: 'Internal', Message: 'Error starting gRPC call. HttpRequestException: An error occurred while sending the request. IOException: The response ended prematurely.'. dbug: Grpc.Net.Client.Internal.GrpcCall[4] Finished gRPC call. dbug: Grpc.Net.Client.Internal.GrpcCall[8] gRPC call canceled. Unhandled exception. Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. IOException: The response ended prematurely.", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: The response ended prematurely. at System.Net.Http.HttpConnection.FillAsync() at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Grpc.Net.Client.Internal.GrpcCall2.RunCall(HttpRequestMessage request, Nullable1 timeout)")
at GrpcGreeterClient.Program.Main(String[] args) in C:\Users\MYUSERNAME\source\repos\grpc\TestgrpcClient\TestgrpcClient\Program.cs:line 64
at GrpcGreeterClient.Program.(String[] args)
s
on the server:
fail: Microsoft.AspNetCore.Server.Kestrel[0]
HTTP/2 over TLS was not negotiated on an HTTP/2-only endpoint.
Since I am succesfully calling the service with BloomRPC I suppose the issue is with my client:
Client code:
AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
var loggerFactory = LoggerFactory.Create(logging =>
{
logging.AddConsole();
logging.SetMinimumLevel(LogLevel.Trace);
});
var httpclient = new HttpClient();
httpclient.DefaultRequestVersion = new Version(2, 0);
using var channel = GrpcChannel.ForAddress("https://www.MYSERVER.com:50005",
new GrpcChannelOptions { LoggerFactory = loggerFactory, HttpClient = httpclient});
var client = new Greeter.GreeterClient(channel);
var reply = await client.SayHelloAsync(
new HelloRequest { Name = "GreeterClient" });
Console.WriteLine("Greeting: " + reply.Message);
Console.WriteLine("Press any key to exit...");
Console.ReadKey();
anyone have any idea what I could try next?
I feel like I have exhausted all my options :(

Thanks to #jdweng I figured out what was causing the issue.
a mandatory company proxy server was interfering, setting the UseProxy = false in the httpclienthandler fixed the issue for me and I am now able to call my gRPC server without problems!

Storage getting blobs TaskCanceledException

My application uses Azure storage blobs in order to download files from remote storage to the local machine.
How it works:
The machine getting a queue message with URL (signed URL) to Azure Storage blob.
The application sending GET to download the file. It's done with HttpClient.
Application logic continues from there.
This is how I downloading the file:
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Get, signedUrl);
HttpClient.Timeout = TimeSpan.FromSeconds(120);
var response = await HttpClient.SendAsync(msg);
response.EnsureSuccessStatusCode();
I found that few times per day, I got exception:
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException: Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStreamInternal.g__InternalFillBufferAsync|38_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial)
at System.Net.Security.SslStreamInternal.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates1 shouldRetryResultPredicates, Func5 onRetryAsync, Int32 permittedRetryCount, IEnumerable1 sleepDurationsEnumerable, Func4 sleepDurationProvider, Boolean continueOnCapturedContext)
at Polly.AsyncPolicy1.ExecuteAsync(Func3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
At this point, I must clarify that this behavior is happening to ~0.1% of the URLs passed in the system.
I was suspecting that the problem is with the signed-URL. I found in all cases the signed URL was correct - I just put it in the browser and it was working.
Now when I know that the problem is not the URL itself, I thought it can be a network problem. So, by using Polly I have added 5 retries with an exponential delay between each one. Up to 5 retries. Delay formula - 3^n (where n is the number of retries). In the worst case, it will wait 4 minutes before trying again... And even now - still same error. All 5 retries were used but the file was not downloaded.
What to do? What else I should check?

Seems to be a network issue. I suggest you use Azure Storage Client SDK to download blob.
// Blob SAS URL
string url = "https://storagetest789.blob.core.windows.net/devops/deploy.ps1?sp=r&st=2020-01-21T09:39:06Z&se=2020-01-21T17:39:06Z&spr=https&sv=2019-02-02&sr=b&sig=tyGX6wpgAooOZK2Po2ntukF0fGx3PosSNTgkLlVysrE%3D";
CloudBlockBlob blob = new CloudBlockBlob(new Uri(url));
Console.WriteLine(blob.DownloadText());
You can also download to file or stream with SDK methods. The SDK may handle all the underlying stuffs.

Using TFS Rest API works from curl but not from client dotnet library

I'm using the following code from the library Microsoft.VisualStudio.Services.Client:
var credential = new VssBasicCredential("", "<PAT>");
var url = "<url>"";
using var vssConnection = new VssConnection(new Uri(url), credential);
await vssConnection.ConnectAsync();
It fails with the following exception, which I assume has to do with server not allowing basic auth:
System.ComponentModel.Win32Exception (0x80090020): GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (SPNEGO cannot find mechanisms to negotiate).
at System.Net.NTAuthentication.GetOutgoingBlob(Byte[] incomingBlob, Boolean throwOnError, SecurityStatusPal& statusCode)
at System.Net.NTAuthentication.GetOutgoingBlob(String incomingBlob)
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
However, I manage to execute REST command successfully from command line
curl <url>/_apis/projects -u :<PAT>
I'm using Azure Devops Server 2019 for which I cannot modify any configurations. I'm running the code from dotnet-core Alpine docker image.
Is it a limitation of the dotnet library or am I using it wrong?

I got this error with an expired AccessToken.
Verified this with curl.
After creating a new Token it worked.

I had the same error in a program that ran in a Docker container. The cause was an expired PAT. This was not very clear from the error message ("SPNEGO cannot find mechanisms to negotiate"). In addition, the expired PAT still worked on my development machine. That's why I didn't even think that the PAT could have expired.
Newly generated PAT: Everything is running.

If you want to use GitHttpClient, As #Shayki Abramczyk pointed out you should use vssConnection.GetClient<GitHttpClient>() to get GitHttpClient connection.
var credential = new VssBasicCredential("", "<PAT>");
var url = "<url>"";
using var vssConnection = new VssConnection(new Uri(url), credential);
GitHttpClient gitClient = vssConnection.GetClient<GitHttpClient>();
You can follow the examples in the QuickStarts and Samples from Microsoft documents

The issue was with the Alpine distribution of dotnet Docker image, it does not yet support gss-ntlmssp. Switching to the standard image (Ubuntu based - mcr.microsoft.com/dotnet/core/aspnet:3.1) solved the issue.