Develop Reference

.NET GoogleDrive API v3: persistent "connection was closed" errors

Uploading files via FilesResource.Update or FilesResource.Create is regularly emitting IOExceptions with message "System.IO.IOException: Unable to read data from the transport connection: The connection was closed." Is there any chance I can prevent these IOExceptions from happening?
The uploads seem to succeed, eventually, probably because this is happening within the ResumableUpload facility in the API. But (maybe anecdotally) it's causing not-insignificant performance issues; at times the queue of work being processed by our upload processes slows to almost a standstill, with these errors being logged incessantly.
Utility for uploading streams to Drive:
public async Task UploadFile(Stream uploadStream, string filename, string parentFolderId, string mimeType="", CancellationToken token = default(CancellationToken))
{
try
{
// File information for where this content will end up - name, folder, mime type
var fileMetaData = new File();
fileMetaData.Name = filename;
fileMetaData.MimeType = mimeType;
// check if file by this name exists with same parent and not trashed
var results = await ListFiles($"mimeType!='{FolderMimeType}' and name='{filename}' and '{parentFolderId}' in parents and trashed=false", token);
if (results.Count > 0)
{
log.Debug($"Overwrite existing fileid {results[0].Id} \"{results[0].Name}\" in parent {parentFolderId}");
var request = driveService.Files.Update(fileMetaData, results[0].Id, uploadStream, fileMetaData.MimeType);
request.Fields = "id, name";
request.ProgressChanged += Upload_ProgressChanged;
request.ResponseReceived += Upload_ResponseReceived;
request.SupportsAllDrives = true; // required in order to be able to upload to shared drive
await request.UploadAsync(token);
}
else
{
log.Debug($"No existing file found, uploading {filename} to parent {parentFolderId}");
fileMetaData.Parents = new List<string> { parentFolderId };
// send the stream to the destination file and dispose when finished
var request = driveService.Files.Create(fileMetaData, uploadStream, fileMetaData.MimeType);
request.Fields = "id, name";
request.ProgressChanged += Upload_ProgressChanged;
request.ResponseReceived += Upload_ResponseReceived;
request.SupportsAllDrives = true; // required in order to be able to upload to shared drive
await request.UploadAsync(token);
}
}
catch (Exception e)
{
throw e;
}
finally
{
uploadStream.Dispose();
}
}
These exceptions are caught in the "ProgressChanged" event, where the upload's status gets marked as Failed even though it does seem to be retried by the ResumableUpload part of the API:
static void Upload_ProgressChanged(IUploadProgress progress)
{
log.Debug($"Status: {progress.Status}, {BytesToString(progress.BytesSent)} sent");
if (progress.Exception != null && progress.Status == Google.Apis.Upload.UploadStatus.Failed)
{
log.Error($"Upload failed: {progress.Exception.Message}", progress.Exception);
}
}
Log w/ stack trace:
2022-05-23 17:25:19,365|ERROR|62|Upload failed: Unable to read data from the transport connection: The connection was closed.
System.IO.IOException: Unable to read data from the transport connection: The connection was closed.
at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload.UploadBuffer.<PopulateFromStreamAsync>d__11.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 844
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload.<PrepareBufferUnknownSizeAsync>d__81.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 720
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload.<SendNextChunkAsync>d__77.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 630
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload.<UploadCoreAsync>d__74.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 572

HTTPClient.PostAsync. Exception: Connection was closed unexpectedly. C#

When I try to PostAsync, no response is coming back, even though the content is coming through.
The connection closes after 20 seconds or so, giving me an exception.
I'm getting a 200 OK, when Posting in Postman.
This is the code calling the method with .Wait:
StringContent sc = new StringContent(xml, Encoding.UTF8, "application/xml");
var t = Task.Factory.StartNew(() => MyAsyncMethod(sc, ao.orderId).Wait());
t.Wait();
t.Dispose();
This is the method:
public async Task MyAsyncMethod(StringContent sc, string orderid)
{
try
{
var response = await client.PostAsync("www.url.com", sc);
}
catch (Exception e)
{
using (StreamWriter writer = new StreamWriter(AppDomain.CurrentDomain.BaseDirectory + "/Log/PostException.txt"))
{
writer.WriteLine(e.Message);
}
}
}
The Exception:
An error occurred while sending the request. at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at ShipNoticeInvoiceAPI.BLL.ShipNoticeGenerator.<SendShipNotice>d__4.MoveNext()
System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
System.Collections.ListDictionaryInternal
-2146233088
mscorlib
Void ThrowForNonSuccess(System.Threading.Tasks.Task)
I hope that you have tried something simular and are able to help me out. I feel like I've tried everything.

HttpWebRequest async call execution order when exception occurs

I used HttpWebRequest to async call a web service, occasionally EndGetResponse() returns some exceptions. However, even I got the exception, the call is successful and the data are returned. The following is my code with error checking code removed. I got 4 log items and the order is like this:
The following result is returned: ...
GetResult is successful!
PostXML exception: ...
GetResult exception: ...
public async Task<string> GetResult(string xmlData)
{
try
{
string response = await PostXML(Url, xmlData).ConfigureAwait(false);
_logger.Log($"GetResult is successful!");
return response;
}
catch (Exception ex)
{
_logger.Log("GetResult exception: " + ex.ToString());
throw;
}
}
private async Task<string> PostXML(string destinationUrl, string requestXml)
{
try
{
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(requestXml);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(destinationUrl);
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
HttpWebResponse response = (HttpWebResponse)await Task.Factory
.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
_logger.Log("The following result is returned: \r\n" + responseStr);
responseStream.Close();
return responseStr;
}
}
catch (Exception ex)
{
_logger.Log("PostXML exception: " + ex.ToString());
throw;
}
return null;
}
The exception is like this:
PostXML() exception: System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MyService.d__7.MoveNext()
I guess I haven't fully understood how HttpWebRequest works in async mode. Anyway I hope these questions can help me to understand it more:
1. Why does the execution continue even an exception occurs? In my understanding, there should only have 2 log items to log the exception.
2. What is the reason the exception "The underlying connection was closed" occurs? Why does the exception occur but the service still gave back the correct result?

Why does the execution continue even an exception occurs? In my understanding, there should only have 2 log items to log the exception.
This isn't possible. I suspect your code is calling GetResult twice. That's what the log seems to indicate.
What is the reason the exception "The underlying connection was closed" occurs?
This happens when the server closes the connection before the client is ready for it to be closed. This is an unusual error to get from a REST API, but not unheard of.

GRPC with C# and streaming response from server results in error: 'Shutdown has already been called'

I'm trying to use server streaming using GRPC with C#, and my client receives one response and then throws error:
Shutdown has already been called
at Grpc.Core.Internal.CompletionQueueSafeHandle.BeginOp()
at Grpc.Core.Internal.CallSafeHandle.StartReceiveMessage(IReceivedMessageCallback callback)
at Grpc.Core.Internal.AsyncCallBase`2.ReadMessageInternalAsync()
at Grpc.Core.Internal.ClientResponseStream`2.<MoveNext>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at GRPCTransferClient.Program.<Test>d__1.MoveNext() in Program.cs:line 25
I tried throttling the responses by putting delays both on the server and client side but I still got the error. But I noticed that if I put a 1 second delay on the client before trying to call MoveNext() I won't even get the first response and it throws the error immediately.
My proto is this:
syntax = "proto3";
package Mega.SplitText;
service SplitText {
rpc Split(Text) returns (stream Line) {}
}
message Text {
string text = 1;
}
message Line {
string line = 1;
}
Server code:
public class SplitTextServiceImpl : SplitTextBase
{
public SplitTextServiceImpl()
{
}
public override async Task Split(Text request, IServerStreamWriter<Line> responseStream, ServerCallContext context)
{
foreach (string line in request.Text_.Split(';'))
{
await responseStream.WriteAsync(new Line { Line_ = line });
Console.WriteLine($"Sent {line}");
}
}
}
Client code:
class Program
{
static void Main(string[] args)
{
var program = new Program();
program.Test();
}
async void Test()
{
Channel channel = new Channel($"127.0.0.1:50051", ChannelCredentials.Insecure);
var client = new SplitTextClient(channel);
using (var response = client.Split(new Text { Text_ = "a;b;c;d;e" }))
{
while (await response.ResponseStream.MoveNext())
{
Console.WriteLine(response.ResponseStream.Current);
}
}
Console.ReadLine();
}
}
I copied this structure from the examples folder on the official GRPC repository, and that works fine on my machine, so I have no idea what I'm doing wrong in this other project.
The projects are both .Net Core 2.0, but I also tried with .Net Framework 4.6.1 just in case. In both cases the result is the same.

Solved, the problem was that I forgot to wait for the client async function. Solution:
static void Main(string[] args)
{
var program = new Program();
program.Test().Wait(); // Need to wait here
}
async Task Test()
{
Channel channel = new Channel($"127.0.0.1:50051", ChannelCredentials.Insecure);
var client = new SplitTextClient(channel);
using (var response = client.Split(new Text { Text_ = "a;b;c;d;e" }))
{
await Task.Delay(1000);
while (await response.ResponseStream.MoveNext())
{
Console.WriteLine(response.ResponseStream.Current);
}
}
Console.ReadLine();
}

WebException: The remote server returned an error: NotFound Windows Phone Youtube Html

I wanted to get elements and tags inside of youtube search results with below code for windows phone application.
public partial class Page1 : PhoneApplicationPage
{
string keyword;
public Page1()
{
InitializeComponent();
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
NavigationContext.QueryString.TryGetValue("parameter", out keyword);
LoadResults();
}
public void LoadResults()
{
WebClient codeSampleReq = new WebClient();
codeSampleReq.DownloadStringCompleted += codeSampleReq_DownloadStringCompleted;
codeSampleReq.DownloadStringAsync(new Uri("https://www.youtube.com/results?search_query=" + keyword));
}
void codeSampleReq_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
try
{
HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.OptionFixNestedTags = true;
htmlDoc.LoadHtml(e.Result);
HtmlNode divContainer = htmlDoc.GetElementbyId("a");
if (divContainer != null)
{
HtmlNodeCollection nodes = divContainer.SelectNodes("a");
foreach (HtmlNode trNode in nodes)
{
}
}
}
catch (Exception ex)
{
MessageBox.Show("Unable to download" + ex.Message);
}
}
}
and get exception below:
{System.Net.WebException: The remote server returned an error:
NotFound. ---> System.Net.WebException: The remote server returned an
error: NotFound. at
System.Net.Browser.ClientHttpWebRequest.InternalEndGetResponse(IAsyncResult
asyncResult) at
System.Net.Browser.ClientHttpWebRequest.<>c__DisplayClasse.b__d(Object
sendState) at
System.Net.Browser.AsyncHelper.<>c__DisplayClass1.b__0(Object
sendState) --- End of inner exception stack trace --- at
System.Net.Browser.AsyncHelper.BeginOnUI(SendOrPostCallback
beginMethod, Object state) at
System.Net.Browser.ClientHttpWebRequest.EndGetResponse(IAsyncResult
asyncResult) at System.Net.WebClient.GetWebResponse(WebRequest
request, IAsyncResult result) at
System.Net.WebClient.DownloadBitsResponseCallback(IAsyncResult
result)}
does anybody know why server return 404 not found?
note: Keyword can only be string like "ali, veli, kadir" or something else.

NotFound is WebClient`s response many kinds of errors. The two most common cases are
bad server certificate (should not be your case, Youtube has a valid certificate)
no Internet connection in your emulator or device.
I almost sure your case is the second one, check your Internet connection.