i'm writing a Telegram bot without using any library but only using HTTP request. The problem is that i keep getting updates endlessly when i start long polling, here's my code:
public async Task StartPolling(string[] allowedUpdates = default!)
{
int messageOffset = 0;
while (true)
{
Console.WriteLine("Sending GetUpdate with offset " + messageOffset);
var updates = await GetUpdatesAsync(offset: messageOffset, timeout: Convert.ToInt32(s_httpClient.Timeout.TotalSeconds), allowedUpdates: allowedUpdates);
Console.WriteLine("Receving " + updates.Length + " updates...");
foreach (var update in updates!)
{
HandleUpdatesAsync(update);
messageOffset = update.UpdateId + 1;
Console.WriteLine("MessageId = " + update.UpdateId);
}
Console.WriteLine("Offsett = " + messageOffset);
}
}
public async Task<Update[]?> GetUpdatesAsync(int offset = 0, int limit = 100, int timeout = 0, string[]? allowedUpdates = default)
{
var request = new GetUpdatesRequest
{
Offset = offset,
Limit = limit,
Timeout = timeout,
AllowedUpdates = allowedUpdates ?? Array.Empty<string>(),
};
return await SendRequestAsync<Update[]>(request);
}
private async Task<T?> SendRequestAsync<T>(IRequest request)
{
var httpRequestMessage = new HttpRequestMessage(method: request.Method, requestUri: $"{_baseUrl}/{request.MethodName}")
{
Content = request.Content
};
try
{
var httpResponseMessage = await s_httpClient.SendAsync(httpRequestMessage);
var updatesJson = await httpResponseMessage.Content.ReadAsStreamAsync();
var response = await JsonSerializer.DeserializeAsync<Response<T>>(updatesJson, options);
return response!.Result;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
throw;
}
}
when i run the code and send a message to my bot it keeps printing the "Sending GetUpdate with offsett etc... here's the screenshot of the console
The only way to stop getting infinite updates is by closing the application or by manualy sending a getUpdate with the SAME offset of the code in the browser like this:
https://api.telegram.org/botXXXXXXXXXXXXXXX/getUpdates?offset=419787029
Sorry for my English :)
Related
I have a api in which I need to post a request and write the response to individual txt files.
Each post request needs to go with a MaterialID which I am reading from a list. The list can contain anything from 1000 to 5000 MaterialIDs
I need a way to do multiple parallel requests.
The below code is what I currently have but is built more for synchronous request.
How do i go about doing parallel requests ?
//reads MaterialID from file
System.IO.StreamReader fileTXT =
new System.IO.StreamReader(#"C:\\Temp\Test.txt");
while ((line = fileTXT.ReadLine()) != null)
{
Material_Count.Add(line.Trim());
UpdateLogTxt("Material_Count = " + line.ToString() + " ");
}
fileTXT.Close();
//loops through MaterialID list and starts post request
foreach (var item in Material_Count)
{
try
{
UpdateLogTxt(" Submitting = " + item.ToString());
var Set = Properties.Settings.Default;
using (var httpClient = new HttpClient())
{
string FCSEndPoint = #"https://www.api.co.za";
string FCSToken = "";
FCSToken = textBoxDescription.Text.Trim();
string uri = $"{FCSEndPoint}/material/pageIndex=" + item.ToString() + "";
HttpResponseMessage response = null;
httpClient.BaseAddress = new Uri(uri);
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SessionToken", FCSToken);
AuthenticationHeaderValue headerObj = new AuthenticationHeaderValue("SessionToken", FCSToken);
httpClient.DefaultRequestHeaders.Authorization = headerObj;
var TaskObj = httpClient.PostAsJsonAsync(uri, String.Empty);
TaskObj.Wait();
HttpResponseMessage messageOut = TaskObj.Result;
response = TaskObj.Result;
if (response.IsSuccessStatusCode)
{
var TaskObj2 = messageOut.Content.ReadAsStringAsync();
TaskObj2.Wait();
string ResponseStr = TaskObj2.Result;
//writes response to a txt file.
string fileName = #"C:\Temp\material\Material_ " + item.ToString() + " " + DateTime.Now.ToString("yyyyMMddmmss") + ".txt";
try
{
// Check if file already exists. If yes, delete it.
if (File.Exists(fileName))
{
File.Delete(fileName);
}
// Create a new file
using (StreamWriter sw = File.CreateText(fileName))
{
sw.WriteLine(ResponseStr.ToString());
}
}
catch (Exception ex)
{
UpdateLogTxt("Exception Write to file Failed = --- " + ex.ToString());
}
}
else if (response.StatusCode == System.Net.HttpStatusCode.Forbidden)
{
UpdateLogTxt("Response Failed (Forbidden) = --- " + response.StatusCode.ToString());
}
else
{
}
}
}
catch (HttpRequestException h)
{
UpdateLogTxt("HttpRequestException Send Failed = --- " + h.ToString());
}
catch (Exception ex)
{
UpdateLogTxt("Exception Send Failed = --- " + ex.ToString());
}
}
Replace the foreach by the Parallel.ForEach in the System.Threading.Tasks namespace
You can find handy sample codes here =>
Parallel.ForEach method
Here's an alternative using Microsoft's Reactive Framework - NuGet System.Reactive and then you can do this:
Start by collecting a couple of constants used in your code to make sure they happen on the UI thread and only once:
string fcsToken = textBoxDescription.Text.Trim();
string now = DateTime.Now.ToString("yyyyMMddmmss");
Now lets create a helper method to handle the HTTP call:
Task<HttpResponseMessage> PostAsJsonAsync(HttpClient hc, string item)
{
string fcsEndPoint = #"https://www.api.co.za";
string uri = $"{fcsEndPoint}/material/pageIndex={item}";
hc.BaseAddress = new Uri(uri);
hc.DefaultRequestHeaders.Accept.Clear();
hc.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
hc.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SessionToken", fcsToken);
AuthenticationHeaderValue authorization = new AuthenticationHeaderValue("SessionToken", fcsToken);
hc.DefaultRequestHeaders.Authorization = authorization;
return hc.PostAsJsonAsync(uri, String.Empty);
}
Now we can write the observable query:
IObservable<(string fileName, string responseText)> query =
from item in
File
.ReadLines(#"C:\\Temp\Test.txt")
.Select(x => x.Trim())
.ToObservable()
from output in
Observable
.Using(
() => new HttpClient(),
hc =>
from response in Observable.FromAsync(() => PostAsJsonAsync(hc, item))
where response.IsSuccessStatusCode
from responseText in Observable.FromAsync(() => response.Content.ReadAsStringAsync())
select
(
fileName: $#"C:\Temp\material\Material_ {item} {now}.txt",
responseText: responseText
))
select output;
And finally we have the subscription:
IDisposable subscription =
query
.Subscribe(x => File.WriteAllText(x.fileName, x.responseText));
That's it. It's all asynchronous and concurrent.
I am getting 400 error code with bad request while request to file upload API.
I built the back-end and front-end for file uploading in asp.net core and it works in localhost when I run it with IIS in my PC (using visual studio 2017).
Both of saving and updating API are working in my local but update API is not working if I deploy the code
front-end code like below:
public static async Task<HttpResponseMessage> UploadFile(string uploadUrl, string filePath, FFFileInfo fileInfo)
{
string fileName = fileInfo.Name + "." + fileInfo.Extension;
string contentType = MimeTypes.GetMimeType(filePath);
using (var hc = new HttpClient())
{
hc.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(TokenType, AccessToken);
hc.DefaultRequestHeaders.Accept.Clear();
hc.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
Stream fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read);
StreamContent streamContent = CreateFileContent(fileStream, fileName, contentType);
// StreamContent streamContent = CreateFileContent(fileStream, "image.jpg", "image/jpeg"); // Multiple file upload
var requestContent = new MultipartFormDataContent("Upload Id" + DateTime.Now.ToString(CultureInfo.InvariantCulture));
requestContent.Add(streamContent, fileInfo.Name, fileName);
var progressContent = new ProgressableStreamContent(
requestContent,
4096,
(sent, total) =>
{
//Console.WriteLine("Uploading {0}/{1}", sent, total);
int percentage = (int) Math.Round((double)(100 * sent) / total);
Console.Write("\r{0}\t{1}%", fileInfo.Path, percentage);
if (sent == total)
{
Console.WriteLine();
}
});
var response = await hc.PostAsync(new Uri(uploadUrl), progressContent);
return response;
}
}
backend code like below:
[HttpPost]
[DisableFormValueModelBinding]
public async Task<IActionResult> UploadFiles([FromQuery] FFFileInfo fileinfo)
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
return BadRequest($"Expected a multipart request, but got {Request.ContentType}");
}
authUser = User.ToAuthUser();
userId = authUser.UserId();
customerId = authUser.CustomerId();
Server.Model.File new_file = new Server.Model.File();
var boundary = MultipartRequestHelper.GetBoundary(MediaTypeHeaderValue.Parse(Request.ContentType), _defaultFormOptions.MultipartBoundaryLengthLimit);
var reader = new MultipartReader(boundary, HttpContext.Request.Body);
var section = await reader.ReadNextSectionAsync();
MemoryStream writeStream = new MemoryStream();
byte[] content = null;
while (section != null)
{
ContentDispositionHeaderValue contentDisposition;
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out contentDisposition);
int chunkSize = 1024;
byte[] byte_file = new byte[chunkSize];
int bytesRead = 0;
new_file.File_Content = byte_file;
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
{
//await section.Body.CopyToAsync(targetStream);
using (var byte_reader = new BinaryReader(section.Body))
{
do
{
bytesRead = byte_reader.Read(byte_file, 0, byte_file.Length);
if(bytesRead <= 0)
{
content = writeStream.ToArray();
}
writeStream.Write(byte_file, 0, bytesRead);
} while (bytesRead > 0);
content = writeStream.ToArray();
}
}
}
// Drains any remaining section body that has not been consumed and
// reads the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
try
{
new_file = new Server.Model.File
{
File_Name = fileinfo.Name,
File_Path = fileinfo.Path,
File_Ext = fileinfo.Extension,
Check_Sum = fileinfo.Checksum,
ToolSerialNumber = fileinfo.ToolSerialNumber,
FileSize = fileinfo.Length,
File_Content = content,
UserId = userId,
CustomerId = customerId
};
}
catch (Exception ex)
{
return BadRequest(ex);
}
try
{
if (!fileService.isExist(new_file.File_Path, userId))
{
fileService.SaveFile(new_file);
}
else
{
Server.Model.File existing = fileService.GetFileByPath(new_file.File_Path, userId);
fileService.UpdateFile(existing, new_file);
}
//set file content to null to response with small data
new_file.File_Content = null;
return Ok(new_file);
}
catch (Exception ex)
{
logger.LogError("DB action error {0}", ex.ToString());
return BadRequest(ex);
}
}
As you can see the above code, saving and updating are using same code but only updating is not working when it is deployed.
It is very strange for me.
I found the solution.
This code was deployed by my client I couldn't check the database that he deployed.
Based on researching and testing, I got an idea that might be related with permission issue.
So, we check it for db.
At the end, we found that current user has insert, delete, select permission but have not update permission.
After granting the update permission, it is working perfectly
I have used the following code to send using the System.Net.WebSocket from WPF after socket has connected
try
{
Console.WriteLine("Sending...");
await _ws.SendAsync(new ArraySegment<byte>(packet.GetBytes()), WebSocketMessageType.Binary, true, CancellationToken.None);
Console.WriteLine("Sent...");
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
Calling code
if (!ValidateSettings()) return;
var hostname = TbHostname.Text.Trim();
var tcpport = TbTcpPort.Text.Trim();
_ws = new AchiWebSocket();
_streamTimer = new System.Threading.Timer(async s =>
{
var socket = (AchiWebSocket)s;
if (!_semaWs.Wait(0)) return;
UpdateBuffer();
var buffer = GetFrameBuffer();
var packet = new BinaryPacket(buffer.GetBytes());
if(socket.WebSocketState == null || socket.WebSocketState == WebSocketState.Closed)
await socket.Connect(new Uri("ws://" + hostname + "/superrfb"));
try
{
await socket.SendAsync(packet);
}
catch(Exception e)
{
Console.WriteLine("--" + e.Message);
}
finally
{
_semaWs.Release();
}
}, _ws, 0, 70);
Receiving Code (Asp.Net Core)
public async Task StartReceiveAsync(Action<Packet> onReceive)
{
WebSocketReceiveResult result;
var buffer = new byte[4 * 1024];
var stream = new MemoryStream();
do
{
result = await _ws.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
await stream.WriteAsync(buffer, 0, result.Count);
if (result.EndOfMessage)
{
var packet = BinaryPacket.FromRaw(stream.ToArray());
onReceive.Invoke(packet);
}
}
while (!result.CloseStatus.HasValue);
Dispose();
}
I do receive some data but the socket gets stuck in SendAsync after a few packets. It doesn't return even after waiting several minutes.
Ok I figured it out. My onReceive.Invoke() function was throwing exception sometimes causing the server to stop receiving. This made the receive buffer fill up and WebSocket.SendAsync to stuck and wait until the buffer is free. After wrapping the onReceive.Invoke() in try catch, the code is working fine now.
try
{
onReceive.Invoke(packet);
}
catch(Exception e)
{
Debug.WriteLine(e.Message);
}
I am creating a twiiter stremaing MVC controller. The controller method is implemented as a web socket.
Ref: Connecting with websockets to windows azure using MVC controllerenter link description here
I want to display the twitter stream in the UI. I am using twitterinvl. My code is included below. The stream event MatchingTweetReceived are not getting triggered from the controller method.
Any help on this is highly appreciated.
Controller action method.
public ActionResult About()
{
if (ControllerContext.HttpContext.IsWebSocketRequest)
{
ControllerContext.HttpContext.AcceptWebSocketRequest(DoTalking);
}
return new HttpStatusCodeResult(HttpStatusCode.SwitchingProtocols);
}
Rest of the code
public async Task DoTalking(AspNetWebSocketContext context)
{
try
{
WebSocket socket = context.WebSocket;
while (true)
{
var buffer = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = await socket.ReceiveAsync(buffer, CancellationToken.None);
if (socket.State == WebSocketState.Open)
{
string userMessage = Encoding.UTF8.GetString(buffer.Array, 0, result.Count);
string outmessage = "Empty";
//userMessage = "You sent: " + userMessage + " at " + DateTime.Now.ToLongTimeString();
Auth.SetUserCredentials("<<credential>>", "<<credential>>", "<<credential>>", "<<credential>>");
var stream = Stream.CreateFilteredStream();
stream.AddTrack("enterhash");
stream.MatchingTweetReceived += (sender, theTweet) =>
{
outmessage = theTweet.Tweet.CreatedBy.ToString() + theTweet.Tweet.CreatedAt + theTweet.Tweet;
buffer = new ArraySegment<byte>(Encoding.UTF8.GetBytes(userMessage));
//Console.WriteLine($"Tweet: {theTweet.Tweet.CreatedAt} {theTweet.Tweet}");
socket.SendAsync(buffer, WebSocketMessageType.Text, true, CancellationToken.None);
};
stream.StartStreamMatchingAllConditions();
Trace.WriteLine(outmessage);
//await socket.SendAsync(buffer, WebSocketMessageType.Text, true, CancellationToken.None);
}
else
{
break;
}
}
}
catch (Exception ex)
{
}
}
This question is a followup to Threading issues when using HttpClient for asynchronous file downloads.
To get a file transfer to complete asynchronously using HttpClient, you need to add HttpCompletionOption.ResponseHeadersRead to the SendAsync request. Thus, when that call completes, you will be able to determine that all was well with the request and the response headers by adding a call to EnsureSuccessStatusCode. However the data is possibly still being transferred at this point.
How can you detect errors which happen after the headers are returned but before the data transfer is complete? How would said errors manifest themselves?
Some example code follows, with the point of the question marked at line 109)with the comment: "// *****WANT TO DO MORE ERROR CHECKING HERE**"
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
namespace TestHttpClient2
{
class Program
{
/* Use Yahoo portal to access quotes for stocks - perform asynchronous operations. */
static string baseUrl = "http://real-chart.finance.yahoo.com/";
static string requestUrlFormat = "/table.csv?s={0}&d=0&e=1&f=2016&g=d&a=0&b=1&c=1901&ignore=.csv";
static void Main(string[] args)
{
var activeTaskList = new List<Task>();
string outputDirectory = "StockQuotes";
if (!Directory.Exists(outputDirectory))
{
Directory.CreateDirectory(outputDirectory);
}
while (true)
{
Console.WriteLine("Enter symbol or [ENTER] to exit:");
string symbol = Console.ReadLine();
if (string.IsNullOrEmpty(symbol))
{
break;
}
Task downloadTask = DownloadDataForStockAsync(outputDirectory, symbol);
if (TaskIsActive(downloadTask))
{
// This is an asynchronous world - lock the list before updating it!
lock (activeTaskList)
{
activeTaskList.Add(downloadTask);
}
}
else
{
Console.WriteLine("task completed already?!??!?");
}
CleanupTasks(activeTaskList);
}
Console.WriteLine("Cleaning up");
while (CleanupTasks(activeTaskList))
{
Task.Delay(1).Wait();
}
}
private static bool CleanupTasks(List<Task> activeTaskList)
{
// reverse loop to allow list item deletions
// This is an asynchronous world - lock the list before updating it!
lock (activeTaskList)
{
for (int i = activeTaskList.Count - 1; i >= 0; i--)
{
if (!TaskIsActive(activeTaskList[i]))
{
activeTaskList.RemoveAt(i);
}
}
return activeTaskList.Count > 0;
}
}
private static bool TaskIsActive(Task task)
{
return task != null
&& task.Status != TaskStatus.Canceled
&& task.Status != TaskStatus.Faulted
&& task.Status != TaskStatus.RanToCompletion;
}
static async Task DownloadDataForStockAsync(string outputDirectory, string symbol)
{
try
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(baseUrl);
client.Timeout = TimeSpan.FromMinutes(5);
string requestUrl = string.Format(requestUrlFormat, symbol);
var request = new HttpRequestMessage(HttpMethod.Post, requestUrl);
var response = await client.SendAsync(request,
HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
using (var httpStream = await response.Content.ReadAsStreamAsync())
{
var timestampedName = FormatTimestampedString(symbol, true);
var filePath = Path.Combine(outputDirectory, timestampedName + ".csv");
using (var fileStream = File.Create(filePath))
{
await httpStream.CopyToAsync(fileStream);
}
}
// *****WANT TO DO MORE ERROR CHECKING HERE*****
}
}
catch (HttpRequestException ex)
{
Console.WriteLine("Exception on thread: {0}: {1}\r\n",
System.Threading.Thread.CurrentThread.ManagedThreadId,
ex.Message,
ex.StackTrace);
}
catch (Exception ex)
{
Console.WriteLine("Exception on thread: {0}: {1}\r\n",
System.Threading.Thread.CurrentThread.ManagedThreadId,
ex.Message,
ex.StackTrace);
}
}
static volatile string lastTimestampedString = string.Empty;
static volatile string dummy = string.Empty;
static string FormatTimestampedString(string message, bool uniquify = false)
{
// This is an asynchronous world - lock the shared resource before using it!
lock (dummy)
//lock (lastTimestampedString)
{
Console.WriteLine("IN - Thread: {0:D2} lastTimestampedString: {1}",
System.Threading.Thread.CurrentThread.ManagedThreadId,
lastTimestampedString);
string newTimestampedString;
while (true)
{
DateTime lastDateTime = DateTime.Now;
newTimestampedString = string.Format(
"{1:D4}_{2:D2}_{3:D2}_{4:D2}_{5:D2}_{6:D2}_{7:D3}_{0}",
message,
lastDateTime.Year, lastDateTime.Month, lastDateTime.Day,
lastDateTime.Hour, lastDateTime.Minute, lastDateTime.Second,
lastDateTime.Millisecond
);
if (!uniquify)
{
break;
}
if (newTimestampedString != lastTimestampedString)
{
break;
}
//Task.Delay(1).Wait();
};
lastTimestampedString = newTimestampedString;
Console.WriteLine("OUT - Thread: {0:D2} lastTimestampedString: {1}",
System.Threading.Thread.CurrentThread.ManagedThreadId,
lastTimestampedString);
return lastTimestampedString;
}
}
}
}
I have copied and slightly cleaned up the relevant code.
var request = new HttpRequestMessage(HttpMethod.Post, requestUrl);
var response = await client.SendAsync(request,
HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
using (var httpStream = await response.Content.ReadAsStreamAsync())
{
var timestampedName = FormatTimestampedString(symbol, true);
var filePath = Path.Combine(outputDirectory, timestampedName + ".csv");
using (var fileStream = File.Create(filePath))
{
await httpStream.CopyToAsync(fileStream);
}
}
The question is, what if something goes wrong during reading the stream and copying it into your file?
All logical errors have already been addressed as part of the HTTP request and response cycle: the server has received your request, it has decided it is valid, it has responded with success (header portion of response), and it is now sending you the result (body portion of response).
The only errors that could occur now are things like the server crashing, the connection being lost, etc. My understanding is that these will manifest as HttpRequestException, meaning you can write code like this:
try
{
using (var httpStream = await response.Content.ReadAsStreamAsync())
{
var timestampedName = FormatTimestampedString(symbol, true);
var filePath = Path.Combine(outputDirectory, timestampedName + ".csv");
using (var fileStream = File.Create(filePath))
{
await httpStream.CopyToAsync(fileStream);
}
}
}
catch (HttpRequestException e)
{
...
}
The documenation doesn't say much, unfortunately. The reference source doesn't either. So your best bet is to start with this and maybe log all exceptions that are not HttpRequestException in case there is another exception type that could be thrown during the download of the response body.
If you want to narrow it down to the part which is between the header read and the content read, you actually leave yourself with the asynchronous buffer read:
var httpStream = await response.Content.ReadAsStreamAsync();
If you look whats going on inside the method, you'll see:
public Task<Stream> ReadAsStreamAsync()
{
this.CheckDisposed();
TaskCompletionSource<Stream> tcs = new TaskCompletionSource<Stream>();
if (this.contentReadStream == null && this.IsBuffered)
{
this.contentReadStream = new MemoryStream(this.bufferedContent.GetBuffer(),
0, (int)this.bufferedContent.Length,
false, false);
}
if (this.contentReadStream != null)
{
tcs.TrySetResult(this.contentReadStream);
return tcs.Task;
}
this.CreateContentReadStreamAsync().ContinueWithStandard(delegate(Task<Stream> task)
{
if (!HttpUtilities.HandleFaultsAndCancelation<Stream>(task, tcs))
{
this.contentReadStream = task.Result;
tcs.TrySetResult(this.contentReadStream);
}
});
return tcs.Task;
}
CreateContentReadStreamAsync is the one doing all the reading, internally, it will call LoadIntoBufferAsync, which you can find here.
Basically, you can see the it encapsulates IOException and ObjectDisposedException, or an ArgumentOutOfRangeException is the buffer is larger than 2GB (although i think that will be highly rare).