Develop Reference

multi-threaded foreach Api Post and write response to a folder

I have a api in which I need to post a request and write the response to individual txt files.
Each post request needs to go with a MaterialID which I am reading from a list. The list can contain anything from 1000 to 5000 MaterialIDs
I need a way to do multiple parallel requests.
The below code is what I currently have but is built more for synchronous request.
How do i go about doing parallel requests ?
//reads MaterialID from file
System.IO.StreamReader fileTXT =
new System.IO.StreamReader(#"C:\\Temp\Test.txt");
while ((line = fileTXT.ReadLine()) != null)
{
Material_Count.Add(line.Trim());
UpdateLogTxt("Material_Count = " + line.ToString() + " ");
}
fileTXT.Close();
//loops through MaterialID list and starts post request
foreach (var item in Material_Count)
{
try
{
UpdateLogTxt(" Submitting = " + item.ToString());
var Set = Properties.Settings.Default;
using (var httpClient = new HttpClient())
{
string FCSEndPoint = #"https://www.api.co.za";
string FCSToken = "";
FCSToken = textBoxDescription.Text.Trim();
string uri = $"{FCSEndPoint}/material/pageIndex=" + item.ToString() + "";
HttpResponseMessage response = null;
httpClient.BaseAddress = new Uri(uri);
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SessionToken", FCSToken);
AuthenticationHeaderValue headerObj = new AuthenticationHeaderValue("SessionToken", FCSToken);
httpClient.DefaultRequestHeaders.Authorization = headerObj;
var TaskObj = httpClient.PostAsJsonAsync(uri, String.Empty);
TaskObj.Wait();
HttpResponseMessage messageOut = TaskObj.Result;
response = TaskObj.Result;
if (response.IsSuccessStatusCode)
{
var TaskObj2 = messageOut.Content.ReadAsStringAsync();
TaskObj2.Wait();
string ResponseStr = TaskObj2.Result;
//writes response to a txt file.
string fileName = #"C:\Temp\material\Material_ " + item.ToString() + " " + DateTime.Now.ToString("yyyyMMddmmss") + ".txt";
try
{
// Check if file already exists. If yes, delete it.
if (File.Exists(fileName))
{
File.Delete(fileName);
}
// Create a new file
using (StreamWriter sw = File.CreateText(fileName))
{
sw.WriteLine(ResponseStr.ToString());
}
}
catch (Exception ex)
{
UpdateLogTxt("Exception Write to file Failed = --- " + ex.ToString());
}
}
else if (response.StatusCode == System.Net.HttpStatusCode.Forbidden)
{
UpdateLogTxt("Response Failed (Forbidden) = --- " + response.StatusCode.ToString());
}
else
{
}
}
}
catch (HttpRequestException h)
{
UpdateLogTxt("HttpRequestException Send Failed = --- " + h.ToString());
}
catch (Exception ex)
{
UpdateLogTxt("Exception Send Failed = --- " + ex.ToString());
}
}

Replace the foreach by the Parallel.ForEach in the System.Threading.Tasks namespace
You can find handy sample codes here =>
Parallel.ForEach method

Here's an alternative using Microsoft's Reactive Framework - NuGet System.Reactive and then you can do this:
Start by collecting a couple of constants used in your code to make sure they happen on the UI thread and only once:
string fcsToken = textBoxDescription.Text.Trim();
string now = DateTime.Now.ToString("yyyyMMddmmss");
Now lets create a helper method to handle the HTTP call:
Task<HttpResponseMessage> PostAsJsonAsync(HttpClient hc, string item)
{
string fcsEndPoint = #"https://www.api.co.za";
string uri = $"{fcsEndPoint}/material/pageIndex={item}";
hc.BaseAddress = new Uri(uri);
hc.DefaultRequestHeaders.Accept.Clear();
hc.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
hc.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SessionToken", fcsToken);
AuthenticationHeaderValue authorization = new AuthenticationHeaderValue("SessionToken", fcsToken);
hc.DefaultRequestHeaders.Authorization = authorization;
return hc.PostAsJsonAsync(uri, String.Empty);
}
Now we can write the observable query:
IObservable<(string fileName, string responseText)> query =
from item in
File
.ReadLines(#"C:\\Temp\Test.txt")
.Select(x => x.Trim())
.ToObservable()
from output in
Observable
.Using(
() => new HttpClient(),
hc =>
from response in Observable.FromAsync(() => PostAsJsonAsync(hc, item))
where response.IsSuccessStatusCode
from responseText in Observable.FromAsync(() => response.Content.ReadAsStringAsync())
select
(
fileName: $#"C:\Temp\material\Material_ {item} {now}.txt",
responseText: responseText
))
select output;
And finally we have the subscription:
IDisposable subscription =
query
.Subscribe(x => File.WriteAllText(x.fileName, x.responseText));
That's it. It's all asynchronous and concurrent.

Getting FullContent from blobBaseClient.DownloadToAsync

I am trying to download blob using BlockBlobClient DownloadToAsync method.
As mentioned in documentation, I provided required param, but getting only partial content.
Related code is:
string messageJson = string.Empty;
try
{
var blobBaseClient = new BlobBaseClient(_connectionString, _containerName, blobName);
Stream stream = new MemoryStream();
CancellationToken cancelToken = new CancellationToken();
StorageTransferOptions storageTransferOptions = new StorageTransferOptions
{
//bytes * 1000000 = MB
InitialTransferSize = blobInitialChunkSize.Value * 1000000,
MaximumConcurrency = blobChunkUploadMaxThreadCount.Value,
MaximumTransferSize = long.MaxValue
};
BlobRequestConditions blobRequestConditions = new BlobRequestConditions();
Response response = await blobBaseClient.DownloadToAsync(stream, blobRequestConditions, storageTransferOptions, cancelToken);
if(response.Status == Convert.ToInt32(HttpStatusCode.OK)
{
using (StreamReader streamReader = new StreamReader(stream))
{
messageJson = streamReader.ReadToEnd();
}
}
else
{
_logger.LogInformation("response.Headers.ContentType : " + response.Headers.ContentType + DateTime.Now.ToString());
_logger.LogInformation("response.Headers.ContentLength : " + response.Headers.ContentLength + DateTime.Now.ToString());
_logger.LogInformation("response.Headers.ETag : " + response.Headers.ETag + DateTime.Now.ToString());
_logger.LogInformation("response.Headers.RequestId : " + response.Headers.RequestId + DateTime.Now.ToString());
_logger.LogInformation("response.Status : " + response.Status + DateTime.Now.ToString());
_logger.LogInformation("response.ReasonPhrase : " + response.ReasonPhrase + DateTime.Now.ToString());
_logger.LogInformation("stream : " + stream + DateTime.Now.ToString());
}
catch (Exception ex)
{
_logger.LogInformation(ex.Message + " " + DateTime.Now.ToString());
_logger.LogInformation(ex.StackTrace + " " + DateTime.Now.ToString());
_logger.LogInformation(ex.Source + " " + DateTime.Now.ToString());
_logger.LogInformation(ex.InnerException + " " + DateTime.Now.ToString());
throw;
}
return messageJson;
the result I got is else block(not exception) & log values :
response.Headers.ContentType : application / octet - stream
response.Headers.ContentLength : 189778220
response.Headers.ETag : 0x8D860678531E07B
response.Status : 206
response.ReasonPhrase : Partial Content
stream: System.IO.MemoryStream
messageJson:
I want to know how can I get full data instead of Partial Data ? Any pointers/Help will be appreciated.

Try the code bellow with DownloadAsync method, it will return 200.
BlobServiceClient blobServiceClient = new BlobServiceClient(_connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(_containerName);
BlobClient blobClient = containerClient.GetBlobClient(blobName);
if (await blobClient.ExistsAsync())
{
var response = await blobClient.DownloadAsync();
using (var streamReader = new StreamReader(response.Value.Content))
{
while (!streamReader.EndOfStream)
{
var line = await streamReader.ReadLineAsync();
Console.WriteLine(line);
}
}
Console.WriteLine(response);
}
Azure Blob storage supports return 206 response if you're using API version 2011-01-18 or later. As the documentation says:
A successful operation to read the full blob returns status code 200
(OK).
A successful operation to read a specified range returns status code
206 (Partial Content).

As suggested on Bug I asked on Github, I have added success condition for HTTP 206 too and it is working fine:
if ((response.Status == Convert.ToInt32(HttpStatusCode.OK))
|| response.Status == Convert.ToInt32(HttpStatusCode.PartialContent))
{
using StreamReader streamReader = new StreamReader(response.ContentStream);
stream.Position = 0;
messageJson = streamReader.ReadToEnd();
}

Output HTTP status codes via popup

I am trying to output a small popup-like window to the user saying either the restful command was successfully accepted or not (and display the Http Code).
I am really looking for direction on how to do this properly in ASP.NET MVC.
Controller Function
public void ExportJira()
{
string postUrl = "https://somewebsite.org/rest/api/2/issue";
var httpWebRequest = (HttpWebRequest)WebRequest.Create(postUrl);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
//Using Service Account for Demo Jira Instance
httpWebRequest.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes("FakeAccount:fakePw"));
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
string json = #"{""fields"":{""project"":{""key"": ""SDR""},""summary"": ""This is SDR Web App"",""issuetype"" : {""id"":""3""}}}";
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
HttpWebResponse httpResponse = null;
try
{
httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
System.Diagnostics.Debug.WriteLine("Errorcode: {0}", (int)httpResponse.StatusCode);
System.Diagnostics.Debug.WriteLine("Support Headers: \n" + httpResponse.SupportsHeaders + "\n");
System.Diagnostics.Debug.WriteLine("Headers: \n" + httpResponse.Headers + "\n");
TempData["msg"] = "<script>alert('Success!" + (int)httpResponse.StatusCode + "');</script>";
}
}
catch (WebException e)
{
if (e.Status == WebExceptionStatus.ProtocolError)
{
httpResponse = (HttpWebResponse)e.Response;
TempData["msg"] = "<script>alert('Failed!" + (int)httpResponse.StatusCode + "');</script>";
}
else
{
TempData["msg"] = "<script>alert('Failed!" + (int)httpResponse.StatusCode + "');</script>";
}
}
finally
{
if (httpResponse != null)
{
httpResponse.Close();
}
}
}
}
View Code
#Html.ActionLink("Export to Jira", "ExportJira", null, new { id = "myLink" })
#Html.Raw(TempData["msg"])

Every get or post controller function should return a view model, that can be used by the view to display whatever is relevant. Rather than using TempData, do something like this
class ExportJiraViewModel
{
public string Message {get;set;}
}
public IActionResult ExportJira()
{
var vm = new ExportJiraViewModel();
// fill out vm where appropriate
return View(vm);
}

Handling a bool Response using HttpClient windows phone

I'm developing a windows phone app that generates a QRcode, when the code is scanned, then response has to come back as true, please look at my code and advice as i'm getting a false response even thought the other app has scanned the code.
private async void queryQRCode(string code)
{
try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var authorisationHeader = GenerateBasicAuthorizationParameter("merchant-" + ObjUserData.MerchantId, ObjUserData.PassApiPassword); // 12ED5A5F2B38ACCBE437731BB2AC1F30 35A09A5C2AE97F055A0FDAEDA5A7093D
//Console.WriteLine ("http content - " + httpContent.ReadAsStringAsync().Result);
HttpResponseMessage httpResponse = new HttpResponseMessage();
// Do the actual request and await the response
client.DefaultRequestHeaders.Authorization = authorisationHeader;
var f = QueryCodeUrl + "/" + code + "/scanned";
httpResponse = await client.GetAsync(new Uri(f));
//httpResponse.EnsureSuccessStatusCode();
// If the response contains content we want to read it!
if (httpResponse.Content != null)
{
var responseContent = await httpResponse.Content.ReadAsStringAsync();
// From here on you could deserialize the ResponseContent back again to a concrete C# type using Json.Net
Debug.WriteLine(responseContent);
try
{
ScannedCodeResult res = JsonConvert.DeserializeObject<ScannedCodeResult>(responseContent);
Debug.WriteLine("RETURNED RESPONSE IS "+res.scanned.ToString());
if (res.scanned == false)
{
queryQRCode(code);
}
else
{
Debug.WriteLine("YAY!!!! The User has successfully scanned the application");
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message + ex.StackTrace);
}
// var qrResponse = (VcsMasterPassService.MasterPassModels.QRCodeResponse)JsonConvert.DeserializeObject (responseContent, typeof(VcsMasterPassService.MasterPassModels.QRCodeResponse));
// CreateQRCodeImage (qrResponse.code);
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message + ex.StackTrace);
}
finally
{
}
}

Above problem is caused by windows phone caching.

HttpClient's response in Asp.Net returning 404(Not found: Could not find the controller)

I have to authenticate and then request a Web Api in asp.net. The authentication is completed properly. But when I pass the session values to a new request to generate a certain response, the response returns an Controller not found error. I am trying to implement it in Asp.Net MVC application.
Here is the code for the main Index:
public async Task<ActionResult> Index()
{
var result = "";
string json = " { \"username\": \"webservice\", \"password\": \"iIU2piH6wpBF92T\" }";
var request = (HttpWebRequest)WebRequest.Create("https://os.mcparcel.com/osapi/user/login.json");
request.Method = "POST";
request.UserAgent = "MCParcel Plugin";
request.ContentType = "application/json";
using (var s = request.GetRequestStream())
{
using (var stw = new StreamWriter(s))
{
stw.Write(json);
}
}
try
{
var response = (HttpWebResponse)request.GetResponse();
var data = new StreamReader(response.GetResponseStream());
result = data.ReadToEnd();//<-this gives the value for session id and name
string key = "WucHEwRuy7trUDE7u3agEqEWrUkajuCr";
string order_id = "MCParcel3";
int labels_number = 1;
var r = await Test(key, order_id, labels_number, result);
}
catch (WebException ex)
{
var errorData = new StreamReader(ex.Response.GetResponseStream());
var errorString = errorData.ReadToEnd();
}
return View();
}
Here are the other functions inside the same controller, the error is in:
var response
public static async Task<string> Test(string key, string order_id, int labels_number, dynamic results, int? shipment_id = 0)
{
var r = await GetShippingOptionRequest(key, order_id, labels_number, results);
Console.WriteLine(r);
return r;
}
public static async Task<string> GetShippingOptionRequest(string key, string order_id, int labels_number, dynamic results, int? shipment_id = 0)
{
string json = " { \"key\": \"" + key + "\", \"order_id\": \"" + order_id + "\", \"labels_no\": " + labels_number + ", \"shipment_id\": " + shipment_id + "";
var dynObj = JsonConvert.DeserializeObject<LoginResponse>(results);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Method", "POST");
client.DefaultRequestHeaders.Add("ContentType", "application/json");
client.DefaultRequestHeaders.Add("UserAgent", "MCParcel Plugin");
client.DefaultRequestHeaders.Add("sessid", dynObj.sessid);
client.DefaultRequestHeaders.Add("session_name", dynObj.session_name);
client.DefaultRequestHeaders.Add("key", key);
client.DefaultRequestHeaders.Add("order_id", order_id);
client.DefaultRequestHeaders.Add("labels_number", Convert.ToString(labels_number));
client.DefaultRequestHeaders.Add("shipment_id", Convert.ToString(shipment_id));
//the code below is the required response that is not returning values and returning 404 error
var response = await client.GetStringAsync("https://os.mcparcel.com/osapi/service_os_api/get_label_info.json");
return response;
}
}
The response should return something similar to following:
{
"source_labels": ["https://os.mcparcel.com/sites/os.mcparcel.com/files/sourcepdflabels/labels_8c1e3033a8d23c632006639f39ef6964.pdf"],
"tracks": ["https://os.mcparcel.com/track_and_trace/(J)JD0000900581338100135004"],
"labels_file": "https://os.mcparcel.com/sites/os.mcparcel.com/files/pdf_labels/70994c37ad2a99d4047e0684c3e05c1f.pdf"
}
Any help will be highly appreciated. Thanks in advance!!!

Are you able to hit other controllers? If yes, then you might try to create another controller and start moving your current code with simple json request and build it up to complex json request. Or it might also be your routing.
If no, then it might be you have not setup config.MapHttpAttributeRoutes().