I'm trying to build a simple twitter HttpClient and MySQL application using .Net Core 3.1, but I'm seeing an issue where result object becomes null before I'm done handling it. What should be the correct way to handle this?
Sample code:
using (HttpClient httpClient = new HttpClient())
{
httpClient.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var requestUri = "https://api.twitter.com/2/tweets/search/stream";
httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer <token>");
var stream = httpClient.GetStreamAsync(requestUri).Result;
using (var reader = new StreamReader(stream))
{
//while (!reader.EndOfStream)
while (reader.Peek() >= 0)
{
//We are ready to read the stream
var ResultObject = JsonConvert.DeserializeObject<Tweet>(reader.ReadLine());
Console.WriteLine(ResultObject);
if (ResultObject != null) // <== ResultObject disappears after this :: NullReferenceException
{
Console.WriteLine(ResultObject);
string sQuery = $"INSERT INTO MySQLTable (tweet_id,text) VALUES ({ResultObject.data.id},\"{ResultObject.data.text}\");";
Client.NonQuery(sQuery);
Console.WriteLine(Client.Query("SELECT * FROM MySQLTable;"));
};
}
}
}
public class Tweet
{
public TweetData data;
}
public class TweetData
{
public string id;
public string text;
}
(This issue did not appear in .Net 5.0)
Problem was in string sQuery.
I was trying to .Select() on a null member. Works perfectly fine when there is validation present for null members.
Related
I want trace the response body of my httpclient requests in dependencies table Application Insight. My application runs with .NET framework 4.8
I created an Initializer to trace the Dependencies telemetry with the following code:
public class TrackResponseBody : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var requestTelemetry = telemetry as DependencyTelemetry;
if (requestTelemetry == null)
return;
if (requestTelemetry.TryGetOperationDetail("HttpResponse", out var responseObj))
{
var response = responseObj as HttpWebResponse;
if (response != null)
{
using (var stream = response.GetResponseStream())
{
var reader = new StreamReader(stream);
string result = reader.ReadToEnd();
requestTelemetry.Properties["ResponseBody"] = result;
}
}
}
}
}
But when i try to call reader.ReadToEnd(), my code generate this exception: System.NotSupportedException: 'The stream does not support concurrent IO read or write operations.'
This code write correctly in application insight dependencies log, if i don't try to get body response.
This is how I implemented my HttpClient:
var client_ = new HttpClient();
client_.BaseAddress = new Uri("https://www.google.com");
using (var request_ = new HttpRequestMessage())
{
request_.Method = new HttpMethod("GET");
var response_ = await client_.SendAsync(request_).ConfigureAwait(false);
var responseData_ = await response_.Content.ReadAsStringAsync().ConfigureAwait(false);
}
Is there a way to get the body response of my HttpClient?
I try the code by myself, in TrackResponseBody, this line of code returns null: var response = responseObj as HttpWebResponse; , the response variable is always null.
And then I make a small change, instead convert responseObj to HttpWebResponse, I just convert responseObj to string, then add it to the requestTelemetry.Properties. The code is as below:
public class TrackResponseBody: ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var requestTelemetry = telemetry as DependencyTelemetry;
if (requestTelemetry == null)
return;
if (requestTelemetry.TryGetOperationDetail("HttpResponse", out var responseObj))
{
var response = responseObj as HttpWebResponse;
//convert responseObj to string
string s = responseObj.ToString();
requestTelemetry.Properties["ResponseBody"] = s;
}
}
}
Test result:
I have a WebAPI post method:
[HttpPost]
public void Submit(ICollection<IFormFile> formFiles)
{
foreach (var formFile in formFiles)
{
if (formFile.Length > 0)
{
using (var stream = new FileStream(#"1.txt", FileMode.Create))
{
formFile.CopyTo(stream);
}
}
}
}
From Client I am trying to do :
HttpClient client = new HttpClient();
CustomFormFile fromFile = new CustomFormFile(new FileInfo("ds.txt"));
ICollection<IFormFile> listFormsFiles = new List<IFormFile>();
listFormsFiles.Add(fromFile);
var response = client.PostAsJsonAsync(requestURL, listFormsFiles)
When I am trying to set a breakpoint on my WebApi it is not getting hit, but if I change the PostAsync to following, breakpoint gets hit:
var response = client.PostAsJsonAsync(requestURL, fromFile);
But on service, this becomes null as it was expecting a list but got a single object.
Any pointers what am I missing?
We are building a web application that consist of an Angular2 frontend, a ASP.NET Core web api public backend, and a ASP.NET Core web api private backend.
Uploading files from Angular2 to the public backend works. But we would prefer to post them forward to the private backend.
Current working code
[HttpPost]
public StatusCodeResult Post(IFormFile file)
{
...
}
From there I can save the file to disk using file.CopyTo(fileStream);
However, I want to re-send that file, or those files, or, ideally, the whole request to my second web api core.
I am not sure how to achieve this with the HttpClient class of asp.net core.
I've tried all kinds of things such as
StreamContent ss = new StreamContent(HttpContext.Request.Body);
var result = client.PostAsync("api/Values", ss).Result;
But my second backend gets an empty IFormFile.
I have a feeling it is possible to send the file(s) as a stream and reconstruct them on the other side, but can't get it to work.
The solution must use two web api core.
Solution
Public backend in DMZ
[HttpPost]
public StatusCodeResult Post(IFormFile file)
{
try
{
if (file != null && file.Length > 0)
{
using (var client = new HttpClient())
{
try
{
client.BaseAddress = new Uri(currentPrivateBackendAddress);
byte[] data;
using (var br = new BinaryReader(file.OpenReadStream()))
data = br.ReadBytes((int)file.OpenReadStream().Length);
ByteArrayContent bytes = new ByteArrayContent(data);
MultipartFormDataContent multiContent = new MultipartFormDataContent();
multiContent.Add(bytes, "file", file.FileName);
var result = client.PostAsync("api/Values", multiContent).Result;
return StatusCode((int)result.StatusCode); //201 Created the request has been fulfilled, resulting in the creation of a new resource.
}
catch (Exception)
{
return StatusCode(500); // 500 is generic server error
}
}
}
return StatusCode(400); // 400 is bad request
}
catch (Exception)
{
return StatusCode(500); // 500 is generic server error
}
}
Private backend
[HttpPost]
public void Post()
{
//Stream bodyStream = HttpContext.Request.Body;
if (Request.HasFormContentType)
{
var form = Request.Form;
foreach (var formFile in form.Files)
{
var targetDirectory = Path.Combine(_appEnvironment.WebRootPath, "uploads");
var fileName = GetFileName(formFile);
var savePath = Path.Combine(targetDirectory, fileName);
using (var fileStream = new FileStream(savePath, FileMode.Create))
{
formFile.CopyTo(fileStream);
}
}
}
}
Hi i had the same issue and this is what worked for me :
My setup is netCore MVC netCoreApi.
My MVC Controller looks like :
[HttpPost("UploadFiles")]
public async Task<IActionResult> Post(List<IFormFile> files)
{
Sp4RestClient dataPovider = new Sp4RestClient("http://localhost:60077/");
long size = files.Sum(f => f.Length);
foreach (var file in files)
{
await dataPovider.ImportFile(file);
}
return Ok();
}
DataProvider Method :
public async Task ImportFile(IFormFile file)
{
RestClient restClient = new RestClient(_queryBulder.BuildImportFileRequest());
using (var content = new MultipartFormDataContent())
{
content.Add(new StreamContent(file.OpenReadStream())
{
Headers =
{
ContentLength = file.Length,
ContentType = new MediaTypeHeaderValue(file.ContentType)
}
}, "File", "FileImport");
var response = await restClient.Post<IFormFile>(content);
}
}
And least my WebApi Controller :
[HttpPost]
[Route("ImportData")]
public IActionResult Import(IFormFile file)
{
return Ok();
}
To see the complete code here is my RestClient Post method :
public async Task<RestResult<T>> Post<T>(HttpContent content)
{
using (HttpClient httpClient = new HttpClient())
{
HttpResponseMessage response = await httpClient.PostAsync(Endpoint, content);
if (response.StatusCode == HttpStatusCode.Created)
{
T result = JsonConvert.DeserializeObject<T>(await response.Content.ReadAsStringAsync());
return new RestResult<T> { Result = result, ResultCode = HttpStatusCode.OK };
}
RestResult<T> nonOkResult =
new RestResult<T> { Result = default(T), ResultCode = response.StatusCode, Message = await response.Content.ReadAsStringAsync() };
return nonOkResult;
}
}
// Yeah i know im not getting HttpStatusCode.Created back ;)
happy coding ;)
API Code
[Route("api/upload/{id}")]
[HttpPost]
public async Task<IActionResult> Post(string id)
{
var filePath = #"D:\" + id; //+ Guid.NewGuid() + ".png";
if (Request.HasFormContentType)
{
var form = Request.Form;
foreach (var formFile in form.Files)
{
if (formFile.Length > 0)
{
using (var stream = new FileStream(filePath, FileMode.Create))
{
await formFile.CopyToAsync(stream);
}
}
}
}
return Ok(new { Path = filePath });
}
Back End
[Route("home/UploadFile")]
[HttpPost]
public IActionResult UploadFile(IFormFile file)
{
if (file == null || file.Length == 0)
return Content("file not selected");
var client = new HttpClient();
byte[] data;
using (var br = new BinaryReader(file.OpenReadStream()))
data = br.ReadBytes((int)file.OpenReadStream().Length);
ByteArrayContent bytes = new ByteArrayContent(data);
MultipartFormDataContent multiContent = new MultipartFormDataContent
{
{ bytes, "file", file.FileName }
};
var result = client.PostAsync("http://localhost:2821/api/upload/" + file.FileName, multiContent).Result;
return RedirectToAction("file");
}
Download Source
I was in a similar situation - I needed a proxy method for forwarding not only files but also JSON data and whatnot. I did not want to do any analysis of the data in my proxy to let the final receiver deal with it.
So with some help from #Anton Tykhyy I came to the following working solution:
byte[] arr = null;
using (var mems = new MemoryStream())
{
// read entire body into memory first because it might be chunked with unknown length
await request.Body.CopyToAsync(mems);
await mems.FlushAsync(); // not sure if needed after CopyToAsync - better safe then sorry
arr = mems.ToArray();
}
msg.Content = new ByteArrayContent(arr);
msg.Content.Headers.ContentLength = arr.Length;
// keep content-type header "as is" to preserve multipart boundaries etc.
msg.Content.Headers.TryAddWithoutValidation("Content-Type", request.ContentType);
var response = await _httpClient.SendAsync(msg);
I tested it with complex request that contained multipart form data with JSON field and multiple attached files, and all the data reached my backend server without any issues.
Ignoring the HttpClient when you call the private backend API, can you reference the private Core API project from the public Core API project and call the controller directly from the Core API project? See the request is still null/empty. If the request comes out with a value then the issue is with the use of the HttpClient.
Ideally, you want to create a package library(kind of SDK) for your private Core API that you want to distribute to consuming clients. This acts like a wrapper/proxy. This way you can isolate the private backend system and you can troubleshoot it in isolation. So you public Core API project(which is the private backend client) can reference it as nuget package.
I found this code which works great for me, which is easy to use but I'm kinda stuck with a simple problem
private static string DoGET(string URL, NameValueCollection QueryStringParameters = null, NameValueCollection RequestHeaders = null)
{
string ResponseText = null;
using (WebClient client = new WebClient())
{
try
{
if (RequestHeaders != null)
{
if (RequestHeaders.Count > 0)
{
foreach (string header in RequestHeaders.AllKeys)
client.Headers.Add(header, RequestHeaders[header]);
}
}
if (QueryStringParameters != null)
{
if (QueryStringParameters.Count > 0)
{
foreach (string parm in QueryStringParameters.AllKeys)
client.QueryString.Add(parm, QueryStringParameters[parm]);
}
}
byte[] ResponseBytes = client.DownloadData(URL);
ResponseText = Encoding.UTF8.GetString(ResponseBytes);
}
catch (WebException exception)
{
if (exception.Response != null)
{
var responseStream = exception.Response.GetResponseStream();
if (responseStream != null)
{
using (var reader = new StreamReader(responseStream))
{
//Response.Write(reader.ReadToEnd());
}
}
}
}
}
return ResponseText;
}
I want to convert this code into a HTTP Post method and when I changed client.DownloadData(URL) to client.UploadValues(URL), it requires the NameValueCollection as the second parameter but the top portion of the code have added them into the client,
Do I not add "QueryStringParameters" into the client and use it as the 2nd parameter instead?
You asked: Do I not add "QueryStringParameters" into the client and use it as the 2nd parameter instead?
The answer is: Yes
Explanation
The QueryString property of the WebClient class is used to build name/value pairs that are appended to the WebClient URI as a query string, and therefore used mostly in Get requests.
Assuming you wanted, you could still do:
byte[] ResponseBytes = client.UploadValues(URL, client.QueryString);
But this will obviously be wasteful since you could have passed it directly as well.
I am getting my Request from a third party application(different domain) to my ASP application. I am handling the request and doing the business part in my application and as a acknowledgement I need to send XML string as Response to the same Page which POSTED the request to my Application. I was successful in retrieving the input from Request using the following code
NameValueCollection postPageCollection = Request.Form;
foreach (string name in postPageCollection.AllKeys)
{
... = postPageCollection[name]);
}
But i am not sure how to send back the response along with XML String to the site(different domain)?
EDIT: How to get the URL from where the POST happened.
You can get the url that come from Request.ServerVariables["HTTP_REFERER"]
For the XML, here are 2 functions that I use
public static string ObjectToXML(Type type, object obby)
{
XmlSerializer ser = new XmlSerializer(type);
using (System.IO.MemoryStream stm = new System.IO.MemoryStream())
{
//serialize to a memory stream
ser.Serialize(stm, obby);
//reset to beginning so we can read it.
stm.Position = 0;
//Convert a string.
using (System.IO.StreamReader stmReader = new System.IO.StreamReader(stm))
{
string xmlData = stmReader.ReadToEnd();
return xmlData;
}
}
}
public static object XmlToObject(Type type, string xml)
{
object oOut = null;
//hydrate based on private string var
if (xml != null && xml.Length > 0)
{
System.Xml.Serialization.XmlSerializer serializer = new System.Xml.Serialization.XmlSerializer(type);
using (System.IO.StringReader sReader = new System.IO.StringReader(xml))
{
oOut = serializer.Deserialize(sReader);
sReader.Close();
}
}
return oOut;
}
And here is an example how I use it
[Serializable]
public class MyClassThatKeepTheData
{
public int EnaTest;
}
MyClassThatKeepTheData cTheObject = new MyClassThatKeepTheData();
ObjectToXML(typeof(MyClassThatKeepTheData), cTheObject)
Cant you just use the following code:
Request.UrlReferrer.ToString();