Develop Reference

ASP.NET - Cannot get error response immediately when uploading very large file?

When I use Postman to try uploading a large file to my server (written in .NET Core 2.2), Postman immediately shows the HTTP Error 404.13 - Not Found error: The request filtering module is configured to deny a request that exceeds the request content length
But when I use my code to upload that large file, it gets stuck at the line to send the file.
My client code:
public async void TestUpload() {
StreamContent streamContent = new StreamContent(File.OpenRead("D:/Desktop/large.zip"));
streamContent.Headers.Add("Content-Disposition", "form-data; name=\"file\"; filename=\"large.zip\"");
MultipartFormDataContent multipartFormDataContent = new MultipartFormDataContent();
multipartFormDataContent.Add(streamContent);
HttpClient httpClient = new HttpClient();
Uri uri = new Uri("https://localhost:44334/api/user/testupload");
try {
HttpResponseMessage httpResponseMessage = await httpClient.PostAsync(uri, multipartFormDataContent);
bool success = httpResponseMessage.IsSuccessStatusCode;
}
catch (Exception ex) {
}
}
My server code:
[HttpPost, Route("testupload")]
public async Task UploadFile(IFormFileCollection formFileCollection) {
IFormFileCollection formFiles = Request.Form.Files;
foreach (var item in formFiles) {
using (var stream = new FileStream(Path.Combine("D:/Desktop/a", item.FileName), FileMode.Create)) {
await item.CopyToAsync(stream);
}
}
}
My client code gets stuck at the line HttpResponseMessage httpResponseMessage = await httpClient.PostAsync(uri, multipartFormDataContent), while the server doesn't receive any request (I use a breakpoint to ensure that).
It gets stuck longer if the file is bigger. Looking at Task Manager, I can see my client program uses up high CPU and Disk as it is actually uploading the file to the server. After a while, the code moves to the next line which is
bool success = httpResponseMessage.IsSuccessStatusCode
Then by reading the response content, I get exactly the result as of Postman.
Now I want to know how to immediately get the error to be able to notify the user in time, I don't want to wait really long.
Note that when I use Postman to upload large files, my server doesn't receive any request as well. I think I am missing something, maybe there is problem with my client code.
EDIT: Actually I think it is the client-side error. But if it is server-side error, then it still doesn't mean too much for me. Because, let me clear my thought. I want to create this little helper class that I can use across projects, maybe I can share it with my friends too. So I think it should be able, like Postman, to determine the error as soon as possible. If Postman can do, I can too.
EDIT2: It's weird that today I found out Postman does NOT know before hand whether the server accepts big requests, I uploaded a big file and I saw it actually sent the whole file to the server until it got the response. Now I don't believe in myself anymore, why I thought Postman knows ahead of time the error, I must be stupid. But it does mean that I have found a way to do my job even better than Postman, so I think this question might be useful for someone.

Your issue has nothing to do with your server-side C# code. Your request gets stuck because of what is happening between the client and the server (by "server" I mean IIS, Apache, Nginx..., not your server-side code).
In HTTP, most clients don't read response until they send all the request data. So, even if your server discovers that the request is too large and returns an error response, the client will not read that response until the server accepts the whole requests.
When it comes to server-side, you can check this question, but I think it would be more convenient to handle it on the client side, by checking the file size before sending it to the server (this is basically what Postman is doing in your case).

Now I am able to do what I wanted. But first I want to thank you #Marko Papic, your informations do help me in thinking about a way to do what I want.
What I am doing is:
First, create an empty ByteArrayContent request, with the ContentLength of the file I want to upload to the server.
Second, surround HttpResponseMessage = await HttpClient.SendAsync(HttpRequestMessage) in a try-catch block. The catch block catches HttpRequestException because I am sending a request with the length of the file but my actual content length is 0, so it will throw an HttpRequestException with the message Cannot close stream until all bytes are written.
If the code reaches the catch block, it means the server ALLOWS requests with the file size or bigger. If there is no exception and the code moves on to the next line, then if HttpResponseMessage.StatusCode is 404, it means the server DENIES requests bigger than the file size. The case when HttpResponseMessage.StatusCode is NOT 404 will never happen (I'm not sure about this one though).
My final code up to this point:
private async Task<bool> IsBigRequestAllowed() {
FileStream fileStream = File.Open("D:/Desktop/big.zip", FileMode.Open, FileAccess.Read, FileShare.Read);
if(fileStream.Length == 0) {
fileStream.Close();
return true;
}
HttpRequestMessage = new HttpRequestMessage();
HttpMethod = HttpMethod.Post;
HttpRequestMessage.Method = HttpMethod;
HttpRequestMessage.RequestUri = new Uri("https://localhost:55555/api/user/testupload");
HttpRequestMessage.Content = new ByteArrayContent(new byte[] { });
HttpRequestMessage.Content.Headers.ContentLength = fileStream.Length;
fileStream.Close();
try {
HttpResponseMessage = await HttpClient.SendAsync(HttpRequestMessage);
if (HttpResponseMessage.StatusCode == HttpStatusCode.NotFound) {
return false;
}
return true; // The code will never reach this line though
}
catch(HttpRequestException) {
return true;
}
}
NOTE: Note that my approach still has a problem. The problem with my code is the ContentLength property, it shouldn't be exact the length of the file, it should be bigger. For example, if my file is exactly 1000 bytes in length, then if the file is successfully uploaded to the server, the Request that the server gets has greater ContentLength value. Because HttpClient doesn't just only send the content of the file, but it has to send many informations in addition. It has to send the boundaries, content types, hyphens, line breaks, etc... Generally speaking, you should somehow find out before hand the exact bytes that HttpClient will send along with your files to make this approach work perfectly (I still don't know how so far, I'm running out of time. I will find out and update my answer later).
Now I am able to immediately determine ahead of time whether the server can accept requests that are as big as the file my user wants to upload.

API GET Request Returns HTML/Text instead of JSON

This is my first time working with a RESTful API and Xamarin and etc. I have so far made a simple REST API. I have written a GET call to it that, if I write http://localhost:[num]/api/Name, it will return a JSON file of the matching Emu's information. I have tested this with Postman, so I know that it works.
I have now written an app that will call this API in order to catch this information and then display it. So far, I've got it connected to the server hosting my API, but I'm unable to get it to return JSON. Instead it seems to be returning text/HTTP.
From what I've searched up on previous Stack Overflow threads, it seems that I was missing Headers requesting that reply be in a JSON format. When I added in code that was on the official .NET documentation on Microsoft's website, it gave me issues with my Json Deserialiser. I have also added in the information in the header to make sure that it returns json.
Here is the code for the function:
async private void Submit_OnClicked(object sender, EventArgs e)
{
var nameValue = EmuName.Text;
var baseAddr = new Uri("http://my_url/HelloEmu/");
var client = new HttpClient { BaseAddress = baseAddr };
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
string url = (string)nameValue;
var returnedJson = await client.GetStringAsync(url);
Models.EmuItemModel MyEmu = JsonConvert.DeserializeObject<Models.EmuItemModel>(returnedJson);
ReturnedName.Text = MyEmu.Name;
ReturnedAge.Text = MyEmu.Age.ToString();
ReturnedWeight.Text = MyEmu.Weight.ToString();
My code actually faults on the line ReturnedWeight.Text = MyEmu.Weight.ToString()
But I'm guessing the more majour issue is occuring during deserialisng the object, because it seemingly "skips" over the preceeding two lines when I run it in the debugger.
When I run it in Visual Studio 2019, the value of "returnedJson" is this:
"<html><head><meta http-equiv=\"refresh\" content=\"0;url=http://lookup.t-mobile.com/search/?q=http://my_url/HelloEmu/Keith&t=0\"/></head><body><script>window.location=\"http://lookup.t-mobile.com/search/?q=\"+escape(window.location)+\"&r=\"+escape(document.referrer)+\"&t=0\";</script></body></html>"
I think this is an HTML output. I would love any hints about what on earth I'm doing wrong!
EDIT: Since it almost seems like the HTML is returning an error message, perhaps it could do with my url??? I've published the website using the File system method. So to access the API in Postman I'll use http://localhost:[port]/api/values, calling my website in a regular ol' browser makes it go http://my_url/HelloEmu. I get a 403 "no directory" method in return...
EDIT: Here is the Postman code:
enter image description here

Usually it happens because there are missing headers or some other malformed request, Download RestSharp DLL from NuGet, and then you can use the following, in postman, go to "Code":
And choose C# you will see a code snippet (Example):

Error deserializing JSON (invalid JSON Primitive) on windows 10 works, on server 2012 does not

I've searched all the existing question/answers concerning the error in the subject but the behaviour let me thing is not something wrong at the code rather on the machine instead.
I have the local dev machine on Windows 10 in wich the deserialization works perfectly. Once I publish on Server 2012 it blows up.
I used 3 version of the code to try to force a resolution, I can get different error messages but the result is the same, on production when try to deserialize blows up
I am using .NET framework 4 and c# with NewtonJson to handle json calls.
What I am unable to find if someone had odd behaviour on different deploy.
below the snippet
foreach(var s in ids) {
i++;
string _endpoint = sbc_url + s;
Uri _uri = new Uri(_endpoint);
WebClient wcClient = new WebClient();
wcClient.BaseAddress = _endpoint;
wcClient.Headers.Add("contentType: 'application/json; charset=utf-8'");
wcClient.Headers.Add("dataType: 'json'");
var response = wcClient.DownloadString(_endpoint);
try {
var jss = new JavaScriptSerializer();
var dict = jss.Deserialize <Dictionary<string, dynamic>> (response); // BLOWS UP HERE
ws_ret r = new ws_ret();
foreach(var tt in dict["result"]) {
r.result.Add(tt);
}
if (r.result != null)
numeri.result.AddRange(r.result);
} catch (Exception ex) {
}
}

Setting contentType: "application/json", the request body will treated as JSON content. This is why "Invalid JSON primitive" error occurs, because of URL encoded format is not same as JSON format.
Remove:
wcClient.Headers.Add("contentType: 'application/json; charset=utf-8'");

I apologize but I've solved. The sysadmin didnt inform me that the connection was under firewall and simply the server machine was out of allowed addresses. BTW I am sure this post still make sense because as I was thinking two identical configurations there are no way to have different behaviour and if someone happen the same issue need to insist to investigate everything.
As I suspected the error message returned "Invalid JSON primitive" does not have anything to see with the permission denyed. Thanks a lot to everybody

How can I make a SOAP call in C# with an old WSDL+XSD and current Web Service location?

My question is similar to this one: need to call soap ws without wsdl except that my application does not use Spring so the answer was not helpful.
Here's what I have:
A web service that only accepts SOAP requests
A current endpoint URL for the web service
An outdated wsdl and xsd file
An outdated sample SOAP request file
What I need to do is:
Successfully make a SOAP request using some combination of the above.
I've tried to approach this from two different angles, with no luck so far. My background is familiarity with web services with POST and GETs, but not SOAP. After googling 'C# SOAP', I wrote the following code:
void soap(String xmlfile)
{
Stream outputStream = null;
StreamReader reader = null;
WebResponse response = null;
try
{
string text = System.IO.File.ReadAllText(xmlfile);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://misapi.ercot.com/2007-08/Nodal/eEDS/EWS");
request.PreAuthenticate = true;
X509Certificate ercotCert = X509Certificate.CreateFromCertFile("D:\\Amigo\\WebSite1\\Nodal_Test_Cert.cer");
request.ClientCertificates.Clear();
request.ClientCertificates.Add(ercotCert);
ServicePointManager.ServerCertificateValidationCallback +=
new System.Net.Security.RemoteCertificateValidationCallback(customValidation);
request.Credentials = CredentialCache.DefaultNetworkCredentials;
// I don't actually have a SOAPAction, but have tried adding a fake one just to see
//request.Headers.Add("SOAPAction", "http://www.ercot.com/Nodal/Payload");
request.Method = "POST";
request.ContentType = "text/xml;charset=\"utf-8\"";
request.ContentLength = text.Length;
StreamWriter writer = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
writer.Write(text);
writer.Close();
response = request.GetResponse();
outputStream = response.GetResponseStream();
reader = new StreamReader(outputStream);
Response.Write(reader.ReadToEnd());
}
catch (WebException e)
{
// This is where it ends up, with Status="ProtocolError"
}
catch (System.Web.Services.Protocols.SoapException soapE)
{
// Never gets in here
}
catch (Exception e)
{
// Never gets in here
}
finally
{
if (outputStream != null)
outputStream.Close();
if(reader != null)
reader.Close();
if(response != null)
response.Close();
}
}
private static bool customValidation(object sender, X509Certificate cert, X509Chain chain, System.Net.Security.SslPolicyErrors error)
{
return true;
}
This yields a 500 Internal Server Error. It throws a WebException which contains no other message or inner exception, but the Status is 'ProtocolError'. I have tried other permutations including using XmlDocument and other content types, but none have worked.
The other thing I've tried is using "Add Web Reference" in Visual Studio. Putting in the endpoint URL doesn't work and gives a soap fault. If, instead, I point to the local copy of my outdated wsdl, then it will add the WebReference but won't let me use it due to numerous errors that I cannot correct. My guess is that these errors are due to the wsdl being outdated, things like the namespace not matching or being unable to find things at the URLs included. If I replace those URLs with the current web service endpoint URL, it still does not work.
If anyone could pinpoint a problem in my code, or direct me on how to get the "Add Web Reference" working, I would be greatly greatly appreciated!
Thanks in advance!

SOAP is just the format for the payload that you POST to the service, that's all. There's a defined specification for fields and namespaces and such, and there are different SOAP versions, but there's really not that much to it. (Other than it being remarkably verbose for it's usage, but that's a different topic.)
You need to start with a current WSDL. If you know you're working WSDL is outdated, that means that something the webservice is expecting (required) or could work with (optional) is different from your definition. And the WSDL is your contractual gateway into the webservice. You need to get a current WSDL.
As luck would have it, when I navigate to https://misapi.ercot.com/2007-08/Nodal/eEDS/EWS/?WSDL (which I derived by appending "?WSDL" to the end of the url), after skipping past the certificate error, bingo -- there's the WSDL used by the service. You can either (a) save it locally, or (b) reference it directly from Visual Studio in building a web service client. Because of the cert error, I would recommend saving it locally and building from there.
This should get you started.

HttpWebRequest.AllowAutoRedirect=false can cause timeout?

I need to test around 300 URLs to verify if they lead to actual pages or redirect to some other page. I wrote a simple application in .NET 2.0 to check it, using HttpWebRequest. Here's the code snippet:
System.Net.HttpWebRequest wr = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create( url );
System.Net.HttpWebResponse resp = (System.Net.HttpWebResponse)wr.GetResponse();
code = resp.StatusDescription;
Code ran fast and wrote to file that all my urls return status 200 OK. Then I realized that by default GetResponse() follows redirects. Silly me! So I added one line to make it work properly:
System.Net.HttpWebRequest wr = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create( url );
wr.AllowAutoRedirect = false;
System.Net.HttpWebResponse resp = (System.Net.HttpWebResponse)wr.GetResponse();
code = resp.StatusDescription;
I ran the program again and waited... waited... waited... It turned out that for each url I was getting a System.Net.WebException "The operation has timed out". Surprised, I checked the URL manually - works fine... I commented out AllowAutoRedirect = false line - and it works fine again. Uncommented this line - timeout. Any ideas what might cause this problem and how to work around?

Often timeouts are due to web responses not being disposed. You should have a using statement for your HttpWebResponse:
using (HttpWebResponse resp = (HttpWebResponse)wr.GetResponse())
{
code = resp.StatusDescription;
// ...
}
We'd need to do more analysis to predict whether that's definitely the problem... or you could just try it :)
The reason is that .NET has a connection pool, and if you don't close the response, the connection isn't returned to the pool (at least until the GC finalizes the response). That leads to a hang while the request is waiting for a connection.