implement cURL command in C# (synchronous, post, TLS 1,2) - c#

Here's a generic cURL command to copy a csv file from a server to c:\temp.
curl.exe --data "ARG1=value&ARG2=value" "https://some.server.com/data.csv" > "c:\temp\data.csv"
How would you code this a synchronous method in C# using a post method and TLS 1.2?

byte[] data;
using (WebClient client = new WebClient()) {
data = client.DownloadData("https://some.server.com/data.csv?arg1=value&ARG2=value");
}
File.WriteAllBytes(#"c:\temp\data.csv", data);

After some fiddling found this to work:
using System.Net.Http;
...
var values = new Dictionary<string, string>() { { "arg1", "value1" }, { "arg2", "value2" } };
var content = new FormUrlEncodedContent(values);
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12; // enable TLS 1.2
HttpResponseMessage responseMsg = new HttpClient().PostAsync(#"http://some.server.com/data.csv", content).Result;
if (responseMsg.StatusCode != System.Net.HttpStatusCode.OK) throw new Exception("err msg");
File.WriteAllText(#"c:\temp\data.csv", responseMsg.Content.ReadAsStringAsync().Result);
...
This code worked for me (I can't show the actual site). The site in question is a data vendor who used to accept simple HTTP GET requests but is now switching to POST with SSL and TSL 1.2 in order to comply with EU rules.
I am not claiming that this code is robust, thread safe, or anything like that. Just that I got it working. So that's a start.

Related

Azure function Error : received an unexpected eof or 0 bytes from the transport stream

I have a requirement where I am calling an API (programmatically PUT Method) from another API.
Both of the APIs are hosted as Azure Function App.
The request has nearly 600 rows.
The below method call is throwing the error: received an unexpected EOF or 0 bytes from the transport stream
If I send a request say 100-150 rows, it processes successfully.
I think that it is nothing to do with the code, it is related to the Azure Function app.
Please let me know if I need to add any configuration to the Azure Function app.
Thanks in Advance.
public async Task<List<CarPricing>> TestMethod(CarPricingModel request, string resourcePath,string token)
{
try
{
using var stream = StreamUtility.GenerateStreamFromString(JsonConvert.SerializeObject(request));
using var data= new StreamContent(stream);
data.Headers.ContentType = new MediaTypeHeaderValue("application/json");
var queryParams = new Dictionary<string, string>()
{
{"id", "XXXXXXXXXXXXXXXXXXXXXX" }
};
var relativeUrl = QueryHelpers.AddQueryString(resourcePath, queryParams);
using var requestMessage = new HttpRequestMessage
{
Method = HttpMethod.Put,
Content = content,
RequestUri = new Uri(relativeUrl, UriKind.Relative)
};
var httpResponseMessage = await _httpClient.SendAsync(requestMessage);
httpStatusCode = httpResponseMessage.StatusCode;
var httpResponse = await httpResponseMessage.Content.ReadAsStreamAsync();
using var responseContent = new JsonTextReader(new StreamReader(httpResponse));
var response = new JsonSerializer().Deserialize<List<CarPricing>>(responseContent);
return response;
}
catch (Exception ex)
{
_log.LogError("API error {err_msg}",ex.Message);
throw;
}
}
Check the below steps that might help to fix the issue:
received an unexpected eof or 0 bytes from the transport stream
This error generally occurs during the HTTP Calls of .NET Core Applications.
TLS/SSL binding is supported in the Azure Function App. You can bind it through Azure Portal and using the Code.
If you’re using the HTTPS Protocol, apply this SSL Call before the request made as mentioned in the comments:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
or
ServicePointManager.SecurityProtocol |= SecurityProtocolType.Ssl3| SecurityProtocolType.Tls| SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12
but the above error might come for many causes such as:
Client IP may be restricted, which you can add in Access Restrictions of the Function app API.
Typo Mistake or Incorrect URL of the API that is called programmatically from another Azure Function API.
Refer to this MS Doc for using the TLS/SSL Certificate Programmatically and SO Thread that shows how to use the TLS/SSL Certificate in Azure Function App.

c# httpclient post force single packet

Using Microsoft Message Analyzer, I can see that post data using the HttpClient is being sent in two tcp packets. One for the header, then one for the post data. This data could easily fit into one packet, however it is being split into two. I have explicitly turned on nagling and expect 100 continue off using the ServicePointManager, though, it doesn't seem to help.
ServicePointManager.Expect100Continue = false;
ServicePointManager.UseNagleAlgorithm = true;
5023 (.Net) shows 2 packets are sent to destination, 8170 (Postman) shows 1 packet being sent. Tests were done with the same payload.
Below is some sample code used to generate the request in .net
public void TestRequest()
{
var uri = new Uri("http://www.webscantest.com/");
ServicePointManager.Expect100Continue = false;
ServicePointManager.UseNagleAlgorithm = true;
var p = ServicePointManager.FindServicePoint(uri);
p.Expect100Continue = false;
p.UseNagleAlgorithm = true;
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Connection", "close");
var values = new Dictionary<string, string>
{
{ "thing1", "hello" },
{ "thing2", "world" }
};
var content = new FormUrlEncodedContent(values);
var response = client.PostAsync("http://www.webscantest.com/", content, CancellationToken.None).Result;
}
Is there a way to force the payload into a single packet?
Using .Net Framework 4.7
related question here
So after looking at the dotnet core source code (can only assume the same in other .net versions), I can see in the WinHttpHandler that the Request Header and Request Body are sent at different points.
The request header is sent with Interop.WinHttp.WinHttpSendRequest. Then the request body with Interop.WinHttp.WinHttpWriteData which according to the WinHttp docs, will "Wait until WinHttpSendRequest has completed before calling this function".
I think this issue could be solved, if the request body was sent with the WinHttpSendRequest, which currently sets the body as IntPtr.Zero.
Request Header here
Request Body here

Differences between using C# HttpClient API and the postman testing? Client call works on postman, but not C# httpClient getAsync

I am testing a REST API post, and it works well when I try it on Postman. However, in some scenario (related to the posting XML data) if I post with HttpClient API, I would receive the following error:
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
But the same XML content works fine on Postman with status OK and proper response.
What is the differences between using the C# HttpClient API and the postman testing? How can I configure my API call to match with the behavior on postman?
Here I attached the source code, and the Postman screenshot
public void createLoan()
{
string baseCreateLoanUrl = #"https://serverhost/create?key=";
var strUCDExport = XDocument.Load(#"C:\CreateLoan_testcase.xml");
using (var client = new HttpClient())
{
var content = new StringContent(strUCDExport.ToString(), Encoding.UTF8, Mediatype);
string createLoanApi = string.Concat(baseCreateLoanUrl, APIKey);
try
{
var response = client.PostAsync(createLoanApi, content).Result;
}
catch (Exception ex)
{
MessageBox.Show("Error Happened here...");
throw;
}
if (response.IsSuccessStatusCode)
{
// Access variables from the returned JSON object
string responseString = response.Content.ReadAsStringAsync().Result;
JObject jObj = JObject.Parse(responseString);
if (jObj.SelectToken("failure") == null)
{
// First get the authToken
string LoanID = jObj["loanStatus"]["id"].ToString();
MessageBox.Show("Loan ID: " + LoanID);
}
else
{
string getTokenErrorMsg = string.Empty;
JArray errorOjbs = (JArray) jObj["failure"]["errors"];
foreach (var errorObj in errorOjbs)
{
getTokenErrorMsg += errorObj["message"].ToString() + Environment.NewLine;
}
getTokenErrorMsg.Dump();
}
}
}
Thanks for Nard's comment, after comparing the header, I found the issue my client header has this:
Expect: 100-continue
While postman doesn't has.
Once I removed this by using the ServicePointManager:
ServicePointManager.Expect100Continue = false;
Everything seems fine now. Thanks all the input!
My gut tells me it's something simple. First, we know the API works, so I'm thinking it's down to how you are using the HttpClient.
First things first, try as suggested by this SO answer, creating it as a singleton and drop the using statement altogether since the consensus is that HttpClient doesn't need to be disposed:
private static readonly HttpClient HttpClient = new HttpClient();
I would think it would be either there or an issue with your content encoding line that is causing issues with the API. Is there something you are missing that it doesn't like, I bet there is a difference in the requests in Postman vs here. Maybe try sending it as JSON ala:
var json = JsonConvert.SerializeObject(strUCDExport.ToString());
var content = new StringContent(json, Encoding.UTF8, Mediatype);
Maybe the header from Postman vs yours will show something missing, I think the real answer will be there. Have fiddler running in the background, send it via Postman, check it, then run your code and recheck. Pay close attention to all the attribute tags on the header from Postman, the API works so something is missing. Fiddler will tell you.
I was struggling with this for 2 days when I stumbled over Fiddler which lets you record the traffic to the service. After comparing the calls I saw that I had missed a header in my code.

'UTF8' is not a supported encoding name

So I'm just playing around with Spotify's Web API and I'm trying to access my top played tracks. Although I've encountered a problem I've been trying to solve for a couple of hours now but I can't find an answer.
When I try to deserialize my response, I get the follwing error:
'UTF8' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.
Parameter name: name
The character set provided in ContentType is invalid. Cannot read content as string using an invalid character set.
The ContentType is application/json; charset=UTF8
Any ideas?
Here's my request code:
private static HttpClient GetHttpClient()
{
HttpClientHandler handler = new HttpClientHandler() {
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
var httpClient = new HttpClient(handler);
httpClient.DefaultRequestHeaders.Add("Accept", "application/json");
return httpClient;
}
public async Task<SearchArtistResponse> GetSelfTopAsync(string type, string userName)
{
var httpClient = GetHttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", GetAccessToken(userName));
var sb = new StringBuilder();
sb.Append(ApiUrl);
sb.Append($"/me/top/{type}");
var query = sb.ToString();
var response = await httpClient.GetAsync(query);
var spotifyResponse = JsonConvert.DeserializeObject<SearchArtistResponse>(await response.Content.ReadAsStringAsync());
return spotifyResponse;
}
Are you using .net core?
You will need to add the following code to make the encodings available in .NET desktop available in your environment:
System.Text.EncodingProvider provider = System.Text.CodePagesEncodingProvider.Instance;
Encoding.RegisterProvider(provider);
More info on CodePagesEncodingProvider.Instance can be found here.
The problem should be a validation of response header Content-Type ,that occur when you call ReadAsStringAsync(), if you call ReadAsByteArrayAsync() instead and parse to string
(System.Text.Encoding.UTF8.GetString())
that will gonna work!!!
I had a same problem while I was trying to get an answer from my API which is built in PHP using C# service. I could fix the issue by changing "charset=UTF8" to "charset=\"UTF-8\"" on the PHP side(the api that sends result to the C# service). Hope this helps someone.

Post a picture using webrequest in c#

I have the following python code how to post a picture through webservice:
product_image = requests.post(
'https://client.planorama.com/tapi/v1/product_image/',
data={ 'product_id': 1784682 },
files={ "file": open(my_image.jpg, 'rb') }
)
Can anyone help me to do the same thing in C#,
Multipart Form Post in C# includes a simple C# class using HttpWebRequest with a provided (working) example.
We are rather spoiled with the requests library in python.
Are you running .NET 4 or 4.5? If so, take a look at Joshcodes answer to Sending Files using HTTP POST in c# - it uses Microsoft.Net.Http which is by far the best HTTP library in the .NET world these days.
UPDATE: I haven't checked this for accuracy yet, but it could go something like this:
static HttpResponseMessage UploadFileWithParam(string requestUri, string fileName, string key1, string val1)
{
using (var client = new HttpClient())
{
using (var content = new MultipartFormDataContent())
{
content.Add(new StringContent(val1), key1);
var fileContent = new StreamContent(File.OpenRead(fileName));
fileContent.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = fileName
};
content.Add(fileContent);
return client.PostAsync(requestUri, content).Result;
}
}
}
// UploadFileWithParam("http://example.com", #"c:\...", "param1", "value1").Dump();

Categories

Resources