i am just working on my first Windows Phone 8.1 app (Universal if this matters, but only Windows Phone implemented at the moment). And at first all is working very smooth but as soon as my app is running for about 25-30 Minutes I can no longer use my HttpClient. I use the Windows.Web.Http.HttpClient.
In my first trys I used a singleHttpClientand reused it all the time. As I became aware that this is not working I started using a newHttpClient` for each request. But still no luck.
This is my method to get a new HttpClient:
private HttpClient GetClient()
{
var filter = new HttpBaseProtocolFilter
{
AllowUI = false,
CacheControl = { WriteBehavior = HttpCacheWriteBehavior.NoCache },
ServerCredential =
new PasswordCredential(
BaseApiUri.ToString(),
credentials.UserName,
credentials.Password),
};
var httpClient = new HttpClient(filter);
var headers = httpClient.DefaultRequestHeaders;
var httpConnectionOptionHeaderValueCollection = headers.Connection;
httpConnectionOptionHeaderValueCollection.Clear();
headers.Accept.TryParseAdd("application/json");
headers.CacheControl.TryParseAdd("no-cache");
headers.Add("Pragma", "no-cache");
headers.Add("Keep-Alive", "false");
headers.Cookie.Clear();
return httpClient;
}
The extra code setting the headers and clearing cookies are my attempts to stop some kind of caching of connections under the surface that might happen. But still no luck.
My method to make requests my API is like the following:
private async Task<bool> PostNoResponseRequestTo(string relativeUri, object requestContent, CancellationToken cancellationToken)
{
var targetUri = new Uri(BaseApiUri, relativeUri);
var requestJson = JsonConvert.SerializeObject(requestContent);
var content = new HttpStringContent(requestJson, UnicodeEncoding.Utf8, "application/json");
try
{
using (var httpClient = this.GetClient())
{
var post =
await httpClient.PostAsync(targetUri, content).AsTask(cancellationToken).ContinueWith(
async request =>
{
using (var response = await request)
{
return response.IsSuccessStatusCode;
}
},
cancellationToken);
return await post;
}
}
catch (Exception)
{
return false;
}
}
This works fine for about 25-30 Minutes after which the calls to the api suddenly start to fail. I start getting a 401 but as you can see i have specified credentials and because those are working and do not change (hardcoded them to test this) i start believing that the problem is on the API side.
This is the response I get:
StatusCode: 401, ReasonPhrase: 'Unauthorized', Version: 2, Content: Windows.Web.Http.HttpStreamContent, Headers:
{
Server: Microsoft-IIS/8.5
Date: Fri, 20 Mar 2015 14:25:06 GMT
WWW-Authenticate: Digest qop="auth",algorithm=MD5-sess,nonce="+Upgraded+NounceRemoved",charset=utf-8,realm="Digest", Negotiate, NTLM
X-Powered-By: ASP.NET
}
{
Content-Length: 1344
Content-Type: text/html
}
My API consists of a Asp.Net project with ServiceStack for its API functionality.
This is running on an IIS with activated digest authentication (all other are disabled).
By inspecting the logs i became aware of a failing API call in front of each successful call. But if i'm right this is by design of digest auth because i have not found a way to tell the client that the other side is using digest auth. I was able to specify this kind of information in my other .Net projects but for some reason Microsoft changed the code (and namespace) for the HttpClient. I am also aware of the HttpClient in the original namespace that you can get through nuget but this is not working for me as i get an error in my output window as soon as i make any call. This closes my app without any kind of information.
Back to the log i was able to get some information with the help of the extended logging and the tool to analyze them. The error is something like (can't access it right now will edit it later):'Invalid token passed to function/method'.
I really hope that someone can help me to solve this problem as it makes the app nearly unusable. My users have to restart the app every 15 Minutes to be on the save site.
Thanks for all advices that help me.
Try Checking the Machine Key setting in IIS. Automatically generate at runtime if tick will generate a new key every time the app pool is restarted. This might be causing your issue. The Machine Key can be set on the server, website or application level. As activated digest authentication is encrypted this might be the issue.
Managing Websites with IIS Manager (part 6) - The Machine Key and Windows Authentication
Related
I have an Azure function that sends a request to a URL and sends back the response. This function kept failing with timeout error for URLs from a particular domain (confidential).
To debug this, I created a very minimal Azure function:
var content = string.Empty;
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
content = await response.Content.ReadAsByteArrayAsync();
}
return new OkObjectResult(content);
This code works fine in local. When I try using the deployed Azure function, it works for all the other domains I tried (ex: https://google.com) but it hits request timeout error for a particular domain after trying for about 90 seconds. The error happens at this particular line: _httpClient.GetAsync(url). Again, it works fine for this (confidential) domain in local.
I have tried deploying the Azure function to two completely different Azure service plans and regions. Same result. It doesn't work for URLs from the required domain. Works for URLs of other domains.
Error:
System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
Update (solution):
I tried sending a request from Postman, copied the code from there for C# and deployed it to the Azure function and it is now working for the problematic domain. Something like below:
var client = new RestClient(url);
client.Timeout = -1;
var request = new RestRequest(Method.GET);
IRestResponse response = client.Execute(request);
The key here is client.Timeout = -1, which seems to have fixed the problem.
Now, in my original code, I tried setting HttpClient's timeout to Timeout.InfiniteTimeSpan both in Startup configuration as well as at individual request level but it did not work.
services.AddHttpClient("AzureTestClient", options =>
{
options.Timeout = Timeout.InfiniteTimeSpan;
});
Am I setting the timeout wrong in the HttpClient solution?
If you are using a Consumption plan then maybe the confidential URL need to whitelist the whole Azure Data center. You can follow the guide here or consider upgrading the Consumption plan to a premium one and have a dedicated linked VNET.
Maybe your local machine is already linked to the domain/whitelisted so azure function operates from different range.
Another reason maybe the URL returns a different HttpStatusCode that is't Successful range (200-299) so it fails with "EnsureSuccessStatusCode" in the old code?
Normally for the http code initialization, I did something like that:
public void Configure(IWebJobsBuilder builder)
{
builder.Services.AddHttpClient("AzureTestClient",
options => { options.Timeout = Timeout.InfiniteTimeSpan; });
}
Then when I want to use it, I do like that in any other function and it worked:
var client = clientFactory.CreateClient("AzureTestClient");
I am getting to the stage of hair pulling with this one, I'm hoping someone can see if I'm doing anything wrong.
I'm trying to POST some form data to website using Azure data factory web activity however whilst I get a response (I get the page and some headers) it is different to the response I get if I make the exact same request using C# and HttpClient code. I've used fiddler to view the request being post'd using my C# script and according to the request information given in data factory they are exactly the same - so same headers, same content format etc...
This POST request is to login to a website which has a custom login mechanism, so no OAuth or anything like that unfortunately. It is supposed to return a cookie, which it does if I use my C# script, but if I make the same POST request using data factory web activity then I get different html sent back (it just returns the same login screen) and also different set of response headers in the "ADFWebActivityResponseHeaders" part of the activity output!?! See below for what is returned in the web activity output response headers:-
"ADFWebActivityResponseHeaders": {
"Pragma": "no-cache",
"Vary": "Accept-Encoding",
"X-Frame-Options": "DENY",
"Cache-Control": "no-store, must-revalidate, no-cache, post-check=0, pre-check=0",
"Date": "Wed, 09 Sep 2020 08:09:30 GMT",
"Server": "Microsoft-IIS/8.5"
}
If I do this via C# I also get a 'Set-Cookie' as well (strangely if I make a 'GET' request for the homepage of this site I do get a 'Set-Cookie' in the response!!!), but never when doing this via data factory. I'm struggling to see how this is possible unless data factory is modifying my request in some fashion? Below is my C# code, pretty simple/standard:-
var handler = new HttpClientHandler();
handler.CookieContainer = new CookieContainer();
handler.UseCookies = true;
handler.UseDefaultCredentials = false;
// Create our http client which will perform our web requests
var HttpClient = new HttpClient(handler);
HttpClient.BaseAddress = new Uri("**REMOVED**");
// Some of the extracts take a LONG time, so set the timeout for default of 30mins
HttpClient.Timeout = TimeSpan.FromMinutes(30);
// Set the 'form' parameters we're going to POST to the server in the request
var parameters = new Dictionary<string, string>
{
{ "username", "**REMOVED**" },
{ "password", "**REMOVED**" }
};
// URL encode the parameters
var content = new FormUrlEncodedContent(parameters);
// Submit our POST with the parameters
var response = await HttpClient.PostAsync("**REMOVED**", content);
Running this code and using fiddler I see the following request with headers, these are the only headers:-
Content-Length: 80
Content-Type: application/x-www-form-urlencoded
username=REMOVED&password=REMOVED
and in the 'input' side of the web activity is the details of the request, I've added the headers in the web activity and these are correct:-
"method": "POST",
"headers": {
"Content-Type": "application/x-www-form-urlencoded",
"Content-Length": 80
},
"body": "username=REMOVED&password=REMOVED"
Note that in the data factory I'm using a self hosted integration runtime as this website blocks addresses that do not come from the specific IP addresses used externally by our on-prem network/firewall. I know that is not the problem as I'm getting a response with the normal login page from the site (if I use the Azure integration runtime I get a denied response).
Here is a screen shot of the web activity in data factory:-
Really hope someone out there can see what I'm missing or whatever...
Turns out this does work and will list the cookies in the JSON output from the activity as shown below (note this is to be found in the output of the ADF activity, so you would pick up the cookie from the output a bit like... #activity('Login and get cookie').output.ADFWebActivityResponseHeaders["Set-Cookie"] )
However, in my case the url I was POSTing to was responding with a 302 (moved temporarily) but the 'Location' header which should be there is not in the ADFWebActivityResponseHeaders - which is why I missed it. I tried using Chrome with the developer tools and looked at the response directly which is where I found the 302 response code. After that, I just used the new URL given in the response headers (i.e. the url in the 'Location') that I found when using the browser dev tools.
Unfortunately at the time of writing, the Azure data factory HTTP activity does not follow redirects (and doesn't list all the response headers either!) so if anyone encounters the same problem they will need to manually find out and get the url's for any redirects. In other words, try using a tool like browser/postman and look at the response if it doesn't work in ADF... you might find there is a redirect going on :-)
There is a feature request logged for this here, be sure to add your vote :)
edited to update the Azure feedback change of URL after MS decided to change things on the feedback site!?!
I use HttpClient in my app to send my user/password to a service that returns some cookies that I can later use for all my other requests. The service is located at https://accounts.dev.example.com/login and returns two cookies that have Domain=.dev.example.com. The issue I'm finding is that, in some machines (Windows Domain Controllers), these cookies are not being used when I request resources in subdomains like https://accounts.dev.example.com/health-check, but according to the MDN docs a cookie for a domain can be used for requesting resources to subdomains:
Domain= Optional
Specifies those hosts to which the cookie will be sent. If not specified, defaults to the host portion of the current document location (but not including subdomains). Contrary to earlier specifications, leading dots in domain names are ignored. If a domain is specified, subdomains are always included.
Do you know how to properly configure HttpClient to pass the domain cookies to subdomain requests?
A bit more of details:
The cookies returned by my authentication service at https://accounts.dev.example.com/login look like this in the HTTP headers:
Set-Cookie: AK=112233;Version=1;Domain=.dev.example.com;Path=/;Max-Age=5400;Secure;HttpOnly,
Set-Cookie: AS=445566;Version=1;Domain=.dev.example.com;Path=/;Max-Age=5400;Secure;HttpOnly,
Then I can query C#'s CookieContainer with either of these calls in normal workstations:
cookies.GetCookies("https://accounts.dev.example.com")
cookies.GetCookies("https://dev.example.com")
Both of which will return the 2 cookies like:
$Version=1; AK=112233; $Path=/; $Domain=.dev.example.com
$Version=1; AS=445566; $Path=/; $Domain=.dev.example.com
But in the other machines (the Domain Controller's) the first call will return an empty list, while the second will return the 2 cookies.
Why this difference on the behaviour of CookieContainer.GetCookies depending on which machine is running the code?
My workstations are using Microsoft Windows 10 Home Single Language (.Net 4.0.30319.42000) and the DCs are using Microsoft Windows Server 2012 R2 Datacenter (.Net 4.0.30319.36399).
The code
This is a modified version of my code:
public static async Task<string> DoAuth(CookieContainer cookies,
Dictionary<string, string> postHeaders,
StringContent postBody)
{
try
{
using (var handler = new HttpClientHandler())
{
handler.CookieContainer = cookies;
using (var client = new HttpClient(handler, true))
{
foreach (var key in postHeaders.Keys)
client.DefaultRequestHeaders.Add(key, postHeaders[key]);
var response = await client.PostAsync("https://accounts.dev.example.com/login", postBody);
response.EnsureSuccessStatusCode();
// This line returns 0 in Domain Controllers, and 2 in all other machines
Console.Write(cookies.GetCookies("https://accounts.dev.example.com").Count);
return await response.Content.ReadAsStringAsync();
}
}
}
catch (HttpRequestException e)
{
...
throw;
}
}
As I couldn't find an answer to this (not in TechNet either), I decided to go with the following solution, which works, but not sure if there is a proper way of solving the issue:
foreach (Cookie cookie in cookies.GetCookies(new Uri("https://dev.example.com")))
{
cookies.Add(new Uri("https://accounts.dev.example.com"), new Cookie(cookie.Name, cookie.Value, cookie.Path, ".accounts.dev.example.com"));
}
So, I'm duplicating the cookie for each one of the subdomains that my app should send these cookies to.
The underlying issue seems to be a bug in the Set-Cookie header. It seems the cause of the issue is the Version= component in the Set-Cookie header. This makes the CookieContainer fall on its face and results in the strange $Version and $Domain cookies then being sent in subsequent client requests. As far as I can tell there is no way to remove these broken cookies either. Iterating GetCookies() with the originating domain does not reveal the erroneous cookies.
I've seen a lot of question about this, and all points to me using ConfigureAwait(false), but even after doing so, it still doesn't returned any response. When I run the debugger, the code stops at the PostAsync and does not continue with my code. Am I doing something wrong here? Does it have to do with me calling an API via HTTPS?
Here's the code:
public async static Task<PaymentModel> AddAsync(Card card)
{
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:", "hidden"))));
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var cardJson = JsonConvert.SerializeObject(card);
var postRequest = new StringContent(cardJson, Encoding.UTF8, "application/json");
var request = await client.PostAsync(new Uri("https://sample-3rd-party-api/api/endpoint/here"), postRequest).ConfigureAwait(false);
var content = await request.Content.ReadAsStringAsync().ConfigureAwait(false);
}
EDIT:
In response to the comments below, the code is contained from a method AddAsync(Card card) called from a button click with a handler:
public async void OnExecute(object sender, EventArgs args)
{
//some code here
payment = await PaymentModel.AddAsync(card).ConfigureAwait(false);
}
EDIT 2:
I tried pinging the API, but it returns a request timed out, but when I tried it using Postman, it's doing fine (the API is just a Sandbox which is open for all, so it's okay to share this):
EDIT 3:
I think the problem lies with where I don't have an SSL certificate to access the API. I have a PHP server that connects to the same API and I have to set SSL_VERIFYPEER to false just so I can access it (don't worry, I added a cacert now so its on true again). Can the same issue be happening here? If so, what can I do to create a valid certificate for my xamarin forms app
You can use this
var json = JsonConvert.SerializeObject(card);
using (var client = new HttpClient())
{
var t = await client.PostAsJsonAsync("https://sample-3rd-party-api/api/endpoint/here", json);
Response R =JsonConvert.DeserializeObject<Response>((JsonConvert.DeserializeObject(t.Content.ReadAsStringAsync().Result.ToString())).ToString());
}
What's most likely happening here is your OnExecute method has a return type of void instead of Task which prevents the UI thread from being able to await it. Try either changing that return type to Task or creating a new UI thread to perform this work. I wouldn't worry about the ping timing out as long as Postman works. Many public web servers disable their ping response.
Does it have to do with me calling an API via HTTPS?
As you are remaining in the same network and calling the same API from POSTMAN and .NET HTTP Client and only getting success with POSTMAN.So this issue gets cancelled.
Next
tried pinging the API, but it returns a request timed out
This is answered on top of mine.
Can you Please try setting the timeout option for HTTPClient while initializing.
client.Timeout = TimeSpan.FromSeconds(10);
and if still Problem persists please setup Fiddler and compare both the req sent from POstman and .NET client
So I think the problem is resolved now since I'm able to receive content from the request, what I did was simply follow the docs here: https://learn.microsoft.com/en-us/xamarin/cross-platform/app-fundamentals/transport-layer-security?tabs=windows
It looks like my settings are outdated in platform level.
Update the HttpClient implementation and SSL/TLS implementation
options to enable TLS 1.2 security.
Update the HttpClient Implementation option to enable TSL 1.2
security. (NSUrlSession (iOS 7.0+)
I was having the same issue and below trick fixed the issue.
Change your var request = await client.PostAsync(...); as below
var task = client.PostAsync(new Uri("https://sample-3rd-party-api/api/endpoint/here"), postRequest);
var request = task.GetAwaiter().GetResult();
I have an asp.net MVC 4 Web API that was created in VS2010. It is currently hosted on a Windows Server 2003 box running IIS 6.0. The API works great when I navigate to it using Google Chrome from my machine (on the same network as the server). The issue I am having is when I try to consume the API from a VS2013 C# console app. I am getting the following response:
{StatusCode: 401, ReasonPhrase: 'Unauthorized', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{ Server: Microsoft-IIS/6.0
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
Date: Mon, 04 Apr 2016 16:09:50 GMT
Content-Length: 1539
Content-Type: text/html
}}
Here is the code for the console app:
static void Main(string[] args) {
RunAsync().Wait();
Console.ReadLine();
}
static async Task RunAsync() {
using (var client = new HttpClient()) {
client.BaseAddress = new Uri("http://server/website/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync("api/rateshop/1000/03/90210/");
if (response.IsSuccessStatusCode) {
RatingResultModel rateResults = await response.Content.ReadAsAsync<RatingResultModel>();
foreach (var rate in rateResults.Rates) {
Console.WriteLine("{0}\t${1}", rate.Service, rate.Rate);
}
} else {
Console.WriteLine("{0}\n{1}", response.StatusCode, response.RequestMessage);
}
}
}
My actual WebAPI has [AllowAnonymous] above the ApiController.
The IIS 6.0 server has Anonymous Authentication enabled, and the current user is set to the server administrator, and I provided full control to the website directory for the administrator. Integrated Windows Authentication is also enabled.
So far everything I have searched for is either documentation on how to "Return a 401 status code" instead of what to do when a 401 status code is returned, or it is documentation on how to add authentication to a Web API instead of how to remove it. There seems to be very little on how to handle a Web API in IIS 6.0. As far as I can tell, anonymous authentication is enabled.
I have already tried this stackoverflow answer and I've followed MSDN's article on adding anonymous access to IIS 6.0. I don't believe I will need authentication or authorization security for this Web API since it is going to be used on a local network.
Given that the API works when I consume it with Google Chrome from my machine, what might be causing my issue with trying to use a console app to consume the Web API?
Thanks to #Nilesh for the answer.
Wrapping my RunAsync method in a using (var clientHandler = new HttpClientHandler() {UseDefaultCredentials = true}) did the trick.
Full solution:
using (var clientHandler = new HttpClientHandler()) {
clientHandler.UseDefaultCredentials = true;
using (var client = new HttpClient(clientHandler)) {
client.BaseAddress = new Uri("http://server/website/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync("api/rateshop/1000/03/90210/");
if (response.IsSuccessStatusCode) {
RatingResultModel rateResults = await response.Content.ReadAsAsync<RatingResultModel>();
foreach (var rate in rateResults.Rates) {
Console.WriteLine("{0}\t${1}", rate.Service, rate.Rate);
}
} else {
Console.WriteLine("{0}\n{1}", response.StatusCode, response.RequestMessage);
}
}
}