Develop Reference

C# HttpClient POST requests from Azure Function with Authorization tag intended for third-party API are stripped of Headers and Body

UPDATE
I was able to get a working request posted. The third-party API has us sending the Token (which is basically a Guid) as a bearer token. Azure appears to do some sort of pre-validation on this. When I swapped out the GUID with a true randomly generated bearer token, it worked.
I do still wonder if there's a way to disable this check-in Azure. The "bad" Bearer token works for GET requests but fails for POST/PUT requests.
Summary of the Application
We have Azure Functions (i.e., Time Trigger, Orchestrator, Activities) that look for items in an on-prem queue table in SQL and then POST it to a third-party API via JSON.
The third-party API requires an Authorization header with the POST request.
Technical Overview
dotnet core 3.1
azure function runtime ~3
Additional Information
This codebase worked fine during UAT back in April-May of this year. It then sat idle until we rebooted the project a couple of weeks ago.
Outbound requests are not proxied through APIM. They're sent directly to the third-party API
Application Insights is configured for the Azure Function
What works
All of the GET requests. No issues at all.
What doesn't work
POST requests. I proxied the requests to a beeceptor to see exactly what was being received. When the Authorization header is included most of the headers are stripped (I.e., Content-Type, Content-Length) and the Body of the request is blank.
If I removed the Authorization header then all headers and body are received as expected.
Question
I can only assume at this point that some Azure service, pre-flight check, security policy is intercepting the Authorization header thinking it's intended for "itself", but I have absolutely no idea what it could be. I've been on Google now for days.
Simplified Version of Code
using var client = new HttpClient();
client.DefaultRequestHeaders.Clear();
// Request params are dynamic and a helper method builds the full request path
var path = PathBuilder(queueItem.RequestParams, queueItem.Request.UrlPath);
// This can change in code not shown if the request is sending files
var contentType = "application/json";
client.BaseAddress = new Uri(queueItem.Request.Client.BaseApiUrl);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue { NoCache = true };
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", queueItem.Request.Client.AuthToken);
// queueItem.Data is JSON
HttpContent json = new StringContent(queueItem.Data, Encoding.UTF8, contentType);
return await client.PostAsync(path, json);
Also...
I've confirmed the JSON body is valid
The code did work and has remain unchanged

Given all that you’ve tried, it might be a long shot, but have you tried to add the token like:
client.DefaultRequestHeaders.TryAddWithoutValidation(“Authorization”, “bearer token here…”);
and then check whether the try succeeded or not?

Why would my REST service .NET clients send every request without authentication headers and then retry it with authentication header?

We happen to run a REST web service with API requiring that clients use Basic authentication. We crafted a set of neat samples in various languages showing how to interface with our service. Now I'm reviewing IIS logs of the service and see that the following pattern happens quite often:
a request comes, gets rejected with HTTP code 401
the same request is resent and succeeds
which looks like the first request is sent without Authorization headers and then the second one is sent with the right headers and succeeds. Most of the time the log record contains "user-agent" which is the same string we planted into our .NET sample.
So I assume the problem is with .NET programs only. The problem is not reproduced with our sample code so I assume the users somehow modified the code or wrote their own from scratch.
We tried contacting the users but apparently they don't want to invest time into research. So it'd be nice to find what the most likely scenario is which leads to this behavior of .NET programs.
Why would they do this? Why would they not attach the headers on the first attempt?

This is the default behavior of HttpClient and HttpWebRequest classes which is exposed the following way.
Note: Below text explains suboptimal behavior causing the problem described in the question. Most likely you should not write your code like this. Instead scroll below to the corrected code
In both cases, instantiate a NetworkCredenatial object and set the username and password in there
var credentials = new NetworkCredential( username, password );
If you use HttpWebRequest - set .Credentials property:
webRequest.Credentials = credentials;
If you use HttpClient - pass the credentials object into HttpClientHandler (altered code from here):
var client = new HttpClient(new HttpClientHandler() { Credentials = credentials })
Then run Fiddler and start the request. You will see the following:
the request is sent without Authorization header
the service replies with HTTP 401 and WWW-Authenticate: Basic realm="UrRealmHere"
the request is resent with proper Authorization header (and succeeds)
This behavior is explained here - the client doesn't know in advance that the service requires Basic and tries to negotiate the authentication protocol (and if the service requires Digest sending Basic headers in open is useless and can compromise the client).
Note: Here suboptimal behavior explanation ends and better approach is explained. Most likely you should use code from below instead of code from above.
For cases when it's known that the service requires Basic that extra request can be eliminated the following way:
Don't set .Credentials, instead add the headers manually using code from here. Encode the username and password:
var encoded = Convert.ToBase64String( Encoding.ASCII.GetBytes(
String.Format( "{0}:{1}", username, password ) ) );
When using HttpWebRequest add it to the headers:
request.Headers.Add( "Authorization", "Basic " + encoded );
and when using HttpClient add it to default headers:
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue( "Basic", encoded );
When you do that the request is sent with the right authorization headers every time. Note that you should not set .Credentials, otherwise if the username or password is wrong the same request will be sent twice both time with the wrong credentials and both times of course yielding HTTP 401.

Credentials on HttpClient is not validated after first successful REST call

I'm creating an application where the user is logging in with a Username, Password and a Domain. I want to make as much as it is reusable across Windows platforms so I'm using the nuget package Microsoft HTTP Client libraries in a Portable Class Library.
Here is how i create the HttpClient with a HttpClientHandler and then calling the GetAsync.
HttpClientHandler handler = new HttpClientHandler();
ICredentials myCredentials = new NetworkCredential("Username", "Password", "Domain");
handler.Credentials = myCredentials;
HttpClient client = new HttpClient(handler);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.BaseAddress = new Uri("https://....");
HttpResponseMessage response = await client.GetAsync("...");
This seems to work fine. The credentials are send in the request and only registered users are allowed to get the data.
In my application the users also have the option to sign out and then sign in again with possibly another username, password or domain. And here is where the problem is.
If I have called the client.GetAsync with some valid credentials one time, the HttpClient seems to remember the old user credentials, although I'm creating a new instance of HttpClient each time and setting the correct credentials for the new user.
So my questions is, Is the HttpClient keeping a network channel open or is there some session problem that I'm not aware of?
--- Update #1 ---
If I make the URLs unique in GetAsync(...), e.g. I could pass some random parameter with the request, the server will validate the credentials and only Authorized users will get access to the resource. It is not really a good solution, so I did some more research.
I looks like the server is sending a response header called Persistent-Auth: true. This tells the client that the Authorization header is not required for the next request. I geuss thats why the credentials are not sent the next I try to call the GetAsync for the same resource. Surprisingly I also noticed in Fiddler that for the second request to this resource, no HTTP request is being sent at all from the client.
One interesting thing is that if I try the same approach in a browser, the Authorization has the same behavior, so its only included in the first request. For the second request to the same resource, I can see in Fiddler that a HTTP request is being sent as you would expect.
So to sum it all. I guess I'm stuck with 2 issues. First, is it possible to change this Persistent-Auth behavior so it is set to false in the server response. Second, why is my application not sending any request at all the second time I'm requesting the same resource.

According to the answer of this question:
How to stop credential caching on Windows.Web.Http.HttpClient?
It should work for Windows build 10586 onwards.
To manual clear all cached credentials, we can also call the method HttpBaseProtocolFilter.ClearAuthenticationCache() which clears all cached credential information. Documentation for this method can be found here: https://learn.microsoft.com/en-us/uwp/api/Windows.Web.Http.Filters.HttpBaseProtocolFilter

HTTPS Request to HTTP Drops Headers?

This is more for curiosity as I'm failing to find any answers or documentation for this phenomenon, but here's the scenario:
There are 2 services/applications, both hosted on IIS 7. Service 1 receives an HTTPS request from an external source (browser, fiddler, etc.) and to validate the request it needs to call service 2, so service 1 makes its own, new, separate call over HTTP to service 2. This call has an Authorization header added to the request object. When service 2 receives this call, the authentication header is gone, as if stripped out. Thus the authentication fails, this returns to service 1 which then rejects the external call.
Does anyone have an explanation why this header, and some others from what I've seen in testing, doesn't make it through with the HTTP call? Is this a behavior of IIS, or ASP.NET, or something? If the call to service 2 was HTTPS then the headers make it through fine. I'm generating the request like so:
string uriendpoint = "http://service.test.com/testService.svc/authtest";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriendpoint);
request.Credentials = CredentialCache.DefaultCredentials;
var authField = MD5Hash("test:test!!2013");
request.Headers.Add(HttpRequestHeader.Authorization, authField.ToString());
request.Method = WebRequestMethods.Http.Get;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Most likely the "service 2" have code similar to "If incoming request is HTTP ignore authorization headers". It is very reasonable behavior as HTTP traffic can be very easily sniffed and replayed - so honest servers block callers from potentially unsecure behavior.

A co-worker of mine came across the root cause of this behavior being IIS's "URL Rewrite" module. We had it setup to do a permanent redirect of http requests to https, and this redirect is where the headers get dropped. It's a bit odd that IIS does this, but I guess I'll just try something else to get around this problem.

Authenticate Android client on WCF Webapi site?

I want to implement a REST-api in C#. I found that WCF Webapi can do that.
My first question is how I can give only authenticated users access to my api?
And the second question is if the client to be authenticated is a Android-device, how do I do the HTTP request to authenticate?
Thanks!

em...
We did similar things, we use basic authentication+HTTPS,
that means the user name and password will be passed along each request, in the http header.
Thus in your web service, you can authenticate then, if it is from not valid user, then kick them out.
Or alternatively you can generate a GUID for each of your client, ask then to pass the GUID back to the search along with each http request, authenticate the GUID.
on Android device , when you send out the http request , add an http header
Authorization:Basic ****
quite easy , here is a codesnipet on android
String baseUrl = this.getValue(ServiceBaseUrlKey);</i>
DefaultHttpClient client = new ConnectionManager().getHttpClient();//create a httpclient
HttpGet request = new HttpGet();
request.setURI(new URI(baseUrl + "Path"));
//TODO need to wrap up how to apply the basic authentication.
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials("UserName", "****");
request.addHeader(new BasicScheme().authenticate(credentials, request));
request.addHeader("Content-Type","Application/JSON");
HttpResponse response = client.execute(request);