The following code takes a target uri and gets its content as stream. Before the network call is made, it checks whether a proxy is required to access the target uri or not.
This works perfectly for .NET apps in full-framework (i.e. v4.8):
var targetUri = new Uri("...");
HttpClient client;
// get the system default web proxy ...
var proxyUri = WebRequest.DefaultWebProxy.GetProxy(targetUri);
// ... and check whether it should be used or not
var proxyAuthorityEqualsTargetAuthority = proxyUri?.Authority?.Equals(targetUri.Authority) == true;
var proxyRequired = !proxyAuthorityEqualsTargetAuthority;
if (proxyRequired)
{
var proxy = new WebProxy()
{
Address = proxyUri,
BypassProxyOnLocal = false,
UseDefaultCredentials = true
};
var httpClientHandler = new HttpClientHandler { Proxy = proxy };
client = new HttpClient(handler: httpClientHandler, disposeHandler: true);
}
else
{
client = new HttpClient();
}
await client.GetStreamAsync(targetUri) ...
However it does not work within a .NET Core app:
Accessing WebRequest.DefaultWebProxy.GetProxy(targetUri) will throw a PlatformNotSupportedException:
Operation is not supported on this platform.
So I tried this line instead which is supported by .NET Core as well:
// deprecated: var proxyUri = new Uri(WebRequest.DefaultWebProxy.GetProxy(targetUri).AbsoluteUri);
var proxyUri = new WebProxy().GetProxy(targetUri);
However, the returning proxyUri.Authority does always return the targetUri now (both, in .NET and .NET Core) instead of the address to the proxy server like WebRequest.DefaultWebProxy.GetProxy(targetUri) does in .NET.
This way, proxyAuthorityEqualsTargetAuthority is always true and therefore, proxyRequired is always false. Accessing the target uri directly throws a 407 (Proxy Authentication Required).
Does anyone know how to get the address of the default web proxy in .NET Core?
Okay, I don't know what I missed as I tried this weeks ago. I tried again and everything seems fine.
With CredentialCache.DefaultCredentials, this short code works with both: .NET and .NET Core.
var targetUri = new Uri("...");
var handler = new HttpClientHandler();
handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;
var client = new HttpClient(handler, disposeHandler: true);
await client.GetStreamAsync(targetUri) ...
While trying to retrieve the Acces Token from a windows server using HttpClient I am getting an error:
"GSSAPI operation failed with error - An invalid status code was supplied (SPNEGO cannot find mechanisms to negotiate)."
private readonly HttpClient client = new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true, AllowAutoRedirect = true }) { Timeout = TimeSpan.FromSeconds(5) };
public async Task<UserAccessToken> GetAuthenticationToken(string accessBrokerHost)
{
try
{
var response = await client.SendAsync(new HttpRequestMessage(HttpMethod.Get, $"{accessBrokerHost}/Token")).ConfigureAwait(false);
//accessBrokerHost is HTTP SPN created internally in a windows server
if (!response.IsSuccessStatusCode)
{
throw new BrokerNotAvailableException();
}
return await response.Content.ReadAsAsync<UserAccessToken>().ConfigureAwait(false);
}
}
system.ComponentModel.win32Exception is throwing as GSSAPI operation failed with error - An invalid status code was supplied (SPNEGO cannot find mechanisms to negotiate)
The above code is working fine in windows but not in Linux (I am using Linux Mint). As of my knowledge, it refers to a problem trying to use Kerberos but no Kerberos ticket is active to authenticate for Linux.
Finally, I found a solution to this question.
Solution 1:
Step 1. According to this, you should add
AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);
Step 2. Since UseDefaultCredentials = true won't work here, you need to pass your Network credentials manually to HttpClient like below
HttpClient client = new HttpClient(new HttpClientHandler{Credentials = new NetworkCredential("UserName" "Password", "Domain")}
Step 3. You should change your HttpRequestMessage version to
HttpVersion.Version11.
Solution 2: You can also fix this issue by converting your entire application into NetcoreApp3.0
I'm new at anglesharp. I tried the proxy change at the documentation but it didnt work.
Now i'm using this, it works well with webclient but it isnt working with AngleSharp.
The code i'm trying is ;
var handler = new HttpClientHandler()
{
Proxy = new WebProxy(String.Format("{0}:{1}", "myProxy", "myPort"), false),
PreAuthenticate = true,
UseDefaultCredentials = false,
};
var config = Configuration.Default.WithJs().WithCookies().WithDefaultLoader().With(handler);
//Create a new context for evaluating webpages with the given config
var context = BrowsingContext.New(config);
var document = await context.OpenAsync("https://api.ipify.org?format=json");
Console.WriteLine(document.DocumentElement.OuterHtml);
I'm not getting any error, proxy is not working thats it. I'm getting my original ip not the proxys.
But with WebClient it works well.
You just add something (handler, i.e., a HttpClientHandler instance) to AngleSharp's configuration - something that will not be used by anything in AngleSharp.
First of all AngleSharp's internal HTTP client is only a default client. For compatibility reasons AngleSharp cannot ship with HttpClient instead if uses HttpWebRequest. This also allows you setting a proxy.
Now if you want to use your code I suggest you use AngleSharp.Io (https://www.nuget.org/packages/AngleSharp.Io or https://github.com/AngleSharp/AngleSharp.Io). It's quite simple and straight forward:
var handler = new HttpClientHandler
{
Proxy = new WebProxy(String.Format("{0}:{1}", "myProxy", "myPort"), false),
PreAuthenticate = true,
UseDefaultCredentials = false,
};
var config = Configuration.Default
.WithRequesters(handler)
.WithDefaultLoader()
.WithJs()
.WithTemporaryCookies()
.WithDefaultLoader();
var context = BrowsingContext.New(config);
var document = await context.OpenAsync("https://api.ipify.org?format=json");
Console.WriteLine(document.DocumentElement.OuterHtml)
Only .WithRequesters(handler) has been added. This adds the requesters from AngleSharp.Io. By providing handler we can configure the HttpClient.
Hope that helps!
I am trying to connect to a proxy via this piece of code using HttpClient.
i would like to connect to a proxy strictly using HttpClient, or if unable to. using any c# library as long as the task is achieved.
HttpClientHandler handler = new HttpClientHandler();
//setup web proxy and credentials
var webproxy =
new WebProxy("94.232.55.98", 8080)//ip and port number
{
UseDefaultCredentials = false,
Credentials = CredentialCache.DefaultCredentials
};
handler = new HttpClientHandler
{
Proxy = webproxy,
UseProxy = true,
PreAuthenticate = true,
UseDefaultCredentials = false
};
container = new CookieContainer();
handler.CookieContainer = container;
handler.UseCookies = true;
client = new HttpClient(handler);
//Query a url and get its contents, but the request was not using any proxy seemingly
HttpResponseMessage responseMessage = client.GetAsync("https://shop.shoprite.com/").Result;
Looking at the code i am in need of guidance on how to connect httpclient to a AU proxy, and how to possibly get a proxy with or without credentials and to make it work all together.
I am trying to access a website geo blocked only in Australia that is why im trying to use a proxy.
Thanks in advance!
EDIT:
I have retrieved my proxy from this site ( looking at AU proxies )
https://free-proxy-list.net/ and i get the first ip and put it on WebProxy, having 8080 as port number as an example
but it doesnt seem to work.
When i am about to request the site, i am having an error like this accessing a site that is geo blocked anywhere else except in australia.
It looks like your code is correct, however the host you are trying to connect to is not accessible through the proxy you are trying to use. You can get a more useful error message by altering the last few lines to utilise the EnsureSuccessStatusCode method. This will throw an exception if the status code is not 2XX.
using (var client = new HttpClient(handler))
using (var responseMessage = await client.GetAsync("http://shop.shoprite.com/"))
{
responseMessage.EnsureSuccessStatusCode();
responseMessage.Dump();
}
I have a web application (hosted in IIS) that talks to a Windows service. The Windows service is using the ASP.Net MVC Web API (self-hosted), and so can be communicated with over http using JSON. The web application is configured to do impersonation, the idea being that the user who makes the request to the web application should be the user that the web application uses to make the request to the service. The structure looks like this:
(The user highlighted in red is the user being referred to in the examples below.)
The web application makes requests to the Windows service using an HttpClient:
var httpClient = new HttpClient(new HttpClientHandler()
{
UseDefaultCredentials = true
});
httpClient.GetStringAsync("http://localhost/some/endpoint/");
This makes the request to the Windows service, but does not pass the credentials over correctly (the service reports the user as IIS APPPOOL\ASP.NET 4.0). This is not what I want to happen.
If I change the above code to use a WebClient instead, the credentials of the user are passed correctly:
WebClient c = new WebClient
{
UseDefaultCredentials = true
};
c.DownloadStringAsync(new Uri("http://localhost/some/endpoint/"));
With the above code, the service reports the user as the user who made the request to the web application.
What am I doing wrong with the HttpClient implementation that is causing it to not pass the credentials correctly (or is it a bug with the HttpClient)?
The reason I want to use the HttpClient is that it has an async API that works well with Tasks, whereas the WebClient's asyc API needs to be handled with events.
You can configure HttpClient to automatically pass credentials like this:
var myClient = new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true });
I was also having this same problem. I developed a synchronous solution thanks to the research done by #tpeczek in the following SO article: Unable to authenticate to ASP.NET Web Api service with HttpClient
My solution uses a WebClient, which as you correctly noted passes the credentials without issue. The reason HttpClient doesn't work is because of Windows security disabling the ability to create new threads under an impersonated account (see SO article above.) HttpClient creates new threads via the Task Factory thus causing the error. WebClient on the other hand, runs synchronously on the same thread thereby bypassing the rule and forwarding its credentials.
Although the code works, the downside is that it will not work async.
var wi = (System.Security.Principal.WindowsIdentity)HttpContext.Current.User.Identity;
var wic = wi.Impersonate();
try
{
var data = JsonConvert.SerializeObject(new
{
Property1 = 1,
Property2 = "blah"
});
using (var client = new WebClient { UseDefaultCredentials = true })
{
client.Headers.Add(HttpRequestHeader.ContentType, "application/json; charset=utf-8");
client.UploadData("http://url/api/controller", "POST", Encoding.UTF8.GetBytes(data));
}
}
catch (Exception exc)
{
// handle exception
}
finally
{
wic.Undo();
}
Note: Requires NuGet package: Newtonsoft.Json, which is the same JSON serializer WebAPI uses.
What you are trying to do is get NTLM to forward the identity on to the next server, which it cannot do - it can only do impersonation which only gives you access to local resources. It won't let you cross a machine boundary. Kerberos authentication supports delegation (what you need) by using tickets, and the ticket can be forwarded on when all servers and applications in the chain are correctly configured and Kerberos is set up correctly on the domain.
So, in short you need to switch from using NTLM to Kerberos.
For more on Windows Authentication options available to you and how they work start at:
http://msdn.microsoft.com/en-us/library/ff647076.aspx
OK, so thanks to all of the contributors above. I am using .NET 4.6 and we also had the same issue. I spent time debugging System.Net.Http, specifically the HttpClientHandler, and found the following:
if (ExecutionContext.IsFlowSuppressed())
{
IWebProxy webProxy = (IWebProxy) null;
if (this.useProxy)
webProxy = this.proxy ?? WebRequest.DefaultWebProxy;
if (this.UseDefaultCredentials || this.Credentials != null || webProxy != null && webProxy.Credentials != null)
this.SafeCaptureIdenity(state);
}
So after assessing that the ExecutionContext.IsFlowSuppressed() might have been the culprit, I wrapped our Impersonation code as follows:
using (((WindowsIdentity)ExecutionContext.Current.Identity).Impersonate())
using (System.Threading.ExecutionContext.SuppressFlow())
{
// HttpClient code goes here!
}
The code inside of SafeCaptureIdenity (not my spelling mistake), grabs WindowsIdentity.Current() which is our impersonated identity. This is being picked up because we are now suppressing flow. Because of the using/dispose this is reset after invocation.
It now seems to work for us, phew!
In .NET Core, I managed to get a System.Net.Http.HttpClient with UseDefaultCredentials = true to pass through the authenticated user's Windows credentials to a back end service by using WindowsIdentity.RunImpersonated.
HttpClient client = new HttpClient(new HttpClientHandler { UseDefaultCredentials = true } );
HttpResponseMessage response = null;
if (identity is WindowsIdentity windowsIdentity)
{
await WindowsIdentity.RunImpersonated(windowsIdentity.AccessToken, async () =>
{
var request = new HttpRequestMessage(HttpMethod.Get, url)
response = await client.SendAsync(request);
});
}
It worked for me after I set up a user with internet access in the Windows service.
In my code:
HttpClientHandler handler = new HttpClientHandler();
handler.Proxy = System.Net.WebRequest.DefaultWebProxy;
handler.Proxy.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
.....
HttpClient httpClient = new HttpClient(handler)
....
Ok so I took Joshoun code and made it generic. I am not sure if I should implement singleton pattern on SynchronousPost class. Maybe someone more knowledgeble can help.
Implementation
//I assume you have your own concrete type. In my case I have am using code first with a class called FileCategory
FileCategory x = new FileCategory { CategoryName = "Some Bs"};
SynchronousPost<FileCategory>test= new SynchronousPost<FileCategory>();
test.PostEntity(x, "/api/ApiFileCategories");
Generic Class here. You can pass any type
public class SynchronousPost<T>where T :class
{
public SynchronousPost()
{
Client = new WebClient { UseDefaultCredentials = true };
}
public void PostEntity(T PostThis,string ApiControllerName)//The ApiController name should be "/api/MyName/"
{
//this just determines the root url.
Client.BaseAddress = string.Format(
(
System.Web.HttpContext.Current.Request.Url.Port != 80) ? "{0}://{1}:{2}" : "{0}://{1}",
System.Web.HttpContext.Current.Request.Url.Scheme,
System.Web.HttpContext.Current.Request.Url.Host,
System.Web.HttpContext.Current.Request.Url.Port
);
Client.Headers.Add(HttpRequestHeader.ContentType, "application/json;charset=utf-8");
Client.UploadData(
ApiControllerName, "Post",
Encoding.UTF8.GetBytes
(
JsonConvert.SerializeObject(PostThis)
)
);
}
private WebClient Client { get; set; }
}
My Api classs looks like this, if you are curious
public class ApiFileCategoriesController : ApiBaseController
{
public ApiFileCategoriesController(IMshIntranetUnitOfWork unitOfWork)
{
UnitOfWork = unitOfWork;
}
public IEnumerable<FileCategory> GetFiles()
{
return UnitOfWork.FileCategories.GetAll().OrderBy(x=>x.CategoryName);
}
public FileCategory GetFile(int id)
{
return UnitOfWork.FileCategories.GetById(id);
}
//Post api/ApileFileCategories
public HttpResponseMessage Post(FileCategory fileCategory)
{
UnitOfWork.FileCategories.Add(fileCategory);
UnitOfWork.Commit();
return new HttpResponseMessage();
}
}
I am using ninject, and repo pattern with unit of work. Anyways, the generic class above really helps.
Set identity's impersonation to true and validateIntegratedModeConfiguration to false in web.config
<configuration>
<system.web>
<authentication mode="Windows" />
<authorization>
<deny users="?" />
</authorization>
<identity impersonate="true"/>
</system.web>
<system.webServer>
<validation validateIntegratedModeConfiguration="false" ></validation>
</system.webServer>
</configuration>
string url = "https://www..com";
System.Windows.Forms.WebBrowser webBrowser = new System.Windows.Forms.WebBrowser();
this.Controls.Add(webBrowser);
webBrowser.ScriptErrorsSuppressed = true;
webBrowser.Navigate(new Uri(url));
var webRequest = WebRequest.Create(url);
webRequest.Headers["Authorization"] = "Basic" + Convert.ToBase64String(Encoding.Default.GetBytes(Program.username + ";" + Program.password));
webRequest.Method = "POST";