The company I work for have a limited internet connection and we use the proxy (example: 10.10.10.10:8080) to access at some restricted connections.
I can use the API in Postman (putting the proxy in the Postman settings) but when putting in the C# WebClient code it gives me a 403-Forbidden error.
I only need the var sensorData field but I splitted in var data and var data2 to understand where was the problem. It gives me the error at the var data = ...
Uri uri = new Uri("https://XXXXXXXX/api/DatasourceData/DatasourceDataHistoryBySerialNumber/");
Token token = new Token();
token = GetToken(tokenAPI);
using (WebClient client = new WebClient())
{
try
{
client.Proxy = new WebProxy("10.10.10.10", 8080);
client.Headers.Add("Authorization", "Bearer " + token.AccessToken);
client.QueryString.Add("serialNumbersDatasource", "I2001258");
client.QueryString.Add("startDate", string.Format("{0:s}", "2019-12-01"));
client.QueryString.Add("endDate", string.Format("{0:s}", DateTime.Now));
client.QueryString.Add("isFilterDatesByDataDate", "false");
var data = client.DownloadData(uri);
var data2 = (Encoding.UTF8.GetString(data));
sensorData = (JsonConvert.DeserializeObject<List<Sensor>>(Encoding.UTF8.GetString(client.DownloadData(uri))))[0];
}
}
Seems the problem at this line
client.Headers.Add("Authorization", "Bearer " + "tokenTest");
here you wil add header Authorization with value Bearer tokenTest
so, 403 Forbidden returns by service which you are addressing, but not a proxy
change to
client.Headers.Add("Authorization", "Bearer " + tokenTest);
and check if tokenTest has valid value
Check to see if you need any additional properties on the proxy. You may possibly need to enable:
UseDefaultCredentials (Boolean) true if the default credentials are
used; otherwise, false. The default value is false
Also, check your full url and query string that you are producing - look at the outgoing request fabricated (in the debugger) or through Fiddler and make sure it all lines up, url, query string, headers, etc.
From the docs:
Address
Gets or sets the address of the proxy server.
BypassArrayList
Gets a list of addresses that do not use the proxy server.
BypassList
Gets or sets an array of addresses that do not use the proxy server.
BypassProxyOnLocal
Gets or sets a value that indicates whether to bypass the proxy server for local addresses.
Credentials
Gets or sets the credentials to submit to the proxy server for authentication.
UseDefaultCredentials
Gets or sets a Boolean value that controls whether the DefaultCredentials are sent with requests.
Probably a problem with authorization header.
Is the token valid? Does it work with the same token in Postman?
I bet the api can't validate the token and and gives you no authorization to the resources. This is what a 403 would mean (but don't know what the api programmer actually intended by giving you 403).
Do you have access to the api's source code?
The token is really a string "tokentest" and that works with Postman?
I would suggest you to go for xNet.dll instead of webclient Because xNet library are considered best for proxy and webrequest.
var request = new HttpRequest()
request.UserAgent = Http.ChromeUserAgent();
request.Proxy = Socks5ProxyClient.Parse("10.10.10.10:8080");//can use socks4/5 http
Based on this
Try adding User-Agent in the header
client.Headers.Add("User-Agent", "PostmanRuntime/7.26.1");
In my case i did not specify security protocol. Paste this line of code before running any WebClient requests.
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
original answer: How to specify SSL protocol to use for WebClient class
I'm using HttpClient but it has problems with DNS resolve (it is using the sync method for this) So I use another lib for doing DNS queries and now I'm tryging to get custom urls by IP but I need to replace Host header. For example I have url http://fb.com but I need to get http://1.1.1.1 with Host set to fb.com I've tryied:
_req = new HttpRequestMessage(HttpMethod.Get, newUri.ToString());
_req.Headers.Host = uri.Host;
_httpClient.DefaultRequestHeaders.Host = uri.Host;
but this doesn't work. Is there any way to set own Host header like in HttpWebRequest?
It's work. The problem was with Fiddler which override Host header based on url. When Fiddler is off everything is going fine.
I have Client + Win service.
Both must work with web server using proxy.
Without proxy all works fine,
With system proxy settings (proxy settings from ie)
WebRequest.DefaultWebProxy
Client works fine, but service don't see this proxy settings (netsh winhttp set proxy don't help me). So - proxy server works OK
When I trying to use manual settings:
var_proxy = new WebProxy(Server + ":" + Port, true)
{Credentials = null};
var request = (HttpWebRequest)WebRequest.Create(target);
request.ContentType = Constants.ContentType; // Default content type
request.UserAgent = _userAgentHeader;
request.Method = "POST";
request.Proxy = _proxy;
And as I see in proxy logs - It works with http server, but not used with https! All requests go directly.
How can I fix it?
I have a large collection of DNS names that have already been resolved to IP addresses. With this collection I need to download HTML from them. It's a very large list and I need to do it as efficiently as possible.
I'm using System.Net.HttpWebRequest to download HTML from the each domain. HttpWebRequest is repeating the DNS lookup, and this is adding to the connection time. I've run tests to see if sockets for those IPs on port 80 would connect faster and they do.
So I'd like to use HttpWebRequest with a known IP address, but I don't know how. All WebRequest factory methods require a URL.
Now I thought I could do something like this (where 1.2.3.4 is the IP)
var req = WebRequest.Create("http://1.2.3.4/");
req.Headers.Add(....); <-- add something here
I need to somehow add to the HTTP header what the target domain is, but I'm not sure how to do it.
Pretty simple:
var ip = "93.184.216.119";
var host = "example.com";
var ipUri = new UriBuilder(Uri.UriSchemeHttp, ip).Uri;
var request = WebRequest.CreateHttp(ipUri);
request.Host = host;
using (var response = request.GetResponse())
{
// do something with response
}
I am working on a project that uses proxy to obtain websites' HTML code.
Now, the trouble I have is that I want to use Username-Password authentication and not IP-Auth when connecting to the proxy.
I have written a sample code and ran it with Snippy. It worked. Then I copied the same code into a Visual Studio .NET 4.5 Project and it failed with the error: An existing connection was forcibly closed by the remote host when trying to get the response.
Here is the code:
WebProxy[] proxies = { new WebProxy("ip", port) };
proxies[0].Credentials = new NetworkCredential { UserName = "username", Password = "password" };
string url = "https://www.google.com/search?q=s&num=50";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.Proxy = proxies[0];
using (HttpWebResponse res = (HttpWebResponse)req.GetResponse())
{
using (StreamReader response_stream = new StreamReader(res.GetResponseStream())
{
string html = response_stream.ReadToEnd();
Console.WriteLine(html);
}
}
I have tried different variations. When I switch to authorization by IP, add mine and comment out the NetworkCredentials assignment, the code works perfectly both in Snippy and Visual Studio.
But why does it fail when using NetworkCredentials?
OK, I found the culprit. It was my proxy provider.
Note for everybody ever running into problems when testing proxy connection with HttpWebRequest -- try another seller. It might save you a couple of minutes. Or hours.