I know that I can change the computers global proxy setting, Software\\Microsoft\\Windows\\CurrentVersion\\Internet Settings, to affect IE instances created using Watin.
But is there any way to intercept requests made by the IE browsers and run them through a proxy? My goal is to run multiple instances of IE, each with its own proxy, which isn't possible with my current solution above.
WatiN IE creates multiple ProcessIDs (single instance IE creates multiple process ids). in order to overwrite proxy settings just for WatiN by using Fiddler Core we need to get all child process ids which are created by WatiN IE. Helper class can be found here PInvoke: Getting all child handles of window – Svett Ralchev class. And then we check all process ids inside the BeforeRequest event and wait for watin process id to overwrite proxy settings.
private void FiddlerApplication_BeforeRequest(Session sess)
{
//Debug.WriteLine("FiddlerApplication_BeforeRequest: " + sess.LocalProcessID.ToString());
if (WatinIEprocessHolder.ContainsKey(sess.LocalProcessID))
{
//see http://stackoverflow.com/questions/14284256/how-to-manually-set-upstream-proxy-for-fiddler-core
sess["X-OverrideGateway"] = WatinIEprocessHolder[sess.LocalProcessID];
}
}
Working Test Application can be downloaded here http://www.rentanadviser.com/downloads/WatiN-2.1.0.1196.zip
Test results with different anonymous proxy below. (ipaddress=browser.Text)
Process Ids:3852,7852,, Your IP address: 119.46.110.17, Proxy:119.46.110.17:8080
Process Ids:2508,6948,, Your IP address: 178.21.112.27, Proxy:178.21.112.27:3128
Process Ids:1348,1368,, Your IP address: 122.96.59.107, Proxy:122.96.59.107:83
Process Ids:7152,5104,, Your IP address: 136.0.16.217, Proxy:136.0.16.217:3127
Process Ids:4128,3480,, Your IP address: 198.52.199.152, Proxy:198.52.199.152:7808
Process Ids:2036,7844,, Your IP address: 122.96.59.107, Proxy:122.96.59.107:82
Sample code:
private void this_FormClosing(object sender, FormClosingEventArgs e)
{
StopFiddler();
}
private void Form1_Load(object sender, EventArgs e)
{
this.FormClosing += this_FormClosing;
ProxyHolder = new List<string>();
ProxyHolder.Add("119.46.110.17:8080");
ProxyHolder.Add("178.21.112.27:3128");
ProxyHolder.Add("122.96.59.107:83");
ProxyHolder.Add("136.0.16.217:3127");
ProxyHolder.Add("198.52.199.152:7808");
ProxyHolder.Add("122.96.59.107:82");
StartFiddler();
System.Threading.Thread.Sleep(500);
for (var i = 0; i < ProxyHolder.Count; i++)
{
WhatIsMyIpThroughProxy(ProxyHolder[i]);
Application.DoEvents();
System.Threading.Thread.Sleep(500);
}
//WhatIsMyIpThroughProxy();
}
private Dictionary<int, string> WatinIEprocessHolder = new Dictionary<int, string>();
private List<string> ProxyHolder = null;
public void WhatIsMyIpThroughProxy(string ProxyIPandPort)
{
using (var browser = new IE(true))// we should not navigate now. Because we need process ids.
{
WindowHandleInfo ChildHandles = new WindowHandleInfo(browser.hWnd);
foreach (var cHandle in ChildHandles.GetAllChildHandles())
{
int pid = new WatiN.Core.Native.Windows.Window(cHandle).ProcessID;
if (WatinIEprocessHolder.ContainsKey(pid) == false)
WatinIEprocessHolder.Add(pid, ProxyIPandPort);
}
System.Text.StringBuilder processIDs = new System.Text.StringBuilder();
foreach (var k in WatinIEprocessHolder.Keys)
{
processIDs.Append(k.ToString() + ",");
//Debug.WriteLine(string.Format("{0}:{1}", k, WatinIEprocessHolder[k]));
}
//we got the process ids above. Navigate now.
browser.GoTo("http://www.rentanadviser.com/en/common/tools.ashx?action=whatismyip");
browser.WaitForComplete();
WatinIEprocessHolder.Clear();
System.Net.IPAddress ip;
if (System.Net.IPAddress.TryParse(browser.Text, out ip))
{
Debug.WriteLine(string.Format("Process Ids:{0}, Your IP address: {1}, Proxy:{2}", processIDs.ToString(), browser.Text, ProxyIPandPort));
}
else
{
Debug.WriteLine(string.Format("Process Ids:{0}, Your IP address: {1}, Proxy:{2}", processIDs.ToString(), "Failed", ProxyIPandPort));
}
}
}
private void StartFiddler()
{
FiddlerApplication.BeforeRequest += FiddlerApplication_BeforeRequest;
FiddlerApplication.Startup(8888, true, true, true);
}
private void StopFiddler()
{
FiddlerApplication.BeforeRequest -= FiddlerApplication_BeforeRequest;
if (FiddlerApplication.IsStarted())
{
FiddlerApplication.Shutdown();
}
}
private void FiddlerApplication_BeforeRequest(Session sess)
{
//Debug.WriteLine("FiddlerApplication_BeforeRequest: " + sess.LocalProcessID.ToString());
if (WatinIEprocessHolder.ContainsKey(sess.LocalProcessID))
{
//see http://stackoverflow.com/questions/14284256/how-to-manually-set-upstream-proxy-for-fiddler-core
sess["X-OverrideGateway"] = WatinIEprocessHolder[sess.LocalProcessID];
}
}
I've created an app called Process Proxifier which uses FiddlerCore to add proxy settings to the Windows applications dynamically. You can find its full source code here: https://processproxifier.codeplex.com/
Also I should mention that this solution is limited to target processes with system's default "CERN" proxy setting (which is pointed at Fiddler/FiddlerCore).
It's not possible to do that with IE or even with WebBrowser (it's just an instance of IE).
But you can manipute WebBrowser behavior to achieve your desired feature.
It's possible to write your custom WebBrowser which fetched data by sending your custom WebRequest that contain your different proxy.
How to load web browser with web response
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://example.com");
webRequest.Proxy = new WebProxy(host, port);
HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse();
Stream receiveStream = response.GetResponseStream();
WebBrowser webBrowser = new WebBrowser();
webBrowser.DocumentStream = receiveStream;
WebRequest.Proxy
I know you are looking for an alternative solution without using the computers global proxy setting but I thought of adding this here so others who don't have this constraint know about it.
The solution was on your question - The Windows Registry.
It is simple to change the proxy settings globally at runtime, you need to change the registry keys you are interested in using the Microsoft.Win32.Registry class in the Microsoft.Win32 namespace.
You can find MSDN documentation for this here: http://msdn.microsoft.com/en-us/library/microsoft.win32.registry(v=vs.110).aspx
See below an example of how to do this.
RegistryKey myKey = Registry.LocalMachine.OpenSubKey("SOFTWARE\\Your key", true);
myKey.SetValue("My String Value", "Test Value", RegistryValueKind.String);
Now to change proxy settings on the box you need to change or create the right proxy registry keys you can find all the available keys at:
MSDN Documentation - 2.2.1.10 Internet Settings.
Below is a few of the keys you need to set. Each version of IE has their own keys but the ones below are identical to all browsers.
UseProxyServer
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyEnable
ProxyServerAndPort
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyServer
ProxyOverride
REG_SZ
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyOverride
HTTP1_1ThroughProxy
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\HTTP1_1ThroughProxy
User Specific
Please bear in mind these are Current user registry entries so you may need to set them in the context of the windows identity. Also the simplest way to see what should be the values for these keys is to apply the proxy changes on the Internet Settings dialog and check them on RegEdit.exe.
Create user automatically
This is your saving grace here because you can run your process for watin on a local windows account with the settings that way you dont need to change your own proxy settings.
You can then have one user called WatinUser that the proxy settings are set against you can automate the creation of this user using the System.DirectoryServices.AccountManagement Namespace classses.
See examples here at SO: create local user account
There are products like Proxifier that let you setup rules to route traffic to different proxies based on application names, IP addresses, hostnames and port numbers. This would not let you use different proxies for multiple IE processes, but if those processes were accessing different URLs you could route the traffic through separate proxy servers. Proxifier works using the WinSocks stack, similar to what many antivirus use, and it is transparent to the application layer.
Another suggestion is to write your own web request interceptor/proxy server that will grab proxy server info from requested url and forward normalized url to the real proxy server.
for e.g. from watin you launch url "someurl?ProxyServer=10.10.10.12" now this will be intercepted by your own proxy server and it will use the proxy server param to redirect requested url i.e. "someurl" to 10.10.10.12 your proxy server implementation can set proxy details at run time and fetch the results from your server using dynamic proxy.
I hope it makes some sense.
Related
We like to enable some hidden features of our software only if it is run inside of the company network. The key requirements are:
no need for a third party library outside of DotNet 4.5.1
easy to implement (should not be more than some dozens of lines .. I don't want to reimplement a crypto library)
It should be reasonable safe:
at least: hard to reverse engineer
at best: "impossible" to break even with read-access to the source code
low maintenance overhead
Win2012-Server is available for installation of additional software (open source or own implementation prefered - server can be assumed to be safe)
What I have thought about:
Check if a specific PC is available with a known MAC or IP (current implementation, not really secure and some other flaws)
Test, if a service is available on a specific response (i.e. I send 'Hello' to MyServer:12345 - server responses with 'World')
Similar to 2nd but a more complex challenge (i.e. send a seed for a RNG to the server, verify the response)
Set up an apache with HTTPS and verify the certificate
If you use ActiveDirectory, you could add a reference to the System.DirectoryServices namespace and check
ActiveDirectorySite currentSite = ActiveDirectorySite.GetComputerSite();
then you can get a bit of information from the currentSite object and check against that. That's how I enable/disable features of an application I'm developing currently.
I also grab:
var client = Dns.GetHostEntry(Dns.GetHostName());
foreach (var ip in client.AddressList)
{
if(ip.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork)
{
ipAddress = ip;
}
}
Which you can check to make sure the client is connected with the proper protocol.
I've choosen the last option: Set up a webserver in the intranet and verify the certificate.
It was easier than expected. There are enough tutorials for setting up an apache with https for every supported OS. The self-signed certificate have a lifetime of 9999 days - should be okay until 2042. The C#-part is also reasonable small:
private static bool m_isHomeLocation = false;
public static bool IsHomeLocation
{
get
{
if (m_isHomeLocation)
return true;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://yourLicenseServer:yourConfiguredPort");
request.ServerCertificateValidationCallback += ((s, certificate, chain, sslPolicyErrors) => true);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
response.Close();
var thumbprint = new X509Certificate2(request.ServicePoint.Certificate).Thumbprint;
m_isHomeLocation = (thumbprint == "WhateverThumbprintYourCertificateHave");
}
catch
{
// pass - maybe next time
}
return m_isHomeLocation;
}
}
I have a .NET Windows service which is using fiddler core to capture content from a set of web sites. I start Internet Explorer with Process.Start("IEXPLORE.EXE", url) which works. The problem is I need to compare the contents of desktop surfing and mobile surfing. I have tried to change the user agent of Internet Explorer without success. My next idea was using Chrome with a User-Agent Switcher to simulate mobile surfing. The problem with this is that I can't get fiddler core to capture any data from Chrome using a service (regardless of the User-Agent switcher). It all works fine from a console application but the service captures nothing. I'm assuming that this shouldn't be a "session 0 isolation" problem as Fiddler and Chrome are running in the same session (the session of the service user). The version of FiddlerCore is 4.4.0.1.
Here is the code for capturing the data:
public class TrafficEvent
{
public string Url { get; set; }
public string ContentType { get; set; }
public byte[] Data { get; set; }
}
private List<TrafficEvent> trafficEvents;
private Fiddler.SessionStateHandler responseHandler;
private void HandleBeforeResponse(Fiddler.Session session)
{
trafficEvents.Add(new TrafficEvent
{
Url = session.fullUrl,
ContentType = session.oResponse.headers.Exists("Content-Type") ? session.oResponse.headers["Content-Type"] : "",
Data = session.responseBodyBytes
});
}
public void StartFiddler()
{
trafficEvents = new List<TrafficEvent>();
responseHandler = new Fiddler.SessionStateHandler(HandleBeforeResponse);
Fiddler.FiddlerApplication.BeforeResponse += responseHandler;
Fiddler.WinINETCache.ClearFiles();
Fiddler.CONFIG.IgnoreServerCertErrors = false;
int port = 8888;
Fiddler.URLMonInterop.SetProxyInProcess("127.0.0.1:" + port.ToString(), "<-loopback>");
Fiddler.FiddlerApplication.Startup(port, true, false);
}
public List<TrafficEvent> StopFiddler()
{
Fiddler.FiddlerApplication.BeforeResponse -= responseHandler;
Fiddler.FiddlerApplication.Shutdown();
return trafficEvents;
}
public async Task CaptureData(string url)
{
StartFiddler();
var proc = Process.Start("chrome.exe", url);
proc.EnableRaisingEvents = true;
await Task.Delay(30 * 1000);
proc.Kill();
List<TrafficEvent> trafficEvents = StopFiddler();
foreach (TrafficEvent trafficEvent in trafficEvents)
{
// Store event in database
}
}
EDIT:
Could not comment the answer of EricLaw.
#EricLaw unfortunatelly the proxy switch for Chrome did not work for the service. Earlier I have tried to set the proxy in the config-file:
<!-- The following section is to force use of Fiddler for all applications, including those running in service accounts -->
<system.net>
<defaultProxy>
<proxy autoDetect="false" bypassonlocal="false" proxyaddress="http://127.0.0.1:8888" usesystemdefault="false" />
</defaultProxy>
</system.net>
That did not work either. Changing the User-Agent via Fiddler worked well with Windows.Forms.WebControl but not with Internet Explorer or Chrome. I am currently running the service as an application, which is not optimal but working for now. Will try the UA Pick, thanks.
SOLUTION
My initial problem was not beeing able to capture data from mobiles. It reallye did not matter whether it was Chrome or Internet Explorer. Thanks to the answer of #EricLaw I used the UA-pick in IE, which works for capturing mobile sites in a service!
Firstly, keep in mind that you can change the User-Agent header sent using Fiddler itself. Alternatively, you can use my UA Pick add-on.
If you want to keep going with the Chrome route, the problem you're encountering is that a service account isn't guaranteed to check the active user account proxy settings, which is where Fiddler registers as the proxy by default.
The simplest approach is to just start Chrome with a command line that points at the proxy server, e.g. chrome.exe --proxy-server=127.0.0.1:8888
I want to read the proxy settings (if any defined) and display them to the user via a windows forms application.
Until now I am using code like the following:
WebProxy defaultProxy = WebProxy.GetDefaultProxy();
if (defaultProxy != null)
{
ProxyAddress = defaultProxy.Address.Host;
ProxyPort = defaultProxy.Address.Port;
ProxyDefaultCredentials = defaultProxy.UseDefaultCredentials;
ProxyByPassLocalAddresses = defaultProxy.BypassProxyOnLocal;
ProxyByPassAddressList = defaultProxy.BypassList;
//...etc...
}
I don't want to use the proxy to proceed calling an internet resource. I just want to read those values and display them to the user on a dialog.
I have tried to use all the (kind of) related WebRequest.DefaultWebProxy, HttpWebRequest.GetSystemWebProxy(), etc. with no luck because they are all related to an upcomming web request to specified web resource. Also, those properties and methods are giving just an indication that if you try to call the "x" uri, a proxy server, may, or may not be used during that request.
Once again, I just want to show a similar form like that one under the internet settings when the poxy server setings are defined (server address, ports, bypass list, ect.).
Thats all...
I'm following a tutorial on this link http://www.codeproject.com/KB/aspnet/ASPNETService.aspx
Now I'm stuck at these codes
private const string DummyPageUrl =
"http://localhost/TestCacheTimeout/WebForm1.aspx";
private void HitPage()
{
WebClient client = new WebClient();
client.DownloadData(DummyPageUrl);
}
My local application address has a port number after "localhost", so how can I get the full path (can it be done in Application_Start method)? I want it to be very generic so that it can work in any cases.
Thanks a lot!
UPDATE
I tried this in the Application_Start and it runs fine, but return error right away when published to IIS7
String path = HttpContext.Current.Request.Url.GetLeftPart(UriPartial.Authority) + VirtualPathUtility.ToAbsolute("~/");
If it is calling back to the same server, perhaps use the Request object:
var url = new Uri(Request.Url, "/TestCacheTimeout/WebForm1.aspx").AbsoluteUri;
Otherwise, store the other server's details in a config file or the database, and just give it the right value.
But a better question would be: why would you talk via http to yourself? Why not just call a class method? Personally I'd be using an external scheduled job to do this.
You need an answer that works when you roll out to a different environment that maybe has a virtual application folder.
// r is Request.Url
var url = new Uri(r, System.Web.VirtualPathUtility.ToAbsolute("~/Folder/folder/page.aspx")).AbsoluteUri;
This will work in all cases and no nasty surprises when you deploy.
I suspect you're using the ASP.NET development server that's built-in to Visual Studio, which has a tendency to change port numbers by default. If that's the case, then you might try simply configuring the development server to always use the same port, as described here. Then just add the port number to your URL, like so:
private const string DummyPageUrl =
"http://localhost:42001/TestCacheTimeout/WebForm1.aspx";
Try as I might, I'm unable to resolve an address to IP. The code snippet is shown below. I keep getting the No such host is known exception, even though I could access google with my browser (The DNS server is almost certainly working). I'm however behind company's firewall.
try
{
foreach (IPAddress address in Dns.GetHostAddresses("www.google.com"))
{
Console.WriteLine(address.ToString());
}
}
catch (SocketException e)
{
Console.WriteLine("Source : " + e.Source); // System
Console.WriteLine("Message : " + e.Message); // No such host is known
}
There is nothing wrong with your code. Given that you can access www.google.com from a web browser the next most likely problem is that the web browser is using a proxy server. The web browser is actually accessing www.google.com through the proxy server which is allowed through the firewall. The simple application you wrote is not allowed through the firewall and is resulting in an exception.
You can verify this by looking at the proxy settings in Internet Explorer.
Tools -> Options -> Connections -> Lan Settings
There will be a proxy server group of settings. If there is a value present, this is almost certainly your problem.
You need to set up the proxy:
here's a snippet that should set it up for all the following calls:
protected void SetupProxy(string proxyUrl, string proxyLogin, string proxyPassword, string[] proxyBypass)
{
WebProxy proxy = new WebProxy(proxyUrl);
proxy.Credentials = new NetworkCredential(proxyLogin, proxyPassword);
proxy.BypassList = proxyBypass;
proxy.BypassProxyOnLocal = true;
WebRequest.DefaultWebProxy = proxy;
}
Rather than try through a browser, try pinging www.google.com (or some other host, of course) from the command line.
The ping itself may well not work, but it should show the IP address resolution first. If you get an error message like this:
Ping request could not find host www.google.com.
Please check the name and try again.
then it's likely that the proxy server is doing the DNS lookup for you when you're browsing, and your DNS server is either not working or your machine's network settings are incorrect.