I know this is possible using Puppeteer in js, but I'm wondering if anyone has figured out how to proxy on a page level in PuppeteerSharp (different proxies for different tabs)?.
it seems I can catch the request, but I'm not sure how to adjust the proxy.
page.SetRequestInterceptionAsync(true).Wait();
page.Request += (s, ev) =>
{
// what to do?
}
Edit
I am aware that I can set the proxy at the browser level like so;
var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
Headless = false,
Args = new[] { "--proxy-server=host:port" }
});
var page = await browser.NewPageAsync();
await page.AuthenticateAsync(new Credentials() { Username = "username", Password = "password" });
But this is not what I'm trying to do. I'm trying to set the proxy for each page within a single browser instance. I want to test lots of proxies so spawning a new instance of the browser just to set the proxy is too much overhead.
You can use different browser instances for each logical instances. I mean instead of trying to set different proxy for each page/tab with different proxy just create new browser instance and set proxy via launch args.
If this solution doesn't fit your needs, check this question. There is library for NodeJS which give ability to use different proxy per each page/tab. You can check that library source code and implement same things inside your C# application.
That library is using very simple method. Instead of sending requests via puppeter's browser/page library send request via nodejs http tools. It can be done by using method page.setRequestInterception. So library intercept each request from page, after that gather data and send request via http tools. I used C# a long time ago. So maybe I am wrong, but you can try to use HttpWebRequest or something similar. After you get result you should use method request.respond and pass response results there. In this way you can put any kind of proxy inside your application. Check here code of library.
Related
How can I extract the X-Pagination header from a response and use the next link to chain requests?
I've tried in both Postman and C# console application with RestSharp. No success.
Easiest would be a small console application to test. I just need to iterate through the pages.
This is what I get back in the headers X-Pagination:
{
"Page":1,
"PageSize":20,
"TotalRecords":1700,
"TotalPages":85,
"PreviousPageLink":"",
"NextPageLink":"www......./api/products/configurations?Filters=productid=318&IncludeApplicationPerformance=true&page=1",
"GotoPageLinkTemplate":"www..../api/products/configurations?Filters=productid=318&IncludeApplicationPerformance=true&page=0"
}
In Postman you simply retrieve the header, parse it into a Json object then use the value to set a link for your next request.
Make your initial request then in the Test tab do something like:
var nextPageLinkJson = JSON.parse(pm.response.headers.get("X-Pagination"));
var nextPageLink = nextPageLinkJson.NextPageLink;
pm.environment.set("nextPageLink", nextPageLink);
If you don't know how many pages you're going to have then you'll have to play with conditions when to set the nextPageLink variable and what not but that's the general idea.
You can set the request to run using the new link with postman.setNextRequest("request_name") as well.
Additionally this approach will only work in collection runner.
I have been contemplating on a dilemma for hours. I have a Visual Studio Solution that contains a WCF, WebForms, UWP, Xamarin and a SharedLibrary Projects.
I intend to use the WCF project as the backend which talks to the database and process Email and SMS integration and feed the other apps.
OPTION A
Currently, The WCF is hosted on an Azure App Service which makes it accessible via POST, GET, etc from the url which is: https://mywcfprojectlink.azurewebsites.net/service1.svc/GetUsers
With such arrangements, I can perform a POST request to get data from the apps:
string response = string.Empty;
string url = "https://mywcfprojectlink.azurewebsites.net/service1.svc/GetUsers";
try
{
var values = new Dictionary<string, string>
{
{ "data", Encryption.EncryptString(dat.ToString()) } //dat is incoming method param
};
string jsonString = JsonConvert.SerializeObject(values);
var cli = new WebClient();
cli.Headers[HttpRequestHeader.ContentType] = "application/json";
response = cli.UploadString($"{url}", jsonString);
var result = JsonConvert.DeserializeObject<string>(response);
topic.InnerText = Encryption.DecryptString(result.ToString());
}
catch (Exception)
{
return string.Empty;
}
The method above is a simple one as I have other ones where I Deserialize with Models/Classes.
OPTION B
I equally have access to the methods defined in service1 by adding the project reference to my WebForms which surprisingly is also compatible with xamarin but not with UWP. Nevertheless, I am interested in the WebForms scenario. Below is an example method:
using BackEnd;
//Service1 service1 = new Service1();
//var send = service1.GetUsers(dat.ToString()); //dat is incoming method param
//topic.InnerText = send;
Obviously, using the Option B would eliminate the need to encrypt, decrypt, serialize or deserialize the data being sent. However, I have serious performance concerns.
I need to know the better option and if there is yet another alternative (probably an Azure Resource), you can share with me.
If you decide to use https endpoint of the Azure website, option A is secure because of SSL encryption. So you don't have to encrypt/decrypt it by yourself. The only tip is to create a proper authorization mechanism. For example use TransportWithMessageCredential. An example is provided in below article https://www.codeproject.com/Articles/1092557/WCF-Security-and-Authentication-in-Azure-WsHttpBin
I have a Controller Action in which I:
Forward a URL to a 3rd party that the 3rd party re-directs the client to if an operation is successful.
The client then connects to the 3rd party and performs an operation.
If successful the 3rd party re-directs (using a 302) the client to the URL we told them at step 1.
This all works. Problem is the URL the Controller on the server tells the 3rd party about us built using:
var urlHelper = new UrlHelper(System.Web.HttpContext.Current.Request.RequestContext);
var responseUrl = urlHelper.Action("Action", "Controller", new { id }, Request.Url.Scheme);
responseURL now looks like:
"https://some.internal.domain/App/Action/1234".
The client however accessed the site using an address like:
"https://example.com/App/Action/1234".
and they cannot access https://some.internal.domain externally.
So how can I build the absolute URL for the 3rd party to re-direct the client to using the URL the client is accessing the site from?
UPDATE: The above method should have worked it turns out the problem is am behind a Reverse Proxy (have re-worded the title to reflect this).
I came across a similar problem. You basically have two options.
Use the URL Rewrite Module to rewrite your entire response by replacing/rewriting all pattern matched with https://some.internal.domain to https://example.com/App. You'll probably want to add certain conditions like only rewrite if the response type is of type html/text and what not.
Use an extension method. This method requires the Reverse Proxy in question to forward additional headers identifying where the original request came from (e.g. Azure Application Gateway sends a header named X-Original-Host but I think most other reverse proxies use X-Forwarded-For or some variant like Winfrasoft X-Forwarded-For). So based on the example provided you could do something like.
The helper could look like this
public static class UrlHelpers
{
public static string GetUrlHostname(this UrlHelper url)
{
var hostname = url.RequestContext.HttpContext.Request.Headers["X-Original-Host"] ?? url.RequestContext.HttpContext.Request.Url.Host;
return hostname;
}
}
And then to use it based on your example.
var urlHelper = new UrlHelper(System.Web.HttpContext.Current.Request.RequestContext);
var responseUrl = urlHelper.Action("Action", "Controller", new { id }, Request.Url.Scheme, urlHelper.GetUrlHostname());
I am trying to register my application, which is written in C#, with Gracenote for my senior seminar class. I believe I have done it correctly, but to my understanding when I call a WebClient.UploadData() method, I should have what is returned displayed in my output window. Nothing came up.
Since you are supposed to register your application with Gracenote only once per installation, I'm kind of at a standstill since I'm not sure if it registered or not. Below is my code. [#####-AAAAAA is my ClientID provided by Gracenote.]
using (var client = new WebClient())
{
var dataToPost = Encoding.Default.GetBytes("<QUERIES><QUERY CMD=\"REGISTER\"><CLIENT>########-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA</CLIENT></QUERY></QUERIES>");
var result = client.UploadData("https://c########.web.cddbp.net/webapi/xml/1.0/", "POST", dataToPost);
}
Try using my C# Gracenote API wrapper here http://www.nuget.org/packages/ParkSquare.Gracenote/
This is a follow-up to Choosing a Connection String based on kind of request for which I got no answer and what I thought worked doesn't.
I have a webservice that needs to choose a specific connection string based on the user calling it from a browser or from a client application.
I tried:
HttpContext.Current != null? ConnectionStrings["Website"].ConnectionString : ConnectionStrings["Client"].ConnectionString
but realized that at some point even if I'm using the client application, there is some HttpContext (if someone can explain why it'd be great) but the Browser field under Request is "Unknown". So, then I tried:
if ( HttpContext.Current != null )
{
if ( HttpContext.Current.Request.Browser != "Unknown" )
{
//browser connection string here
}
else
//client app connection string here
}
else
//client app connection string here
This worked wonders when debugging, but on testing environment it still points to Browser connection string even when calling from the client app, as if at some point the Browser isn't "Unknown" ...
Is there a MUCH easier/simpler way to do this? The way I'm doing it seems really ugly.
I'm quite desperate at the moment as I have no idea why this is happening..
Rather than detecting and switching on the browser type, consider these two suggestions:
Add Custom Request Headers
In your various callers, define a new custom header in your Http request.
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Headers.Add("CallerType", "ClientApp"); // "Browser", etc.
Then you know exactly and reliably what type of client is calling. This would be hard to get wrong, and couldn't be spoofed/mistaken.
Include The Caller Type in the QueryString
myService.asmx?BrowserType=1
Add a simple new querystring parameter to your .asmx webmethod. This will work just the same in a controlled environment, but if other users/developers get it wrong, or malform the expected values, you'd have to take other measures to correct/handle.
Both allow you to easily determine the connString on the incoming value. Perhaps the absense of a modifier/header, you could assume a default. Your sample question has 2 basic outcomes, and either suggested solution will be easy to extend (browser, client app, iPhone, whathaveyou).