Develop Reference

PuppeteerSharp and Page Level Proxies

I know this is possible using Puppeteer in js, but I'm wondering if anyone has figured out how to proxy on a page level in PuppeteerSharp (different proxies for different tabs)?.
it seems I can catch the request, but I'm not sure how to adjust the proxy.
page.SetRequestInterceptionAsync(true).Wait();
page.Request += (s, ev) =>
{
// what to do?
}
Edit
I am aware that I can set the proxy at the browser level like so;
var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
Headless = false,
Args = new[] { "--proxy-server=host:port" }
});
var page = await browser.NewPageAsync();
await page.AuthenticateAsync(new Credentials() { Username = "username", Password = "password" });
But this is not what I'm trying to do. I'm trying to set the proxy for each page within a single browser instance. I want to test lots of proxies so spawning a new instance of the browser just to set the proxy is too much overhead.

You can use different browser instances for each logical instances. I mean instead of trying to set different proxy for each page/tab with different proxy just create new browser instance and set proxy via launch args.
If this solution doesn't fit your needs, check this question. There is library for NodeJS which give ability to use different proxy per each page/tab. You can check that library source code and implement same things inside your C# application.
That library is using very simple method. Instead of sending requests via puppeter's browser/page library send request via nodejs http tools. It can be done by using method page.setRequestInterception. So library intercept each request from page, after that gather data and send request via http tools. I used C# a long time ago. So maybe I am wrong, but you can try to use HttpWebRequest or something similar. After you get result you should use method request.respond and pass response results there. In this way you can put any kind of proxy inside your application. Check here code of library.

Rest GET request in C# with body

I am trying to make a GET request with a body to a REST API - without success.
The API sits on top of Elastisearch, which requires a body for requests.
The body contains the filters and requests for specific fields for the request.
I have tried multiple different requests in Postman and they work fine. The body is forwarded and the results are filtered as expected.
The API is state run which means i have no control over it what so ever.
It refuses any request that is not GET (POST, PUT etc.).
I have tried with RestSharp, but it refuses to make a GET request with a body.
Error message is: "Http verb GET does not support body".
(i tried to get the C# code from Postman, but it throws the same exception).
I have also tried with HttpClient, HttpWebRequest and WebClient. Same error in all of them.
To try a new platform i created a PowerShell script to run from C#, but it throws the same exception.
Is this just impossible to do in C#?
My only option now is to create e.g. a PHP (or other language) script that can query the API succefully, and then access this script from C#. But it seems like quite a workaround only because of the GET/body problem.
Any advice or ressources you could provide or refer me to would be greatly appreciated! Thank you.
Btw. i have seen a lot of posts on Stackoverflow on this problem. Many of them are quite old so i am hoping that perhaps something has changed. Also many of them end up with a solution where you use POST instead of GET - which is not a option for me.

You can use NEST (https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/nest.html) - client for elastic for .net
Example code:
var elasticUrls = new Uri[]{new Uri("http://localhost:9200") };
var connect = new ConnectionSettings(
new SniffingConnectionPool(elasticUrls, true, null),
(builtin, settings) =>
new JsonNetSerializer(
builtin,
settings,
null,
null,
new JsonConverter[] {new StringEnumConverter()})
);
var client = new ElasticClient(connect);
var searchResult = await client.SearchAsync<ElasticModel>(
d => d
.Query(q => q.Bool(qd => qd
.Must(m => m.MatchAll())
))
.From(0)
.Size(10)
).ConfigureAwait(false);
/// <summary>
/// Some model from Elastic
/// </summary>
class ElasticModel
{
public int Id { get; set; }
}

Using RestSharp, in the request.AddParameter use ParameterType.GetOrPost instead of ParameterType.RequestBody
request.AddParameter("application/json", "things", ParameterType.GetOrPost);

Getting started with soap web service client c#

My boss asked how long it would take to build a client to access a web service that will send and receive some basic data and embedded documents. Just starting playing with it to see what's involved. I have been doing web and desktop development for about 20 years but have literally never touched a web service so with that I'm at the extreme newb level.
So far I used the wsdl to create the ServiceReference1 and I can see the methods in intellisense but I don't have the first clue where to start with calling the methods, passing parameters and consuming the response. I feel stupid because I'm sure it's pretty simple but just flailing at the code and looking for on point examples has gotten me nowhere. Usually I can find something through google in minutes that is exactly on point but not having luck here. Would appreciate a push in the right direction.
So basic questions. Proper way to make the calls. How and where to land the returned data. How to add parameters.
Here is my first attempt. This gets a simple list and has no parameters. The result in fiddler returns data but there is a runtime type mismatch error which I think is caused by some stray characters leading the response which appear to be caused by chucking, what ever that is. The response starts with 1ffs every time then contains the remainder of the xml. Secondarily I need to get the list into a dataset or some other container but I was hoping to just be able to step into the code and see a result
ServiceReference1.FilingInfoClient webservice = new FilingInfoClient();
ServiceReference1.courtListRequest cr = new ServiceReference1.courtListRequest();
ServiceReference1.courtListResponse lr = new ServiceReference1.courtListResponse();
lr = webservice .getCourtList(cr);
This is essentially the same but takes a date param. When I run this fiddler shows the parameter is not being sent. No other errors but I'm sure only because it exploded immediately.
ServiceReference1.FilingInfoClient webservice = new FilingInfoClient();
ServiceReference1.messageListRequest mr = new ServiceReference1.messageListRequest();
ServiceReference1.MessageListResponse mlr = new ServiceReference1.MessageListResponse();
mr.latestMessagePullTimestamp = DateTime.Now.AddDays(-5);
mr.endTimestamp = DateTime.Now;
mlr.latestMessagePullTimestamp = DateTime.Now;
mlr = webservice.getMessageList(mr);
This is the info provided by the web service host
<x:Envelope xmlns:x="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:urn1="urn:green:partner:ws:schema:FilingInfo">
<x:Header/>
<x:Body>
<urn1:getcourtList>
<urn1:courtListRequest/>
</urn1:getcourtList>
</x:Body>
</x:Envelope>
<x:Envelope xmlns:x="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:urn1="urn:green:partner:ws:schema:FilingInfo">
<x:Header/>
<x:Body>
<urn1:getMessageList>
<urn1:messageListRequest>
<urn1:latestMessagePullTimestamp>?</urn1:latestMessagePullTimestamp>
</urn1:messageListRequest>
</urn1:getMessageList>
</x:Body>
</x:Envelope>

we've got request and response pairs for each operation in the webservice. think like request => input, response => output, operation => method.
the webservice is an API. things that consume APIs are clients. the WSDL describes the API's operations and their requests and responses. tools like visual studio know how to read WSDLs and build C# code to perform those (SOAP) operations under-the-hood. this is the client (here FilingInfoClient). visual studio'll also generate classes representing each request and response.
this allows for a familiar programming experience. you call a method, give it some input, and it returns some output.
using (var client = new FilingInfoClient())
{
var request = new courtListRequest
{
//TODO fill in relevant properties
};
var response = client.getCourtList(request);
}

Zendesk - How to get users based on roles?

I am using Elizabeth's wrapper from https://github.com/mozts2005/ZendeskApi_v2
I want to pull a list of agents. I don't see any built in Function that will allow that.
I have tried using the endpoint of /api/v2/users.json?role=agent with the GetAllUsers() function but it still returns all of them.
Right now, I am going to add a custom field to be able to search for them, but that should not be the case, especially since Zendesk's API does have an option for returning users based on their role: /api/v2/users.json?role[]=admin&role[]=end-user
Can anyone help me out?

You can give a try to the Zendesk Search API:
from urllib.parse import urlencode
import requests
results = [] # Empty list to collect pagination results
credentials = 'your_zendesk_email', 'your_zendesk_password'
session = requests.Session()
session.auth = credentials
params = {
'query': 'type:user role:agent'
}
url = 'https://your_subdomain.zendesk.com/api/v2/search.json?' + urlencode(params)
while url:
response = session.get(url)
data = response.json()
results += data['results']
url = data['next_page'] # should return false according to the doc when the last page is reached
Useful resources:
zendesk api python tutorial
zendesk api pagination
Search endpoint seems to be supported also in the c# library you are using.

How to track MongoDB requests from a console application

I have a Console Application project written in C# which I've added Application Insights to with the following NuGet packages.
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Agent.Intercept
Microsoft.ApplicationInsights.DependencyCollector
Microsoft.ApplicationInsights.NLogTarget
Microsoft.ApplicationInsights.PerfCounterCollector
Microsoft.ApplicationInsights.Web
Microsoft.ApplicationInsights.WindowsServer
Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel
I've configured my InstrumentationKey in the config file and I'm firing up a TelemetryClient on startup using the with the following code:
var telemetryClient = new TelemetryClient();
telemetryClient.Context.User.Id = Environment.UserName;
telemetryClient.Context.Session.Id = Guid.NewGuid().ToString();
telemetryClient.Context.Device.OperatingSystem = Environment.OSVersion.ToString();
Everything is working well except AI is not capturing any requests that get sent to Mongo, I can see requests going off to SQL server in the 'Application map' but no sign of any other external requests. Is there any way that I can see telemetry of requests made to Mongo?
EDIT - Thanks to Peter Bons I ended up with pretty much the following which works like a charm and allows me to distinguish between success and failure:
var telemetryClient = new TelemetryClient();
var connectionString = connectionStringSettings.ConnectionString;
var mongoUrl = new MongoUrl(connectionString);
var mongoClientSettings = MongoClientSettings.FromUrl(mongoUrl);
mongoClientSettings.ClusterConfigurator = clusterConfigurator =>
{
clusterConfigurator.Subscribe<CommandSucceededEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", e.CommandName, DateTime.Now.Subtract(e.Duration), e.Duration, true);
});
clusterConfigurator.Subscribe<CommandFailedEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", $"{e.CommandName} - {e.ToString()}", DateTime.Now.Subtract(e.Duration), e.Duration, false);
});
};
var mongoClient = new MongoClient(mongoClientSettings);

I am not familiar with MongoDB but as far as I can tell there is no default support for it when it comes to Application Insights. But that does not mean you cannot do this, it will just involve some more code.
Again, I am not familiar with MongoDB but according to http://www.mattburkedev.com/logging-queries-from-mongodb-c-number-driver/ there is built-in support for logging the generated queries. Now, we only need to hook this up to Application Insights.
Since you already know how to use the TelemetryClient we can use the custom tracking methods provided by that class. See https://learn.microsoft.com/nl-nl/azure/application-insights/app-insights-api-custom-events-metrics for the available custom tracking methods.
All you need to do is to insert some code like this:
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
query, // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
The class telemetryClient is thread-safe so you can reuse it.
Now, according to the referenced blogpost you should be able to do something like this:
var client = new MongoClient(new MongoClientSettings()
{
Server = new MongoServerAddress("localhost"),
ClusterConfigurator = cb =>
{
cb.Subscribe<CommandStartedEvent>(e =>
{
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
e.Command.ToJson() // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
});
}
});
Again, I am not familiar with MongoDB but I hope this is a starting point for your imagination on how to adapt it to your needs using your knowledge of MongoDB.
EDIT:
If there is also a CommandCompletedEvent or similar event as opposed to the CommandStartedEvent event you should probably track the dependency there because you should then be able to calculate (or simpel read) the time spent and maybe get the actual value for the success indicator.