Develop Reference

How to fill a form inside an iframe with Puppeteer-Sharp

I am trying to scrape Taobao website with Puppeteer Sharp.
Here is the code:
private static async Task SurfWithPuppeteer()
{
var options = new LaunchOptions{ Devtools = true };
Console.WriteLine("Downloading chromium");
await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
Console.WriteLine("Navigating to Hacker News");
using (var browser = await Puppeteer.LaunchAsync(options))
using (var page = await browser.NewPageAsync())
{
page.DefaultNavigationTimeout = 50000;
await page.GoToAsync("https://login.tmall.com/?spm=875.7931836/B.a2226mz.1.66144265pHmhvt&redirectURL=https%3A%2F%2Fwww.tmall.com%2F");
var frameElement= await page.QuerySelectorAsync("#J_loginIframe");
//var frameElement = await page.QuerySelectorAsync("div#mallPage iframe");
//var frameElement = await page.Frames.Select(f=>f.QuerySelectorAsync("#J_loginIframe")).FirstOrDefault();
var frame = await frameElement.ContentFrameAsync();
var frameContent = await frame.GetContentAsync();
await frame.TypeAsync("#TPL_username_1", "compuwizpiyu");
await frame.TypeAsync("#TPL_password_1", "Priyanka24$");
var btn = await frame.QuerySelectorAsync("#J_SubmitStatic");
await btn.ClickAsync();
var res= await frame.WaitForNavigationAsync();
var t= await frame.GetContentAsync();
//var url = page.Url;
}
}
But I am unable to navigate to the frame that has the login form (frame has no name, only src and id).
I have tried to check the frames with page.Frames, but since the iframes have no name, difficult to find the correct frame I am looking for.
I have tried couple of other options too like :
var frameElement = await page.QuerySelectorAsync("div#mallPage iframe");
var frameElement = await page.Frames.Select(f=>f.QuerySelectorAsync("#J_loginIframe")).FirstOrDefault()
But still unable to get the intended frame. Please help me with what is wrong here.

This may be due to CORS.
Try the following code.
var options = new LaunchOptions
{
Devtools = true,
Args = new[]
{
"--disable-web-security",
"--disable-features=IsolateOrigins,site-per-process",
},
};

collect output from Task.Whenall to a list/data-type

I have the following c# code where i am sending a series of requests and reponses
public static async Task AuthenticateQvpx2()
{
var handshake = new Handshake();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
await Task.WhenAll(Receive(_webSocket), Send(_webSocket, buffer));
}
}
The async functions Send and Receive, has the following code.
await webSocket.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true, CancellationToken.None);
var result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
I wish to collect the requests and responses into an array/ any form of data type.
I having trouble as I am not particularly sure of what i should do next?

I wish to collect the requests and responses
It's kind of odd to collect the requests, as that data is already right there (in the buffer variable).
Assuming you meant that you just need the response, you can do that using await:
public static async Task AuthenticateQvpx2()
{
var handshake = new Handshake();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
var receiveTask = Receive(_webSocket);
await Task.WhenAll(receiveTask, Send(_webSocket, buffer));
var response = await receiveTask;
}
}

Not sure based on your snippets what type of Tasks your Send and Receive return but generally you can get your results from multiple tasks after using Task.WhenAll using LINQ this way:
var handshake = new Handshake();
List<Task<WebsocketReceiveResult>> tasks = newList<Task<WebsocketReceiveResult>>();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
tasks.Add(webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None));
}
await Task.WhenAll(tasks);
var resultArray = tasks.Select(t => t.Result).ToArray();

Try something like
ConcurrentDictionary<Guid, (System.Byte[], WebSocketReceiveResult)> x = new ConcurrentDictionary<Guid, (byte[], WebSocketReceiveResult)>();
Generate GUID in your foreach a pass it to your methods:
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
var guid = Guid.NewGuid();
await Task.WhenAll(Receive(_webSocket, guid), Send(_webSocket, buffer, guid));
}
Then you can work with dictionary from within your Receive and Send methods in natural way.
void Send(WebSocket webSocket, byte[] buffer, Guid guid)
{
x.GetOrAdd(guid, new ValueTuple<System.Byte[], WebSocketReceiveResult>(buffer, null));
await webSocket.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true, CancellationToken.None)
}
void Receive(WebSocket webSocket, Guid guid)
{
var result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
x[guid].Item2 = result;
}

ASP.NET Core with WebSockets - WebSocket handshake never occurs

I am very new to C# programming, having previously only worked with Java. This project I am building should be very straightforward - we have a web page with a selection of foreign currency pairs. The element chosen is sent to the server, which responds with a hardcoded value of their exchange rate. The requirement is that both actions are implemented through the use of WebSockets. Here is the JS code on my page:
var protocol;
var wsUri;
var socket;
window.onload = function(e) {
e.preventDefault();
protocol = location.protocol === "https:" ? "wss:" : "ws:";
wsUri = protocol + "//" + window.location.host;
socket = new WebSocket(wsUri);
socket.onopen = e => {
console.log("socket opened", e);
};
document.getElementById("currencypair").onchange = function()
{
var selector = document.getElementById("currencypair");
var text = selector.options[selector.selectedIndex].text;
socket.send(text);
};
socket.onmessage = function (evt) {
var receivedMessage = evt.data;
document.getElementById("output").html(receivedMessage);
};
};
Here is a snippet of the Startup.cs class Configure method:
app.UseWebSockets();
app.UseMiddleware<WebSocketMiddleware>();
And here is the middleware class to process requests.
public class WebSocketMiddleware
{
private readonly RequestDelegate _next;
public WebSocketMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
if (!context.WebSockets.IsWebSocketRequest)
{
await _next.Invoke(context);
return;
}
var ct = context.RequestAborted;
using (var socket = await context.WebSockets.AcceptWebSocketAsync())
{
while (true)
{
var stringReceived = await ReceiveStringAsync(socket, ct);
if (CurrencyPairCollection.CurrencyPairs.TryGetValue(stringReceived, out var value))
{
await SendStringAsync(socket, value.ToString(), ct);
}
else
{
throw new Exception("Unexpected value");
}
await Task.Delay(1000, ct);
}
}
}
private static async Task<string> ReceiveStringAsync(WebSocket socket, CancellationToken ct = default(CancellationToken))
{
var buffer = new ArraySegment<byte>();
using (var ms = new MemoryStream())
{
WebSocketReceiveResult result;
do
{
ct.ThrowIfCancellationRequested();
result = await socket.ReceiveAsync(buffer, ct);
ms.Write(buffer.Array, buffer.Offset, result.Count);
}
while (!result.EndOfMessage);
ms.Seek(0, SeekOrigin.Begin);
if (result.MessageType != WebSocketMessageType.Text || result.Count.Equals(0))
{
throw new Exception("Unexpected message");
}
using (var reader = new StreamReader(ms, Encoding.UTF8))
{
return await reader.ReadToEndAsync();
}
}
}
private static Task SendStringAsync(WebSocket socket, string data, CancellationToken ct = default(CancellationToken))
{
var segment = new ArraySegment<byte>(Encoding.UTF8.GetBytes(data));
return socket.SendAsync(segment, WebSocketMessageType.Text, true, ct);
}
}
Please mind I was working with the following example which contains mistakes listed by people in the comment section. I did my best to resolve them, however due to my limited experience, that may be where the fault lies.
https://www.softfluent.com/blog/dev/Using-Web-Sockets-with-ASP-NET-Core
Basically, upon running the app the browser console immediately reports this:
WebSocket connection to 'ws://localhost:51017/' failed: Error during WebSocket handshake: Unexpected response code: 200

I have been able to answer my own question. So in Startup.cs of which I provided only a snippet, a call to app.UseMvc() is made right before the lines I have already shared. This is generated by the default template. The trick was to move this call to below the following:
app.UseWebSockets();
app.UseMiddleware<WebSocketMiddleware>();
as otherwise the request pipeline is disrupted.
This will allow our socket to open, however without changing the following line in async Task ReceiveStringAsync(...)
var buffer = new ArraySegment<byte>();
to
var buffer = new ArraySegment<byte>(new byte[8192]);
it will still close prematurely. Next, just needed to correct JS syntax error. Changed
document.getElementById("output").html(receivedMessage);
to
document.getElementById("output").value = receivedMessage;
That's it, it works.

async task cancellation c# xamarin

I have a functionality of search users. I have provided a textview and on that textview changed method I'm firing a method to get data from web server. But I'm facing problem when user types letter, because all the api hits done in async task. Service should be hit after 100 milli-sec of wait, means if user types a letter "a" then doesn't type for 100 milli-sec then We have to hit the service. But if user types "a" then "b" then "c", so one service should be hit for "abc", not for all.
I followed the official link, but it doesn't help me
https://msdn.microsoft.com/en-us/library/jj155759.aspx
So basically here is my code
textview.TextChange+= (sender,e) =>{
CancellationTokenSource cts = new CancellationTokenSource();
await Task.Delay(500);
// here some where I have to pass cancel token
var lst = await APIClient.Instance.GetUserSearch("/user/get?searchTerm=" + newText, "application/json",cts);
if (lst != null && lst.Count > 0){
lstSearch.AddRange(lst);
}
}
Here is my method to GetUser
public async Task<JResponse> GetUserSearch<JResponse>(string uri, string contentType,CancellationToken cts)
{
try
{
Console.Error.WriteLine("{0}", RestServiceBaseAddress + uri);
string url = string.Format("{0}{1}", RestServiceBaseAddress, uri);
var request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = contentType;
if (Utility.CurrentUser != null && !string.IsNullOrWhiteSpace(Utility.CurrentUser.AuthToken))
{
request.Headers.Add("api_key", Utility.CurrentUser.AuthToken);
}
request.Method = "POST";
var payload = body.ToString();
request.ContentLength = payload.Length;
byte[] byteArray = Encoding.UTF8.GetBytes(body.ToString());
request.ContentLength = byteArray.Length;
using (var stream = await request.GetRequestStreamAsync())
{
stream.Write(byteArray, 0, byteArray.Length);
stream.Close();
}
using (var webResponse = await request.GetResponseAsync())
{
var response = (HttpWebResponse)webResponse;
using (var reader1 = new StreamReader(response.GetResponseStream()))
{
Console.WriteLine("Finished : {0}", uri);
var responseStr = reader1.ReadToEnd();
var responseObj = JsonConvert.DeserializeObject<JResponse>(
responseStr,
new JsonSerializerSettings()
{
MissingMemberHandling = MissingMemberHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore
});
return responseObj;
}
}
}
catch (System.Exception ex)
{
Utility.ExceptionHandler("APIClient", "ProcessRequestAsync", ex);
}
return default(JResponse);
}

In your example, you are creating a CancellationTokenSource - you need to hold a reference to it, so that the next time the handler is invoked, the previous search can be cancelled. Here is an example console app that you should be able to run, but the important piece is in the handler.
private CancellationTokenSource _cts;
private async void TextChangedHandler(string text) // async void only for event handlers
{
try
{
_cts?.Cancel(); // cancel previous search
}
catch (ObjectDisposedException) // in case previous search completed
{
}
using (_cts = new CancellationTokenSource())
{
try
{
await Task.Delay(TimeSpan.FromSeconds(1), _cts.Token); // buffer
var users = await _userService.SearchUsersAsync(text, _cts.Token);
Console.WriteLine($"Got users with IDs: {string.Join(", ", users)}");
}
catch (TaskCanceledException) // if the operation is cancelled, do nothing
{
}
}
}
Be sure to pass the CancellationToken into all of the async methods, including those that perform the web request, this way you signal the cancellation right down to the lowest level.

Try to use timer. First time then you change text - you create it. Then you change text after that - you restart timer. If you don't change text for 700 milliseconds - timer will fire PerformeSearch method. Use Timeout.Infinite for timer period parameter to prevent it from restarting.
textview.TextChange += (sender,e) =>
{
if (_fieldChangeTimer == null)
_fieldChangeTimer = new Timer(delegate
{
PerformeSearch();
}, null, 700, Timeout.Infinite);
else
{
_fieldChangeTimer.Change(700, Timeout.Infinite);
}
};

Instantiate the CancellationTokenSource.
cts = new CancellationTokenSource(); Example method
private void cancelButton_Click(object sender, RoutedEventArgs e)
{
if (cts != null)
{
cts.Cancel();
}
}

Shared HttpClient blocks on async requests

I'm having an issue with HttpClient and async requests. Basically i'm having an async method that is creating async requests with a shared HttpClient that is initialized in the ctor.
My problem is that it seems that the HttpClient blocks when calling my method in an async manner.
Here is my calling code:
var tasks = trips.Select(u => api.Animals.GetAsync(u * 100, 100).ContinueWith(t =>
{
lock (animals)
{
if (t.Result != null)
{
foreach (var a in t.Result)
{
animals.Add(a);
}
}
}
}));
await Task.WhenAll(tasks);
Here is the method that blocks with a shared HttpClient:
//HttpClient blocks on each request
var uri = String.Format("animals?take={0}&from={1}", take, from);
var resourceSegmentUri = new Uri(uri, UriKind.Relative);
var response = await _client.GetAsync(resourceSegmentUri);
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
var animals = JsonConvert.DeserializeObject<T>(content);
return animals;
}
This snippet does not block, when using a client for each request:
using (var client = new HttpClient(){BaseAddress = new Uri(_config.BaseUrl)})
{
var uri = String.Format("animals?take={0}&from={1}", take, from);
var resourceSegmentUri = new Uri(uri, UriKind.Relative);
var response = await client.GetAsync(resourceSegmentUri);
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
var animals = JsonConvert.DeserializeObject<T>(content);
return animals;
}
}
Is a shared HttpClient a no go? Or can I utilize it in some other way?

Using a shared HttpClient is actually recommended.
See my answer why - What is the overhead of creating a new HttpClient per call in a WebAPI client?