I'm using hybrid connections to request data from a listener. If I can write and read to the connection, how can I know that the response I've read from the connection matches the request I've given it? For example:
private HybridConnectionClient _client = new HybridConnectionClient(***);
public override async Task<RelayResponse> SendAsync(RelayRequest request)
{
var stream = await _client.CreateConnectionAsync();
var writer = new StreamWriter(stream) { AutoFlush = true };
var reader = new StreamReader(stream);
var reqestSerialized = JsonConvert.SerializeObject(request);
await writer.WriteLineAsync(reqestSerialized);
string responseSerialized = await reader.ReadLineAsync();
var response = JsonConvert.DeserializeObject<RelayResponse>(responseSerialized);
return response;
}
If the listener on this connection is reading and responding to many requests at the same time, is there anyway to know that the next Readline() we do on the client side to get the response is the one that is associated with the request? Or is that something that has to be managed?
Understanding a bit more about azure relay hybrid connections, I understand this now.
There isn't really any concept of a synchronous request/response in the framework, but if you use a new connection for each request, and respond on that same connection in the listener, you can be sure the response is for the request you sent.
Spawn a new connection for each request, then make sure the response is written to that connection. So looking at Microsoft's listener example code, whenever listener.AcceptConnectionAsync() fires do all the message response on relayConnection, then go back to waiting at await listener.AcceptConnectionAsync();
while (true)
{
var relayConnection = await listener.AcceptConnectionAsync();
if (relayConnection == null)
{
break;
}
ProcessMessagesOnConnection(relayConnection, cts);
}
Related
I must individually test each server of a cluster. Each server have one IP address but I must use same URL. I am using a console project.
static string RequestGet(string requestUrl, string ipspecify)
{
HttpWebRequest request = HttpWebRequest.CreateHttp(requestUrl);
// Submit the request, and get the response body.
string responseBodyFromRemoteServer;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
responseBodyFromRemoteServer = reader.ReadToEnd();
}
}
return responseBodyFromRemoteServer;
}
Locally, I can modify my host file, but I want to do this automatically through my program.
RequestGet("https://toto.org/myservice", "172.2.240.16")
RequestGet("https://toto.org/myservice", "172.2.240.17")
IP address is server address.
What is the solution?
Turns out, as of .NET 5, this is possible using HttpClient. SocketsHttpHandler, the default handler used by HttpClient, gained a ConnectCallback property which lets you override how a connection to the remote machine is established.
Here's where it's called, which you can use as inspiration for writing your own. The following seems to work fine:
public static async Task Main()
{
using var client = new HttpClient(new SocketsHttpHandler() { ConnectCallback = ConnectCallback });
await client.GetStringAsync("http://example.com");
}
private static async ValueTask<Stream> ConnectCallback(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
{
var endPoint = new DnsEndPoint("1.2.3.4", context.DnsEndPoint.Port);
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
await socket.ConnectAsync(endPoint, cancellationToken);
return new NetworkStream(socket, ownsSocket: true);
}
The request is made to 1.2.3.4 (or whatever IP you specify), but everything above the level of the TCP socket itself, including TLS, the Host header, etc, is still set to example.com, meaning that you shouldn't get certificate errors.
I am sending cURL request using HttpClient through the method described here under.
The parameter used for this method are:
SelectedProxy = a custom class that stores my proxy's parameters
Parameters.WcTimeout = the timeout
url, header, content = the cURL request (based on this tool to convert to C# https://curl.olsh.me/).
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var httpClient = new HttpClient(handler))
{
httpClient.Timeout = Parameters.WcTimeout;
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), url))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await httpClient.SendAsync(request);
}
catch (Exception e)
{
//Here the exception happens
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
If I am running this without proxy, it works like a charm.
When I send a request using a proxy which I tested first from Chrome, I have the following error on my try {} catch {}. Here is the error tree
{"An error occurred while sending the request."}
InnerException {"Unable to connect to the remote server"}
InnerException {"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond [ProxyAdress]"}
SocketErrorCode: TimedOut
By using a Stopwatch I see that the TimedOut occurred after around 30 sec.
I tried a few different handler based on the following links What's the difference between HttpClient.Timeout and using the WebRequestHandler timeout properties?, HttpClient Timeout confusion or with the WinHttpHandler.
It's worth noting that WinHttpHandler allow for a different error code, i.e. Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'. The underlying reason is the same though it helped to target where it bugs (i.e. WinInet) which confirms also what #DavidWright was saying regarding that timeouts from HttpClient manages a different part of the request sending.
Hence my issue is coming from the time it takes to establish a connection to the server, which triggers the 30sec timeout from WinInet.
My question is then How to change those timeout?
On a side note, it's worth noting that Chrome, which uses WinInet, does not seem to suffer from this timeout, nor Cefsharp on which a big part of my app is based, and through which the same proxies can properly send requests.
So thanks to #DavidWright I understand a few things:
Before that the HttpRequestMessage is sent and the timeout from HttpClient starts, a TCP connection to the server is initiated
The TCP connection has its own timeout, defined at OS level, and we do not identified a way to change it at run time from C# (question pending if anyone want to contribute)
Insisting on trying to connect works as each try benefits from previous tries, though proper exception management & manual timeout counter needs to be implemented (I actually considered a number of tries in my code, assuming each try is around 30sec)
All this together ended up in the following code:
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
var sp = ServicePointManager.FindServicePoint(endpoint);
sp.ConnectionLeaseTimeout = (int)Parameters.ConnectionLeaseTimeout.TotalMilliseconds;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var client = new HttpClient(handler))
{
client.Timeout = Parameters.WcTimeout;
int n = 0;
back:
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), endpoint))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await client.SendAsync(request);
}
catch (Exception e)
{
if(e.InnerException != null)
{
if(e.InnerException.InnerException != null)
{
if (e.InnerException.InnerException.Message.Contains("A connection attempt failed because the connected party did not properly respond after"))
{
if (n <= Parameters.TCPMaxTries)
{
n++;
goto back;
}
}
}
}
// Manage here other exceptions
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
On a side note, my current implementation of HttpClient may be problematic in the future. Though being disposable, HttpClient should be defined at App level through a static, and not within a using statement. To read more about this go here or there.
My issue is that I want to renew the proxy at each request and that it is not set on a per request basis. While it explains the reasdon of the new ConnectionLeaseTimeout parameter (to minimize the time the lease remains open) it is a different topic
I have had the same problem with HttpClient. Two things need to happen for SendAsync to return: first, setting up the TCP channel over which the communication occurs (the SYN, SYN/ACK, ACK handshake, if you're familiar with that) and second getting back the data that constitutes the HTTP response over that TCP channel. HttpClient's timeout only applies to the second part. The timeout for the first part is governed by the OS's network subsystem, and it's quite difficult to change that timeout in .NET code.
(Here's how you can reproduce this effect. Set up a working client/server connection between two machines, so you know that name resolution, port access, listening, and client and server logic all works. Then unplug the network cable on the server and re-run the client request. It will time out with the OS's default network timeout, regardless of what timeout you set on your HttpClient.)
The only way I know around this is to start your own delay timer on a different thread and cancel the SendAsync task if the timer finishes first. You can do this using Task.Delay and Task.WaitAny or by creating a CancellationTokenSource with your desired timeone (which essentially just does the first way under the hood). In either case you will need to be careful about cancelling and reading exceptions from the task that loses the race.
I'm using an http client and an http server(listener)
In some cases when i'm having big delay-traffic in my network, http client sends request but it nevers takes a response from http listener.As a result i'm trying to resend my request to server. But, server already has took previous one request and re-runs the new one. In this case, server runs my request twice.
httplistener
HttpListener listener;
Thread t;
private void Form1_Load(object sender, EventArgs e)
{
t = new Thread(new ThreadStart(MyThreadMethod));
t.IsBackground = true;
listener = new HttpListener();
listener.Prefixes.Add("http://192.168.0.214:8282/");
listener.Start();
t.Start();
// label1.Text = "Opened";
}
void MyThreadMethod()
{
while (true)
{
IAsyncResult result = listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener);
result.AsyncWaitHandle.WaitOne();
}
}
public void ListenerCallback(IAsyncResult result)
{
HttpListener listener = (HttpListener)result.AsyncState;
HttpListenerContext context = listener.EndGetContext(result);
string methodName = Convert.ToString(context.Request.Url);
//Here is my code
string Response = "OK";
HttpListenerResponse response = context.Response;
string responseString = Convert.ToString(Response);
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
response.ContentLength64 = buffer.Length;
System.IO.Stream output = response.OutputStream;
output.Write(buffer, 0, buffer.Length);
output.Close();
}
And here is my httpClient
HttpClient client = new HttpClient();
var content = new StringContent("my Data", Encoding.UTF8, "application/json");
var result = client.PostAsync("http://192.168.0.214:8282/", content).Result;
How can i prevent, when i'm having big delay_ms in my network, http listener running twice my requests?
This is a very common issue for distributed applications. You make a request, someone gets it, sometimes complete the request sometimes can not. Furthermore, sometimes you get the response and sometimes not. To deal with such situations, the rule of thumb is "your commands/requests should be idempotent". I mean, even if you send a request/command several times to a service, the result should not change and the command should only be executed only once. Accomplishing this sometimes can be very complicated but for simple scenarios you can simply accomplish this by adding a command id to your requests and a command log to your server. When server receives the request, it first checks whether this command already executed. If it was executed before, it returns the same success response, otherwise runs the command.
I have tried to create a simple console application.
We have a call system from 8x8 that provide a web streaming API but their documentation is very limited and nothing in C#.
The api service streams call statuses in near real time and I would like to get that 'stream' and be able to read and process it in realtime if possible. The response or Content Type is 'text/html'. But the actual body of the response can be declared as json - sample below:
{"Interaction":{"attachedData":{"attachedDatum":[{"attachedDataKey":"#pri","attachedDataValue":100},{"attachedDataKey":"callingName","attachedDataValue":999999999999},{"attachedDataKey":"cha","attachedDataValue":99999999999},{"attachedDataKey":"cnt","attachedDataValue":0},{"attachedDataKey":"con","attachedDataValue":0},{"attachedDataKey":"med","attachedDataValue":"T"},{"attachedDataKey":"pho","attachedDataValue":9999999999},{"attachedDataKey":"phoneNum","attachedDataValue":9999999999},{"attachedDataKey":"tok","attachedDataValue":999999999}]},"event":"InteractionCreated","inboundChannelid":9999999999,"interactionEventTS":9999999,"interactionGUID":"int-15b875d0da2-DJOJkDhDsrh3AIaFP8VkICv9t-phone-01-testist","resourceType":0}}
I have seen several posts concerning httpClient and the GetAsync methods but none of these appear to work as they appear to be for calls when a response is made, not something that constantly has a response.
Using fiddler for the call it does not appear to close so the stream is constantly running, so fiddler does not display any data until a separate user or instance connects.
When I use a browser the content is 'streamed' to the page and updates automatically and shows all the content (as above).
The api contains authentication so when another client connects and retrieves data the connected client closes and finally I am able to see the data that was gathering.
This is the code so and does return the big stream when another client connects but ideally I want a real time response and appears to just get stuck in the GETASYNC method:
var response = await client.GetAsync(address, HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode)
{
var responseContent = response.Content;
string responseString = await responseContent.ReadAsStringAsync();
Console.WriteLine(responseString);
}
Hopefully that's enough information for one of you clever people to help me in my predicament.
I was also having an issue consuming their streaming API and the examples I found that worked with the Twitter and CouchBase streaming API's did not work with 8x8. Both Twitter and CouchBase send line terminators in their pushes so the solution relied on ReadLine to pull in the feed. Since 8x8 does not send terminators you'll need to use ReadBlock or better ReadBlockAsync.
The following code shows how to connect using credentials and consume their feed:
private static async Task StreamAsync(string url, string username, string password)
{
var handler = new HttpClientHandler()
{
Credentials = new NetworkCredential {UserName = username, Password = password},
PreAuthenticate = true
};
// Client can also be singleton
using (var client = new HttpClient(handler))
{
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Connection.Add("keep-alive");
using (var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead))
{
using (var body = await response.Content.ReadAsStreamAsync())
{
using (var reader = new StreamReader(body))
{
while (!reader.EndOfStream)
{
var buffer = new char[1024];
await reader.ReadBlockAsync(buffer, 0, buffer.Length);
Console.WriteLine(new string(buffer));
}
}
}
}
}
}
I started down the path of using HttpClient as I thought the service I was accessing is a REST service. Turns out it's a JSON service running on port 80 but is a socket application.
The HttpClient opens the remote port but when it sends the JSON request it never gets a response. I was having the hardest time getting fiddler to get a response back as well. But I was able to get wget and curl to send/receiving a response. That's when I talked to the original developer and he mentioned that it wasn't a true "REST" service, but just a socket application that sends/receives JSON.
Is there something I can do to tweak HttpClient to access a socket application or am I going to have to take a step back and use WebSockets?
This is the test code that sends/receives the JSON packet.
private async Task ProcessZone(string szIPAddress)
{
string responseData = string.Empty;
Uri baseAddress = new Uri(#"http://" + szIPAddress + "/player");
try
{
using (var httpClient = new HttpClient { BaseAddress = baseAddress })
{
var _req = new SendRequest();
_req.id = "rec-100";
_req.url = "/stable/av/";
_req.method = "browse";
var _json = JsonConvert.SerializeObject(_req);
using (var content = new StringContent(_json,Encoding.UTF8, "application/json"))
{
using (var response = await httpClient.PostAsync(baseAddress, content))
{
responseData = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
}
}
}
var _Response = JsonConvert.DeserializeObject<Response>(responseData);
var item = new ZoneInfo();
item.szIPAddress = szIPAddress;
item.szZoneName = _Response.result.item.title;
lstZones.Add(item);
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
private async Task ProcessZones()
{
foreach (var item in ZoneSearch)
{
await ProcessZone(item.IPAddress);
}
}
The connection hangs on this line:
using (var response = await httpClient.PostAsync(baseAddress, content))
I should also mention that the code above does work fine on a true rest service...
That's when I talked to the original developer and he mentioned that it wasn't a true "REST" service, but just a socket application that sends/receives JSON.
Knowing the protocol is the first step towards making a working client.
Is there something I can do to tweak HttpClient to access a socket application or am I going to have to take a step back and use WebSockets?
Neither, unfortunately. HttpClient - as the name implies - only works with HTTP services. Since the server is not an HTTP server, it won't work with HttpClient. WebSockets have a rather confusing name, since they are not raw sockets but instead use the WebSocket protocol, which require an HTTP handshake to set up. Since the server is not an HTTP/WebSocket server, it won't work with WebSockets.
Your only choices are to either pressure the developer to write a real REST service (which makes your job orders of magnitude easier), or use raw sockets (e.g., Socket). Correctly using raw sockets is extremely difficult, so I recommend you pressure the developer to write a REST service like the entire rest of the world does today.