Develop Reference

Modify Headers In Selenium test framework

I'm trying to create an automated test framework using specflow and selenium and I need to pass basic auth, as well as x-forwarded-for header information to the browser for some of the tests. I see solutions for Browser mob proxy but it seems like that project is dead. I tried using Titanium Proxy but I'm having a heck of a time getting it to work. I'm mostly having trouble understanding what I need to do with certificates to get my webdriver to use the proxy and load the sites without fussing. Currently I can't get it to open http or https sites. When I run my test case I can see that the browser is correctly setting the proxy so I don't think that's the case. I left commented out code for things I tried to do to set a certificate but I'm not 100% sure what I'm actually doing there. Any help would be appreciated.
Alternatively if there is a better, easier way to add headers to a request I'm all ears.
public class ProxyService
{
private ProxyServer server;
public int port { get; }
public ProxyService(int port)
{
this.server = new ProxyServer();
//server.CertificateManager.CreateServerCertificate("rootCert.pfx");
//server.CertificateManager.LoadRootCertificate();
//server.CertificateManager.CertificateEngine = Titanium.Web.Proxy.Network.CertificateEngine.BouncyCastle;
//server.CertificateManager.TrustRootCertificate(true);
//server.CertificateManager.EnsureRootCertificate(true,true,false);
this.port = port;
}
public void AddHeader(string key, string value)
{
}
private async Task OnRequest(object sender, SessionEventArgs e) => await Task.Run(
() =>
{
e.HttpClient.Request.Headers.AddHeader("Authorization", "Basic xxxxxxxxxx");
});
public void Start()
{
var endpoint = new ExplicitProxyEndPoint(System.Net.IPAddress.Any, port, true);
server.AddEndPoint(endpoint);
server.Start();
server.BeforeRequest += OnRequest;
}
public void Stop()
{
server.Stop();
}
}

How to switch a ConnectionHandler to UDP

My ASP.Net Core application provides a TCP listener, implemented with a custom ConnectionHandler, to receive binary data from another process (let's call it Datasource) on another host. This data is then sent to the browser through a WebSocket (called DataSink in the code).
Since the process Datasource has changed from a single TCP connection to UDP datagrams, I need to adapt (its internals are out of my reach).
How can I switch the current implementation to an UDP listener? Is there a canonical way how this is done with ASP.Net Core?
public class MySpecialConnectionHandler : ConnectionHandler
{
private readonly IMyDataSink DataSink;
public MySpecialConnectionHandler(IMyDataSink dataSink)
{
DataSink = dataSink;
}
public override async Task OnConnectedAsync(ConnectionContext context)
{
TransportConnection connection = context as TransportConnection;
Console.WriteLine("new connection: " + connection.RemoteAddress + ":" + connection.RemotePort);
while (true)
{
var result = await connection.Transport.Input.ReadAsync().ConfigureAwait(false);
var buffer = result.Buffer;
foreach (var segment in buffer)
{
await DataSink.RelayData(segment.Span.ToArray()).ConfigureAwait(false);
}
if (result.IsCompleted)
{
break;
}
connection.Transport.Input.AdvanceTo(buffer.End);
}
Console.WriteLine(connection.ConnectionId + " disconnected");
}
}
The UDP listener must be available while the ASP.Net Core application is running.
EDIT:
Order and reliability of the datagram transmission is not that important (perhaps not at all), since the transmitted data is a MPEG1-stream multiplexed into MPEG-TS. The data source is on the first host, the ASP.Net Core application is on a second host and the receiver / consumer is a third host. The host creating the stream and the receiving process on the third host are in separate networks. The ASP.Net Core application acts as a relay. The sender is sending all time, but does not care about whether the data is received or not.
EDIT 2:
The main problem right now is where to put the UdpClient. The previous implementation (back when we used TCP) configured the Kestrel server for additional TCP listening and used the already presented ConnectionHandler:
return WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureKestrel((_, options) =>
{
// HTTP
options.Listen(networkInterface, httpPort);
// HTTPS
options.Listen(networkInterface, httpsPort, builder => builder.UseHttps());
// stream sink
options.Listen(streamInterface, streamPort, builder => builder.UseConnectionHandler<MySpecialConnectionHandler >());
});
The ConnectionHandler accepts the incoming TCP connection and then forwards the streaming data to a number of connected WebSockets. Since this is not usable with UDP datagrams, I need to place the UdpClient somewhere where it continuously (i.e. while(true)) receives datagrams and forwards them to the WebSockets. My problem is that I don't know where to put it, run the background thread and have the communication span threads without having any problems with this inter-thread data flow (like race conditions).

So, to conclude this:
We used a combination of a BackgroundWorker with an UdpClient. The BackgroundWorker is only instantiated when there is at least one receiver:
StreamReceiver = new BackgroundWorker();
StreamReceiver.DoWork += ReceiveStream;
StreamReceiver.RunWorkerAsync();
ReceiveStream is a private method that establishes the UdpClient and then waits for incoming data that needs to be relayed.
private async void ReceiveStream(object sender, DoWorkEventArgs e)
{
// DataSinkPort is a const int
UdpClient datasource = new UdpClient(_DataSinkPort);
while (true)
{
var rec = await datasource.ReceiveAsync();
await RelayData(rec.Buffer);
if (_CancellationToken.IsCancellationRequested)
{
return;
}
}
}
The method RelayData just uses the outgoing TCP connection of each subscribed receiver.

Correct pattern for ZeroMQ Server/Client type

I have a server that needs to get instructions to run processes for clients on another machine.
The clients send a job message, the Server processes the job and later sends the back results.
I tried using the NetMQ Request-Response pattern (see below)
This works nicely for 1 client, BUT if a second client sends a request before previous client job is finished - I get an error.
I really need to be able to receive ad-hoc messages from clients, and send results when they are completed. Clearly, I am using the wrong pattern, but reading the ZeroMQ docs has not highlighted a more appropriate one.
namespace Utils.ServerMQ
{
class ServerMQ
{
public static void Go()
{
using (var responseSocket = new ResponseSocket("#tcp://*:393"))
{
while (true)
{
Console.WriteLine("Server waiting");
var message = responseSocket.ReceiveFrameString();
Console.WriteLine("Server Received '{0}'", message);
//System.Threading.Thread.Sleep(1000);
var t2 = Task.Factory.StartNew(() =>
{
RunProcMatrix(message, responseSocket);
});
}
}
}
public static void RunProcMatrix(object state, ResponseSocket responseSocket)
{
var process = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = Path.Combine(#"H:\Projects\Matrix\Matrix\bin\Debug\", "Matrix001.exe"),
Arguments = (string)state,
WindowStyle = ProcessWindowStyle.Normal,
CreateNoWindow = false
}
};
process.Start();
process.WaitForExit();
responseSocket.SendFrame((string)state);
}
}
}

You want a ROUTER socket on the server side, so it can receive multiple requests at a time. (Guide) REQ sockets on the client side are still fine unless the server may arbitrarily push data to them, then they need to be DEALER sockets.
Note that for sockets beyond REQ/RESP you need to manually handle the message envelope (the first frame of the message indicating its destination). Guide
The 0MQ docs are incredibly dense... I don't blame you for not intuiting this from them :)
This example from the NetMQ docs is full ROUTER-DEALER: https://netmq.readthedocs.io/en/latest/router-dealer/#router-dealer, you can take just the router side and it should work the same though.

Connecting to a SignalR server

On the server I create a Hub
public class SGHub : Hub
{
public static List<string> Users = new List<string>();
public void Send(string name, string message)
{
Clients.All.broadcastMessage(name, message);
Console.WriteLine(SGHub.Users.Count);
}
}
On the client I connect to the Hub
void Start()
{
hubConnection = new HubConnection(serverURL);
hubConnection.Error += HubConnectionError;
iHubProxy = hubConnection.CreateProxy("SGHub");
Subscription subscription = iHubProxy.Subscribe("broadcastMessage");
hubConnection.Start();
}
If the server is not running, and the client tries to connect to the Hub, the application hangs, how to avoid this?

You can try to start flow with an http request to serverURL
and only if server returns Ok, you start signalr connection flow.

I would suggest to put you clientside connection code in a thread. That's how I did it. Every time you have a long running task (like waiting for a timeout in your case) and you do this in the UI thread the app will freeze.

TcpClient dispose order

I am writing a network layer on top of TCP and I am facing some troubles during my UnitTest phase.
Here is what I'm doing (My library is composed of multiple classes but I only show you the native instructions causing my problems, to limit the size of the post):
private const int SERVER_PORT = 15000;
private const int CLIENT_PORT = 16000;
private const string LOCALHOST = "127.0.0.1";
private TcpClient Client { get; set; }
private TcpListener ServerListener { get; set; }
private TcpClient Server { get; set; }
[TestInitialize]
public void MyTestInitialize()
{
this.ServerListener = new TcpListener(new IPEndPoint(IPAddress.Parse(LOCALHOST), SERVER_PORT));
this.Client = new TcpClient(new IPEndPoint(IPAddress.Parse(LOCALHOST), CLIENT_PORT));
this.ServerListener.Start();
}
// In this method, I just try to connect to the server
[TestMethod]
public void TestConnect1()
{
var connectionRequest = this.ServerListener.AcceptTcpClientAsync();
this.Client.Connect(LOCALHOST, SERVER_PORT);
connectionRequest.Wait();
this.Server = connectionRequest.Result;
}
// In this method, I assume there is an applicative error within the client and it is disposed
[TestMethod]
public void TestConnect2()
{
var connectionRequest = this.ServerListener.AcceptTcpClientAsync();
this.Client.Connect(LOCALHOST, SERVER_PORT);
connectionRequest.Wait();
this.Server = connectionRequest.Result;
this.Client.Dispose();
}
[TestCleanup]
public void MyTestCleanup()
{
this.ServerListener?.Stop();
this.Server?.Dispose();
this.Client?.Dispose();
}
First of all, I HAVE TO dispose the server first if I want to connect earlier to the server on the same port from the same endpoint:
If you run my tests like this, it will run successfully the first time.
The second time, it will throw an exception, in both tests, on the Connect method, arguing the port is already in use.
The only way I found to avoid this exception (and to be able to connect on the same listener from the same endpoint) is to provoke a SocketException within the Server by sending bytes to the disposed client twice (on the first sending, there is no problem, the exception is thrown only on the second sending).
I don't even need to Dispose the Server if I provoke an Exception ...
Why is the Server.Dispose() not closing the connection and freeing the port ??? Is there a better way to freeing the port than by provoking an Exception ?
Thanks in advance.
(Sorry for my English, I am not a native speaker)
Here is an example within a main fonction, to be checkout more easily:
private const int SERVER_PORT = 15000;
private const int CLIENT_PORT = 16000;
private const string LOCALHOST = "127.0.0.1";
static void Main(string[] args)
{
var serverListener = new TcpListener(new IPEndPoint(IPAddress.Parse(LOCALHOST), SERVER_PORT));
var client = new TcpClient(new IPEndPoint(IPAddress.Parse(LOCALHOST), CLIENT_PORT));
serverListener.Start();
var connectionRequest = client.ConnectAsync(LOCALHOST, SERVER_PORT);
var server = serverListener.AcceptTcpClient();
connectionRequest.Wait();
// Oops, something wrong append (wrong password for exemple), the client has to be disposed (I really want this behavior)
client.Dispose();
// Uncomment this to see the magic happens
//try
//{
//server.Client.Send(Encoding.ASCII.GetBytes("no problem"));
//server.Client.Send(Encoding.ASCII.GetBytes("oops looks like the client is disconnected"));
//}
//catch (Exception)
//{ }
// Lets try again, with a new password for example (as I said, I really want to close the connection in the first place, and I need to keep the same client EndPoint !)
client = new TcpClient(new IPEndPoint(IPAddress.Parse(LOCALHOST), CLIENT_PORT));
connectionRequest = client.ConnectAsync(LOCALHOST, SERVER_PORT);
// If the previous try/catch is commented, you will stay stuck here,
// because the ConnectAsync has thrown an exception that will be raised only during the Wait() instruction
server = serverListener.AcceptTcpClient();
connectionRequest.Wait();
Console.WriteLine("press a key");
Console.ReadKey();
}
You may need to restart Visual Studio (or wait some time) if you trigger the bug and the program refuse to let you connect.

Your port is already in use. Run netstat and see. You'll find ports still open in the TIME_WAIT state.
Because you have not gracefully closed the sockets, the network layer must keep these ports open, in case the remote endpoint sends more data. Were it to do otherwise, the sockets could receive spurious data meant for something else, corrupting the data stream.
The right way to fix this is to close the connections gracefully (i.e. use the Socket.Shutdown() method). If you want to include a test involving the remote endpoint crashing, then you'll need to handle that scenario correctly as well. For one, you should set up an independent remote process that you can actually crash. For another, your server should correctly accommodate the situation by not trying to use the port again until an appropriate time has passed (i.e. the port is actually closed and is no longer in TIME_WAIT).
On that latter point, you may want to consider actually using the work-around you've discovered: TIME_WAIT involves the scenario where the status of the remote endpoint is unknown. If you send data, the network layer can detect the failed connection and effect the socket cleanup earlier.
For additional insights, see e.g.:
Port Stuck in Time_Wait
Reconnect to the server
How can I forcibly close a TcpListener
How do I prevent Socket/Port Exhaustion?
(But do not use the recommendation found among the answers to use SO_REUSEADDR/SocketOptionName.ReuseAddress…all that does is hide the problem, and can result in corrupted data in real-world code.)