I want to get data from this wss://ws-feed.gdax.com
I don't know anything about websocket. I am reading some tutorials and it uses terms like websocket server, TCP, etc which I have no idea about. Can anyone please advice how should I proceed, how to write a c# code to get data from the above.
This is the document which I reading to fetch real time data - https://docs.gdax.com/#websocket-feed
Started by creating a window app. Read here that the WebSocketSharp library can be used to connect WebSockets so installed it and so far written this code:
using (var ws = new WebSocket("wss://ws-feed.gdax.com"))
{
ws.Connect();
string json = "{\"type\": \"subscribe\", \"product_ids\": [\"BTC-USD\"]}";
ws.Send(json); //gives error -Function evaluation disabled because a previous function evaluation timed out. You must continue execution to reenable function evaluation.
}
Any help would be much appreciated.
ClientWebSocket socket = new ClientWebSocket();
Task task = socket.ConnectAsync(new Uri("wss://ws-feed.gdax.com"), CancellationToken.None);
task.Wait();
Thread readThread = new Thread(
delegate(object obj)
{
byte[] recBytes = new byte[1024];
while (true) {
ArraySegment<byte> t = new ArraySegment<byte>(recBytes);
Task<WebSocketReceiveResult> receiveAsync = socket.ReceiveAsync(t, CancellationToken.None);
receiveAsync.Wait();
string jsonString = Encoding.UTF8.GetString(recBytes);
Console.Out.WriteLine("jsonString = {0}", jsonString);
recBytes = new byte[1024];
}
});
readThread.Start();
string json = "{\"product_ids\":[\"btc-usd\"],\"type\":\"subscribe\"}";
byte[] bytes = Encoding.UTF8.GetBytes(json);
ArraySegment<byte> subscriptionMessageBuffer = new ArraySegment<byte>(bytes);
socket.SendAsync(subscriptionMessageBuffer, WebSocketMessageType.Text, true, CancellationToken.None);
Console.ReadLine();
Basically, what you do is in the right direction. Just check websocket state before you read the message.
The below example is from https://github.com/sefbkn/gdax.netcore, with little modification
var webSocketClient = new ClientWebSocket();
var cancellationToken = new CancellationToken();
await webSocketClient.ConnectAsync(new Uri("wss://ws-feed.gdax.com"), cancellationToken).ConfigureAwait(false);
if (webSocketClient.State == WebSocketState.Open)
{
var requestString = JsonConvert.SerializeObject(new {
type = "subscribe",
product_ids = new[]{"ETH-EUR"},
channels = new[]{"ticker"}
});
var requestBytes = UTF8Encoding.UTF8.GetBytes(requestString);
var subscribeRequest = new ArraySegment<byte>(requestBytes);
var sendCancellationToken = new CancellationToken();
await webSocketClient.SendAsync(subscribeRequest, WebSocketMessageType.Text, true, sendCancellationToken).ConfigureAwait(false);
while (webSocketClient.State == WebSocketState.Open)
{
var receiveCancellationToken = new CancellationToken();
using(var stream = new MemoryStream(1024))
{
var receiveBuffer = new ArraySegment<byte>(new byte[1024*8]);
WebSocketReceiveResult webSocketReceiveResult;
do
{
webSocketReceiveResult = await webSocketClient.ReceiveAsync(receiveBuffer, receiveCancellationToken).ConfigureAwait(false);
await stream.WriteAsync(receiveBuffer.Array, receiveBuffer.Offset, receiveBuffer.Count);
} while(!webSocketReceiveResult.EndOfMessage);
var message = stream.ToArray().Where(b => b != 0).ToArray();
messageReceived(Encoding.ASCII.GetString(message, 0, message.Length));
}
}
}
Related
I don't understand why the response comes only if I use CompleteAsync().
using var call = _apiClient.GetToken(headers: _threadContext.Metadata, deadline: DateTime.UtcNow.AddSeconds(5));
var keyReq = new GetTokenRequest()
{
Key = publicKey
};
var readTask = Task.Run(async () =>
{
await foreach(var message in call.ResponseStream.ReadAllAsync())
{
if (message.Challenge != null)
{
var challenge = message.Challenge.ToByteArray();
var signature = await VerifySignature(challenge);
var signReq = new GetTokenRequest
{
Signature = ByteString.CopyFrom(signature)
};
await call.RequestStream.WriteAsync(signReq);
await call.RequestStream.CompleteAsync();
}
else if (message.Token != null)
{
token = message.Token;
}
}
});
await call.RequestStream.WriteAsync(keyReq);
await readTask;
If I change the end with this, I receive messages but in the response the next WriteAsync fails because the stream is closed.
await call.RequestStream.WriteAsync(keyReq);
await call.RequestStream.CompleteAsync();
await readTask;
And if I doesn't complete the request, response message never comes.
Any idea ?
Note: the server is in go.
This code doesn't work with Grpc.Net.Client.Web only. With classic SocketHttpHandler it's ok. Problem is solved. thanks.
The example shown on the following page doesn't work:
Using c# ClientWebSocket with streams
It hangs on the following line:
await ws.ConnectAsync(serverUri, CancellationToken.None);
It appears the connection is not made.
Please indicate the simplest modification to make the following code work. I do not wish to use any 3rd party tools or libraries.
private static async Task DoClientWebSocket()
{
using (ClientWebSocket ws = new ClientWebSocket())
{
Uri serverUri = new Uri("wss://echo.websocket.org/");
await ws.ConnectAsync(serverUri, CancellationToken.None);
while (ws.State == WebSocketState.Open)
{
string msg = "hello123";
ArraySegment<byte> bytesToSend = new ArraySegment<byte>(Encoding.UTF8.GetBytes(msg));
await ws.SendAsync(bytesToSend, WebSocketMessageType.Text, true, CancellationToken.None);
ArraySegment<byte> bytesReceived = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = await ws.ReceiveAsync(bytesReceived, CancellationToken.None);
Console.WriteLine(Encoding.UTF8.GetString(bytesReceived.Array, 0, result.Count));
}
}
}
You are correct. You don't need to add any header in order to use wss://echo.websocket.org/. Your code run just fine at my end. But I'll suggest one improvement to include timeout for your ConnectAsync, SendAsync and ReceiveAsync calls so that it do not get stuck for long.
I have restricted code to call SendAsync to just 5 times so that its easier to verify output.
[Edited:] Include logic to receive complete response by calling `ReceiveAsync multiple times.
private static async Task DoClientWebSocket()
{
using (ClientWebSocket ws = new ClientWebSocket())
{
Uri serverUri = new Uri("wss://echo.websocket.org/");
//Implementation of timeout of 5000 ms
var source = new CancellationTokenSource();
source.CancelAfter(5000);
await ws.ConnectAsync(serverUri, source.Token);
var iterationNo = 0;
// restricted to 5 iteration only
while (ws.State == WebSocketState.Open && iterationNo++ < 5)
{
string msg = "hello0123456789123456789123456789123456789123456789123456789";
ArraySegment<byte> bytesToSend =
new ArraySegment<byte>(Encoding.UTF8.GetBytes(msg));
await ws.SendAsync(bytesToSend, WebSocketMessageType.Text,
true, source.Token);
//Receive buffer
var receiveBuffer = new byte[200];
//Multipacket response
var offset = 0;
var dataPerPacket = 10; //Just for example
while (true)
{
ArraySegment<byte> bytesReceived =
new ArraySegment<byte>(receiveBuffer, offset, dataPerPacket);
WebSocketReceiveResult result = await ws.ReceiveAsync(bytesReceived,
source.Token);
//Partial data received
Console.WriteLine("Data:{0}",
Encoding.UTF8.GetString(receiveBuffer, offset,
result.Count));
offset += result.Count;
if (result.EndOfMessage)
break;
}
Console.WriteLine("Complete response: {0}",
Encoding.UTF8.GetString(receiveBuffer, 0,
offset));
}
}
}
static void Main(string[] args)
{
var taskWebConnect = Task.Run(() => DoClientWebSocket());
taskWebConnect.Wait();
}
Output on command prompt:
Data:hello01234
Data:5678912345
Data:6789123456
Data:7891234567
Data:8912345678
Data:9123456789
Complete response: hello0123456789123456789123456789123456789123456789123456789
I am receiving JSON through a websocket. At least: I am partially. Using an online websocket service I receive the full JSON response (all the HTML markup is ignored). When I look at the JSON that I receive in my console I can see the HTML markup (viewing it with the HTML viewer during debugging removes the HTML) but it ends abruptly (incomplete data).
My buffer has plenty of space and I am using async-await to (supposedly) wait for the entire response to come in before continuing.
private async Task Receive()
{
var buffer = new byte[4096 * 20];
while (_socket.State == WebSocketState.Open)
{
var response = await _socket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
if (response.MessageType == WebSocketMessageType.Close)
{
await
_socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Close response received",
CancellationToken.None);
}
else
{
var result = Encoding.UTF8.GetString(buffer);
var a = buffer[1000];
var b = buffer[10000];
var c = buffer[50000];
var d = buffer[81000];
Console.WriteLine(result);
var responseObject = JsonConvert.DeserializeObject<Response>(result, _requestParameters.ResponseDataType);
OnSocketReceive.Invoke(this, new SocketEventArgs {Response = responseObject });
buffer = new byte[4096 * 20];
}
}
}
Things to note: The buffer is perfectly big enough and b, c and d are never filled. I should also note that this only happens for the 1-questions-newest-tag-java request, 155-questions-active works perfectly fine.
After doing some digging I have found that response.CloseStatus and response.CloseStatusDescription are always null, response.Count is always 1396 (copy-pasting the result in Word does show that there are always 1396 characters) and response.EndOfMessage is false.
Digging through some source code I have found that the DefaultReceiveBufferSize is 16 * 1024 (big enough) and the WebSocketGetDefaultKeepAliveInterval() refers to an external implementation (but the debugger shows 00:00:30).
It is not a matter of timeout since the debugger halts at the same moment the online service receives its response.
Why is my method continuing to execute when the socket has not yet received all data?
Just to complete #Noseratio response, the code would be something like this:
ArraySegment<Byte> buffer = new ArraySegment<byte>(new Byte[8192]);
WebSocketReceiveResult result= null;
using (var ms = new MemoryStream())
{
do
{
result = await socket.ReceiveAsync(buffer, CancellationToken.None);
ms.Write(buffer.Array, buffer.Offset, result.Count);
}
while (!result.EndOfMessage);
ms.Seek(0, SeekOrigin.Begin);
if (result.MessageType == WebSocketMessageType.Text)
{
using (var reader = new StreamReader(ms, Encoding.UTF8))
{
// do stuff
}
}
}
Cheers.
I might be wrong, but I don't think you're always supposed to receive a complete WebSocket message at once. The server may be sending the message in chunks (that'd correspond to calling SendAsync with endOfMessage: false).
So, do await _socket.ReceiveAsync() in a loop and accumulate the received chunks, until WebSocketReceiveResult.EndOfMessage is true or an error has occured.
On a side note, you probably should be using WebSocket.CreateClientBuffer instead of new ArraySegment<byte>(buffer).
// Read the bytes from the web socket and accumulate all into a list.
var buffer = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = null;
var allBytes = new List<byte>();
do
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
for (int i = 0; i < result.Count; i++)
{
allBytes.Add(buffer.Array[i]);
}
}
while (!result.EndOfMessage);
// Optional step to convert to a string (UTF-8 encoding).
var text = Encoding.UTF8.GetString(allBytes.ToArray(), 0, allBytes.Count);
Following Noseratio's answer I have implemented a temporary buffer that will construct the data of the entire message.
var temporaryBuffer = new byte[BufferSize];
var buffer = new byte[BufferSize * 20];
int offset = 0;
WebSocketReceiveResult response;
while (true)
{
response = await _socket.ReceiveAsync(
new ArraySegment<byte>(temporaryBuffer),
CancellationToken.None);
temporaryBuffer.CopyTo(buffer, offset);
offset += response.Count;
temporaryBuffer = new byte[BufferSize];
if (response.EndOfMessage)
{
break;
}
}
Full implementation here
Try this:
try
{
WebSocketReceiveResult result;
string receivedMessage = "";
var message = new ArraySegment<byte>(new byte[4096]);
do
{
result = await WebSocket.ReceiveAsync(message, DisconectToken);
if (result.MessageType != WebSocketMessageType.Text)
break;
var messageBytes = message.Skip(message.Offset).Take(result.Count).ToArray();
receivedMessage += Encoding.UTF8.GetString(messageBytes);
}
while (!result.EndOfMessage);
if (receivedMessage != "{}" && !string.IsNullOrEmpty(receivedMessage))
{
ResolveWebSocketResponse.Invoke(receivedMessage, Connection);
Console.WriteLine("Received: {0}", receivedMessage);
}
}
catch (Exception ex)
{
var mes = ex.Message;
}
I have the following c# code where i am sending a series of requests and reponses
public static async Task AuthenticateQvpx2()
{
var handshake = new Handshake();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
await Task.WhenAll(Receive(_webSocket), Send(_webSocket, buffer));
}
}
The async functions Send and Receive, has the following code.
await webSocket.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true, CancellationToken.None);
var result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
I wish to collect the requests and responses into an array/ any form of data type.
I having trouble as I am not particularly sure of what i should do next?
I wish to collect the requests and responses
It's kind of odd to collect the requests, as that data is already right there (in the buffer variable).
Assuming you meant that you just need the response, you can do that using await:
public static async Task AuthenticateQvpx2()
{
var handshake = new Handshake();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
var receiveTask = Receive(_webSocket);
await Task.WhenAll(receiveTask, Send(_webSocket, buffer));
var response = await receiveTask;
}
}
Not sure based on your snippets what type of Tasks your Send and Receive return but generally you can get your results from multiple tasks after using Task.WhenAll using LINQ this way:
var handshake = new Handshake();
List<Task<WebsocketReceiveResult>> tasks = newList<Task<WebsocketReceiveResult>>();
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
tasks.Add(webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None));
}
await Task.WhenAll(tasks);
var resultArray = tasks.Select(t => t.Result).ToArray();
Try something like
ConcurrentDictionary<Guid, (System.Byte[], WebSocketReceiveResult)> x = new ConcurrentDictionary<Guid, (byte[], WebSocketReceiveResult)>();
Generate GUID in your foreach a pass it to your methods:
foreach (var request in handshake.AutheticateStrings)
{
var buffer = _encoder.GetBytes(request);
var guid = Guid.NewGuid();
await Task.WhenAll(Receive(_webSocket, guid), Send(_webSocket, buffer, guid));
}
Then you can work with dictionary from within your Receive and Send methods in natural way.
void Send(WebSocket webSocket, byte[] buffer, Guid guid)
{
x.GetOrAdd(guid, new ValueTuple<System.Byte[], WebSocketReceiveResult>(buffer, null));
await webSocket.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true, CancellationToken.None)
}
void Receive(WebSocket webSocket, Guid guid)
{
var result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
x[guid].Item2 = result;
}
I am receiving JSON through a websocket. At least: I am partially. Using an online websocket service I receive the full JSON response (all the HTML markup is ignored). When I look at the JSON that I receive in my console I can see the HTML markup (viewing it with the HTML viewer during debugging removes the HTML) but it ends abruptly (incomplete data).
My buffer has plenty of space and I am using async-await to (supposedly) wait for the entire response to come in before continuing.
private async Task Receive()
{
var buffer = new byte[4096 * 20];
while (_socket.State == WebSocketState.Open)
{
var response = await _socket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
if (response.MessageType == WebSocketMessageType.Close)
{
await
_socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Close response received",
CancellationToken.None);
}
else
{
var result = Encoding.UTF8.GetString(buffer);
var a = buffer[1000];
var b = buffer[10000];
var c = buffer[50000];
var d = buffer[81000];
Console.WriteLine(result);
var responseObject = JsonConvert.DeserializeObject<Response>(result, _requestParameters.ResponseDataType);
OnSocketReceive.Invoke(this, new SocketEventArgs {Response = responseObject });
buffer = new byte[4096 * 20];
}
}
}
Things to note: The buffer is perfectly big enough and b, c and d are never filled. I should also note that this only happens for the 1-questions-newest-tag-java request, 155-questions-active works perfectly fine.
After doing some digging I have found that response.CloseStatus and response.CloseStatusDescription are always null, response.Count is always 1396 (copy-pasting the result in Word does show that there are always 1396 characters) and response.EndOfMessage is false.
Digging through some source code I have found that the DefaultReceiveBufferSize is 16 * 1024 (big enough) and the WebSocketGetDefaultKeepAliveInterval() refers to an external implementation (but the debugger shows 00:00:30).
It is not a matter of timeout since the debugger halts at the same moment the online service receives its response.
Why is my method continuing to execute when the socket has not yet received all data?
Just to complete #Noseratio response, the code would be something like this:
ArraySegment<Byte> buffer = new ArraySegment<byte>(new Byte[8192]);
WebSocketReceiveResult result= null;
using (var ms = new MemoryStream())
{
do
{
result = await socket.ReceiveAsync(buffer, CancellationToken.None);
ms.Write(buffer.Array, buffer.Offset, result.Count);
}
while (!result.EndOfMessage);
ms.Seek(0, SeekOrigin.Begin);
if (result.MessageType == WebSocketMessageType.Text)
{
using (var reader = new StreamReader(ms, Encoding.UTF8))
{
// do stuff
}
}
}
Cheers.
I might be wrong, but I don't think you're always supposed to receive a complete WebSocket message at once. The server may be sending the message in chunks (that'd correspond to calling SendAsync with endOfMessage: false).
So, do await _socket.ReceiveAsync() in a loop and accumulate the received chunks, until WebSocketReceiveResult.EndOfMessage is true or an error has occured.
On a side note, you probably should be using WebSocket.CreateClientBuffer instead of new ArraySegment<byte>(buffer).
// Read the bytes from the web socket and accumulate all into a list.
var buffer = new ArraySegment<byte>(new byte[1024]);
WebSocketReceiveResult result = null;
var allBytes = new List<byte>();
do
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
for (int i = 0; i < result.Count; i++)
{
allBytes.Add(buffer.Array[i]);
}
}
while (!result.EndOfMessage);
// Optional step to convert to a string (UTF-8 encoding).
var text = Encoding.UTF8.GetString(allBytes.ToArray(), 0, allBytes.Count);
Following Noseratio's answer I have implemented a temporary buffer that will construct the data of the entire message.
var temporaryBuffer = new byte[BufferSize];
var buffer = new byte[BufferSize * 20];
int offset = 0;
WebSocketReceiveResult response;
while (true)
{
response = await _socket.ReceiveAsync(
new ArraySegment<byte>(temporaryBuffer),
CancellationToken.None);
temporaryBuffer.CopyTo(buffer, offset);
offset += response.Count;
temporaryBuffer = new byte[BufferSize];
if (response.EndOfMessage)
{
break;
}
}
Full implementation here
Try this:
try
{
WebSocketReceiveResult result;
string receivedMessage = "";
var message = new ArraySegment<byte>(new byte[4096]);
do
{
result = await WebSocket.ReceiveAsync(message, DisconectToken);
if (result.MessageType != WebSocketMessageType.Text)
break;
var messageBytes = message.Skip(message.Offset).Take(result.Count).ToArray();
receivedMessage += Encoding.UTF8.GetString(messageBytes);
}
while (!result.EndOfMessage);
if (receivedMessage != "{}" && !string.IsNullOrEmpty(receivedMessage))
{
ResolveWebSocketResponse.Invoke(receivedMessage, Connection);
Console.WriteLine("Received: {0}", receivedMessage);
}
}
catch (Exception ex)
{
var mes = ex.Message;
}