GDAX/Coinbase API Level3 orderbook - skipping messages - c#

I'm using the GDAX API Websocket Stream to try and create a copy of the full LEVEL3 orderbook.
I've got a very simple implementation using WebSocketSharp and Im basically doing something like this.
private readonly WebSocket _webSocket = new WebSocket("wss://ws-feed.gdax.com");
_webSocket.OnMessage += WebSocket_OnMessage;
_webSocket.Connect();
_webSocket.Send(JsonConvert.SerializeObject(new BeginSubscriptionMessage()));
private void WebSocket_OnMessage(object sender, MessageEventArgs e)
{
var message = JsonConvert.DeserializeObject<BaseMessage>(e.Data);
switch (message.Type)
{
case "match": //A trade occurred between two orders.
MatchMessage matchMessage = JsonConvert.DeserializeObject<MatchMessage>(e.Data);
_receivedMatchQueue.Enqueue(matchMessage);
break;
case "received": //A valid order has been received and is now active. This message is emitted for every single valid order as soon as the matching engine receives it whether it fills immediately or not.
ReceivedMessage receivedMessage = JsonConvert.DeserializeObject<ReceivedMessage>(e.Data);
_receivedMessageQueue.Enqueue(receivedMessage);
break;
case "open": //The order is now open on the order book. This message will only be sent for orders which are not fully filled immediately. remaining_size will indicate how much of the order is unfilled and going on the book.
OpenMessage openMessage = JsonConvert.DeserializeObject<OpenMessage>(e.Data);
_receivedOpenQueue.Enqueue(openMessage);
break;
case "done": //The order is no longer on the order book. Sent for all orders for which there was a received message. This message can result from an order being canceled or filled.
DoneMessage doneMessage = JsonConvert.DeserializeObject<DoneMessage>(e.Data);
_receivedDoneQueue.Enqueue(doneMessage);
break;
case "change": //Existing order has been changed
ChangeMessage changeMessage = JsonConvert.DeserializeObject<ChangeMessage>(e.Data);
_receivedChangeQueue.Enqueue(changeMessage);
break;
case "activate": //Stop order placed
//Console.WriteLine("Stop Order Placed");
//ActivateMessage activateMessage = JsonConvert.DeserializeObject<ActivateMessage>(e.Data);
break;
case "subscriptions":
break;
case "ticker":
TickerMessage tickerMessage = JsonConvert.DeserializeObject<TickerMessage>(e.Data);
_receivedTickerQueue.Enqueue(tickerMessage);
break;
case "l2update":
break;
}
}
The problem I am running into is that when I look at the sequence numbers as received through both the RECEIVED and OPEN messages I can see they are not sequential which (based on the following information) suggests that messages are being skipped.
Basically you end up with something like this
Open Message SequenceId: 5359746354
Open Message SequenceId: 5359746358
Open Message SequenceId: 5359746361
Open Message SequenceId: 5359746363
Open Message SequenceId: 5359746365
Open Message SequenceId: 5359746370
Open Message SequenceId: 5359746372
I have tried testing this on Azure, just to make sure that it wasn't a bandwidth limitation on my end and the results were largely similar.
So given this, how is it possible to build a complete 'real-time' orderbook using the 'full' websocket stream if messages are dropped? Can I just safely ignore them? Or do I just somehow clear orphaned values?
Any advice from anyone having done something similar would be extremely appreciated.

Most likely messages are not dropped, you just have wrong impression of what "sequence" those sequence numbers represent.
As stated in api documentation
Most feed messages contain a sequence number. Sequence numbers are
increasing integer values for each product with every new message
being exactly 1 sequence number than the one before it.
So every channel has separate sequence number for each product (like ETH-USD), not for each message type (like "open" or "receive"). Suppose you subscribed to "full" channel, for products ETH-USD and ETH-EUR. Then you should expect sequence like this:
receive `ETH-EUR` X
open `ETH-EUR` X + 1
receive `ETH-USD` Y
done `ETH-EUR` X + 2
open `ETH-USD` Y + 1
For full channel, message types are: received, open, done, match, change, activate (note that ticker message belongs to different channel, so has separate sequence). So to ensure no messages are not skipped you need to track all those message types and ensure that last sequence number you received is exactly 1 less than new sequence number, per product (in case you subscribed to multiple products).
Proof code:
class Program {
private static readonly WebSocket _webSocket = new WebSocket("wss://ws-feed.gdax.com");
private static long _lastSequence = 0;
private static readonly HashSet<string> _expectedTypes = new HashSet<string>(
new[] { "received", "open", "done", "match", "change", "activate" });
static void Main(string[] args) {
var subMsg = "{\"type\": \"subscribe\",\"product_ids\": [\"ETH-USD\"],\"channels\": [\"full\"]}";
_webSocket.OnMessage += WebSocket_OnMessage;
_webSocket.Connect();
_webSocket.Send(subMsg);
Console.ReadKey();
}
private static void WebSocket_OnMessage(object sender, MessageEventArgs e) {
var message = JsonConvert.DeserializeObject<BaseMessage>(e.Data);
if (_expectedTypes.Contains(message.Type)) {
lock (typeof(Program)) {
if (_lastSequence == 0)
_lastSequence = message.Sequence;
else {
if (message.Sequence > _lastSequence + 1) {
Debugger.Break(); // never hits, so nothing is dropped
}
_lastSequence = message.Sequence;
}
}
}
}
}
public class BaseMessage {
[JsonProperty("type")]
public string Type { get; set; }
[JsonProperty("product_id")]
public string ProductId { get; set; }
[JsonProperty("sequence")]
public long Sequence { get; set; }
}

Related

RTCPeerConnection.CreateAnswer() fails

I am trying to set up the signaling between a browser client and unity client. the two clients use Webrtc and I am using a websocket to make the signaling happen. First, I am creating an offer on the click of a button in the browser. the offer reaches the c# client and the localConnection.setRemoteDescription works fine, then I need to create an answer. Instead the answer is an empty object recieved
{"type":0,"sdp":null}
Any Ideas if there is something wrong?
public void handleIncommingMessages(object sender, MessageEventArgs e) {
SignalingMessage message = JsonConvert.DeserializeObject<SignalingMessage>(e.Data);
switch (message.type){
case "offer": handleOffer(message); break;
case "answer": handleAnswer(message);break;
case "ice-candidate": handleIceCandidateMessage(message);break;
}
}
void handleOffer(SignalingMessage offerMessage)
{
localConnection.SetRemoteDescription(ref offerMessage.sessionDescription);
RTCSessionDescriptionAsyncOperation answer = localConnection.CreateAnswer();
Debug.Log(answer.Desc.sdp);
SignalingMessage answerMessage = new SignalingMessage {type = "answer" , sessionDescription = answer.Desc , iceCandidate = null};
socket.Send(JsonConvert.SerializeObject(answerMessage));
}
this is the implementation of the SignalingMessage class which represents any message to be sent over the websocket
public class SignalingMessage
{
public string type;
public RTCSessionDescription sessionDescription;
public RTCIceCandidate iceCandidate;
}

C# Threading and events for pin pad device

I am new in C# and currently working on the backend code to support PIN pad. Basically, my code
OpenDevice() -> RequestPIN()
-> key in PIN on PIN PAD -> GetResultPIN()
-> ConsolePrintOutPIN() -> Print keyed PIN on the Console
I don't know how to write thread for this, so that once the "Enter key" is hit on the device after PIN, the system would automatically rolls to function GetResultPIN(). So, with my elementary knowledge, I wrote the following codes using Console.ReadLine() to separate each procedure:
static void Main(string[] args)
{
// 1. Open PIN Pad device
OpenDevice();
Console.ReadLine();// to hold up from the previous procedure, it is *not* for input data purpose
// 2. Request PIN from PIN Pad device.
// On the PIN Pad device, it reads:
// "Key in the PIN: "
RequestPIN();
Console.ReadLine();// to hold up from the previous procedure, it is *not* for input data purpose
// 3. get PIN from the device
GetResultPIN();
// 4. Print out on the Console
ConsolePrintOutPIN();
Console.ReadLine();// to hold up from the previous procedure, it is *not* for input data purpose
}
Question: Can anyone give me any suggestions on how to use threading/event/delegate that can avoid using Console.ReadLine()?
As commended above, Console.ReadLine() is used just to stop the procedure (sorry about my naivety of using it this way....) Once I use Console.ReadLine(), between RequestPIN() and GetResult(), the system would at least wait for me to Key in the PIN from the PIN PAD (connected to the computer through USB, not from key board), and then I would hit any key on the keyboard to pass Console.ReadLine() and GetResultPIN() would be able to get my PIN number from PIN Pad.....the whole program works now, it is not customer ready, because it is very choppy, doesn't flow due to Console.ReadLine() I added.....
So ideally, all the method would flow together. Once the device is opened, RequestPIN() should show on the PIN Pad screen asking for PIN number, some one can key in and hit enter on PIN Pad and it naturally flow to GetResultPIN() and read the result, and then it prints the PIN on the console...`
or
if the person doesn't key in PIN, the device would wait for 30s and directly goes to GetResultPIN() and print out "0000" on the Console
I have looked up treading and delegate, but am not sure how to use them in this situation.... Thank you!
Reference: RequestPin() and GetResultPIN are listed down below:
mIPAD.requestPIN(waitTime, pinMsg, minLen, maxLen, tone, option, ",");
//This function wraps device command 0x04.
//It directs the device to prompt the user to enter a PIN
//by displaying one of five predetermined messages and playing
// a specified sound.
//The messages on the device’s screen look like the figures below.
//The event associated with this function is
//OnPINRequestCompleteEvent.
waitTime: Time the device should wait for the user to begin PIN entry
pinMsg: Message to display as a user prompt, such as "Enter PIN", "ReEnter PIN", "Verify PIN", etc
minLen and maxLen: minimum length and maximum length of PIN
tone: beeping tone option
Option: Verify PIN, not Verify PIN, ISO0 FOrmat, ISO3 Format
Output would be: an integer, 0: Success, Non-Zero: Error
public void GetResultPIN()
{
StringBuilder sb = new StringBuilder();
sb.Append(mIPAD.pin.KSN);
// Key Serial Number:
//a given number from the device, unique for each device
sb.Append("," + mIPAD.pin.EPB);
// EPB: encryption of PIN after Dubpt TripleDES,
// essentially, EPB is PIN
sb.Append("," + mIPAD.getStatusCode());
//status code: Zero is good/done
// None-Zero is Error
sb.Append("\r\n");
result = sb.ToString();
}
Basically, the GetResultPIN() returns a string of random code, for example:
9A00030000047A2000AB,AD781711481B08A2,0 when PIN is successful. If the pin-input part is skipped, it would return ,,0.
Really hard to know if this will work or not without hardware to play with...
This is the way I envisioned it working:
static void Main()
{
OpenDevice();
RequestPIN();
if (GetResultPIN())
{
// do something with the PIN:
var pin = mIPAD.pin.EPB;
// ...
}
else
{
Console.WriteLine("0000");
}
}
public static bool GetResultPIN()
{
TimeSpan timeout = TimeSpan.FromSeconds(30);
System.Diagnostics.Stopwatch SW = new System.Diagnostics.Stopwatch();
SW.Start();
while (mIPAD.getStatusCode() != 0 && SW.Elapsed < timeout)
{
System.Threading.Thread.Sleep(50); // small call to prevent CPU usage ramping to 100%
}
return (mIPAD.getStatusCode() == 0);
}
You can rewrite your api to:
make GetResultPIN() return a value
use this value as input for ConsolePrintOutPIN()
In GetResultPIN you need to make a Task To ReadYour Pin and wait for it.
See : https://msdn.microsoft.com/en-us/library/dd537610(v=vs.110).aspx
You can do something like this:
public string GetResultPIN()
{
StringBuilder sb = new StringBuilder();
sb.Append(mIPAD.pin.KSN);
// Key Serial Number:
//a given number from the device, unique for each device
sb.Append("," + mIPAD.pin.EPB);
// EPB: encryption of PIN after Dubpt TripleDES,
// essentially, EPB is PIN
sb.Append("," + mIPAD.getStatusCode());
//status code: Zero is good/done
// None-Zero is Error
sb.Append("\r\n");
Thread.Sleep(20*1000); // it is in milliseconds
return sb.ToString();
}
Thanks for posting... The solution is still not ideal....
I also did some more testing regarding the function RequestPIN(). I have the following four scenarios:
User finishes keying in PIN sooner than the waitTime goes out.
onPINRequestComplete :
OpStatus:0
KSN:9A00030000047A2000C8
EPB:39DED176D3EA40B9
..............................
User doesn't finish keying in PIN when the waitTime is going out.
onPINRequestComplete :
OpStatus:2
KSN:00000000000000000000
EPB:0000000000000000
..............................
User cancels the PIN pad option by pressing "Cancel X" key on the PIN Pad.
onPINRequestComplete :
OpStatus:1
KSN:00000000000000000000
EPB:0000000000000000
..............................
User doesn't key in PIN at all during the waitTime, and then waitTime goes out.
onPINRequestComplete :
OpStatus:2
KSN:00000000000000000000
EPB:0000000000000000
..............................
So, scenario 1 and 3 would require the thread to wake up right away, while 2 and 4 would require the thread to wake up when the waiTime goes out. So using Thread.sleep(20*1000) within GetResultPIN() would work perfectly for scenario 2 and 4. As for 1 and 3, the user has to wait for a long time....
On the other hand, I found some code about Event
Within Car.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace WaitOnTasksToComplete
{
class Car
{
public event Action OnChange;
private double speed;
public double Speed
{
get { return speed; }
set { speed = value;
if (speed >= 60)
{
if (OnChange != null)
{
OnChange();
}
}
}
}
}
}
Within Program.cs:
using System;
namespace WaitOnTasksToComplete
{
class Program
{
static void Main(string[] args)
{
Car c = new Car();
c.OnChange += C_OnChange;
c.Speed = 5;
c.Speed = 55;
c.Speed = 65;
c.Speed = 75;
}
private static void C_OnChange()
{
Console.WriteLine("Event fired: Car goes higher than 60 MPH.");
}
}
}
So, basically once the Car.speed jumps above 60, the alarm would show. I am considering borrowing the condition into my situation: Initialize OpStatus = -999. When OpStatus=0 or 1, keep executing GetResultPIN() and PrintMessagePIN(). If OpStatus=2 or others, keep waiting...
That is just my thoughts.... still have no clue how to implement it.... Any related ideas or suggestions would be appreciated.....
Ah, I figured out. I am basically using threading here. The main flow is OpenDevice()->RequestPIN()->Thread(()=>CheckOpStatus(getResultPIN)) -> Thread.Start(). Within the Thread, a loop is set to check every half second what the OpStatus is. Per my previous post, OpStatusis the output parameter of PIN Pad, zero- success; non-zero: failure. That said, the loop would proceed until either bool condition_WithinWaitTime or bool condition_NoKeyEvent breaks. After breaking out, invoke the getResultPIN and so on....
Here is my source code, as PIN input is one of my functions, the rest of which have very similar behavior in terms programming (request->manual operation->feedback), so I also included a delegate variable to represents all functions (card swiping, PIN, signature bla bla).
static void Main(string[] args)
{
OpenDevice();
EventGetPIN();
}
static void EventGetPIN()
{
myDel getResult = new myDel(GetResultPIN);
Thread thread1 = new Thread(() => CheckOpStatus(getResult));
myDel requestDel = new myDel(RequestPIN); requestDel();
thread1.Start();
}
static void CheckOpStatus(Delegate getResult)
{
int count = 0;
int checkingPeriod = 500;
int totalWaitTime = waitTime * 1000 + offsetTime;
string OpStatus;
string ksnStart = mIPAD.getKSN();
string ksn = ksnStart;
bool condition_WithinWaitTime = true;
bool condition_NoKeyEvent = true;
while (condition_WithinWaitTime & condition_NoKeyEvent)
{
count++;
OpStatus = mIPAD.getStatusCode().ToString();
ksn = mIPAD.getKSN();
//Console.WriteLine(OpStatus);
condition_WithinWaitTime = (count * checkingPeriod) < totalWaitTime;
condition_NoKeyEvent = (ksn == ksnStart);
Thread.Sleep(checkingPeriod);
}
getResult.DynamicInvoke();
}

using EWS Managed API with ASP.NET MVC

I'm trying to create a web app which does many things but the one that I'm currently focused in is the inbox count. I want to use EWS StreamSubscription so that I can get notification for each event and returns the total count of items in the inbox. How can I use this in terms of MVC? I did find some code from Microsoft tutorial that I was gonna test, but I just couldn't figure how I could use it in MVC world i.e. What's the model going to be, if model is the count then how does it get notified every time an event occurs in Exchange Server, etc.
Here's the code I downloaded from Microsoft, but just couldn't understand how I can convert the count to json and push it to client as soon as a new change event occurs. NOTE: This code is unchanged, so it doesn't return count, yet.
using System;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.Exchange.WebServices.Data;
namespace StreamingNotificationsSample
{
internal class Program
{
private static AutoResetEvent _Signal;
private static ExchangeService _ExchangeService;
private static string _SynchronizationState;
private static Thread _BackroundSyncThread;
private static StreamingSubscriptionConnection CreateStreamingSubscription(ExchangeService service,
StreamingSubscription subscription)
{
var connection = new StreamingSubscriptionConnection(service, 30);
connection.AddSubscription(subscription);
connection.OnNotificationEvent += OnNotificationEvent;
connection.OnSubscriptionError += OnSubscriptionError;
connection.OnDisconnect += OnDisconnect;
connection.Open();
return connection;
}
private static void SynchronizeChangesPeriodically()
{
while (true)
{
try
{
// Get all changes from the server and process them according to the business
// rules.
SynchronizeChanges(new FolderId(WellKnownFolderName.Inbox));
}
catch (Exception ex)
{
Console.WriteLine("Failed to synchronize items. Error: {0}", ex);
}
// Since the SyncFolderItems operation is a
// rather expensive operation, only do this every 10 minutes
Thread.Sleep(TimeSpan.FromMinutes(10));
}
}
public static void SynchronizeChanges(FolderId folderId)
{
bool moreChangesAvailable;
do
{
Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem calls.
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems, _SynchronizationState);
// Update the synchronization cookie
_SynchronizationState = changes.SyncState;
// Process all changes
foreach (var itemChange in changes)
{
// This example just prints the ChangeType and ItemId to the console
// LOB application would apply business rules to each item.
Console.Out.WriteLine("ChangeType = {0}", itemChange.ChangeType);
Console.Out.WriteLine("ChangeType = {0}", itemChange.ItemId);
}
// If more changes are available, issue additional SyncFolderItems requests.
moreChangesAvailable = changes.MoreChangesAvailable;
} while (moreChangesAvailable);
}
public static void Main(string[] args)
{
// Create new exchange service binding
// Important point: Specify Exchange 2010 with SP1 as the requested version.
_ExchangeService = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
Credentials = new NetworkCredential("user", "password"),
Url = new Uri("URL to the Exchange Web Services")
};
// Process all items in the folder on a background-thread.
// A real-world LOB application would retrieve the last synchronization state first
// and write it to the _SynchronizationState field.
_BackroundSyncThread = new Thread(SynchronizeChangesPeriodically);
_BackroundSyncThread.Start();
// Create a new subscription
var subscription = _ExchangeService.SubscribeToStreamingNotifications(new FolderId[] {WellKnownFolderName.Inbox},
EventType.NewMail);
// Create new streaming notification conection
var connection = CreateStreamingSubscription(_ExchangeService, subscription);
Console.Out.WriteLine("Subscription created.");
_Signal = new AutoResetEvent(false);
// Wait for the application to exit
_Signal.WaitOne();
// Finally, unsubscribe from the Exchange server
subscription.Unsubscribe();
// Close the connection
connection.Close();
}
private static void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
// Cast the sender as a StreamingSubscriptionConnection object.
var connection = (StreamingSubscriptionConnection) sender;
// Ask the user if they want to reconnect or close the subscription.
Console.WriteLine("The connection has been aborted; probably because it timed out.");
Console.WriteLine("Do you want to reconnect to the subscription? Y/N");
while (true)
{
var keyInfo = Console.ReadKey(true);
{
switch (keyInfo.Key)
{
case ConsoleKey.Y:
// Reconnect the connection
connection.Open();
Console.WriteLine("Connection has been reopened.");
break;
case ConsoleKey.N:
// Signal the main thread to exit.
Console.WriteLine("Terminating.");
_Signal.Set();
break;
}
}
}
}
private static void OnNotificationEvent(object sender, NotificationEventArgs args)
{
// Extract the item ids for all NewMail Events in the list.
var newMails = from e in args.Events.OfType<ItemEvent>()
where e.EventType == EventType.NewMail
select e.ItemId;
// Note: For the sake of simplicity, error handling is ommited here.
// Just assume everything went fine
var response = _ExchangeService.BindToItems(newMails,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.DateTimeReceived,
ItemSchema.Subject));
var items = response.Select(itemResponse => itemResponse.Item);
foreach (var item in items)
{
Console.Out.WriteLine("A new mail has been created. Received on {0}", item.DateTimeReceived);
Console.Out.WriteLine("Subject: {0}", item.Subject);
}
}
private static void OnSubscriptionError(object sender, SubscriptionErrorEventArgs args)
{
// Handle error conditions.
var e = args.Exception;
Console.Out.WriteLine("The following error occured:");
Console.Out.WriteLine(e.ToString());
Console.Out.WriteLine();
}
}
}
I just want to understand the basic concept as in what can be model, and where can I use other functions.
Your problem is that you are confusing a service (EWS) with your applications model. They are two different things. Your model is entirely in your control, and you can do whatever you want with it. EWS is outside of your control, and is merely a service you call to get data.
In your controller, you call the EWS service and get the count. Then you populate your model with that count, then in your view, you render that model property. It's really that simple.
A web page has no state. It doesn't get notified when things change. You just reload the page and get whatever the current state is (ie, whatever the current count is).
In more advanced applications, like Single Page Apps, with Ajax, you might periodically query the service in the background. Or, you might have a special notification service that uses something like SignalR to notify your SPA of a change, but these concepts are far more advanced than you currently are. You should probably develop your app as a simple stateless app first, then improve it to add ajax functionality or what not once you have a better grasp of things.
That's a very broad question without a clear-cut answer. Your model could certainly have a "Count" property that you could update. The sample code you found would likely be used by your controller.

Random message "The thread has exited with code 0" in managed threads

I've a got a problem with the infamous message "The thread xxx has exited with code 0 (0x0)".
In my code I have a main class called "Load" that starts with a Windows Form load event:
public class Load
{
public Load()
{
Device[] devices = GetDevices(); // Get an array of devices from an external source
for (int i = 0; i < devices.Length; i++)
{
DeviceDiagnosticCtrl deviceDiagnostic = new DeviceDiagnosticCtrl(devices[i].name);
}
}
}
Inside the constructor, for each generic device read from an external source, I initialize a custom diagnostic class that runs a thread:
public class DeviceDiagnosticCtrl
{
private Thread diagnosticController;
private volatile bool diagnosticControllerIsRunning = false;
public DeviceDiagnosticCtrl(string _name)
{
// Thread initialization
this.diagnosticController = new Thread(new ThreadStart(this.CheckDiagnostic));
this.diagnosticController.Start();
this.diagnosticControllerIsRunning = true;
}
private void CheckDiagnostic()
{
while (this.diagnosticControllerIsRunning)
{
try
{
// Custom 'Poll' message class used to request diagnostic to specific device
Poll poll = new Poll();
// Generic Message result to diagnostic request
IGenericMessage genericResult;
// Use a custom driver to send diagnostic request
SendSyncMsgResult res = this.customDriver.SendSyncMessage(poll, out genericResult);
switch (res)
{
case SendSyncMessageResult.GOOD:
{
// Log result
}
break;
case SendSyncMessageResult.EXCEPTION:
{
// Log result
}
break;
}
Thread.Sleep(this.customDriver.PollScantime);
}
catch (Exception ex)
{
// Loggo exception
}
}
}
}
When I run the above code in debug mode I always read 8 devices from external source, and for each of them I continuously run a managed thread to retrieve diagnostic.
My problem is that randomly one or more of the 8 threads I expect from the code above exit with code 0, without any exception.
I've started/restarted the code in debug mode a lot of time, and almost everytime one of the thread exits.
I've read somewhere (i.e. this SO question) that it could depends of Garbage Collector action, but I'm not too sure if this is my case - and how to prevent it.
Do someone see something strange/wrong in the sample code I posted above? Any suggestion?
'while (this.diagnosticControllerIsRunning)' is quite likely to fail immediate, in which case the thread drops out. It's no good starting the thread and THEN setting 'this.diagnosticControllerIsRunning = true;' - you're quite likely to be too late.
Bolt/stable-door. Something like:
do{
lengthyStuff with Sleep() in it
}
while (this.diagnosticControllerRun);
Copied from Here
Right click in the Output window when you're running your program and
uncheck all of the messages you don't want to see (like Thread Exit
messages).

Multiple publishers sending concurrent messages to a single subscriber in Retlang?

I need to build an application where some number of instances of an object are generating "pulses", concurrently. (Essentially this just means that they are incrementing a counter.) I also need to track the total counters for each object. Also, whenever I perform a read on a counter, it needs to be reset to zero.
So I was talking to a guy at work, and he mentioned Retlang and message-based concurrency, which sounded super interesting. But obviously I am very new to the concept. So I've built a small prototype, and I get the expected results, which is awesome - but I'm not sure if I've potentially made some logical errors and left the software open to bugs, due to my inexperience with Retlang and concurrent programming in general.
First off, I have these classes:
public class Plc {
private readonly IChannel<Pulse> _channel;
private readonly IFiber _fiber;
private readonly int _pulseInterval;
private readonly int _plcId;
public Plc(IChannel<Pulse> channel, int plcId, int pulseInterval) {
_channel = channel;
_pulseInterval = pulseInterval;
_fiber = new PoolFiber();
_plcId = plcId;
}
public void Start() {
_fiber.Start();
// Not sure if it's safe to pass in a delegate which will run in an infinite loop...
// AND use a shared channel object...
_fiber.Enqueue(() => {
SendPulse();
});
}
private void SendPulse() {
while (true) {
// Not sure if it's safe to use the same channel object in different
// IFibers...
_channel.Publish(new Pulse() { PlcId = _plcId });
Thread.Sleep(_pulseInterval);
}
}
}
public class Pulse {
public int PlcId { get; set; }
}
The idea here is that I can instantiate multiple Plcs, pass each one the same IChannel, and then have them execute the SendPulse function concurrently, which would allow each one to publish to the same channel. But as you can see from my comments, I'm a little skeptical that what I'm doing is actually legit. I'm mostly worried about using the same IChannel object to Publish in the context of different IFibers, but I'm also worried about never returning from the delegate that was passed to Enqueue. I'm hoping some one can provide some insight as to how I should be handling this.
Also, here is the "subscriber" class:
public class PulseReceiver {
private int[] _pulseTotals;
private readonly IFiber _fiber;
private readonly IChannel<Pulse> _channel;
private object _pulseTotalsLock;
public PulseReceiver(IChannel<Pulse> channel, int numberOfPlcs) {
_pulseTotals = new int[numberOfPlcs];
_channel = channel;
_fiber = new PoolFiber();
_pulseTotalsLock = new object();
}
public void Start() {
_fiber.Start();
_channel.Subscribe(_fiber, this.UpdatePulseTotals);
}
private void UpdatePulseTotals(Pulse pulse) {
// This occurs in the execution context of the IFiber.
// If we were just dealing with the the published Pulses from the channel, I think
// we wouldn't need the lock, since I THINK the published messages would be taken
// from a queue (i.e. each Plc is publishing concurrently, but Retlang enqueues
// the messages).
lock(_pulseTotalsLock) {
_pulseTotals[pulse.PlcId - 1]++;
}
}
public int GetTotalForPlc(int plcId) {
// However, this access takes place in the application thread, not in the IFiber,
// and I think there could potentially be a race condition here. I.e. the array
// is being updated from the IFiber, but I think I'm reading from it and resetting values
// concurrently in a different thread.
lock(_pulseTotalsLock) {
if (plcId <= _pulseTotals.Length) {
int currentTotal = _pulseTotals[plcId - 1];
_pulseTotals[plcId - 1] = 0;
return currentTotal;
}
}
return -1;
}
}
So here, I am reusing the same IChannel that was given to the Plc instances, but having a different IFiber subscribe to it. Ideally then I could receive the messages from each Plc, and update a single private field within my class, but in a thread safe way.
From what I understand (and I mentioned in my comments), I think that I would be safe to simply update the _pulseTotals array in the delegate which I gave to the Subscribe function, because I would receive each message from the Plcs serially.
However, I'm not sure how best to handle the bit where I need to read the totals and reset them. As you can see from the code and comments, I ended up wrapping a lock around any access to the _pulseTotals array. But I'm not sure if this is necessary, and I would love to know a) if it is in fact necessary to do this, and why, or b) the correct way to implement something similar.
And finally for good measure, here's my main function:
static void Main(string[] args) {
Channel<Pulse> pulseChannel = new Channel<Pulse>();
PulseReceiver pulseReceiver = new PulseReceiver(pulseChannel, 3);
pulseReceiver.Start();
List<Plc> plcs = new List<Plc>() {
new Plc(pulseChannel, 1, 500),
new Plc(pulseChannel, 2, 250),
new Plc(pulseChannel, 3, 1000)
};
plcs.ForEach(plc => plc.Start());
while (true) {
Thread.Sleep(10000);
Console.WriteLine(string.Format("Plc 1: {0}\nPlc 2: {1}\nPlc 3: {2}\n", pulseReceiver.GetTotalForPlc(1), pulseReceiver.GetTotalForPlc(2), pulseReceiver.GetTotalForPlc(3)));
}
}
I instantiate one single IChannel, pass it to everything, where internally the Receiver subscribes with an IFiber, and where the Plcs use IFibers to "enqueue" a non-returning method which continually publishes to the channel.
Again, the console output looks exactly like I would expect it to look, i.e. I see 20 "pulses" for Plc 1 after waiting 10 seconds. And the resetting of the counters after a read also seems to work, i.e. Plc 1 has 20 "pulses" after each 10 second increment. But that doesn't reassure me that I haven't overlooked something important.
I'm really excited to learn a bit more about Retlang and concurrent programming techniques, so hopefuly someone has the time to sift through my code and offer some suggestions for my specific concerns, or else even a different design based on my requirements!

Categories

Resources