I have an HTTP server written in C# based off the HttpListenerContext class. The server is for processing binary log files and converting them to text, and can take quite a long time to do the conversion. I would like to indicate progress back to the user, but I am unsure on the best way to do this. On the server side, in handling my HTTP request, I essentially have this function:
public async Task HandleRequest()
{
try
{
await ProcessRequest();
}
catch (HttpListenerException)
{
// Something happened to the http connection, don't try to send anything
}
catch (Exception e)
{
SendFailureResponse(500);
}
}
Currently, ProcessRequest() sends the HTML response when finished, but I would like to essentially add the IProgress interface to the function and somehow indicate that progress back to the Web client. What is the best way to do this?
One way of doing it would be to store progress on server side and periodically pull the information from client.
However, if you want the server to notify the client ( push ), then you will need to implement some kind of bi-directional communication between the server and client (I am currently using ASP.NET Web API and SignalR to achieve this at work).
Here is what I got I'll try to explain and I hope you notice its not FULL FULL complete, you'll have to understand the logic behind this and accept or not as a plausible option.
The Method: Set a custom object to store progress of your ongoing operations, make a global static list containing this metadata. Notice how I track them with Ids: I don't store that on DB, the natural act of instantiating the class will auto_increment their Id.
Then, you can add a new controller to respond the progress of a particular ongoing process.
Now that you have a controller to respond the progress of an ongoing process by Id, you can create a javascript timer to call it and update the DOM.
When creating your process, dont hold the htmlrequest until its over, open a background operation instead and just respond with the newly created ProgressTracker.Id, through that class/list you can keep track of the progress and reply accordingly.
As said in another answer, when an operation finishes you can send a push notification and the clientside javascript will interrupt the timers and proceed to the next view/result/page, or you can increment the looping timer to detect when its done and call the results from another controller. (this way you can avoid using push if needed.)
Here is the partial code:
public class ProgressTracker {
private static GlobalIdProvider = 0;
public int _id = ++GlobalIdProvider;
public int Id { get { return _id; } }
bool IsInProgress = false;
bool IsComplete = false;
float Progress;
public YourProgressObject Data;
}
public class GlobalStatic {
public static List<ProgressTracker> FooOperations = new List<ProgressTracker>();
}
public class SomeWebApiController {
[HttpGet]
[Authorize]
public HttpResponseMessage GetProgress(int Id) {
var query = (from a in GlobalStatic.FooOperations where a.Id==Id select a);
if(!query.Any()) {
return Request.CreateResponse(HttpStatusCode.NotFound, "No operation with this Id found.");
} else {
return Request.CreateResponse(HttpStatusCode.Ok, query.First());
}
}
}
// this is javascript
// ... Your code until it starts the process.
// You'll have to get the ProgressTracker Id from the server somehow.
var InProgress = true;
window.setTimeout(function(e) {
var xmlhttp = new XMLHttpRequest();
var url = "<myHostSomething>/SomeWebApiController/GetProgress?Id="+theId;
xmlhttp.setRequestHeader("Authentication","bearer "+localStorage.getItem("access_token"));
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
var data = JSON.parse(xmlhttp.responseText);
updateProgressBar(data);
}
}
xmlhttp.open("GET", url, true);
xmlhttp.send();
function updateProgressBar(data) {
document.getElementById("myProgressText").innerHTML = data.Progress;
}
}, 3000);
Disclaimer: If my javascript is shitty, pardon me but I'm too used to using jQuery and all this fancy stuff x_x
Related
I have successfully implemented the trivial example documented here, which demonstrates how to transmit server-sent events to a client, but I'm having difficulty going beyond the example.
The server I wrote (IIS/ASP.Net/C#/Web API 2) can handle multiple instances of a game being played at once. Each game has a unique handle.
The client provides the UI for the game. It makes an API request to start a game and is given a handle in the response. It then calls the Subscribe method, passing the handle. As in the example, it uses PushStreamContent to do so. My implementation looks like this;
private Game mGame;
private static readonly ConcurrentBag<Tuple<Game, StreamWriter>> mscolListenerStreams;
// GET: api/v1.0/subscription/{gameId:int}
[HttpGet]
[Route("{gameId:int}")]
public HttpResponseMessage Subscribe(int gameId) {
mGame = Game.GetGame(gameId);
if (mGame is null) {
return this.Request.CreateErrorResponse(HttpStatusCode.NotFound, "Unknown GameId");
}
mGame.GameEvent += game_Event; // Assign an event handler to follow game events.
HttpResponseMessage response = this.Request.CreateResponse(HttpStatusCode.OK);
response.Content = new PushStreamContent(
(a, b, c) => { this.OnStreamAvailable(a, b, c); },
"text/event-stream");
return response;
}
Instead of just saving off the stream as in the example, I need to tie a stream to a game. I do that like this:
private void OnStreamAvailable(Stream stream, HttpContent content, TransportContext context) {
mscolListenerStreams.Add(new Tuple<Game, StreamWriter>(mGame, new StreamWriter(stream)));
}
Instead of the timer event in the example, I use the game event handler to send messages to the relevant streams:
private async static void game_Event(Game game, EventId eventId, int index) {
foreach (Tuple<Game, StreamWriter> tuple in mscolListenerStreams) {
if (!object.ReferenceEquals(tuple.Item1, game)){
continue;
}
try {
await tuple.Item2.WriteAsync($"event:Game\ndata:{game.Id}/{eventId}/{index}\n\n");
await tuple.Item2.FlushAsync();
}
catch {
Tuple<Game, StreamWriter> ignore;
if (mscolListenerStreams.TryTake(out ignore)) {
ignore.Item2.Dispose();
}
}
}
}
As soon as the game fires an event (instead of the timer that's in the example), the clients locks up and stop receiving server messages or they get CORS errors whenever they try to make subsequent API calls. (The client domain is allowed.) The timer still works.
What am I missing here?
I am trying to receive the JSON value from the Realtime Database of Firebase using Unity.
I do the following:
FirebaseDatabase.DefaultInstance
.GetReference("Leaders").OrderByChild("score").GetValueAsync().ContinueWith(task =>
{
if (task.IsFaulted)
{
Debug.LogError("error in reading LeaderBoard");
return;
}
else if (task.IsCompleted)
{
Debug.Log("Received values for Leaders.");
string JsonLeaderBaord = task.Result.GetRawJsonValue();
callback(JsonLeaderBaord);
}
}
});
Trying to Read the CallBack :
private string GetStoredHighScores()
{
private string JsonLeaderBoardResult;
DataBaseModel.Instance.RetriveLeaderBoard(result =>
{
JsonLeaderBoardResult = result; //gets the data
});
return JsonLeaderBoardResult; //returns Null since it doesn't wait for the result to come.
}
Question is how do i wait for the callback to return the value and afterwards return the value of the JsonLeaderBoardResult.
return JsonLeaderBoardResult; //returns Null since it doesn't wait
for the result to come.
The RetriveLeaderBoard function doesn't return immediately. You can either use coroutine to wait for it or return the JsonLeaderBoardResult result via Action. Using Action make more sense in your case.
Change the string return type to void then return the result through Action:
private void GetStoredHighScores(Action<string> callback)
{
string JsonLeaderBoardResult;
DataBaseModel.Instance.RetriveLeaderBoard(result =>
{
JsonLeaderBoardResult = result; //gets the data
if (callback != null)
callback(JsonLeaderBoardResult);
});
}
Usage:
GetStoredHighScores((result) =>
{
Debug.Log(result);
});
EDIT:
That is great, but still need to do some stuff after getting the
result in `GetStoredHighScores' outside the Action, otherwise i can
get an error like: get_transform can only be called from the main
thread.
You get this error because RetriveLeaderBoard is running from on another Thread. Grab UnityThread from this post then do the callback on the main Thread with UnityThread.executeInUpdate.
Your new code:
void Awake()
{
UnityThread.initUnityThread();
}
private void GetStoredHighScores(Action<string> callback)
{
string JsonLeaderBoardResult;
DataBaseModel.Instance.RetriveLeaderBoard(result =>
{
JsonLeaderBoardResult = result; //gets the data
UnityThread.executeInUpdate(() =>
{
if (callback != null)
callback(JsonLeaderBoardResult);
});
});
}
You're seeing a classical confusion with asynchronous APIs. Since loading data from Firebase may take some time, it happens asynchronously (on a separate thread). This allows your main code to continue, while the Firebase client is waiting for a response from the server. When the data is available, the Firebase client calls your callback method with that data, so that you can process it.
It's easiest to see what this does in your code by placing a few logging statements:
Debug.Log("Before starting to load data");
FirebaseDatabase.DefaultInstance
.GetReference("Leaders").OrderByChild("score").GetValueAsync().ContinueWith(task => {
Debug.Log("Got data");
}
});
Debug.Log("After starting to load data");
When you run this code, it prints:
Before starting to load data
After starting to load data
Got data
This is probably not the order in which you expected the output. Due to the asynchronous nature of the call to Firebase, the second log line gets printed last. That explains precisely why you're seeing an empty array when you return it: by that time the data hasn't been loaded from Firebase yet, and your ContinueWith hasn't executed yet.
The solution to this is always the same: since you can't return data that hasn't loaded yet, you have to implement a callback function. The code you shared does that twice already: once for the ContinueWith of Firebase itself and one in RetriveLeaderBoard.
Any code that needs to have the up to date leaderboard, will essentially have to call RetriveLeaderBoard and do any work with the leaderboard data in its callback. For example, if you want to print the leaderboard:
DataBaseModel.Instance.RetriveLeaderBoard(result => {
Debug.Log(result);
});
This question has been answered multiple times, but I'm still not getting it.
I am using an example from this page: https://www.codeproject.com/Articles/1124691/SignalR-Progress-Bar-Simple-Example-Sending-Live-D
However, I've modified the SendProgress method a bit, to send the message only to the specific connection, not all clients:
public static void SendProgress(string connectionId, string progressMessage, int progressCount, int totalItems)
{
var hubContext = GlobalHost.ConnectionManager.GetHubContext<RealTimeProgressBar.ProgressHub>();
var percentage = (progressCount * 100) / totalItems;
hubContext.Clients.Client(connectionId).AddProgress(progressMessage, percentage + "%");
}
I am getting connection Id from overrided OnConnected method:
public override Task OnConnected()
{
Environments.ConnectionId = Context.ConnectionId;
return base.OnConnected();
}
I'm currently storing it in a static class Environments, I guess there is a better way for doing so, but I don't know it yet :)
The question is, when I open for example two instances of my web page in browser, the progress bar is still being shown for all of these instances, though the connectionIds on these pages are different (checked it in SendProgress function).
What is wrong here?
Perhaps, I should change something in js code?
$(function () {
// Reference the auto-generated proxy for the hub.
var progress = $.connection.progressHub;
console.log(progress);
// Create a function that the hub can call back to display messages.
progress.client.AddProgress = function (message, percentage) {
ProgressBarModal("show", message + " " + percentage);
$('#ProgressMessage').width(percentage);
if (percentage == "100%") {
ProgressBarModal();
}
};
$.connection.hub.start().done(function () {
var connectionId = $.connection.hub.id;
console.log(connectionId);
});
});
As i'm not really sure if js funcs really uses Environments.ConnectionId or smthing else..
Thanks!
Edit: What I want to achive is to send unique progress bars for every opened page. e.g. when I press the button on a first page, I'm starting to collect data and progress bar reflects that. I also open another page in a browser, and collect data from another source, so the progress bar should reflect that for this page as well.
my controller method:
public ActionResult Generate(InputDataViewModel viewModel, string connectionId)
{
...
var Tasks = getResultsWithProgressBar(viewModel.jobId, connectionId);
...
return RedirectToAction("Details", new { id = job.JobDataId });
}
and somewhere inside this "getResult" is a function that actually calls progress bar(SendProgress)
Your change should work fine.
You are using a static ConnectionId, so only your last browser, the browser you opened last(or connected last) will see the progress bar as well as updates.
This is the only one i made apart apart from adding in the static Environment.ConnectionId.
//PUSHING DATA TO ALL CLIENTS
// hubContext.Clients.All.AddProgress(progressMessage, percentage + "%");
hubContext.Clients.Client(Environments.ConnectionId).AddProgress(progressMessage, percentage + "%");
So if you open 10 browser windows and click on the button on all those, one by one, only the browser you initially opened last will see the the progress bar as well as updates, but it will see updates from all the 10 browsers shown on that single progress bar!
It is fun to see though!
When I tried your code, I saw the behavior you observed initially. But when I opened two separate Incognito browsers, I see the correct behavior. Close the IIS express site, rebuild and retry it.
Edit:
If you need each tab to have its own progress bar, then you will need one of the two approaches below:
A way to identify which rest request came from which connectionId. If you want to continue using the current code. Just pass in the connection Id to the LongRunningProcess. Client code below:
var progress = $.connection.progressHub;
var connectionId = $.connection.hub.id;
$.getJSON("/Home/LongRunningProcess",
{"connectionId":connectionId},
function (data) {
if (!data) {
alert("Success");
}
else
{
alert(data);
}
});
Server code would be:
public JsonResult LongRunningProcess(string connectionId)
{
//THIS COULD BE SOME LIST OF DATA
int itemsCount = 100;
for (int i = 0; i <= itemsCount; i++)
{
//SIMULATING SOME TASK
Thread.Sleep(500);
//CALLING A FUNCTION THAT CALCULATES PERCENTAGE AND SENDS THE DATA TO THE CLIENT
Functions.SendProgress(connectionId, "Process in progress...", i , itemsCount);
}
return Json("", JsonRequestBehavior.AllowGet);
}
The other option is to call a signalr method on the server instead of a rest method. Create a new hub method in the ProgressHub like below:
public void LongRunningHubMethod()
{
//THIS COULD BE SOME LIST OF DATA
int itemsCount = 100;
for (int i = 0; i <= itemsCount; i++)
{
//SIMULATING SOME TASK
Thread.Sleep(500);
//CALLING A FUNCTION THAT CALCULATES PERCENTAGE AND SENDS THE DATA TO THE CLIENT
Functions.SendProgress(Context.ConnectionId, "Process in progress...", i, itemsCount);
}
}
The client would then just call in the hub method on click like below:
var progress = $.connection.progressHub;
var connectionId = $.connection.hub.id;
progress.invoke('LongRunningHubMethod');
Let me start by saying this all works perfectly at the moment except for one thing - the notification update for the progress bar goes to all clients (as you would expect given that the code example I used sends to ...Clients.All).
All I want to do is send the notification back to the client that initiated the current call to the hub. That's it, nothing else. There is no concept of "logging in" on this website so there's no user identity stuff to work with.
My method is:
public void NotifyUpdates(decimal val)
{
var hubContext = GlobalHost.ConnectionManager.GetHubContext<EventsForceHub>();
if (hubContext != null)
{
//"updateProgress" is javascript event trigger name
await hubContext.Clients.All.updateProgress(val);
}
}
So back in my view I subscribe to "updateProgress" and it works fine - the progress bar updates as desired. But if another client happens to be connected to the hub, when one runs the async task and causes the NotifyUpdates method to run, then ALL connected clients see the taskbar update, which is a bit confusing for them!
In debug, if I inspect hubContext at runtime I can see a Clients property and that has a Connection property which has an Identity property with a unique GUID. Perfect! Just what I want to use. But... I cannot access it! If I try:
var currentConnection = hubContext.Clients.Connection;
...then I simply get a
"does not contain a definition for 'Connection'"
error, which I simply don't understand.
I've tried accessing Context.ConnectionId from the method too, but Context is null at that point so I'm a bit confused. The server method that uses NotifyUpdates to send information back to the client is called via a normal asp.net button, not via AJAX.
Clarification on structure
I think there is a degree of confusion here. It's a very simply webpage with an asp.net button control on it. The eventhandler for that button invokes an async method to return data from the server via a service call/repository.
Inside the async method, it has to process each returned data line and make three or four remote web api calls. On each loop I make a call back to the NotifyUpdates SignalR method shown above with a percentage complete number so this can update back to the client via an eventhandler for the method name specified (updateProgress as shown above). There could be dozens of data lines and each data line requires several Web API calls to a remote server to add data. This can take several seconds per iteration, hence me sending back the "update progress" to the client via that updateProgress method call.
NEW ANSWER
Based on your comments I made the following little test:
It will allow clients to connect to the hub with a clientName, and every client is listening to updates send to them. We will have a group defined for them to be able to notify them from the server side.
I made a dummy progress simulator class to throw some update values to the users.
The code:
Hub class:
public class EventsForceHub : Hub {
public static IHubContext hubContext = GlobalHost.ConnectionManager.GetHubContext<EventsForceHub>();
// allow users to join to hub and get s dedicated group/channel for them, so we can update them
public async Task JoinGroup(string clientName) {
string clientID = Context.ConnectionId;
ClientInfo.clients.Add(clientID, new MyAppClient(clientID, clientName));
await Groups.Add(clientID, clientName);
// this is just mockup to simulate progress events (this uis not needed in real application)
MockupProgressGenerator.DoJob(clientName, 0);
}
public static void NotifyUpdates(decimal val, string clientName) {
// update the given client on his group/channel
hubContext.Clients.Group(clientName).updateProgress(val);
}
}
Some little helper classes:
// client "storage"
public static class ClientInfo {
public static Dictionary<string, MyAppClient> clients = new Dictionary<string, MyAppClient>();
// .. further data and methods
}
// client type
public class MyAppClient {
public string Id { get; set; }
public string Name { get; set; }
// .. further prooerties and methods
public MyAppClient(string id, string name) {
Id = id;
Name = name;
}
}
// this a completely made up and dumb class to simulate slow process and give some simple progress events
public static class MockupProgressGenerator {
public static void DoJob(string clientName, int status) {
if (status < 100) {
Task.Delay(1000).ContinueWith(a =>
{
EventsForceHub.NotifyUpdates(status += 20, clientName);
DoJob(clientName, status);
});
}
}
}
Let's see two simple clients in JS:
$(function () {
var eventsForceHub = $.connection.eventsForceHub;
$.connection.hub.start().done(function () {
$('body').append("Joining with Name: Jerry");
eventsForceHub.server.joinGroup("Jerry");
});
eventsForceHub.client.updateProgress = function (val) {
// message received
$('body').append('<br>').append("New Progress message: " + val);
};
});
For simplicity, same code, with different params, I even put this in two different html pages and stated execution in slightly different timing.
$(function () {
var eventsForceHub = $.connection.eventsForceHub;
$.connection.hub.start().done(function () {
$('body').append("Joining with Name: Tom");
eventsForceHub.server.joinGroup("Tom");
});
eventsForceHub.client.updateProgress = function (val) {
// message received
$('body').append('<br>').append("New Progress message: " + val);
};
});
See it in action:
FIRST ANSWER
I made a small web application to verify your claim. You may create the following to be able to isolate the issue from other possible problems.
I created an empty Web Application and included SignalR.
This is the hub class:
public class EventsForceHub : Hub {
public void NotifyUpdates(decimal val) {
var hubContext = GlobalHost.ConnectionManager.GetHubContext<EventsForceHub>();
if (Context != null) {
string clientID = Context.ConnectionId; // <-- on debug: Ok has conn id.
object caller = Clients.Caller; // <-- on debug: Ok, not null
object caller2 = Clients.Client(clientID); // <-- on debug: Ok, not null
Clients.Caller.updateProgress(val); // Message sent
Clients.Client(clientID).updateProgress(val); // Message sent
}
if (hubContext != null) {
//"updateProgress" is javascript event trigger name
hubContext.Clients.All.updateProgress(val); // Message sent
}
}
}
This is the web page:
<script src="Scripts/jquery-1.10.2.min.js"></script>
<script src="Scripts/jquery.signalR-2.2.2.min.js"></script>
<script src="signalr/hubs"></script>
<script type="text/javascript">
$(function () {
var eventsForceHub = $.connection.eventsForceHub;
$.connection.hub.start().done(function () {
// send mock message on start
console.log("Sending mock message: " + 42);
eventsForceHub.server.notifyUpdates(42);
});
eventsForceHub.client.updateProgress = function (val) {
// message received
console.log("New Progress message: " + val);
};
});
</script>
Try to build an application as little as this to isolate the issue. I have not had any of the issues you mentioned.
For the sake of simplicity and using the debugger I took away the await and async.
Actually, SignalR will take care of that for you. You will get a new instance of your Hub class at every request, no need to force asynchrony into the methods.
Also, GlobalHost is defined as static which should be shared between instances of your Hub class. Using in an instance method does not seem like a very good idea. I think you want to use the Context and the Clients objects instead. However, while debugging we can verify that using GlobalHost also works.
Some debugger screenshots showing runtime values of callerId, Clients.Caller and Clients.Client(clientID):
Understanding SignalR better will help you a lot in achieving your goal.
Happy debugging!
If you want to send the notification back to the client
you should not call
hubContext.Clients.All.updateProgress(val);
instead try
accessing the current user's ConnectionId and use Clients.Client
hubContext.Clients.Client(Context.ConnectionId);
I'm working with .NET 3.5 with a simple handler for http requests. Right now, on each http request my handler opens a tcp connection with 3 remote servers in order to receive some information from them. Then closes the sockets and writes the server status back to Context.Response.
However, I would prefer to have a separate object that every 5 minutes connects to the remote servers via tcp, gets the information and keeps it. So the HttpRequest, on each request would be much faster just asking this object for the information.
So my questions here are, how to keep a shared global object in memory all the time that can also "wake" an do those tcp connections even when no http requests are coming and have the object accesible to the http request handler.
A service may be overkill for this.
You can create a global object in your application start and have it create a background thread that does the query every 5 minutes. Take the response (or what you process from the response) and put it into a separate class, creating a new instance of that class with each response, and use System.Threading.Interlocked.Exchange to replace a static instance each time the response is retrieved. When you want to look the the response, simply copy a reference the static instance to a stack reference and you will have the most recent data.
Keep in mind, however, that ASP.NET will kill your application whenever there are no requests for a certain amount of time (idle time), so your application will stop and restart, causing your global object to get destroyed and recreated.
You may read elsewhere that you can't or shouldn't do background stuff in ASP.NET, but that's not true--you just have to understand the implications. I have similar code to the following example working on an ASP.NET site that handles over 1000 req/sec peak (across multiple servers).
For example, in global.asax.cs:
public class BackgroundResult
{
public string Response; // for simplicity, just use a public field for this example--for a real implementation, public fields are probably bad
}
class BackgroundQuery
{
private BackgroundResult _result; // interlocked
private readonly Thread _thread;
public BackgroundQuery()
{
_thread = new Thread(new ThreadStart(BackgroundThread));
_thread.IsBackground = true; // allow the application to shut down without errors even while this thread is still running
_thread.Name = "Background Query Thread";
_thread.Start();
// maybe you want to get the first result here immediately?? Otherwise, the first result may not be available for a bit
}
/// <summary>
/// Gets the latest result. Note that the result could change at any time, so do expect to reference this directly and get the same object back every time--for example, if you write code like: if (LatestResult.IsFoo) { LatestResult.Bar }, the object returned to check IsFoo could be different from the one used to get the Bar property.
/// </summary>
public BackgroundResult LatestResult { get { return _result; } }
private void BackgroundThread()
{
try
{
while (true)
{
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://example.com/samplepath?query=query");
request.Method = "GET";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.UTF8))
{
// get what I need here (just the entire contents as a string for this example)
string result = reader.ReadToEnd();
// put it into the results
BackgroundResult backgroundResult = new BackgroundResult { Response = result };
System.Threading.Interlocked.Exchange(ref _result, backgroundResult);
}
}
}
catch (Exception ex)
{
// the request failed--cath here and notify us somehow, but keep looping
System.Diagnostics.Trace.WriteLine("Exception doing background web request:" + ex.ToString());
}
// wait for five minutes before we query again. Note that this is five minutes between the END of one request and the start of another--if you want 5 minutes between the START of each request, this will need to change a little.
System.Threading.Thread.Sleep(5 * 60 * 1000);
}
}
catch (Exception ex)
{
// we need to get notified of this error here somehow by logging it or something...
System.Diagnostics.Trace.WriteLine("Error in BackgroundQuery.BackgroundThread:" + ex.ToString());
}
}
}
private static BackgroundQuery _BackgroundQuerier; // set only during application startup
protected void Application_Start(object sender, EventArgs e)
{
// other initialization here...
_BackgroundQuerier = new BackgroundQuery();
// get the value here (it may or may not be set quite yet at this point)
BackgroundResult result = _BackgroundQuerier.LatestResult;
// other initialization here...
}