I'm looking for a way to retrieve a collection of DTOs from my WCF data service in a way that will allow me to be informed every time a whole DTO from the collection has finished downloading, also I want to be able to read it of course.
Means, if I want to get a collection of users, every time a user from the collection is downloaded completely to the client (serializably-speaking), I want the client-side to be notified and be able to read it.
Is it at all possible?
Thanks!
Edit:
Is passing a callback from the client to the server, which the server will use to send the client each user through iteration, a possible/correct direction? Or is there a better solution?
You’ll probably have to split it into multiple requests in order to do this. For example, one request to retrieve the size of the collection, and then a separate request for each item in the collection. Then you know when each item completes. (If you do this, you can even parallelise the whole thing.)
You can't really subdivide a single call easily, so you'd best making one or two concurrent calls, and getting the objects individually. Using some kind of manager class, and some multithreading, you could have an event fired when a call was completed - and map that to an 'object downloaded' event.
Hope that helps.
Related
I have huge in memory object and I am just wondering, if I can create a ConsistentHashing router and be able split load to underling actors.
The main problem is that I need to ingest actors with data after creation, so just wondering how I can achieve that.
The master object has a good number of records, which will be grouped by Id field.
This is my ideas:
when actor is created - can I get it's hashPool to retrieve required ids from master object?
when actor is created I can wrap ingestion message ConsistentHashableEnvelope and then when queering using same wrap to ask for data- will this work?
reference
As per the comments on your question, I understand better what you're trying to do.
I've done something similar, but slightly different in the way the actors have to start up: a caching system where I use a user's ID in a ConsistentHashableEnvelope to route the request to the actor that should hadndle that user's request. If the user's data isn't available, it is loaded into memory from a third-party service. All following requests will operate on this data in memory.
In your case I'd tackle the problem in this manner:
Setup your ConsistentHashing router with actors that startup in a state that can Receive<> the individual entries they need.
Simply Tell() the router all the individual entries in this large object you want to slice up using a ConsistentHashableEnvelope as a wrapper, and the router will send each entry to its correct destination.
In the actors, Receive<> the entries and use some method to merge the received data into the existing internal structure. This means that when the actor first starts up it will simply store the slices and if it receives entries later on you can decide what to do (replace, update, whatever you wish).
When handing off subsequest messages to the router, always make sure you're using the correct key ConsistentHashableEnvelope, otherwise the message will be routed to an actor that doesn't have the data!
From your post I don't know much about the rest of your project requirements, but if you need to be able to first populate the actors with their slices before handling other requests that depend on this data, you may want to start the routees in an AwaitingData state using Become(), and then move them to a Ready state once their data has been received.
If you add more info about your project I may be able to help you some more.
For a call tracking application I'm developing, I want to maintain local database.
As it stands, the application searches for new records in Twilio and inserts them into my database every time it loads. This is very time consuming.
In order to avoid that runtime expense, is there a way I can use usage triggers in Twilio to automatically populate my database in real time? Or even just daily?
If not, how can I achieve something like this?
Since Twilio is already calling your servers (unless there's some way to use it without doing that, but I don't think there is), can't you implement logging there? For instance, before you feed back your greeting, pop in a logging routine to note that you've received a call?
I'm not sure if they offer any other sorts of APIs or callbacks, but I really don't see why anything like that would be necessary. It'd just tie up your servers with more requests at no additional gain. I was just going through their documentation and I don't see anything like this. I could be just totally glossing over it, but again it just seems redundant. The entire Twilio system is based effectively on event hooks, so having separate ones wouldn't serve much additional use.
On the other hand, if for some reason you have absolutely no access whatsoever to the code or people behind the code that serves TwiML back, unless someone else is seeing an event hook API, you might want to just set up a scheduled job on your server (or in Azure, or whatever you're using) to query Twilio daily, since I know you mentioned that that would be sufficient. You could also, of course, set it more frequently. But that really seems like a waste of resources and effort when they're already telling you everything about every call through the massive list of query parameters they pass with every request.
I am currently working on a project in ASP.NET MVC 4 and came along a module where a progress bar is needed. The question I am having right now is "What is the best way to implement an async progress bar?".
After some lookup I came across the following method:
Create a startEvent() and getProgress() in C# code.
Use javascript setTimeout() to call the getProgress() method asynchronously.
(Example: https://www.devexpress.com/Support/Center/Example/Details/E4244)
My remark with this method is that that causes the code to be dependent on the timeout you choose. So it would take some fiddling to find the best and most performant timeout.
Now the method that I would most likely have used before I researched the matter is the following:
In code behind, create a method handleItem(int index) which takes an index and does everything you want to do with the item at that index.
Determine the number of items you want to handle and pass that to your javascript.
In javascript, initiate a for loop that loops from 0 to the amount - 1, and for each index, it initiates an ajax-call to handleItem(i).
On that ajax-call's complete-statement, you can update the progress bar with the new amount.
My questions here are the following:
Does this expose too much of the program logic?
Does this create too much overhead seeing as every call goes to the server and back?
Are there any other reasons why I should refrain from using this method?
Thanks in advance
Koen Morren
This is not a recommended strategy, because the client drives the process. If there is any discontinuation of connectivity or maybe the user closes the browser, the process will stop.
Generally, if you use straight HTTP you will need to poll (aka pull) from javascript. The pseudo code is pretty much this:
Call Creates Task ID and sends it to client
Client queries the status of task with given ID
Another possibility are WebSockets, which allow your client to listen for changes that are pushed by the server.
There are many options to store the progress of a given state. You can index the progress by the HttpContext, task id, or some user id, or even store it in a database and use SqlDependency to get notifications of when the status is changed.
In summary, polling has more lag than push mechanisms. Clients should not drive an asynchronous process, but they should be either notified or provided some mechanisms on the status of an async process.
Unlike ASP.NET, there is few way to push data from server to client in MVC, WebSockets or SingnalR like api(s) can work for you.
The ajax approach is good and give you reliable mechanism to update data no matter user go to other page or closes the browser, every time ajax launched it will update UI. So there is nothing wrong there just have a fair interval in javascript.
Does this expose too much of the program logic?
Code will be written only in class file to calculate current %age.
2.Does this create too much overhead seeing as every call goes to the server and back?
No, ajax are light-weight calls
3.Are there any other reasons why I should refrain from using this method?
This method will allow user to freely navigate to other resources as ajax will work independently.
I have this scenario, and I don't really know where to start. Suppose there's a Web service-like app (might be API tho) hosted on a server. That app receives a request to proccess some data (through some method we will call processData(data theData)).
On the other side, there's a robot (might be installed on the same server) that procceses the data. So, The web-service inserts the request on a common Database (both programms have access to it), and it's supposed to wait for that row to change and send the results back.
The robot periodically check the database for new rows, proccesses the data and set some sort of flag to that row, indicating that the data was processed.
So the main problem here is, what should the method proccessData(..) do to check for the changes of the data row?.
I know one way to do it: I can build an iteration block that checks for the row every x secs. But i don't want to do that. What I want to do is to build some sort of event listener, that triggers when the row changes. I know it might involve some asynchronous programming
I might be dreaming, but is that even possible in a web enviroment.?
I've been reading about a SqlDependency class, Async and AWait classes, etc..
Depending on how much control you have over design of this distributed system, it might be better for its architecture if you take a step back and try to think outside the domain of solutions you have narrowed the problem down to so far. You have identified the "main problem" to be finding a way for the distributed services to communicate with each other through the common database. Maybe that is a thought you should challenge.
There are many potential ways for these components to communicate and if your design goal is to reduce latency and thus avoid polling, it might in fact be the right way for the service that needs to be informed of completion of this work item to be informed of it right away. However, if in the future the throughput of this system has to increase, processing work items in bulk and instead poll for the information might become the only feasible option. This is also why I have chosen to word my answer a bit more generically and discuss the design of this distributed system more abstractly.
If after this consideration your answer remains the same and you do want immediate notification, consider having the component that processes a work item to notify the component(s) that need to be notified. As a general design principle for distributed systems, it is best to have the component that is most authoritative for a given set of data to also be the component to answer requests about that data. In this case, the data you have is the completion status of your work items, so the best component to act on this would be the component completing the work items. It might be better for that component to inform calling clients and components of that completion. Here it's also important to know if you only write this data to the database for the sake of communication between components or if those rows have any value beyond the completion of a given work item, such as for reporting purposes or performance indicators (KPIs).
I think there can be valid reasons, though, why you would not want to have such a call, such as reducing coupling between components or lack of access to communicate with the other component in a direct manner. There are many communication primitives that allow such notification, such as MSMQ under Windows, or Queues in Windows Azure. There are also reasons against it, such as dependency on a third component for communication within your system, which could reduce the availability of your system and lead to outages. The questions you might want to ask yourself here are: "How much work can my component do when everything around it goes down?" and "What are my design priorities for this system in terms of reliability and availability?"
So I think the main problem you might want to really try to solve fist is a bit more abstract: how should the interface through which components of this distributed system communicate look like?
If after all of this you remain set on having the interface of communication between those components be the SQL database, you could explore using INSERT and UPDATE triggers in SQL. You can easily look up the syntax of those commands and specify Stored Procedures that then get executed. In those stored procedures you would want to check the completion flag of any new rows and possibly restrain the number of rows you check by date or have an ID for the last processed work item. To then notify the other component, you could go as far as using the built-in stored procedure XP_cmdshell to execute command lines under Windows. The command you execute could be a simple tool that pings your service for completion of the task.
I'm sorry to have initially overlooked your suggestion to use SQL Query Notifications. That is also a feasible way and works through the Service Broker component. You would define a SqlCommand, as if normally querying your database, pass this to an instance of SqlDependency and then subscribe to the event called OnChange. Once you execute the SqlCommand, you should get calls to the event handler you added to OnChange.
I am not sure, however, how to get the exact changes to the database out of the SqlNotificationEventArgs object that will be passed to your event handler, so your query might need to be specific enough for the application to tell that the work item has completed whenever the query changes, or you might have to do another round-trip to the database from your application every time you are notified to be able to tell what exactly has changed.
Are you referring to a Message Queue? The .Net framework already provides this facility. I would say let the web service manage an application level queue. The robot will request the same web service for things to do. Assuming that the data needed for the jobs are small, you can keep the whole thing in memory. I would rather not involve a database, if you don't already have one.
First of all, I am creating a something like a client/server solution using a standard ASP.NET website - I do know this method is not adviced, and most people would love to scream "COMET!" or "HTML5 Sockets!" - but please don't ;-) !
What I am doing...
I am creating an MMORPG on a website.
I have several clients whom need to be in contact at the same time. This is done by a global object in the Application scope.
My problem
I need to invoke an event to several clients. For instance, when an attack has been performed, I need to update some graphics. The attack logic is resolved in the global object, but each of the clients has to respond to this.
Right now I do the following:
fightTrace.Reciever.InvokeMoveEnded(this);
fightTrace.FiredBy.InvokeMoveEnded(this);
(This is a kind of observer pattern)
What now happends is a race condition. The one who loads the page_load event will get both of these events, and the one who is not running them, will experience no changes in the UI.
So what is it I really want?
What I really need is some genuine and nice way to create an observer pattern through the application state. I need to send an event out to every "listener" which is in this case is a client, and then do some update.
One way to do this is some session-thing, with true/false.. But I would really like some better way!
Thanks!
If I understood your context correctly then whenever the state of your application state object is changed you want to synchronize all the clients of your applications. What you are forgetting here is the stateless behavior of HTTP protocol. Once a response is sent the connection is lost you need to send an HTTP request again to be served again. However you can emulate some thing using State Management and Ajax based short and timely updates to simulate a connected environment. However I've to utter the words which you don't want to hear. Not recommended.
Instead what you can do is to save the state of application object and whenever a request comes serve the response based on updated state of your object. Any how a client has to initiate a request.