I'm doing a simple Messaging system for a Windows Mobile in C#. The application consists in sending and receiving simple text messages using a Web service communication. The messages queue should be persistent, avoiding data lost if the connection with the web service fail or the application crash.
I know about MSMQ, RabbitMQ, DotNetMQ, but they should be installed in the device and this are really simple devices, I don't want to install any other tool in each of the mobiles just for this simple task.
I already implemented the function to write an XML serialized queue with the messages into a file and I read and write all the time from this file.
I'd appreciate any better idea to solve this problem.
Thanks
MSMQ does not need to be installed is supported natively on Windows Mobile 6.5 devices. BTW: there are still many vendors in industrial area providing WM65 based devices, so this is not yet outdated.
The Windows Mobile (CE) based MSMQ is persistent and simple to use. It is normally used for interprocess-communication on the device or for client server communication (which requires MSMQ installed on the 'server').
So, the main thread creates a MSMQ, one thread in your process fills the MSMQ and another can 'peek' and, after successful transmission, 'dequeue' messages from the same MSMQ. See here for a simple example.
I don't really know what's available for Windows Mobile but you can try using basic queue (normal or concurrent, depends on your app) accompanied by two files. Write everything that is enqueued to one "Enqueue log" file and write everything that is dequeued to another "Dequeue log" file.
This two files can always give you enough information to restore your queue, and you don't need to fully rewrite/fully serialize your queue. It needs to be implemented by hand though.
About dequeue:
for example, lets say I have a queue with 3 messages: "one", "two", "three". Now I want to send the next (also the first) message "one". I append the line "one - starting removal from queue" to my "dequeue log", then I dequeue "one" from my queue object and send it where I want it to be sent to. When it is sent, I append " - finished removal from queue" to my "dequeue log". Now I have a line "one - starting removal from queue - finished removal from queue" in my log file.
It doesn't matter when do I crash, I'll always will be able to restore the state of queue object (at least for now I fail to see any logic mistakes in this process). So imho it's not tricky but still... some code should be coded. And it would be a few pages of code.
Sure, there is a better idea is to use SQLite.
I hope this will help you.
Related
I'm working on a task where I'm trying to insure delivery of data to a database in the order in which it is written. The database will be located on another machine, and it's possible that the database machine could fail while the sending computer continues to queue up database messages.
I'm trying to use WCF for this task and from my initial reading believed that this might be possible using WCF with message queue and a ReceiveContext enabled channel. The documentation states that ReceiveContext 'enables an application to decide whether to access the message or leave it in the queue for further processing.'
The problem that I've encountered is that if I call the Abandon method indicating that the message has not been successful processed and should be left on the queue, WCF appears to place the message at the back of the queue instead of leaving it at the front of the queue. Since I need to write the messages to the database in the order they were originally sent this solution will not work.
Is there any way to force WCF to 'peek' at a message before removing it or cause an abandon message to be placed at the front of the queue? If not could anyone suggest a method of accomplishing what I'm attempting to do without having to write a lot of code from scratch.
Thanks,
Al
I am working on one Asp.Net application and need to send mails periodically based on some event. First I thought of creating a thread in global.asax and start thread in application_start. But that becomes a bit of problem when application pool crashes or something. So I implemented a windows service and started thread in that and log any errors in windows event log. This works fine. But I need to know whether I am implementing it correctly or is there a better way of doing it?
I think you are moving (or moved already) to the right direction.
We have similar architectures as well, in some cases we used MSMQ to queue outgoing notifications from the ASP.NET application then the Windows Service, usually called Messaging Manager, can grab asynchronously the incoming messages and send the emails or alerts out.
this proves to be effective and robust, if anything crashes after the message has been queued, nothing will be lost because the windows service will always process the messages in the queue, so you can have ASP.NET recycling or the machine with the windows service being rebooted, nothing is lost ever. And in fact in normal production mode, messages are sent out instantly, the decoupling or loose of sync is mostly hidden when everything is working smooth and servers are not overloaded or suffering anything.
In a later project we are now implementing something similar using TIBCO technologies, EMS for the queues and Business Works for queue subscribers.
Using a Windows Service for this kind of tasks is the preferred way instead of doing it in the ASP.NET application. You may also take a look at Quartz.NET which could simply your code for scheduling the task execution and dealing with threads. But if you don't want to write Windows Services probably the simplest would be to have a console application that will do the job of sending emails and then simply use the Windows Task Scheduler to run it at regular intervals.
Another option is a message-based approach. You could have a Windows Service/Console Application reading messages of a message queue (like msmq) and send email when a message is recieved. You can then have your ASP.NET application publish messages to this queue.
Minibuss is a lightweight client for msmq which is very easy to work with. Another options is NServiceBus.
I have an Application that collects actions and sends them off to a remote server. As these actions aren't time critical (think of them as log lines), I want to queue them up and send them in batches.
That way, I also want to ensure that no message is ever lost (unless the hard drive crashes).
MSMQ seems rather heavyweight, arcane and weird to use. Also, it needs to be installed as a system component.
Serializing my messages into JSON and storing them in SQLite is trivial and straight forward, but before I do that, I wonder if there is a standardized (preferably AMQP compatible) queue that I doesn't require installation and can be embedded into an app?
I really think you should reconsider MSMQ.
It is installed by default in the Server versions of Windows.
Installation on non Server versions of windows is trivial.
It provides a built-in UI for observing the queue
I don't know what your standards are of 'heavy and arcane' - but I just used it for the first time in a project and it was the easiest part of the application. I certainly don't think its much more heavyweight than storing the queue in a database yourself.
If you prefer to use JSON, you can serialize the messages yourself and store as a string
You can configure a queue to be recoverable - so storing the queue on disk not in memory
The only serious objection that I can see is having to install MSMQ. If you are having to deploy this application far and wide on different versions of Windows, I can see that as a significant problem.
Graylog2 is a centralized logging solution that accepts log entries from AMQP messages. Perhaps you could adapt it to your use-case.
In any event, Graylog2 shows that AMQP works for jobs like collecting log messages without losing any.
AMQP doesn't require installation, because it is a protocol. You just need the client library for .NET. However you would need to install an MQ broker on a server somewhere on your LAN to manage the message flow. RabbitMQ is widely used because it is easy to install.
Also, once you start sending messages, then you will also need to have a process somewhere on the network, that receieves them and does something with them such as write to a db.
If you want a homebrew solution, you could install RabbitMQ on the logging server, embed RabbitMQ's .NET client into your application, then write a small program to read from the queue and write the events to disk.
RabbitMQ is fairly lightweight: the default install is only a few Mb and it normally uses about 11Mb of memory to run. It also provides an extension to AMQP, Publisher Confirms, which can be used to ensure that once the server accepts the log message, it will not be lost, unless the hard disk dies. The extension is non-standard, though, and it's probably not supported by other brokers.
The project that I'm working on uses a commercially available package to route audio to various destinations. With this package is a separate application that can be used to log events generated by the audio routing software to a database e.g. connect device 1 to device 3.
I have been tasked with writing an application that reacts to specific events generated by the audio routing software such as reacting to any connections to device 3.
I have noted that the audio routing sofware uses MSMQ to post event information to the event recorder. This means that event data can build up if the recorder software has not run for a while.
I have located the queue - ".\private$\AudioLog" and would like to perform the following actions:
Detect and process new messages as
they are entered onto the queue.
Allow the current event recording
software to continue to
work as before - therefore messages
can not be removed by my
application.
Ensure that I always get to see a
message.
Now I note that I can use MessageQueue to Peek at the queue in order to read messages without deletion and also GetAllMessages() to peek at all messages not removed by the event recorder.
If the recording software isn't connected then I can see that I can gather message data easily enough, but I can't see how I can ensure that I get to see a message before the recorder removes a message when it is connected.
Ideally I would like to add my application as a second destination for the message queue. Is this possible programmatically?
If not as I have administrator privilege, access to the machine with the queue is it possible to configure the queue manually to branch a second copy of the queue to which I can connect my software?
Msmq has a journaling feature. You can configure the queue to have a journal. Then, every message that is removed from the queue( by a read operation) is moved to the journal queue and not deleted. You can then read (or peek) from the journal. If you are using peek operation, make sure that you have a job that delete the journal from time to time.
I'm developing some kind of server service who gets in action and stop on an specific range of time, I need to write a monitoring app to receive some status text sent from the service in the same local host.
It's the first time I need to comm between processes, I've been looking for some methods, wich do you believe is the best for this case? why? pros and cons for each one?
Remoting
Named pipes
Windows messages
local tcp connection
Database table log (I'm using a database so it's another option)
Datafile on disk
Edit:
I'm using .net 2 so the great wcf solution doesn't work for me :(
See this SO question:
C# IPC Best Practises
You almost certainly want to use WCF. This replaces other .Net technologies such as remoting. Also, if you use WCF:
Other clients could also subscribe to that status information in the future.
The service listening to status updates could be run on another physical box if you needed to scale out.
If you're really just looking for "status messages" - as opposed to an actual client app - then you can use Trace/Debug.Write* statements. DebugView will pick them up if it's running, and can also be used across hosts. And, of course, you can add other listeners via config file if you later wanted to save the messages.