1) i like to know how sql server establish a channel between client & db. i guess there must be a channel and that why sql server can send notification to client through that channel. please discuss this issue in detail. because i saw many article on sql dependency but every body gives the code but no body explaining how it works in details.
What is Service Broker?
Service Broker architecture allows you to build loosely coupled SQL Server instances so that the instances talk with each other using normal form of messaging. Service Broker uses TCP/IP to transmit messages form the network and hence allows encrypted messaging. It is both for applications which use SQL Server instance or for applications that distribute the work into more than one SQL server instance. Service Broker allows to use Queue to hold messages and hence the messages are processed one by one without the caller to wait to receive the message.
1) i like to know service broker pass the message always in encrypted format?
2) Service Broker allows to use Queue to hold messages. what is the name of that queue used by service broker. how can i see what is stored in that queue?
3) i saw many people create queue but did not mention why they created? they also did not use that queue in their code. here is one url & sample code
http://www.dreamincode.net/forums/topic/156991-using-sqldependency-to-monitor-sql-database-changes/
CREATE QUEUE NameChangeQueue;
CREATE SERVICE NameChangeService ON QUEUE NameChangeQueue ([http://schemas.microsoft.com/SQL/Notifications/PostQueryNotification]);
GRANT SUBSCRIBE QUERY NOTIFICATIONS TO YourUserName;
ALTER DATABASE YourDatabaseName SET ENABLE_BROKER;
they never use NameChangeQueue queue why ?? how do i know who will use this queue?
4) even i saw people create role but never know why role would be require in this case?
so please discuss ALL my points in detail because i need to understand all the points. thanks
First read The Mysterious Notification to understand how Query Notifications work. SqlDependency is just a .Net wrapper leveraging Query Notifications. This should answer most of your questions.
Query Notifications deliver the notifications using Service Broker (SSB) locally to a queue in the database. While SSB can encrypt traffic, this is irrelevant for SqlDependency since the delivery is local, within the server process. The client application gets the notifications by posting a WAITFOR(RECEIVE) on the queue using an ordinary SqlConnection.
In the example you posted the NameChangeQueue is never used, indeed. By using a SqlDependency object the author is actually using the temporary queue deployed just-in-time when it called SqlDependency.Start(). The author could had used instead the lower level SqlNotificationRequest as described in Using SqlNotificationRequest to Subscribe to Query Notifications which allows you to specify the queue to be used.
The permissions required are described in Query Notification Permissions, but if you use SqlDependency then you will also need permissions to create the temporary queue and stored procedure used by SqlDependency.
Read the articles linked and if you have more questions, ask them as new questions here (don't post more questions as comments please).
Related
I Am trying to play with MQTT and RabbitMQ but with some problems.
Introduction
I have several physical machines that generate data and I want to do something with this data, for simplicity imagine that I just want to write those messages to a separate file for each machine identified by an ID. (do not focus on the details, it is just an example)
What I have
I have an MQTT broker (i.e. mosquitto broker) that handles several messages coming from those machines.
I have a (complex) windows service written in C#. An object instantiation exists for each machine (i.e. machine with id = 1000 leads to an object that represents the physical machine in the service program and so on). This machine object has a RabbitMQ queue of messages that contains every message that is delivered by the machine.
The problem
How can I populate this queue?
I thought that there was the possibility, of using the rabbitMQTT plugin, to instantiate an exchange or something similar to listen for the MQTT topics and to forward the received messages to the appropriate queue as usual, but I cannot find anything on the net.
I hope I've been clear enough in proposing the problem that I Am facing.
Assume that I MUST use RabbitMQ since it is already used for the communication of the different modules of the service.
Hope you can help me understand if there is any possibility to use RabbitMQ to listen for messages from an external MQTT broker that I cannot decide on and then push those messages in an exchange that will route the message based on the routing key extracted from the message.
Practical case:
Real-world machine 10000 produces a message A
Real-world machine 10000 publishes on topic machines/10000 the message A
The service (that is subscribed to the machine/# topic) gets the message A
The service publishes message A in the exchange with routing key machines.10000
The machine 10000's callback processes the message A and does something
Thanks, please be as clear as possible since I need to understand the entire process (if possible)
In want to use RabbitMQ to send events to a server from a mobile C# app. The user records a lot of events in the whole day (number of products manufactured, consumed water, ...) and they need to be delivered in a server to be processed. I understand that RabbitMQ creates a queue in the server but also, I would like to have a queue en in client side, in the mobile app. It is usual that in some parts of the factory Internet fails, so when the user records any event, it needs to be sent using the RabbitMQ client, but if Internet fails, it should remain in an "internal" queue, waiting to be sent in the next synchronization.
Which is the best approach for this problem? Does have RabbitMQ client library a feature for this purpose?
No RabbitMQ does not provide any such thing , typically for a user case like your it is best to use a local light weight database. You can go for something like SQLite.
Keep the data locally till it is synchronized and once done you may delete it from local.
I am currently using a SqlDependency with a SQL Server 2012 Service Broker and I want to be able to have two servers configured both listening to the service broker and pull off the queue but message should only be pulled off the queue once total. Each machine should try and pull down what it can but if too many are coming in it should share a balance in pulling in what it can. Right now I start two instances of the program and both are listening. Once a new message is added they both pull off the same message off the queue and run the code.
Is SqlDependency not the solution to what I want to do? What is the better solution to something like this?
Once a new message is added they both pull off the same message off the queue and run the code
The behavior you describe is how SQLDependency is designed to work. If there are multiple listeners, all listeners are notified. For example, you can see this described in the SignalR SQL Backplane documentation
Notice how all VMs receive notification from SQL Server, including the VM that initiated the update.
If you want to distribute SQL Notifications across a pool of worker VMs, you need a way to share state. Note that the SQL Notification is only an indication that something changed and doesn't indicate what changed. One approach is to add a table to the database to act as a queue of jobs or actions. Subscribers can query this queue on each notification and claim the action by updating or deleting from this table. (Appropriate locks would have to be configured on the table)
Alternatively, you can do this using other tools for shared state, such as a message queue (eg. RabbitMQ), or distributed cache (eg. Redis)
You don't need SQL Notifications or SQLDependency. Each instance can execute:
WAITFOR(
RECEIVE TOP(1) * FROM {NameOfQueue}
), TIMEOUT #timeoutvalue;
This command will WAIT, leaving the connection open, until either a message is available or the timeout has occurred. On the timeout you receive no message so just connect and try again.
Each message can only be RECEIVED by a single process. Internally the row in the Server Broker queue is locked, and other readers will READPAST locked rows.
Because the SQL can be a little bit tricky, I've written what I think is a helpful wrapper class that you are free to use.
Im just starting to work on a particular piece of development.
We have a .NET WCF application, MySql/EF DAL/ORM, that is called by a threaded job scheduler that pulls data from one client, stores it in our DB and passes the latest data to a another client and vice versa.
So to think about it as messages,
ClientB sends an order to ClientA through our system which transforms the order into a readable format for ClientA.
ClientA then can send messages to the ClientB through our system to say stuff like "your order is shipped" or "your order is late".
I need to take these messages and relay them onto ClientB but I want it to be transactional and for us to have full control over failed messages etc.
My current thoughts are, for simplicity sake, to have a OrderMessages table in our DB which receives messages, with a state of "Ready" which can then be processed by a factory and forwarded to the relevant client using a configuration stored against the clients.
Sorry for this being all over the place, but hopefully I've explained what im trying to do :/
Neil
Your proposed architecture is a classic queue table pattern. Remus Rusanu is the canonical resource for building such a thing with SQL Server. The ideas apply to other databases as well.
There is nothing wrong with this architecture. Note, that in case of an error when messaging a client you cannot know whether the message was received or not. There is no 100% solution for this problem. It is the Two Generals Problem.
If you make the clients pull directly from the database you can avoid this problem. Clients can use their own transactions in this case.
Consider leveraging a message platform for publishers and subscribers.
Specifically, consider using a hub and spoke pattern.
Also, BizTalk specializes in workflows across distributed systems.
Also consider the effort involved:
Transactions (short and long)
Error handling
Expected message formats
Orchestrations
I have the need from an asp.net web site to send out many SMS messages at once, and also poll a POP3 account for an incoming mail, and then SMS that out to many recipients, one at a time.
The way I am thinking of doing this is a windows service that would connect to my sql back-end to see if there are SMS messages to be sent out, like every 10-20 seconds or so. If so, get all the messages in a list, delete them from the table, and then proceed to send them out.
Same way with the pop account.
Any ideas on how to best provide this service without causing blocking in the asp.net web page when it is kicked off (e.g. messages added to sql server)?
Platform is windows server 2003 R2, sql 2008 standard, asp.net 3.5 SP1.
Thanks for your advice.
We have implemented similar scenarios using SQL Server service broker's Queueing mechanism. The idea is that every inserted SMS record is caught by a trigger which inserts a message containing the SmsID into the service broker Queue.
You then need a stored procedure which receives messages from the Queue. If there are no messages, your procedure will run until the next entry is inserted. That's OK, since it does not take up resources to listen to the Queue.
Next you'll need a Windows service who continuously (recursively) calls the STP, assembles the SMS and sends it.
The Advantage of the Service Broker Queue over a flag in a table is thread safety. This way you could have as many instances of your Service as you want w/o having to worry too much about concurrency issues.
You can find a nice Service Broker tutoial here: http://www.developer.com/db/article.php/3640771
Instead of using an Sql Server for the queuing you could use MSMQ (Microsoft Message Queuing) for this.
MSMQ is quite easy to set up and once it is up and running it is more scalable than Sql Server.
So what you could do was to setup a new queue in MSMQ that would receive the messages you wanted to send. The message would normally be some sort of Message object that describe the message, the sender and the recipient.
Then you would either setup a service that would poll the queue at a regular interval or you could setup MSMQ to start a class of your choice each time a new Message was sent to the queue.
If you need a log of the messages you could have the service / sender object write to a log in sql server when the message was sent.