This may be a trivial question, but I will ask it anyway: can updates in three or four tables in one database trigger updates to one or two tables in another database (in MS SQL 2005?) I am happy to create any amount of T-SQL necessary, and I also have VS 2008 with C# ability but have never written a trigger like that before.
Essentially, I had the "GetProducts()" of the data repository call my stored procedure anytime any data was loaded with what scope they needed, and I physically changed the "cached" data. Everything in life was good.
Now, they don't want the data update as part of the repository at all. They feel that it is external to the project and should be handled without interaction.
Anyone have any suggestions on what to pursue? Links to ideas already out there would be fantastic.
A trigger only kicks off when one table is updated,inserted or deleted. If you havea specific order the tables must be inserted, you could put the trigger on the last one.
Alternatively you could write the trigger to examine the other tables as well to ensure all have records. Or you could write one trigger for each table. Or if real time updates are not required, you could have a job that runs periodically to handle the changes needed. Not knowing exactly what you want to do, it is hard to say what is the best way to handle your particular situation. Whatever you do with the triggers, remember triggers operate on sets of data not one row at a time. They should always be written to handle multiple row inserts,updates or deletes or sooner or later your trigger will cause data integrity problems. Do not do this in a cursor unless you like having your production tables locked for hours at a time when someone needs to put in 3,000,000 new records for a new client.
if this is what you want:
check database A for updates of table1, table2, table3, and or table4
then update database B table5 and/or table6
you need to use a stored procedure to encapsulate all of the necessary logic and transactions for the original updates in database A and the resulting updates in database B
Are you asking if you update these three tables, then fire a trigger? But if you only update two of the three tables do not fire the trigger?
Your triggers can update any number of tables, they can also cause other triggers to fire, and if you like to live on the dangerous side you can even have these be recursive causing the original trigger to fire again.
However, nothing exists that can cordinate what I think you described. Not to say it can't be done.
Related
As in the title, I am developing an application using C# and WPF which acts as a client on many computers and handling a data using SQL within a company. I want it to refresh the views of the items on all computers using this application when one person adds or deletes something from the server's db. I know I might need to use SQL triggers, but I am kind of confused where to start.
just this type of idea:
private void Timer_Tick(object sender, EventArgs e)
{
if (trigger1.triggered == true)
{
RefreshView();
trigger1.triggered == false;
}
}
It's not a trigger you want. It's Query Notification.
SQL Triggers occur when something happens in the database, and allow you to do things like delete related rows in tables without a foreign key to maintain data integrity. I don't think that they will be of any benefit here because they are intended to do things within the database rather than externally.
The problem with your request and your proposed way of doing it is that it could cause a lot of network traffic. If you don't have a problem with that, one way would be to use a timestamp. When the record changes, save the new timestamp in the database.
Now, in your timer on each machine, check to see if the timestamp has changed since the last time you checked. If it has, reload the data. If not, continue running.
There will be lots of things to think about with this proposition, though. For example, what if the data on your screen has changed but not been saved when someone else changes it? Do you lose your changes? Do you list the changes and ask what to do? Are you actually entitled to make those changes or does it require administrator approval?
It would be more normal when using Optimistic Locking (last record written is the correct one) to check for a clash at save time rather than polling for the changes. That way network traffic is reduced but you are told if the record has changed since you loaded it and given the available options on how to proceed.
In the case of lists of records, the simple way to avoid huge network traffic is a simple Refresh button to reload the list.
This article may give you some ideas on ensuring data integrity in multi-user environments: https://www.codeproject.com/Articles/1178358/What-You-See-Is-What-You-Update
I hope that this gives you some food for thought.
What I wanted to do is to avoid calling the method on refresh because as #oldcoder said, the network traffic will be unnecessarily big. Just was thinking of something like INSERT from pc1 -> DB -> send information about the insert to pc2,pc3. If this is not possible or too complicating, I will just refresh the function every 10sec or so.
I'm trying to record log files on my database. My question is which has the less load on making logs on the database. I'm thinking of storing long term log files ,maybe 3-5 years maximum, for an Inventory Program.
Process: I'll be using a barcode scanner.
After scanning a barcode, I'll get all the details of who is logged in, date and time, product details then saved per piece.
I came up with two ideas.
After the scanning event, It will be saved on a DataTable then after finishing a batch.. DataTable will be written on a *.txt file and then uploaded to my database.
After every scanned barcode, an INSERT query will be executed. I suspect this option will be heavy on the server side since I'm not the only one using this server
What are the pros and cons of the two options?
Are there more efficient ways of storing logs?
Based on your use case, I also think you need to consider at least 2 additional factors, the first being how important is it that the scanned item is logged in the database immediately. If you need the scanned item to be logged because you'll be checking to see if its been scanned, for example to prevent other scans, then doing a single insert is probably a very good idea. The second thing to consider is will you ever need to "unscan" an item, and at which part of the process? If the person scanning needs the ability to revert the scan immediately, it might be a good idea to wait until theyre done all their scannings before dumping the data to the database, as this will let you avoid ever having to delete from the table.
Overall I wouldnt worry too much about what the database can handle, sql-server is very good at handling simultaneous single inserts into a table thats designed for that use case. If youre only going to be inserting new data to the end of the table, and not updating or deleting existing records, performance is going to scale very well. The same goes for larger batch inserts, theyre very efficient no matter how many rows you want to bring in, assuming your table is designed for that purpose.
So overall I would probably pick the more efficient solution from the application side for your specific use case, and then once you have decided that, you can shape the database around the code, rather than trying to shape your code around suspected limitations of the database.
What are the pros and cons of the two options?
Basically your question is which way is more efficient (bulk insert or multiple single insert)?
The answers is always depends and always be situation based. So unfortunately, I don't think there's a right answer for you
The way you structure the log table.
If you choose bulk insert, how many rows do you want to insert at 1 time?
Is it read-only table? And if you want to read from it, how often do you do the read?
Do you need to scale it up?
etc...
Are there more efficient ways of storing logs?
There're some possible ways to improve I can think of (not all of them can work together)
If you go with the first option, maybe you can schedule the insert to non-peak hours
If you go with the first option, chunk the log files and do the insert
Use another database to do the logging
If you go with the second option, do some load testing
Personally, I prefer to go with second option if the project is small to medium size and the logging is critical part of the project.
hope it helps.
Go with the second option, and use transactions. This way the data will not be sent to the db until you call the transaction to complete. (Which can be scheduled.) This will also prevent broken data from getting into your database when a crash or something occurs.
Transactions in .net
Transaction Tutorial in C#
I would like to have optimized version of my WinForms C# based application for slower connections. For this reason I wanted to introduce timestamp column into all tables (that change) and load most of things the first time it's needed and then just read updates/inserts/deletes that could have been done by other people using application.
For this question to have an example I've added a timestamp column into Table called Konsultanci. Considering that this table might be large I would like to load it once and then check for updates/inserts. In a simple way to load it all I do it like this:
private void KonsultantsListFill(ObjectListView listView)
{
using (var context = new EntityBazaCRM(Settings.sqlDataConnectionDetailsCRM)) {
ObjectSet<Konsultanci> listaKonsultantow = context.Konsultancis;
GlobalnaListaKonsultantow = listaKonsultantow.ToList(); // assign to global variable to be used all around the WinForms code.
}
}
How would I go with checking if anything changed to the table? Also how do I handle updates in WinForms c#? Should I be checking for changes on each tabpage select, opening new gui's, saving, loading of clients, consultants and so on? Should I be refreshing all tables all the time (like firing a background thread that is executed every single action that user does? or should it only be executed prior to eventual need for the data).
What I'm looking here is:
General advice on how to approach timestamp problem and refreshing data without having to load everything multiple times (slow connection issues)
A code example with Entity Framework considering timestamp column? Eventually code to be used prior executing something that requires data?
Timestamps are not well suited to help you detect when your cache needs to be updated. First off, they are not datetimes (read here) so they don't give you any clue as to when a record was updated. Timestamps are geared more towards assisting in optimistic locking and concurrency control, not cache management. When trying to update your cache you need a mechanism like a LastModified datetime field on your tables (make sure it's indexed!) and then a mechanism to periodically check for rows that have been modified since the last time you checked.
Regarding keeping your data fresh, you could run a separate query (possibly on another thread) that finds all records with the LastModified > than the last time you checked and then "upsert" (update or insert) them into your cache context. Another mechanism with Entity Framework is to use the Context.Refresh() method.
What would be the most efficent way to Loop through 25,000 records, and based on some prewritten vb logic that wont ever change(99% sure), update the Result column in a table to a value of 1, 2 or 3?
Performance and reliabilty is most important here. This most likely will get called via a client server app on the network but would be nice to be able to call it from a web app. I am thinking about 3 different ways to do it with T-SQL, C#.
a. Write an object that executes a stored procedure gets the 25,000 records, use the foreach collection to go through each record and based on some c# logic, call an object at each record that executes a stored procedure to update that row. This would call the object 25,000 times (and the proc I assume would just reuse the execution plan)
or
b. Write a stored procedure that gets the 25,000 records, use the forbidden cursor to go through each record and based on some T-SQL logic, update that row in this stored procedure.
or
UPDATED: MY SOLUTION IS THIS
For what it's worth I am going with persisited computed columns, and breaking the loop into smaller update statements to update the column (all wrapped in a transaction). See article below. I think it will be really fast, compared to a loop..
http://technet.microsoft.com/en-us/library/cc917696.aspx
You obviously have some condition that determines wheter the value should be 1,2 or 3. You could just do 3 update queries. Each query would update the records based on the condition that determines if the value should be 1, 2 or 3. Don't pull all the data down to your machine if you can help it.
My first choice would be to do it all in SQL if I could, i.e. update xxx set col=1 where (your logic here), update xxx set col=2 where (logic) etc.
If you need to do the logic in the vb client, either in a web app or client server, my choice would be to use a datareader to pass thru the records (pulling down only the columns that are required, not the whole row) and the either execute either a TSQL update or stored procedure to call to update those records that need to be updated, one at a time).
the datareader will give you the best performance; the SP should perform at least as good if not better than a TSQL update, (but probably not by much).
EDIT: Avoid server-side cursors at (almost) any cost...they are true hogs.
Solving this without entering c# is actually the best option if performance is key.
Run your queries outside c#.
If it's really necessary use DataReaders.
I would not go with option B. In my experience using cursors is extremely slow.
C. Use a DataReader and update the records with an ExecuteNonQuery
How about option (C) A stored procedure that updates the table using set-based logic rather than a cursor:
...
update x set col = f(x)
from x
...
Depending on how the updates work you have a couple options.
Have a computed column where the results are persisted. That way when the record changes it will be updated in one place.
Instead of running 25,000 update queries, just use sqlbulk load.
(and this is my preference). Have your app send the parameters to SQL server on what to update. In this case I'd lean towards using a static cursor as it would be a bit faster, as long as one record doesn't necessarily affect the next one.
You can either:
Go with the 3 separate UPDATEs
suggested by #Andrew
Pull the records into a Temporary
Table and loop through them in
batches of maybe 1000 records at a
time in a WHILE loop for the UPDATE
statement (so, 25 loops / UPDATEs)
Or, if you are using SQL Server 2008
(or newer) and the algorithm to
determine the change is complex, you
can pull the 25,000 rows into a
collection on the .Net side and
stream the changes back into a Proc
that has a Table-Valued Parameter
and do a single update. You can find an example of this at:
http://www.sqlservercentral.com/articles/SQL+Server+2008/66554/
In each case, you want to avoid 25,000 UPDATE calls.
I have similar situation. Actually, i have > 10.000.000 records. Business logic was rather complex, and there was old code purely written in SQL. Managers told me that with old code, it take 15+ hours per 1.000.000 records. With my solution, i took only 5 mins, literally ! I have done this in loop which have 3 steps in iteration, and each iteration took one batch of records:
Bulk load of invoices. I don't remember batch size, i think it was about few thousands.
Performing business logic on loaded records
Bulk insert. Because it was bulk, it couldn't be update. So it was bulk into temporary table, with almost same structure as original table, and then update by key in original table. Temporary table was emptied/deleted with every time of bulk insert. It is much faster than standard update.
I have an SQL Server 2008 Database and am using C# 4.0 with Linq to Entities classes setup for Database interaction.
There exists a table which is indexed on a DateTime column where the value is the insertion time for the row. Several new rows are added a second (~20) and I need to effectively pull them into memory so that I can display them in a GUI. For simplicity lets just say I need to show the newest 50 rows in a list displayed via WPF.
I am concerned with the load polling may place on the database and the time it will take to process new results forcing me to become a slow consumer (Getting stuck behind a backlog). I was hoping for some advice on an approach. The ones I'm considering are;
Poll the database in a tight loop (~1 result per query)
Poll the database every second (~20 results per query)
Create a database trigger for Inserts and tie it to an event in C# (SqlDependency)
I also have some options for access;
Linq-to-Entities Table Select
Raw SQL Query
Linq-to-Entities Stored Procedure
If you could shed some light on the pros and cons or suggest another way entirely I'd love to hear it.
The process which adds the rows to the table is not under my control, I wish only to read the rows never to modify or add. The most important things are to not overload the SQL Server, keep the GUI up to date and responsive and use as little memory as possible... you know, the basics ;)
Thanks!
I'm a little late to the party here, but if you have the feature on your edition of SQL Server 2008, there is a feature known as Change Data Capture that may help. Basically, you have to enable this feature both for the database and for the specific tables you need to capture. The built-in Change Data Capture process looks at the transaction log to determine what changes have been made to the table and records them in a pre-defined table structure. You can then query this table or pull results from the table into something friendlier (perhaps on another server altogether?). We are in the early stages of using this feature for a particular business requirement, and it seems to be working quite well thus far.
You would have to test whether this feature would meet your needs as far as speed, but it may help maintenance since no triggers are required and the data capture does not tie up your database tables themselves.
Rather than polling the database, maybe you can use the SQL Server Service broker and perform the read from there, even pushing which rows are new. Then you can select from the table.
The most important thing I would see here is having an index on the way you identify new rows (a timestamp?). That way your query would select the top entries from the index instead of querying the table every time.
Test, test, test! Benchmark your performance for any tactic you want to try. The biggest issues to resolve are how the data is stored and any locking and consistency issues you need to deal with.
If you table is updated constantly with 20 rows a second, then there is nothing better to do that pull every second or every few seconds. As long as you have an efficient way (meaning an index or clustered index) that can retrieve the last rows that were inserted, this method will consume the fewest resources.
IF the updates occur in burst of 20 updates per second but with significant periods of inactivity (minutes) in between, then you can use SqlDependency (which has absolutely nothing to do with triggers, by the way, read The Mysterious Notification for to udneratand how it actually works). You can mix LINQ with SqlDependency, see linq2cache.
Do you have to query to be notified of new data?
You may be better off using push notifications from a Service Bus (eg: NServiceBus).
Using notifications (i.e events) is almost always a better solution than using polling.