I read many posts saying multithreaded applications must use a separate session per thread. Perhaps I don't understand how the locking works, but if I put a lock on the session in all repository methods, would that not make a single static session thread safe?
like:
public void SaveOrUpdate(T instance)
{
if (instance == null) return;
lock (_session)
using (ITransaction transaction = _session.BeginTransaction())
{
lock (instance)
{
_session.SaveOrUpdate(instance);
transaction.Commit();
}
}
}
EDIT:
Please consider the context/type of applications I'm writing:
Not multi-user, not typical user-interaction, but a self-running robot reacting to remote events like financial data and order-updates, performing tasks and saves based on that. Intermittently this can create clusters of up to 10 saves per second. Typically it's the same object graph that needs to be saved every time. Also, on startup, the program does load the full database into an entity-object-graph. So it basically just reads once, then performs SaveOrUpdates as it runs.
Given that the application is typically editing the same object graph, perhaps it would make more sense to have a single thread dedicated to applying these edits to the object graph and then saving them to the database, or perhaps a pool of threads servicing a common queue of edits, where each thread has it's own (dedicated) session that it does not need to lock. Look up producer/consumer queues (to start, look here).
Something like this:
[Producer Threads]
Edit Event -\ [Database Servicer Thread]
Edit Event ------> Queue -> Dequeue and Apply to Session -> Database
Edit Event -/
I'd imagine that a BlockingCollection<Action<Session>> would be a good starting point for such an implementation.
Here's a rough example (note this is obviously untested):
// Assuming you have a work queue defined as
public static BlockingCollection<Action<Session>> myWorkQueue = new BlockingCollection<Action<Session>>();
// and your eventargs looks something like this
public class MyObjectUpdatedEventArgs : EventArgs {
public MyObject MyObject { get; set; }
}
// And one of your event handlers
public MyObjectWasChangedEventHandler(object sender, MyObjectUpdatedEventArgs e) {
myWorkQueue.Add(s=>SaveOrUpdate(e.MyObject));
}
// Then a thread in a constant loop processing these items could work:
public void ProcessWorkQueue() {
var mySession = mySessionFactory.CreateSession();
while (true) {
var nextWork = myWorkQueue.Take();
nextWork(mySession);
}
}
// And to run the above:
var dbUpdateThread = new Thread(ProcessWorkQueue);
dbUpdateThread.IsBackground = true;
dbUpdateThread.Start();
At least two disadvantages are:
You are reducing the performance significantly. Having this on a busy web server is like having a crowd outside a cinema but letting people go in through a person-wide entrance.
A session has its internal identity map (cache). A single session per application means that the memory consumption grows as users access different data from the database. Ultimately you can even end up with the whole database in the memory which of course would just not work. This requires then calling a method to drop the 1st level cache from time to time. However, there is no good moment to drop the cache. You just can't drop in at the beginning of a request because other concurrent sessions could suffer from this.
I am sure people will add other disadvantages.
Related
I have some data that needs to be processed. The data is a tree. The processing goes like this: Take a node N. Check if all of its children have already been processed. If not, process them first. If yes, process N. So we go from top to bottom (recursively) to the leaves, then process leaves, then the leaves' parent nodes and so on, upwards until we arrive at the root again.
I know how to write a program that runs on ONE computer that takes the data (i.e. the root node) and processes it as described above. Here is a sketch in C#:
// We assume data is already there, so I do not provide constructor/setters.
public class Data
{
public object OwnData { get; }
public IList<Data> Children { get; }
}
// The main class. We just need to call Process once and wait for it to finish.
public class DataManager
{
internal ISet<Data> ProcessedData { get; init; }
public DataManager()
{
ProcessedData = new HashSet<Data>();
}
public void Process(Data rootData)
{
new DataHandler(this).Process(rootData);
}
}
// The handler class that processes data recursively by spawning new instances.
// It informs the manager about data processed.
internal class DataHandler
{
private readonly DataManager Manager;
internal DataHandler(ProcessManager manager)
{
Manager = manager;
}
internal void Process(Data data)
{
if (Manager.ProcessedData.Contains(data))
return;
foreach (var subData in data.Children)
new DataHandler(Manager).Process(subData);
... // do some processing of OwnData
Manager.ProcessedData.Add(data);
}
}
But how can I write the program so that I can distribute the work to a pool of computers (that are all in the same network, either some local one or the internet)? What do I need to do for that?
Some thoughts/ideas:
The DataManager should run on one computer (the main one / the sever?); the DataHandlers should run on all the others (the clients?).
The DataManager needs to know the computers by some id (what id would that be?) which are set during construction of DataManager.
The DataManager must be able to create new instances of DataHandler (or kill them if something goes wrong) on these computers. How?
The DataManager must know which computers currently have a running instance of DataHandler and which not, so that it can decide on which computer it can spawn the next DataHandler (or, if none is free, wait).
These are not requirements! I do not know if these ideas are viable.
In the above thoughts I assumed that each computer can just have one instance of DataHandler. I know this is not necessarily so (because CPU cores and threads...), but in my use case it might actually be that way: The real DataManager and DataHandler are not standalone but run in a SolidWorks context. So in order to run any of that code, I need to have a running SolidWorks instance. From my experience, more than one SolidWorks instance on the same Windows does not work (reliably).
From my half-knowledge it looks like what I need is a kind of multi-computer-OS: In a single-computer-setting, the points 2, 3 and 4 are usually taken care of by the OS. And point 1 kind of is the OS (the OS=DataManager spawns processes=DataHandlers; the OS keeps track of data=ProcessedData and the processes report back).
What exactly do I want to know?
Hints to words, phrases or introductory articles that allow me to dive into the topic (in order to become able to implement this). Possibly language-agnostic.
Hints to C# libraries/frameworks that are fit for this situation.
Tips on what I should or shouldn't do (typical beginners issues). Possibly language-agnostic.
Links to example/demonstration C# projects, e.g. on GitHub. (If not C#, VB is also alright.)
You should read up on microservices and queues. Like rabbitmq.
The producer/ consumer approach.
https://www.rabbitmq.com/getstarted.html
If you integrate your microservices with Docker, you can do some pretty nifty stuff.
I have a question, I hope you can help me, Thank you in advance.
I am working in a project, a WEB Application hosted in IIS; the approach is that I have a LogIn for users, but the LogIn must allow one user to login at time, so if two users are trying to access to the site at the same time only one should access, while the other one waits until the first is logged in. I thought of using threads, with a lock statement in the Sign In validation, but I don't know if it is a good practice to use threads in this scenario, due to multiple users may try to Log In at the same time, and only one must access at time. Also, I need to have a log for the users in the order they have accessed the site, to verify that two users did not access at the same time.
Is multithreading a good practice or recommendation for making this?
Any suggestions? Thank you so much.
First off, when using threads its good practice to avoid anything that will block a thread, if at all possible.
You could use a lock which would cause incoming threads to block until the first thread has completed the login process, although I can't see how this would help in understanding multithreading. This will only help in learning how to block threads, which you should try to avoid at all costs, threads are expensive.
IMHO you should never have more threads than CPU cores, use the threadpool, understand the difference between compute bound and I\O bound threads. I say again threads are expensive, in both time and memory.
Well, this is solution is not so much about multithreading - but i would do something like this:
public class SingleUserLock : IDisposable
{
private SingleUserSemaphore _parent;
public SingleUserLock(SingleUserSemaphore parent)
{
_parent = parent;
}
public bool IsLoggedIn => _parent?.CurrentUser == this;
public void Unlock()
{
_parent?.Unlock();
_parent = null;
}
public void Dispose()
{
Unlock();
}
}
public class SingleUserSemaphore
{
private readonly object _lockObject = new object();
public SingleUserLock CurrentUser { get; private set; }
public bool TryLogin()
{
if (Monitor.TryEnter(_lockObject))
{
CurrentUser = new SingleUserLock(this);
return true;
}
return false;
}
public void Unlock()
{
try
{
Monitor.Exit(_lockObject);
CurrentUser = null;
}
catch (Exception ex) { };
}
}
Register an instance of SingleUserSemaphore as a Singleton in your DependecyInjection framework for the Web application. Every time a user logs in, you get the singleton SingleUserSemaphore instance and call TryLogin. If true you can the store SingleUserLock in the Session if possible.
For every request check the session for IsLoggedIn == true.
When the user logs out you call the returned SingleUserLock.Unlock(); from the session or directly SingleUserSemaphore.Unlock();
Now the challenge will be, if the user never Logs out. Your web application will be locked forever. To avoid this you could make an Update method on SingleUserSemaphore with a timestamp for every request made by the logged in user. So when a user logs in, you also check for last activity...
Good luck with you homework.
Short question
How can I lock my entity so that only one operation by only one user can be performed on it at a time in MVC project?
Long question
I have MVC project where I want my action methods to be [SessionState(SessionStateBehavior.ReadOnly)]. But when doing this users can execute another action methods even before one long running action method has not completed. As I have a lot calculations and action methods have to be executed in predefined order, executing another Action method before one ends creates lots of problems. To give example I have main entity called Report, I have to somehow ensure that one report undergoes only one operation by only one user at a time. So I have to lock my Report. Even if I do not use [SessionState(SessionStateBehavior.ReadOnly)] I have to lock report so that multiple users do not edit same reports at a time and for other specific reasons. Currently I am writing this information to database roughly something like:
ReportId
LockedUserId
IsInPorcess
I have to set IsInProcess to true every time before operation begins and reset it to false after operation completed. As I have lots of action methods I created ActionFilter something like below:
public class ManageReportLockAttribute
: FilterAttribute, IActionFilter
{
public ManageReportLockAttribute()
{
}
public void OnActionExecuting(ActionExecutingContext filterContext)
{
...
ReportLockInfo lockInfo = GetFromDatabase(reportId);
if(lockInfo.IsInProcess)
RedirectToInformationView();
lockInfo.IsInProcess = true;
SaveToDatabase(lockInfo);
...
}
public void OnActionExecuted(ActionExecutedContext filterContext)
{
...
ReportLockInfo lockInfo = GetFromDatabase(reportId);
lockInfo.IsInProcess = false;
SaveToDatabase(lockInfo);
...
}
}
It works, for most part, but it has some strange problems (see this question for more info).
My question is that "How can I achieve same functionality (locking report) by different more acceptable way?".
I feel like it is something similar to locking when using multithreading, but it is not exactly same IMO.
Sorry for long, broad and awkward question, but I want a direction to follow. Thanks in advance.
One reason why OnActionExecuted is not called though OnActionExecuting runs as expected is that there are unhandled exceptions that occur in OnActionExecuting. Especially when dealing with the database, there are various reasons that could lead to an exception, e.g.:
User1 starts the process and locks the entity.
User2 also wants to start the process before User1 has saved the change. So the check of IsInProcess does not lead to the redirection and User2 also wants to save the lock. In this case, a concurrency violation should occur because User1 has saved the entity in the meantime.
To illustrate the process over time (C is the check whether IsInProcess is set, S is SaveChanges): first a good case:
User1 CS
User2 CS (check is done after save, no problem)
Now a bad case:
User1 CS
User2 CS (check takes place after check for User1, but before SaveChanges becomes effective ==> concurrency violation)
As the example shows, it is critical to make sure that only one user can place the lock. There are several ways to handle this. In all cases make sure that there are as few reasons for exceptions in OnActionExecuting as possible. Handle and log the exceptions.
Please note that all synchronisation methods will have a negative impact on the performance of your application. So if you haven't already thought about whether you could avoid having to lock the report by restructuring your actions or the data model, this would be the first thing to do.
Easy approach: thread synchronisation
An easy approach is to use thread synchronisation. This approach will only work if the application runs in a single process and not in a web farm/the cloud. You need to decide whether you will be able to change the application if it will be installed in a farm at a later point in time. This sample shows an easy approach (that uses a static object for locking):
public class ManageReportLockAttribute
: FilterAttribute, IActionFilter
{
private static readonly object lockObj = new object();
// ...
public void OnActionExecuting(ActionExecutingContext filterContext)
{
...
ReportLockInfo lockInfo = GetFromDatabase(reportId);
if(lockInfo.IsInProcess)
RedirectToInformationView();
lock(lockObj)
{
// read anew just in case the lock was set in the meantime
// A new context should be used.
lockInfo = GetFromDatabase(reportId);
if(lockInfo.IsInProcess)
RedirectToInformationView();
lockInfo.IsInProcess = true;
SaveToDatabase(lockInfo);
...
}
}
public void OnActionExecuted(ActionExecutedContext filterContext)
{
...
lock(lockObj)
{
lockInfo = GetFromDatabase(reportId);
if (lockInfo.IsInProcess) // check whether lock was released in the meantime
{
lockInfo.IsInProcess = false;
SaveToDatabase(lockInfo);
}
...
}
}
}
For details on using lock see this link. If you need more control, have a look at the overview of thread synchronization with C#. A named mutex is an alternative that provides locking in a more fine coarsed manner.
If you want to lock on reportId instead of a static object, you need to use a lock object that is the same for the same reportId. A dictionary can store the lock objects:
private static readonly IDictionary<int, object> lockObjectsByReportId = new Dictionary<int, object>();
private static object GetLockObjectByReportId(int reportId)
{
int lockObjByReportId;
if (lockObjectsByReportId.TryGetValue(reportId, out lockObjByReportId))
return lockObjByReportId;
lock(lockObj) // use global lock for a short operation
{
if (lockObjectsByReportId.TryGetValue(reportId, out lockObjByReportId))
return lockObjByReportId;
lockObjByReportId = new object();
lockObjectsByReportId.Add(reportId, lockObjByReportId);
return lockObjByReportId;
}
}
Instead of using lockObj in OnActionExecuting and OnActionExecuted, you'd use the function:
// ...
lock(GetLockObjectByReportId(reportId))
{
// ...
}
Database approach: Transactions and isolation levels
Another way to handle this is to use database transactions and isolation levels. This approach will also work in a multi-server environment. In this case, you'd not use the entity framework for database access but move the code to a stored procedure that is run on the database server. By running the stored procedure in a transaction and picking the right isolation level, you can avoid that a user can read the data while another one is changing them.
This link shows an overview of isolation levels for SQL Server.
I have following scenario:
Lets say we have two different webparts operating on the same data - one is a pie chart, another is a data table.
in their Page_Load they asynchronously load data from the database and when loaded place it in application cache for further use or use by other web parts. So each o the web parts has code similar to this:
protected void Page_Load(object sender, EventArgs e)
{
if (Cache["dtMine" + "_" + Session["UserID"].ToString()]==null)
{
...
Page.RegisterAsyncTask(new PageAsyncTask(
new BeginEventHandler(BeginGetByUserID),
new EndEventHandler(EndGetByUserID),
null, args, true));
}
else
{
get data from cache and bind to controls of the webpart
}
}
Since both webparts operate on the same data it does not make sense for me to execute the code twice.
What is the best approach to have one web part communicate to the other "i am already fetching data so just wait until i place it in cache"?
I have been considering mutex, lock, assigning temporary value to the cache item and waiting until that temporary value changes... many options - which one should I use.
You will want to take advantage of the lock keyword to make sure that the data is loaded and added to the cache in an atomic manner.
Update:
I modified the example to hold the lock accessing Cache for as short as possible. Instead of storing the data directly in the cache a proxy will be stored instead. The proxy will be created and added to the cache in an atomic manner. The proxy will then use its own locking to make sure that the data is only loaded once.
protected void Page_Load(object sender, EventArgs e)
{
string key = "dtMine" + "_" + Session["UserID"].ToString();
DataProxy proxy = null;
lock (Cache)
{
proxy = Cache[key];
if (proxy == null)
{
proxy = new DataProxy();
Cache[key] = proxy;
}
}
object data = proxy.GetData();
}
private class DataProxy
{
private object data = null;
public object GetData()
{
lock (this)
{
if (data == null)
{
data = LoadData(); // This is what actually loads the data.
}
return data;
}
}
}
Why don't you load the data and put it in the cache in the Application_Start in Global.asax, then no lock will be needed since locking Cache is a serious thing.
You could use a mutex around the test Cache["key"]==null:
The first thread acquires the lock, tests and sees that there's nothing in the cache and goes off to fetch the data. The second thread has to wait for the first to release the mutex. Once the second thread gets in the mutex, it tests, sees the data is there and then continues.
But this would lock up the thread that is running the Page_Load() method - probably a bad thing.
Perhaps a better solution would be to also test if the PageAsyncTask to fetch the data has been started? If not, start it. If so, you shouldn't start another so you may want to register your own event handler to catch when it completes...
This link http://msdn.microsoft.com/en-us/library/aa772153(VS.85).aspx says:
You can register up to five notification requests on a single LDAP connection. You must have a dedicated thread that waits for the notifications and processes them quickly. When you call the ldap_search_ext function to register a notification request, the function returns a message identifier that identifies that request. You then use the ldap_result function to wait for change notifications. When a change occurs, the server sends you an LDAP message that contains the message identifier for the notification request that generated the notification. This causes the ldap_result function to return with search results that identify the object that changed.
I cannot find a similar behavior looking through the .NET documentation. If anyone knows how to do this in C# I'd be very grateful to know. I'm looking to see when attributes change on all the users in the system so I can perform custom actions depending on what changed.
I've looked through stackoverflow and other sources with no luck.
Thanks.
I'm not sure it does what you need, but have a look at http://dunnry.com/blog/ImplementingChangeNotificationsInNET.aspx
Edit: Added text and code from the article:
There are three ways of figuring out things that have changed in Active Directory (or ADAM). These have been documented for some time over at MSDN in the aptly titled "Overview of Change Tracking Techniques". In summary: Polling for Changes using uSNChanged. This technique checks the 'highestCommittedUSN' value to start and then performs searches for 'uSNChanged' values that are higher subsequently. The 'uSNChanged' attribute is not replicated between domain controllers, so you must go back to the same domain controller each time for consistency. Essentially, you perform a search looking for the highest 'uSNChanged' value + 1 and then read in the results tracking them in any way you wish. Benefits This is the most compatible way. All languages and all versions of .NET support this way since it is a simple search. Disadvantages There is a lot here for the developer to take care of. You get the entire object back, and you must determine what has changed on the object (and if you care about that change). Dealing with deleted objects is a pain. This is a polling technique, so it is only as real-time as how often you query. This can be a good thing depending on the application. Note, intermediate values are not tracked here either. Polling for Changes Using the DirSync Control. This technique uses the ADS_SEARCHPREF_DIRSYNC option in ADSI and the LDAP_SERVER_DIRSYNC_OID control under the covers. Simply make an initial search, store the cookie, and then later search again and send the cookie. It will return only the objects that have changed. Benefits This is an easy model to follow. Both System.DirectoryServices and System.DirectoryServices.Protocols support this option. Filtering can reduce what you need to bother with. As an example, if my initial search is for all users "(objectClass=user)", I can subsequently filter on polling with "(sn=dunn)" and only get back the combination of both filters, instead of having to deal with everything from the intial filter. Windows 2003+ option removes the administrative limitation for using this option (object security). Windows 2003+ option will also give you the ability to return only the incremental values that have changed in large multi-valued attributes. This is a really nice feature. Deals well with deleted objects. Disadvantages This is .NET 2.0+ or later only option. Users of .NET 1.1 will need to use uSNChanged Tracking. Scripting languages cannot use this method. You can only scope the search to a partition. If you want to track only a particular OU or object, you must sort out those results yourself later. Using this with non-Windows 2003 mode domains comes with the restriction that you must have replication get changes permissions (default only admin) to use. This is a polling technique. It does not track intermediate values either. So, if an object you want to track changes between the searches multiple times, you will only get the last change. This can be an advantage depending on the application. Change Notifications in Active Directory. This technique registers a search on a separate thread that will receive notifications when any object changes that matches the filter. You can register up to 5 notifications per async connection. Benefits Instant notification. The other techniques require polling. Because this is a notification, you will get all changes, even the intermediate ones that would have been lost in the other two techniques. Disadvantages Relatively resource intensive. You don't want to do a whole ton of these as it could cause scalability issues with your controller. This only tells you if the object has changed, but it does not tell you what the change was. You need to figure out if the attribute you care about has changed or not. That being said, it is pretty easy to tell if the object has been deleted (easier than uSNChanged polling at least). You can only do this in unmanaged code or with System.DirectoryServices.Protocols. For the most part, I have found that DirSync has fit the bill for me in virtually every situation. I never bothered to try any of the other techniques. However, a reader asked if there was a way to do the change notifications in .NET. I figured it was possible using SDS.P, but had never tried it. Turns out, it is possible and actually not too hard to do. My first thought on writing this was to use the sample code found on MSDN (and referenced from option #3) and simply convert this to System.DirectoryServices.Protocols. This turned out to be a dead end. The way you do it in SDS.P and the way the sample code works are different enough that it is of no help. Here is the solution I came up with:
public class ChangeNotifier : IDisposable
{
LdapConnection _connection;
HashSet<IAsyncResult> _results = new HashSet<IAsyncResult>();
public ChangeNotifier(LdapConnection connection)
{
_connection = connection;
_connection.AutoBind = true;
}
public void Register(string dn, SearchScope scope)
{
SearchRequest request = new SearchRequest(
dn, //root the search here
"(objectClass=*)", //very inclusive
scope, //any scope works
null //we are interested in all attributes
);
//register our search
request.Controls.Add(new DirectoryNotificationControl());
//we will send this async and register our callback
//note how we would like to have partial results
IAsyncResult result = _connection.BeginSendRequest(
request,
TimeSpan.FromDays(1), //set timeout to a day...
PartialResultProcessing.ReturnPartialResultsAndNotifyCallback,
Notify,
request);
//store the hash for disposal later
_results.Add(result);
}
private void Notify(IAsyncResult result)
{
//since our search is long running, we don't want to use EndSendRequest
PartialResultsCollection prc = _connection.GetPartialResults(result);
foreach (SearchResultEntry entry in prc)
{
OnObjectChanged(new ObjectChangedEventArgs(entry));
}
}
private void OnObjectChanged(ObjectChangedEventArgs args)
{
if (ObjectChanged != null)
{
ObjectChanged(this, args);
}
}
public event EventHandler<ObjectChangedEventArgs> ObjectChanged;
#region IDisposable Members
public void Dispose()
{
foreach (var result in _results)
{
//end each async search
_connection.Abort(result);
}
}
#endregion
}
public class ObjectChangedEventArgs : EventArgs
{
public ObjectChangedEventArgs(SearchResultEntry entry)
{
Result = entry;
}
public SearchResultEntry Result { get; set;}
}
It is a relatively simple class that you can use to register searches. The trick is using the GetPartialResults method in the callback method to get only the change that has just occurred. I have also included the very simplified EventArgs class I am using to pass results back. Note, I am not doing anything about threading here and I don't have any error handling (this is just a sample). You can consume this class like so:
static void Main(string[] args)
{
using (LdapConnection connect = CreateConnection("localhost"))
{
using (ChangeNotifier notifier = new ChangeNotifier(connect))
{
//register some objects for notifications (limit 5)
notifier.Register("dc=dunnry,dc=net", SearchScope.OneLevel);
notifier.Register("cn=testuser1,ou=users,dc=dunnry,dc=net", SearchScope.Base);
notifier.ObjectChanged += new EventHandler<ObjectChangedEventArgs>(notifier_ObjectChanged);
Console.WriteLine("Waiting for changes...");
Console.WriteLine();
Console.ReadLine();
}
}
}
static void notifier_ObjectChanged(object sender, ObjectChangedEventArgs e)
{
Console.WriteLine(e.Result.DistinguishedName);
foreach (string attrib in e.Result.Attributes.AttributeNames)
{
foreach (var item in e.Result.Attributes[attrib].GetValues(typeof(string)))
{
Console.WriteLine("\t{0}: {1}", attrib, item);
}
}
Console.WriteLine();
Console.WriteLine("====================");
Console.WriteLine();
}