I have the problem that my database can contain invalid values that come from a previous version of the software. For these values we have to decide individually how to handle them.
I have created a corresponding Entity Framework that can be used to access the database. Internally on the test systems everything worked as well, but during the first test installation on a customer system the program always crashed due to the invalid values in the database.
Now the question is can I detect the errors in the database using the Entity Framework?
I would like to know in which records, which columns lead to errors.
I can't assume any Keys to get each Entity on its own. So my first idea was to check each repository (DbSet) from the DbContext for each entity (QueryingEnumerable). However, with DbSet you can't really access a single entity, you can only go through the query until the first error occurs.
Also, I still don't know which columns led to the error since the exceptions don't contain any information.
Any suggestions what else I can try?
If you have ids or keys for given entities held by dbSet, then you can get single entity.
// In repository or somewhere else, where you have access to this particular DbSet
DbSetOfSomeEntity.FirstOrDefault(x => x.Id == /* Some id number, guid etc */)
Access them in try-catch block, read exception and decide what to do next.
I'm using Entity Framework as my way of communicating with the database and fetching/writing information on it, on a ASP.NET CORE application. (Used as a very basic API, acting as a server for a separate application.)
There comes a time when the clients make requests to join a given lobby. I've just confirmed that if 4 requests at the same time enter, they will all be signed up on the lobby, but the player count did not update, and if it did - it'd go over the top/limit.
Am I using entity framework wrong? Is there an alternative tool to be used for such things, or should I just make it so it uses a single thread (If someone can remind me how), or encapsulate all my actions/endpoints with a lock block statement?
No matter how I structure my code, it's all prone to these same-timed http requests, moving parallelly through my repository/context.
It'd be great if I could make some kind of a queue, which I believe is what encapsulating everything in a lock would do.
EDIT:
As answered by vasily.sib, I can resolve this with the use of concurrency tokens. Please check his comment for some amazing information on how to use them!
Your problem is that operations like these...
_context.Sessions.FirstOrDefault(s => s.Id == sessionId).PlayerCount += 1;
_context.SaveChanges();
...are not atomic. With FirstOrDefault you get the session (which you dereference then without a null check, so First would be a better option here, since you will get a better error message). Then you save the changes in another step. Between those steps another concurrent thread could have changed and already saved a new value for PlayerCount.
There are multiple ways to resolve this, and most of them would require some changes on DB level.
One way to resolve it is to write a stored procedure that can do the update atomically. You would need an UPDATE statement similar to this one:
UPDATE Sessions
SET PlayerCount = PlayerCount + 1
FROM Sessions
WHERE Id = #SessionId
If you don't want to use stored procedures you could send this SQL directly to the DB.
One problem with this solution is that later you also do this step:
if (thisSession.Size == thisSession.PlayerCount)
{
thisSession.Status = SessionStatus.Active;
_context.SaveChanges(); // this can be trimmed and added at the previous scope
}
You would have to integrate this in your UPDATE statement to keep the operation atomic. If you want to add additional steps, things can get complicated.
Another way is to use optimistic concurrency built into EF core. In a nutshell, this means that when you save the data ef will first check whether the destination row is still in the same version compared to the moment you retrieved it.
To achieve that, your session needs a column that will contain a version counter. For SQL Server this would be a colun of type rowversion. Once you have this column, EF can do its optimistic concurrency magic. EF will throw a DbUpdateConcurrencyException which you would have to handle. There are different ways to resolve the error. One easy way would be to repeat your entire operation until it works out.
In my scenario, I typically put all my database persistence logic in a PersistChangeSet override, since it makes it easier for me to synchronize my save stored procedure wrapper calls involving entities that reference ids in other entities (for example, in case I have an entity A that references a temporary id of -1 in another entity B, I want to be able save entity B first, then update its id reference in entity A with the id returned from the database for entity B).
When one stored procedure returns an error for whatever reason, I just throw a custom exception with the message returned by the stored procedure and leverage the SubmitOperation object's HasError and Error properties on the client side (view model) in order to inform the user what went wrong.
The problem I'm having with the approach just described is that persistence errors rolls back the entire ChangeSet, even if , say, 2 out of 3 "persistence operations" (i.e. calls to stored procedure wrappers) succeed. So, for instance, if I had set a temporary id of -1 for an entity on the client side and then in PersistChangeSet I updated that id to a new value after a successful save to the database, by the time I get back to the SubmitChanges callback that id will have reverted to -1 (because some other entitity failed to save in PersistChangeSet). Ideally, entities that were persisted successfully should be removed from the ChangeSet by the time SubmitChanges completed.
I hope I have described my issue sufficiently well, but feel free to ask for any clarifications. I'm open to suggestions in case the approach I described above is not ideal or otherwise violates the governing principles of RIA.
Thanks in advance for your help!
I resolved this issue by using a TransactionScope in an override of the Submit method in my domain service. This allows the PersistChangeSet to behave as a unit of work (in other words, changes are only persisted if all operations are successful).
Details here: http://msdn.microsoft.com/en-us/library/ee707364(v=vs.91).aspx
Say I have an SQL table that has a unique key (or other) constraint on one of the columns.
Then, I have an application (lets go ASP.NET MVC) that allows a user to edit this column.
When the user attempts to save, and the constraint is/will be broken, I need show a user-friendly error message to the user. As such, which of the following is better practice?
Have the application query the database to ensure the constraints are not broken, and then insert/update the row (results in two queries to the db)
OR
Immediately attempt to perform the insert/update, and catch the SqlException should the constraint be broken. I like this, because of only one trip to the db, however how should you then extract which index/constraint was broken and affix the appropriate message? (apart from inspecting Exception.Message?)
1 is the better all-around solution. Db calls are not the only expensive operation, exceptions are costly too. You should really only let your db throw an exception when something goes wrong due to your code, not a user action.
For example, you might later want to install Elmah and get exceptions logged and/or mailed to you. Elmah would log/mail such an exception unless you explicitly told it not to.
Plus like you said, exceptions don't always have the most businessy information to communicate to users (especially SQL exceptions, which are SQL-specific). You have these unique and other constraints for a reason. For those reasons, validate the information before trying to store it.
Take twitter, gmail, etc. When you go to get a username, the application first checks to see if it is taken. These are uniqueness constraints at the application level, which may or may not ultimately be realized as SQL constraints. Since it is your application that faces the user, you should not try to communicate to them by translating from SQL to English.
The problem with option 1 is that it requires the application to have separate distinct knowledge of what the constraints are, which conflicts with a DRY principle, and could lead to problems later when the database constraint gets changed, and the application is not updated - a tight coupling between the data logic, and the application. Also, there is no guarantee of the constraint checks that were performed are still valid at the subsequent pont when you then attempt the update.
This doesn't preclude augmenting your application with some UI layer validation to aid the user before the commit is attempted.
So if that is discarded we are left with option 2, and some translation is necessary somewhere in the application. Where you put that translation and that logic, whether in a Stored Procedurce or a Business Logic layer, is up to you.
I work on a database application written in C# with sql server as backend. And for data integrity, I try to enforce as much as possible on database level - relations, check constraints, triggers.
Due to them, if data is not consistent, the save / update / insert can fail, and app throw SqlException.
I do various validations both in UI (to present user with meaningful info if data entered is not valid), also in BL, which reports errors back to UI which presents it to user.
However, there are things that really cannot be checked in the app, and are handled by the db: I mean errors on delete when no cascade delete and user try to delete a entity from a master table, etc.
E.g. Employees table acts as master in lot of relations - manager of employee, manager of department, cashier, team leader, teams members etc, etc. If I add anew employee which is not involved in any relation I can delete it, but of user try to delete one that is master oin such relation, the delete fails (as it should) due to RI rules enforced at DB level, and that's ok.
I write delete code in a try ... catch and handle the exception, telling user he cannot delete that employee. But I want to give user more meaningful info - the reason the record cannot be deleted. Maybe it was just a test employee record, which was also added to a test team. But user forget where added that and if I could tell "Cannot delete employee because it is part of team T1", user will know to go first to Team T1, remove user then try to delete it again. That's a simple example, since as I said an employee can be involved in lot of relations - in my app I have at least 20.
The solution is to display the Message reported by SqlException, but that's not elegant at all. First, that msg is very technical - it talks about FK, PK, Triggers, which are meaningless for users and will scare them. Second my app is uses multi-lang UI and all menus and msgs are shown in user selected language (selected either at login time or in user profile). And the msg from SqlException is english (if I use english version) or worst, less common languages, like german or dutch, if it happens that sql server is in that language.
Is there any common or recommended approach to extract meaningful info from sql exception to be able to present user a meaningful msg (e.g. what relation or child table caused the failure, or what trigger, etc). but something I can test in program in a lang-independent fashion and then format my own error msg in a user-friendly way?
How do you handle this situation?
Thanks for all answers
(PS: Sorry for the long post)
Unfortunately, there isn't an easy answer here.
The amount of work involved will depend on the consistency of your error messages coming from your business layer. You are going to need to do some form of translation from the "technical" error message to your user oriented message.
This should be a matter of making some forms of lookups from your error messages to a resource key, which can be used to pull out your language-specific error message. However, if you need to parse the messages for more information (ie: table names, etc), then it gets a bit trickier. In that case, you'll probably need to have something that maps an error message to a regex/processor of some form as well as a new resource string. Then you could format the user's string with the information you extract from the original error, and present it to the user.
Well, from the database, you'll only ever get these technical messages, e.g. "violation of foreign key relation FK_something_to_another" or such.
Typically, in the SqlException, you also get a SQL error code or something else.
The best approach would probably be to have a separate table in your database which basically maps those technical SQL errors that can happen to more meaningful, user-oriented messages. E.g. if your SQL error says something like "fk violation blablabaal", you could have an entry in your "UserErrorTable" which maps this to a user message saying "could not delete user (this.and.that), most likely because ..... (he's still member of a team)" or whatever.
You could then try to catch those SqlExceptions in your business layer, translate those technical infos into a custom exception for your users, put in a user-friendly message, and stick the technical exception into the .InnerException of your custom exception type:
public class UserFriendlyException : Exception
{
public string UserErrorMessage { get; set; }
public UserFriendlyException(string message, SqlException exc) : base(message, exc)
{
UserErrorMessage = MapTechnicalExecptionToUserMessage(exc);
}
}
Marc
The short answer is "don't." Let the errors bubble out to the global error handling/logging. The validation and business rules should generally preclude database exceptions, so you are better off failing hard and fast and not submitting dirty data. Transactions help too.
Error Messages do not equate Exceptions. An error message is something that the user should find informative and most important actionable. There are some guidelines around error messages in the User Experience Guidelines I recommend you read up. Apple also has good generic guidelines in Writing Good Alert Messages.
You'll immediately notice that most, if not all, of SQL errors are not good end user messages. 'Constraint FKXF#455 violation' - Bad error for end user. 'Filegroup is full' - Bad error for end user. 'Deadlock' - same. What good applications do is they separate the roles of users. Administrators need to see these errors, not end users. So the application always logs the full SQL error with all the details, eventually notifies the administrator, and then display a different error to the user, something like 'A system error occurred, the administrator was notified'.
If the SQL error can is actionable by the end user, then you can display him an error message instruct him what to do to fix the problem (eg. change the invoice date in the input to satisfy a constraint). But even in this case most times you should not display the SQL error straight to the user ( you already seen a very good reason why not: localization). I understand that this creates a much bigger workload for your development team. You must know what errors are user actionable in each case, from the multitude of errors you may catch. That is well known, and that's exactly why good program managers know that about 80% of the code is handling the error cases, and why the application being 'done' usually means is 20% done. This is what differentiates great apps from ordinary ones: how they behave when things go wrong.
My suggestion is to start up with the principle of progressive disclosure. Display an generic error message stating 'the operation has failed'. Offer to display more details if the user presses a 'Show details...' button on the error message dialog, and display the SqlError collection (btw, you should always log and display the entire SqlError collection of SqlException.Errors, not the SqlException). Use the SqlError.Number property to add logic in the catch block that decides if the user can do anything about the error (decide if the error is actionable) and add appropriate information.
Unfortunately there is no pixie dust. You are touching the foot hils of what is probably the most difficult part of your project.
The way to do this is to write a stored procedure, and use TRY and CATCH. Use RAISERROR to raise your own custom messages, and check the error code in the SqlException.
We usually write some sort of translator in our projects.We compare the SQL exception message with some predefine patterns and show the equivalent message to user
About generic technical > userfriendly errors i can only support the answers already giving.
But to your specific example with the employer i must encourage you to not rely only on the SqlException. Before trying to delete the Employee you should make some check to see if he/she is a part of any teams, is manager etc. It will tremendiosuly improve the usability of your application.
Pseudo:
Employee e;
try {
IEnumerable<Team> teams = Team.GetTeamsByEmployee(e);
if (teams.Count() > 0) {
throw new Exception("Employee is a part of team "+ String.Join(",", teams.Select(o => o.Name).ToArray());
}
IEnumerable<Employee> managingEmployees = Employee.GetEmployeesImManaging(e);
if (managingEmployees.Count() > 0) {
throw new Exception("Employee is manager for the following employees "+ String.Join(",", managingEmployees.Select(o => o.Name).ToArray());
}
Employee.Delete(e);
} catch (Exception e) {
// write out e
}
Errors happen. When it doesn't particularly matter what kind or type of error you hit, or how you handle it, slapping your code in a TRY...CATCH... block and write a generic error reporting system is fine. When you want (or are required to) write something better than that, it can take serious effort, as outlined in some prior posts (which I too have upvoted).
I tend to classify errors as anticipatable or unanticipatable. If you can anticipate an error and you want to handle it with a clear an consice message, such as with your "delete employee" situation, you will have to plan and code accordingly. By their definition, you cannot do this for unanticipatable errors--that's usually where TRY/CATCH comes in.
For your situation, one way could be before deleting the row, check via queries against the child tables whether or not the delete could succeed. If it won't, you'll know precisely why (and for all child tables, not just the first one that would block the delete), and can present the user with a clear message. Alas, you must consider whether the data can change between the check and the actual delete -- perhaps not a problem here, but it could be in other situations.
An alternative based on the TRY...CATCH... would be to check the error number within the catch block. If it's a "can't delete due to foreign key", you could then query the children tables and generate the message, and if it was some other unanticipated error, you'd have to fall back on a generic message.
(A caveat: some times when it hits an error, SQL will raise two error messages in a row [is the FK constraint violation one of them?] and in these situations the various ERROR() functions will only return data for the second and invariably less useful message. This is incredibly aggravating, but there's not too much you can do about it.)
In summary I would caution against relying on SQL exceptions and errors. For me, if we rely on such errors we are storing up trouble. Such error messsages are normally not user readable. In addition UI developers may say "oh well, it will be caught by the database guys, I do not need to validate that". Totally incorrect!
Perhaps a better approach would be to ensure that validation prevents these issues in the first place. e.g. If you cannot delete object A because it is referenced by object B, then we should implement some kind of dependency service at a level higher than the database. It depends what kind of application it is.
Another example would be string length validation. Do we really want to rely on the database validation for checking string length of fields entered by the user. Of course, this is an extreme case!
Normally, if the database layer is throwing an exception it can be a sign of a bug elsewhere. With a client/server setup we can assert that validation is the responsibility of both. If it makes it to the database then normally it is too late.
You may want to use RAISEERROR so the calling code can be fed with an exception from a stored procedure. From that you can provide a sensible error message.
There is never a one size fits all approach. My motto here would be prevention is better than cure! Validate earlier rather than later.