In a previous question I presented these models:
public class Calendar
{
public int ID { get; set; }
public ICollection<Day> Days { get; set; }
}
public class Day
{
public int ID { get; set; }
public DateTime Date { get; set; }
public int CalendarID { get; set; }
}
There is a uniqueness constraint so that you can't have more than one Day with the same Date and CalendarID.
My question now is what if I want to move all days one day into the future (or whatever). The easiest code is just a for loop like
for(Day day in days) {
day.Date = day.Date.AddDays(1);
db.Entry(day).State = EntityState.Modified;
}
await db.SaveChangesAsync();
This will fail, however, because it thinks you are creating duplicates of some of the dates, even though once they're all updated it will work out.
Calling SaveChangesAsync after each day (assuming you process the days in the correct order) would work, but seems massively inefficient for this basic task.
An alternative to updating the Date would be to transfer all the other data of each day to the next one, but this also seems inefficient and could in some cases be undesirable because it means that data is dissociated from the Day's primary key value.
Is there a way to update all the dates while keeping the uniqueness constraint?
The number of SQL UPDATE statements won't change if you call SaveChanges() for each record instead of calling it only once, but at least you'll get the correct order. There's some overhead because of state cleaning and connection management but it's not massively inefficient.
If date shifting is an isolated business transaction you could use a simpler solution instead of fighting with ORM - call a stored procedure or execute SQL directly with something similar to:
var sql = "UPDATE d SET Date = DATEADD(d, 1, Date) FROM (SELECT * FROM Day WHERE CalendarID=#calendarId ORDER BY Date DESC) d";
var updateCnt = db.Database.ExecuteSqlCommand(sql, new SqlParameter("#calendarId", calendar.Id);
if (updateCnt != days.Count)
{
//oops
}
One of the many possible solutions is removing all the records before you do the update.
You can first get your days, store them in memory.
var days = db.Day.Tolist();
Truncate the table, so they won't collide with the new list coming:
db.ExecuteCommand("TRUNCATE TABLE Day");
Do your stuff:
foreach(var day in days)
{
day.Date=day.Date.AddDays(1);
}
Insert your new list.
Now you should be able to save it:
db.SaveChanges();
This should be efficient enough since the quickest way to wipe data is to truncate, and your day objects are child objects.
HOWEVER
If a property is changing a lot, probably it's not a good idea to make it a primary key.
If you find yourself in a conflict with fundamentals, it's quite possible that you made an architectural mistake.
I strongly recommend you to change your primary key to something else, you can even roll a uniqueidentifier column to store Id.
Related
I'm building an app using Xamarin.Forms, and I'm running into a really slow query in the data that I need to optimize if possible.
In order to understand the question I'm trying to frame, I need to do a good job of explaining the database relationships. The business software I'm trying to build allows the user to schedule employees onto crews and crews onto job operations. For the purposes of this explanation, we can ignore jobs, even though the database objects include 'job' in their name.
For scheduling employees day by day, I created a kanban that allows the user to drag employee names to the crew that they want to schedule them onto for the day, and when they are finished editing they use toolbar buttons to navigate to the next date they want to schedule. In the background, the code is creating database objects that create the link between an employee, a date, and a crew.
For scheduling operations day by day, I created a side-scrolling gantt-style scheduler that allows the user to drag operation blocks onto a crew for the day. In the background, the code is creating database objects that create the link between an operation, a date and a crew.
Here's a simple version of what the database objects look like.
public interface IEmployee
{
long? Id { get; set; }
string Name { get; }
string Password { get; set; }
}
public interface ICrewMember
{
long? Id { get; set; }
IEmployee Employee { get; set; }
bool IsLeader { get; set; }
ICrew Crew { get; set; }
DateTime Date { get; set; }
}
public interface IJobSchedule
{
long? Id { get; set; }
IOperation Operation { get; set; }
DateTime Date { get; set; }
ICrew Crew { get; set; }
}
public interface IOperation
{
long? Id { get; set; }
int Priority { get; set; }
}
So the complexity of the scenario comes when I want to find what operations an employee has been scheduled for. I have to first query to find the employee's schedule objects, create a list of crews/dates that they've been schedule for, then get a list of the job schedules that match the date/crew list, and then boil it down to a distinct list of operations (since an operation could get scheduled across multiple days). Here's my current code to do this:
public async Task<List<IOperation>> GetOperationsByEmployee(IDataService<IJobSchedule> JobScheduleRepository)
{
JobScheduleRepository = JobScheduleRepository ??
throw new ArgumentNullException(nameof(JobScheduleRepository));
var result = new List<IOperation>();
var empSchedMatches = await GetEmployeeSchedules().ConfigureAwait(false);
var jobSchedules = await GetJobSchedules(JobScheduleRepository, empSchedMatches).ConfigureAwait(false);
result = jobSchedules.Select(x => x.Operation).Distinct().ToList();
return result;
}
private async Task<IEnumerable<ICrewMember>> GetEmployeeSchedules()
{
//Get complete list of employee schedules to sort through
var allEmpSched = await CrewMemberRepository.GetItemsAsync().ConfigureAwait(false);
//Get schedules with date greater than or equal to Date for this employee
var empSchedMatches = allEmpSched.Where(x => x.Date >= Date && x.Employee == Employee);
return empSchedMatches;
}
private async Task<IEnumerable<IJobSchedule>> GetJobSchedules(IDataService<IJobSchedule> JobScheduleRepository, IEnumerable<ICrewMember> employeeSchedules)
{
//Get complete list of job schedules to sort through
var allJobSched = await JobScheduleRepository.GetItemsAsync().ConfigureAwait(false);
allJobSched = allJobSched.Where(x => x.Date >= Date && x.Crew != null && x.Operation != null);
int count = allJobSched.Count();
var result = new List<IJobSchedule>();
foreach (var empSched in employeeSchedules)
{
//For each employee schedule, there should be 1 matching job schedule
//if the crew was assigned a job for that day
var matches = allJobSched.Where(x => x.Date == empSched.Date && x.Crew == empSched.Crew);
result.AddRange(matches);
string message = $"GetJobSchedules() comparing ({count}) Job Schedules " +
$"to empSched.{empSched.Id} crew.{empSched.Crew.Id} date.{empSched.Date:M/d}";
System.Diagnostics.Debug.WriteLine(message);
}
return result;
}
In order to try to view the process, I had added in a number of different bits of code that printed the steps to the debugger, including a stopwatch. Here's the debugger output:
[0:] Method Called: GetOperationsByEmployee
[0:] GetOperationsByEmployee() executing query...
[0:] Method Called: GetEmployeeSchedules
[0:] Method Called: GetJobSchedules
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.17196 crew.3 date.2/6
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.18096 crew.3 date.2/4
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.18221 crew.3 date.2/3
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.18902 crew.3 date.2/7
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21243 crew.3 date.1/27
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21321 crew.3 date.1/28
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21360 crew.3 date.1/29
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21399 crew.3 date.1/30
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21438 crew.3 date.1/31
[0:] GetJobSchedules() comparing (51) Job Schedules to empSched.21528 crew.3 date.2/5
[0:] Data loaded 6391 ms
So when I'm running the app using mock data, with like 10 objects in memory, it runs in ~100 ms. When I've got 50,000 objects in the real database, with 30 employees, 10 crews, 500 jobs, and 1500 operations to sort through, it takes ~7,000 ms. That comparison was probably obvious, but the point is that I need to find some way, if possible, to optimize the query. I'd like to get it closer to 1 second load time if it can be done.
As always, thanks for any help!
Edit
I'm afraid I'm not getting the answers I was hoping for, because I'm not really looking for advice on the data access side of the question, I'm looking for advice on the LINQ side of the question. I don't know if it helps to understand the data access scenario, so I'll explain briefly.
I'm coding in Xamarin.Forms, using Autofac as my dependency injector. I'm trying to use interfaces to allow the calls to the data service to be abstracted from the data service.
The data is being stored in SQL Server on a server here at the office. The app is using an API for SQL to SQLite called Zumero. Zumero syncs the requested tables off the SQL Server and deposits them into a local file on the mobile device.
I'm using Entity Framework Core to serve the data to the program, again by using interfaces and field mapping to try to abstract the calls for the database objects apart from the database objects themselves.
Edit 2
I'm going to try to re-ask the question here so that it becomes more clear what I'm looking for:
I have a SQLite file that has employees, operations, daily employee schedules, and daily operation schedules. What are some ways that I could write a query to get an employee's list of operations that they've been schedule for?
Imaginary Question:
What are Bob's currently scheduled operations?
Imaginary rows in the data tables:
Employees
Bob
Jim
Larry
Employee Schedules
Bob 1/1/2020 Concrete Crew 1
Bob 1/2/2020 Concrete Crew 1
Bob 1/3/2020 Mill Crew 2
Operation Schedules
Crew 1 1/1/2020 Concrete Operation 1
Crew 2 1/1/2020 Mill Operation 1
Crew 1 1/2/2020 Concrete Operation 1
Crew 1 1/3/2020 Concrete Operation 3
Operations
Concrete Operation 1
Mill Operation 1
Concrete Operation 3
Desired Result:
Bob currently has the following operations on the schedule: Concrete Operation 1
It's a relational database question, of sorts, because I'm asking about the best way to figure out the link from employees through employee schedules through operation schedules to operations.
Thanks for any help!
The answer is simple, you are creating entire tables to runtime and matching them via C# and it is wrong.
This is what database is made for and you must use it.
You have many choices, queries, views, stored procedures, but for sure asking the entire db and performing matches via code is a wrong way.
There are a number of things that you could do to speed up your query. There is one obvious change that I would implement(if I understand the flow enough):
//change to something where you pass your parameters as to not need to load all of your jobs every time
var allJobSched = await JobScheduleRepository.GetItemsAsync().ConfigureAwait(false);
//change to
var matchingJobSched = await JobScheduleRepository.FindMatchingJobSchedules(DateTime date, int crewId).ConfigureAwait(false);
You seem to be doing this for each object, so this refactor should be done over all of your code.
Something else you could try is to write a stored procedure for this action and leave the ORM-time out.
At least from the way I am reading this you are doing a lot of queries where you're essentially fetching the whole set of data from your database before querying in memory. This works fine if you have a few records as you've discovered, however if your front loading whole tables every request you're going to find yourself completely bottlenecked, to the point where your appliacation will eventually cease to function entirely.
There's a lot of code here but just as an example.
var allJobSched = await JobScheduleRepository.GetItemsAsync().ConfigureAwait(false);
allJobSched = allJobSched.Where(x => x.Date >= Date && x.Crew != null && x.Operation != null);
int count = allJobSched.Count();
In the above snippet first you pull all of the Job schedules from the database this will invariably involve a large over the wire data pull from your database (not good).
You'd be much better if you write something similar to the following.
var jobs= await JobScheduleRepository.GetValidJobsAfterDate(Date);
The implementation of the query itself should be sent by your repostitory to your database an handled there, you should never be pulling back huge collections of data to work from in memory (in most cases).
IF, as you insist, this must be done 100% in LINQ, which I do not recommend as you need to do a better job pre-filtering the data coming in, then your issue is that you are performing an exhaustive search for every row in the table. Instead, I recommend creating a dictionary and first loading your rows into that.
Instead of:
private async Task<IEnumerable<ICrewMember>> GetEmployeeSchedules()
{
//Get complete list of employee schedules to sort through
var allEmpSched = await CrewMemberRepository.GetItemsAsync().ConfigureAwait(false);
//Get schedules with date greater than or equal to Date for this employee
var empSchedMatches = allEmpSched.Where(x => x.Date >= Date && x.Employee == Employee);
return empSchedMatches;
}
Do this:
var dict = new Dictionary<Employee, List<IJobSchedule>>
foreach(var item in allEmpSched) {
if (dict.TryGetValue(item.Employee, out var schedules)) {
schedules.Add(item)
} else {
dict[item.Employee] = new List<IJobSchedule>() { item };
}
}
Then, using that dictionary, look up your employee. That's essentially what the database will do for you automatically if you have a properly configured index along with an appropriate SELECT clause, but there you go.
Edit: I see that you're hoping to maybe relocate this to the ORM, such that it will write the correct SQL for you. I don't know how to do that, because I don't use ORMs.
Why don't I use ORMs? There are three reasons. First, data access is not something that should be given over to a machine to build into your application. Data access is often the most-used code you'll have in the software package. Pay attention to it and design it well. Computers cannot provide an effective substitute for proper design.
Second, the SQL language itself is an abstraction upon the physical means to access the underlying data. When a SQL query is executed, the first thing that the database engine does is to come up with a plan to execute it. In essence, the SQL is being interpreted and compiled down to generated code. If you throw another code generator on top of it (the ORM), the results are naturally going to vary, and rarely does an ORM produce good results without spending quite a bit of time tweaking. Spend your time writing good SQL queries.
Finally, ORMs don't really eliminate the problem of impedance mismatch between strong object-oriented models and a relational database. You have to start by addressing the problem in the data model itself, and write your code to deal with relational objects, not deeply-nested objects. Once you do this, you'll find that writing your own queries isn't really that hard.
There are two parts to the question asked.
Question 1: How do I write the code to get from one to many to many to one?
I didn't get any answers here. I'll re-post the code I'm using.
public async Task<List<IOperation>> GetOperationsByEmployee(
IDataService<IJobSchedule> JobScheduleRepository,
DateTime Date,
IEmployee Employee)
{
var result = new List<IOperation>();
var empSchedMatches = allEmployeeSchedules.Where(x => x.Date >= Date && x.Employee == Employee);
var jobSchedules = new List<IJob>();
foreach (var empSched in empSchedMatches)
{
var matches = allJobSched.Where(x => x.Date == empSched.Date && x.Crew == empSched.Crew);
jobSchedules.AddRange(matches);
}
result = jobSchedules.Select(x => x.Operation).Distinct().ToList();
return result;
}
Question 2: How do I speed up this query?
I got a bit of haranguing over ORM choice here, but I didn't have any concrete implementation steps to improve the code. Then, I had a conversation with a programmer who explained to me that he has a ton of experience using Entity Framework with SQLite. He gave me a couple of concrete pointers, and a code sample for how to improve the database loading speed. He explained that Entity Framework tracks objects that it loads, but if the operation is reading data that doesn't need to be tracked, the loading speed can be improved by turning off the tracking. He gave me the following code sample.
internal class ReadOnlyEFDatabase : EFDatabase
{
public ReadOnlyEFDatabase(string dbPath, DbContextOptions options) : base(dbPath, options)
{
this.ChangeTracker.AutoDetectChangesEnabled = false;
this.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
}
public override int SaveChanges()
{
throw new Exception("Attempting to save changes from a read-only connection to the database.");
}
}
Thanks to Jeremy Sheeley from zumero for this help!!!!
I have a situation where i have a counter field in a table named Profile, and on form submit, i will retrieve the counter field and +1 to the counter and update profile table. The incremented counter will be stored in a variable where i will then use to create new records in another table [Bidder]. The problem is when there are multiple form submit at the same time, duplicate record values will be created in the Bidder table
Profile profile = db.Profile.Where(w => w.TenderId == tender_Id && w.IsDeleted == false).FirstOrDefault();
int submission = profile.TotalSubmission + 1;
if (profile != null) {
profile.TotalSubmission = submission;
profile.ModifiedBy = user_id;
profile.ModifiedOn = DateTime.Now;
db.Entry(profile).State = EntityState.Modified;
db.SaveChanges();
}
bid.ROId = string.Format("RO{0}", submission);
db.Entry(bid).State = EntityState.Modified;
db.SaveChanges();
How do i prevent duplicate ROId to be created?
The uniqueness should be enforced using a unique index or a unique constraint.
You can create these using code first (from MSDN):
public class User
{
public int UserId { get; set; }
[Index(IsUnique = true)]
public string Username { get; set; }
public string DisplayName { get; set; }
}
or directly via the database.
The counter should be protected using optimistic concurrency:
public class MyEntity
{
[Key]
public Guid Id { get; set; }
// Add a timestamp property to your class
[Timestamp]
[Required]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[ConcurrencyCheck]
public byte[] VersionTimestamp { get; set; }
public int Counter { get; set; }
}
If you try to update the row with the VersionTimestamp after it has been changed without re-reading it from the database, you'll get an OptimisiticConcurrencyException e.g. in this test scenario:
// Read the entity
MyEntity entity;
using (var context = new MyContext())
{
entity = context.MyEntities.Single(e => e.Id == id1);
}
// Read and update the entity
using (var context = new MyContext())
{
var entity2 = context.MyEntities.Single(e => e.Id == id1);
entity2.Counter++;
context.SaveChanges();
}
// Try to update stale data
// - an OptimisticConcurrencyException will be thrown
using (var context = new MyContext())
{
entity.Counter++;
context.SaveChanges();
}
If you are using SQL Server 2012 or newer, you can use a Sequence to accomplish this. You would also want to enforce uniqueness through a unique constraint.
public partial class YourEfContext : DbContext
{
.... (other EF stuff) ......
// get your EF context
public int GetNextSequenceValue()
{
var rawQuery = Database.SqlQuery<int>("SELECT NEXT VALUE FOR dbo.SomeSequence;");
var task = rawQuery.SingleAsync();
int nextVal = task.Result;
return nextVal;
}
}
Another option, if you don't have a version that supports sequences, is to use a stored procedure on the database to issue Id numbers. The stored proc can work in conjunction with an ID table, which it can place an explicit lock on. This means you can request an id from the proc, it can lock the table, read the current number, increment it, store it back in the table, release the lock, and return the id. You would need to call your proc from code to get the new id to assign. The lock on the db side ensures that you are only ever assigned unique values. As long as your id column is only ever given a value assigned by the proc, you will have unique values. You will still be able to assign arbitrary numbers though, which could include duplicates, that can be solved with a unique constraint.
None of this in Entity-Framework specific, though you can still access all this through entity-framework in one way or another.
You can not rely only on entity framework for your solution. Only the database has a full picture of the stored data. Your different entity context instances don't even know if other instances exist, so coordinating sequence numbers on a global scale is extremely difficult on EF level.
Depending on the frequency of conflicts, two options come to my mind to enforce the uniqueness of the sequence number:
Unique constraint
Stored procedure for writing the data
Unique constraint
You can create a UNIQUE constraint over the ProfileId and Sequence columns. When you store the data with a duplicate sequence number, you will get an exception. Either the exception itself or one of its inner exceptions will be an SqlException. You can examine the error number of that exception and if it's error number 2627 (if your DBMS is SQL Server; if it is not, check for the similar error in your DBMS), you know it's a unique key constraint violation. In this case you get the current sequence number from the DB and write the data again with a new sequence. You have to repeat that until the insert was successful.
In case you're using SQL server, you can selectively handle a UNIQUE KEY constraint violation like this (using C# 6.0 exception filters):
private bool IsUniqueKeyViolation(Exception exception) {
Exception currentException = exception;
while (currentException != null) {
SqlException sqlException = exception as SqlException;
if (sqlException != null) {
return sqlException.Errors.Cast<SqlError>().Any(error => error.Number == 2627);
}
currentException = currentException.InnerException;
}
return false;
}
//...
//...Code to set up the POCOs before Save...
while(true) {
try {
context.SaveChanges();
}
catch(Exception exc) when (IsUniqueKeyViolation(exc)) {
//...Code to update the sequence number...
continue;
}
break;
}
This solution is only practical if the number of conflicts is expected to be small. If the number of conflicts is large, you will see a lot of unsuccessful UPDATE requests to the DB, which can become a performance issue.
EDIT:
As some other answers suggested, you could also use optimistic concurrency with a timestamp column. As long as you only update the DB from your own code, this works fine. However, a UNIQUE KEY constraint will protect the integrity of your data also from changes that don't originate from your application (like migration scripts etc.). Optimistic concurrency does not give you the same guarantee.
Stored procedure
You can create a stored procedure that will set the new sequence number from the last existing number in the same INSERT or UPDATE statement. The stored procedure can return the new sequence number back to the client and you can process it accordingly.
Since this solution will always update the DB in a single statement, it works well for a larger amount of conflicting updates. The disadvantage is that you have to write a part of your program logic in SQL on the DB level.
I am trying to solve an issue I have with pulling large ints (22+ digits) into ASP.Net with Entity Framework Core from a MySQL database.
EF Core does not support BigInteger and the suggestions I received where to use decimal instead. However, when using decimal types on my entities, I always receive the following exception when trying to select from the DB:
System.InvalidCastException: Unable to cast object of type
'System.Int32' to type 'System.Decimal'
In the Database the columns are INT(25) and in my models the type is decimal, here is an example model:
[Table("alliance_reputation_rankings")]
public class AllianceReputationRank
{
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Column("date")]
public DateTime Date { get; set; }
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Column("world")]
public int World { get; set; }
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Column("alliance")]
public string Alliance { get; set; }
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Column("rank")]
public int Rank { get; set; }
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Column("reputation")]
public decimal Reputation { get; set; }
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
[Key]
[Column("entry_id")]
public int EntryId { get; set; }
}
I cannot select the Reputation property using EF Core. Even if I try and use (decimal) before the property to cast it:
Rough example of a select:
_context.AllianceReputationRankings
.Where(p => p.Date == rank.Date && p.Alliance== rank.Alliance && p.World == rank.World)
.Select(pl => new AllianceReputationRank
{
Date = pl.Date,
World = pl.World,
Alliance = pl.Alliance,
Reputation = (decimal)pl.Reputation
}
).FirstOrDefault();
How can I use decimals in my models to bring in large ints from the database? If I cannot use decimals, how can I use large numbers with EF Core?
Entity Framework expects a very tight type-constraint between the database and model, it really doesn't like to see a numeric column with a decimal property. I've outlined several options here, each with it's own benefits and drawbacks. Feel free to use whichever one is the best for you.
Since you're using MySQL, the first option I'm outlining is that you could alter the column type from INT(25) to DECIMAL(25, 0). Then you should be able to use decimal in Entity Framework for that as much as you want.
If you can't do that, then, sadly, you're in a very tight corner. Entity Framework Core just isn't the right tool for this job, it's not mature enough. In comments you clarified that you are using this numeric column for math in the DB, which means string and VARCHAR(25) are out of the playing book, unless you can take that math out of the DB.
This solution relies on the assumption that this entire model is read-only. If it is (and you don't need to update the database from Entity Framework Core) then you can build a VIEW in MySQL that casts the INT(25) column to a VARCHAR(25), and do something like the following:
[NotMapped]
public BigInteger ReputationValue { get; set; }
public string Reputation
{
get
{
return ReputationValue.ToString();
}
set
{
ReputationValue = BigInteger.Parse(value);
}
}
The problem is that you can't really update the database through a VIEW, so if you wanted to update these records (anything to do with this entire model, basically) you would need to write manual SQL, or build a stored procedure. This is just a limitation of Entity Framework Core that can't really be gotten around easily.
Lastly, the final option is to use a stored procedure / method for reading and writing. You could then pass a WHERE clause to it (if you want to take the challenge of building a C#-style conversion, go for it, otherwise just add a WHERE sqlConditionCode ... string that you pass to the method to filter things by.
Then build a second stored procedure / method to do the updates. You could call dbContext.Update(model) which would pass everything to the stored procedure to do the update, and dbContext.Get("WHERE EntryId = #EntryId", new List<SqlParamter> { new SqlParamter("#EntryId", ...) }) I'm sure you get the idea by this point.
Something like:
public IEnumerable<AllianceReputationRank> GetAllianceReputationRanks(string whereClause, IEnumerable<MySqlParameter> parameters)
{
// Get a connection string from somewhere
var connectionString = string.Empty;
using (var connection = new MySqlConnection(connectionString))
using (var command = new MySqlCommand("SELECT * FROM alliance_reputation_rankings " + (whereClause ?? string.Empty), connection))
{
connection.Open();
foreach (var parameter in parameters)
{
command.Parameters.Add(parameter);
}
using (var reader = command.ExecuteReader())
{
if (reader.HasRows)
{
// Build a list or use `yield return`, if building a list instance here
var result = new List<AllianceReputationRank>();
while (reader.Read())
{
// Build a model of `AllianceReputationRank` here
var model = new AllianceReputationRank();
// Use reflection or just add each property manually
model.Date = reader.GetDate("date");
// ...
// Make sure you read `reputation` as a string, then `BigInteger.Parse` it to your model
model.Reputation = BigInteger.Parse(reader.GetString("reputation"));
// Either add the model to the list or `yield return model`
result.Add(model);
}
// If you built a list then `return` it
return result;
}
}
}
}
Building the opposite method is, well, just the opposite. The ?? string.Empty might be superfluous, I don't have an IDE in front of me to check if string + null will throw an exception, but you can remove it if you don't like it. (Better safe than sorry here, in my opinion.) I really hope all my types and usage is correct, if not, I apologize for any modifications needed other than adding the connection string and additional properties.
Honestly, the DECIMAL(25, 0) option should work for you, if that doesn't then the stored procedure / method option should. Both should keep your math in the DB and hopefully not break anything else while also fixing the Entity Framework Core issues at the same time. Is it less than ideal? Absolutely, and I wish it weren't what was necessary. But, unfortunately, Entity Framework Core is very new and still requires a lot of updates just to add simple functionality that Entity Framework not-Core has. (Like the lack of a .Find method in Entity Framework Core, for example.)
I wish we had better news, but without the ability to build our own mappable-types (I.e. build our own BigInteger that is supported by Entity Framework Core) there just isn't a lot to be done with this problem, we're stuck to nasty work-arounds to make it do what we need to.
What I did that seemed to work is to multiply by 1m:
context.AllianceReputationRankings
.Where(p => p.Date == rank.Date && p.Alliance== rank.Alliance && p.World == rank.World)
.Select(pl => new AllianceReputationRank
{
Date = pl.Date,
World = pl.World,
Alliance = pl.Alliance,
Reputation = pl.Reputation * 1m
}
).FirstOrDefault();
This seems to allow decimal operations while not taking performance to the floor.
Suppose you have a database table such that
public partial class myTable
{
[Key]
public Guid rowID { get; set; }
[Column(TypeName = "datetime2")]
public DateTime? eventTime { get; set; }
}
Now suppose you're looking for a data in there that has happened 5 or fewer days ago:
DateTime dTVal = DateTime.Now.AddDays(-5);
var event = db.myTable.Where(evt => evt.eventTime >= dTVal);
The above query will NOT work. The reason being is that DateTime.Now gives hours, minutes and seconds of course. However, instead of giving an error from sql or the like the results are just returned with 0 rows.
In order to resolve and retrieve the expected data:
var dtVal = DateTime.Now.AddDays(-5).Date; // This wouldn't work in the LINQ to Entities query because AddDays() method...
var events = db.myTable.Where(evt => evt.eventTime >= dtVal);
Perhaps my relative newness with EF is to blame, but this seems VERY unintuitive and somewhat a pain, because Intellisense doesn't pick it up as anything other than DateTime? and the hover tooltip on the property is also DateTime? myTable.eventTime... thereby causing me to have to go find every date property I am comparing against to make sure I a converting that correctly.
Should not EF take the DateTime object in this case and convert it to the correct format prior to constructing the query, and throw an exception prior actually performing the query?
Does anyone have a familiarity with this type of problem and what have you done in the past to work with it?
Similar Answered Question
Workaround is to use a DateTime.Date for date2 type entities.
var compareDate = DateTime.Now.AddDays(-5).Date;
Thanks.
I'm using entity framework to setup a table using fluent-api configuration:
Property(g => g.DateTime).IsRequired().HasColumnType("datetime2").HasPrecision(0);
The table does indeed get created successfully:
CREATE TABLE [dbo].[Foo] (
[DateTime] DATETIME2 (0) NOT NULL,
);
The precision of the datetime2 column has been set to 0 as you can see. I thus expect the retrieved date-time values to not include milliseconds at all, aka dates should look like '13 March 2016 18:35:37.0000'. However the retrieved dates always include milliseconds. Here's the code I'm using:
var dbcontext = new ApplicationDbContext(); //foo table is empty
dbcontext.Foo.Add(new Entry { DateTime = DateTime.Now });
dbcontext.SaveChanges();
var date = dbcontext.Foo.First().DateTime; //this should be identical to DateTime.Now above except for milliseconds which should be set to zero right?
How can I achieve the desired effect without resorting to zeroing-out milliseconds manually (via C# code either before insertion or after retrieval)?
If you need just date and time without any milliseconds, use smalldatetime MS SQL type instead. It has accuracy of 1 seconds.
If for some reason you want to have datetime2 in the database, there's no automatic way you can achieve desired behavior. You can create a (calculated) property MyDateTimeWithoutMs that get and set the correct value for the database connected property.
internal DateTime databaseDateTime { get; set; }
public DateTime MyDateTimeWithoutMs
{
get
{
return databaseDateTime.DateTimeWithoutMs();
}
set
{
databaseDateTime= value.ToDateTimeWithoutMs();
}
}
In your model mapping add ignore for calculated property and map the database property to the actual column name.
public class EntryMap : EntityTypeConfiguration<Entry >
{
public Entry Map()
{
Property(t => t.databaseDateTime)
.HasColumnName("DateTime");
Ignore(t => t.MyDateTimeWithoutMs );
Hats off to #JonSkeet who tipped me off as to what was amiss. Turns out that the first dbcontext I instantiated had some sort of caching going on which in turn was causing the date-time value provided to be returned as-is with its milliseconds component intact (go figure ...). One way to go about this, in order to get the desired behavior is to re-instantiate a db-context and start on a tabula-rasa basis to guarantee that no cached values will be return on datetime:
var dbcontext = new ApplicationDbContext(); //foo table is empty
dbcontext.Foo.Add(new Entry { DateTime = DateTime.Now });
dbcontext.SaveChanges();
var dbcon2 = new ApplicationDbContext(); //vital
var date = dbcon2.Foo.First().DateTime;
Alternatively you may use .Entry().Reload() which has the benefit that it doesn't need a new db-context to be instantiated:
var dbcontext = new ApplicationDbContext(); //foo table is empty
var entry = new Entry { DateTime = DateTime.Now };
dbcontext.Foo.Add();
dbcontext.SaveChanges();
dbcontext.Entry(entry).Reload(); //doesnt suffer from the quirks of dbcontext.Gigs.First()
P.S.: Last but not least if you are using this code in a unit-test project make sure to rebuild the project before giving it a go (at least that's what I had to do to make things work in my project)