Datetime milliseconds getting wiped in C# SQLite connection [duplicate] - c#

I have a SQLite database where I store the dates as ticks. I am not using the default ISO8601 format. Let's say I have a table defined as follows:
CREATE TABLE TestDate (LastModifiedTime DATETIME)
Using SQL, I wish to insert the current date and time. If I execute any of the below statements, I end up getting the date and time stored as a string and not in ticks.
INSERT INTO TestDate (LastModifiedTime) VALUES(CURRENT_TIMESTAMP)
INSERT INTO TestDate (LastModifiedTime) VALUES(DateTime('now'))
I have looked at the SQLite documenation, but I do not seem to find any option to obtain the current timestamp in ticks.
I can of course define a parameter in C# and store the value as a System.DateTime. This does result in the datetime getting stored to the database in ticks.
What I would like to do is be able to insert and update the current timestamp directly from within the SQL statement. How would I do this?
Edit:
The reason I want the data stored as ticks in the database, is that the dates are stored in the same format as stored by the ADO.Net data provider, and so that when the data is also queried using the ADO.Net provider it is correctly retrieved as a System.DataTime .Net type.

This particular oddity of SQLite caused me much anguish.
Easy way - store and retrieve as regular timestamp
create table TestDate (
LastModifiedTime datetime
);
insert into TestDate (LastModifiedTime) values (datetime('now'));
select datetime(LastModifiedTime), strftime('%s.%f', LastModifiedTime) from TestDate;
Output: 2011-05-10 21:34:46|1305063286.46.000
Painful way - store and retrieve as a UNIX timestamp
You can use strftime to retrieve the value in ticks. Additionally, to store a UNIX timestamp (roughly equivalent to ticks), you can can surround the number of seconds in single-quotes.
insert into TestDate (LastModifiedTime) values ('1305061354');
SQLite will store this internally as some other value that is not a UNIX timestamp. On retrieval, you need to explicitly tell SQLite to retrieve it as a UNIX timestamp.
select datetime(LastModifiedTime, 'unixepoch') FROM TestDate;
To store the current date and time, use strftime('%s', 'now').
insert into TestDate (LastModifiedTime) VALUES (strftime('%s', 'now'));
Full example:
create table TestDate (
LastModifiedTime datetime
);
insert into TestDate (LastModifiedTime) values (strftime('%s', 'now'));
select datetime(LastModifiedTime, 'unixepoch') from TestDate;
When executed by sqlite3, this script with print:
2011-05-10 21:02:34 (or your current time)

After further study of the SQLite documentation and other information found on date number conversions, I have come up with the following formula, which appears to produce correct results:
INSERT INTO TestDate(LastModifiedTime)
VALUES(CAST((((JulianDay('now', 'localtime') - 2440587.5)*86400.0) + 62135596800) * 10000000 AS BIGINT))
Seems like a painful way to produce something that I would expect to be available as a built-in datetime format, especially that the database supports the storing of datetime values in ticks. Hopefully, this becomes useful for others too.
Update:
The above formula is not perfect when it comes to daylight savings. See section Caveats And Bugs in SQLite docs regarding local time calculation.

The following will return the number of milliseconds since the UNIX Epoch:
SELECT (strftime('%s', 'now') - strftime('%S', 'now') + strftime('%f', 'now')) * 1000 AS ticks
It works by grabbing the number of seconds since the Unix Epoch (%s), subtracting the number of seconds in the current time (%S), adding the number of seconds with decimal places (%f), and multiplying the result by 1000 to convert from seconds to milliseconds.
The subtraction and addition are to add precision to the value without skewing the result. As stated in the SQLite Documentation, all uses of 'now' within the same step will return the same value.

Related

problem on formatting DateTime in parametised query [duplicate]

I am using C# and MySql. I have a requirement where I need to save DateTime.MaxValue to one of the column.
ADO.NET code gives me below value for DateTime.MaxValue
12/31/9999 11:59:59 PM
When I save this in mysql, I see that the value for that datetime(3) column is saved as:
0000-00-00 00:00:00.000
Sample ADO.NET Code
DateTime time = DateTime.MaxValue;
sqlCommand.Parameters.AddWithValue("Expires", time);
sqlCommand.ExecuteNonQuery();
DataType of the column is datetime(3)
I still cannot figure it out why DateTime.MaxValue is saved as 0000-00-00 00:00:00.000
Any thoughts around this?
A DATETIME column can store values up to '9999-12-31 23:59:59'. DateTime.MaxValue is actually 9999-12-31 23:59:59.9999999. When you try to insert it, the fractional seconds overflow the maximum size of the field.
Normally (in STRICT mode), MySQL Server would issue a datetime field overflow error. But if you're running your server in ANSI mode, the overflow is silently converted to the "invalid" date time value 0000-00-00.
One way to fix this problem is to use STRICT mode in your MySQL Server.
Another way is to specify the column type as DATETIME(6), which allows the fractional seconds to be stored.
A third way is to truncate the fractional seconds from your DateTime objects in C# before inserting them in the database.
Maybe some trigger prevents from saving such a high date to your column?
Have u tried inserting that date from SQL query ?
I did some tests in Oracle DB, and all went smoothly.
It shouldnt be different in mysql ...

sql compare two dates, when working with C#

I have a problem. Using C#, I retrieve a date from SQL Server.
I then use that same date to try to retrieve the same row it was in.
Unfortunately that didn't work. The reason is that SQL Server stores dates with millisecond precision, while C# DateTime only stores dates with second precision. The millisecond part was preventing me from selecting the row.
How do we deal with this minute discrepancy?
The problem we are trying to solve is, how do we select rows, when a loss of precision is preventing us from selecting the correct row?
The answer I found was to use the SQL function DATEDIFF (datepart, startdate, enddate).
SELECT *
FROM Table1
WHERE DATEDIFF (second, SellByDate, SellByDate) = 0
This guarantees we select the correct row, since:
The dates are the same, with only a discrepancy of less than a second.
DATEDIFF will reduce that difference to 0

sql select duplicates by ignore seconds in date format [duplicate]

Which method provides the best performance when removing the time portion from a datetime field in SQL Server?
a) select DATEADD(dd, DATEDIFF(dd, 0, getdate()), 0)
or
b) select cast(convert(char(11), getdate(), 113) as datetime)
The second method does send a few more bytes either way but that might not be as important as the speed of the conversion.
Both also appear to be very fast, but there might be a difference in speed when dealing with hundreds-of-thousands or more rows?
Also, is it possible that there are even better methods to get rid of the time portion of a datetime in SQL?
Strictly, method a is the least resource intensive:
a) select DATEADD(dd, DATEDIFF(dd, 0, getdate()), 0)
Proven less CPU intensive for the same total duration a million rows by someone with way too much time on their hands: Most efficient way in SQL Server to get a date from date+time?
I saw a similar test elsewhere with similar results too.
I prefer the DATEADD/DATEDIFF because:
varchar is subject to language/dateformat issues
Example: Why is my CASE expression non-deterministic?
float relies on internal storage
it extends to work out first day of month, tomorrow, etc by changing "0" base
Edit, Oct 2011
For SQL Server 2008+, you can CAST to date i.e. CAST(getdate() AS date). Or just use date datatype so no time to remove.
Edit, Jan 2012
A worked example of how flexible this is: Need to calculate by rounded time or date figure in sql server
Edit, May 2012
Do not use this in WHERE clauses and the like without thinking: adding a function or CAST to a column invalidates index usage. See number 2 here Common SQL Programming Mistakes
Now, this does have an example of later SQL Server optimiser versions managing CAST to date correctly, but generally it will be a bad idea ...
Edit, Sep 2018, for datetime2
DECLARE #datetime2value datetime2 = '02180912 11:45' --this is deliberately within datetime2, year 0218
DECLARE #datetime2epoch datetime2 = '19000101'
select DATEADD(dd, DATEDIFF(dd, #datetime2epoch, #datetime2value), #datetime2epoch)
In SQL Server 2008, you can use:
CONVERT(DATE, getdate(), 101)
Of-course this is an old thread but to make it complete.
From SQL 2008 you can use DATE datatype so you can simply do:
SELECT CONVERT(DATE,GETDATE())
In SQL Server 2008, there is a DATE datetype (also a TIME datatype).
CAST(GetDate() as DATE)
or
declare #Dt as DATE = GetDate()
SELECT CAST(FLOOR(CAST(getdate() AS FLOAT)) AS DATETIME)
...is not a good solution, per the comments below.
I would delete this answer, but I'll leave it here as a counter-example since I think the commenters' explanation of why it's not a good idea is still useful.
Here's yet another answer, from another duplicate question:
SELECT CAST(CAST(getutcdate() - 0.50000004 AS int) AS datetime)
This magic number method performs slightly faster than the DATEADD method. (It looks like ~10%)
The CPU Time on several rounds of a million records:
DATEADD MAGIC FLOAT
500 453
453 360
375 375
406 360
But note that these numbers are possibly irrelevant because they are already VERY fast. Unless I had record sets of 100,000 or more, I couldn't even get the CPU Time to read above zero.
Considering the fact that DateAdd is meant for this purpose and is more robust, I'd say use DateAdd.
SELECT CAST(CAST(GETDATE() AS DATE) AS DATETIME)
I really like:
[date] = CONVERT(VARCHAR(10), GETDATE(), 120)
The 120 format code will coerce the date into the ISO 8601 standard:
'YYYY-MM-DD' or '2017-01-09'
Super easy to use in dplyr (R) and pandas (Python)!
BEWARE!
Method a) and b) does NOT always have the same output!
select DATEADD(dd, DATEDIFF(dd, 0, '2013-12-31 23:59:59.999'), 0)
Output: 2014-01-01 00:00:00.000
select cast(convert(char(11), '2013-12-31 23:59:59.999', 113) as datetime)
Output: 2013-12-31 00:00:00.000
(Tested on MS SQL Server 2005 and 2008 R2)
EDIT: According to Adam's comment, this cannot happen if you read the date value from the table, but it can happen if you provide your date value as a literal (example: as a parameter of a stored procedure called via ADO.NET).
See this question:
How can I truncate a datetime in SQL Server?
Whatever you do, don't use the string method. That's about the worst way you could do it.
Already answered but ill throw this out there too...
this suposedly also preforms well but it works by throwing away the decimal (which stores time) from the float and returning only whole part (which is date)
CAST(
FLOOR( CAST( GETDATE() AS FLOAT ) )
AS DATETIME
)
second time I found this solution... i grabbed this code off
CAST(round(cast(getdate()as real),0,1) AS datetime)
This method does not use string function. Date is basically a real datatype with digits before decimal are fraction of a day.
this I guess will be faster than a lot.
For me the code below is always a winner:
SELECT CONVERT(DATETIME, FLOOR(CONVERT(FLOAT,GETDATE())));
select CONVERT(char(10), GetDate(),126)
Strip time on inserts/updates in the first place. As for on-the-fly conversion, nothing can beat a user-defined function maintanability-wise:
select date_only(dd)
The implementation of date_only can be anything you like - now it's abstracted away and calling code is much much cleaner.
I think you mean
cast(floor(cast(getdate()as float))as datetime)
real is only 32-bits, and could lose some information
This is fastest
cast(cast(getdate()+x-0.5 as int)as datetime)
...though only about 10% faster(about 0.49 microseconds CPU vs. 0.58)
This was recommended, and takes the same time in my test just now:
DATEADD(dd, DATEDIFF(dd, 0, getdate()), 0)
In SQL 2008, the SQL CLR function is about 5 times faster than using a SQL function would be, at 1.35 microseconds versus 6.5 microsections, indicating much lower function-call overhead for a SQL CLR function versus a simple SQL UDF.
In SQL 2005, the SQL CLR function is 16 times faster, per my testing, versus this slow function:
create function dateonly ( #dt datetime )
returns datetime
as
begin
return cast(floor(cast(#dt as float))as int)
end
How about select cast(cast my_datetime_field as date) as datetime)? This results in the same date, with the time set to 00:00, but avoids any conversion to text and also avoids any explicit numeric rounding.
I think that if you stick strictly with TSQL that this is the fastest way to truncate the time:
select convert(datetime,convert(int,convert(float,[Modified])))
I found this truncation method to be about 5% faster than the DateAdd method. And this can be easily modified to round to the nearest day like this:
select convert(datetime,ROUND(convert(float,[Modified]),0))
Here I made a function to remove some parts of a datetime for SQL Server. Usage:
First param is the datetime to be stripped off.
Second param is a char:
s: rounds to seconds; removes milliseconds
m: rounds to minutes; removes seconds and milliseconds
h: rounds to hours; removes minutes, seconds and milliseconds.
d: rounds to days; removes hours, minutes, seconds and milliseconds.
Returns the new datetime
create function dbo.uf_RoundDateTime(#dt as datetime, #part as char)
returns datetime
as
begin
if CHARINDEX( #part, 'smhd',0) = 0 return #dt;
return cast(
Case #part
when 's' then convert(varchar(19), #dt, 126)
when 'm' then convert(varchar(17), #dt, 126) + '00'
when 'h' then convert(varchar(14), #dt, 126) + '00:00'
when 'd' then convert(varchar(14), #dt, 112)
end as datetime )
end
Just in case anyone is looking in here for a Sybase version since several of the versions above didn't work
CAST(CONVERT(DATE,GETDATE(),103) AS DATETIME)
Tested in I SQL v11 running on Adaptive Server 15.7
If possible, for special things like this, I like to use CLR functions.
In this case:
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlDateTime DateOnly(SqlDateTime input)
{
if (!input.IsNull)
{
SqlDateTime dt = new SqlDateTime(input.Value.Year, input.Value.Month, input.Value.Day, 0, 0, 0);
return dt;
}
else
return SqlDateTime.Null;
}
I, personally, almost always use User Defined functions for this if dealing with SQL Server 2005 (or lower version), however, it should be noted that there are specific drawbacks to using UDF's, especially if applying them to WHERE clauses (see below and the comments on this answer for further details). If using SQL Server 2008 (or higher) - see below.
In fact, for most databases that I create, I add these UDF's in right near the start since I know there's a 99% chance I'm going to need them sooner or later.
I create one for "date only" & "time only" (although the "date only" one is by far the most used of the two).
Here's some links to a variety of date-related UDF's:
Essential SQL Server Date, Time and DateTime Functions
Get Date Only Function
That last link shows no less than 3 different ways to getting the date only part of a datetime field and mentions some pros and cons of each approach.
If using a UDF, it should be noted that you should try to avoid using the UDF as part of a WHERE clause in a query as this will greatly hinder performance of the query. The main reason for this is that using a UDF in a WHERE clause renders that clause as non-sargable, which means that SQL Server can no longer use an index with that clause in order to improve the speed of query execution. With reference to my own usage of UDF's, I'll frequently use the "raw" date column within the WHERE clause, but apply the UDF to the SELECTed column. In this way, the UDF is only applied to the filtered result-set and not every row of the table as part of the filter.
Of course, the absolute best approach for this is to use SQL Server 2008 (or higher) and separate out your dates and times, as the SQL Server database engine is then natively providing the individual date and time components, and can efficiently query these independently without the need for a UDF or other mechanism to extract either the date or time part from a composite datetime type.
I would use:
CAST
(
CAST(YEAR(DATEFIELD) as varchar(4)) + '/' CAST(MM(DATEFIELD) as varchar(2)) + '/' CAST(DD(DATEFIELD) as varchar(2)) as datetime
)
Thus effectively creating a new field from the date field you already have.

SQL Server Timestamp using with C#

In my database I have used Timestamp in each table to see when data was inserted.
It stores data in byte[] of 8 byte.
Now I want to read that time using C#.
How can I get DateTime object from Timestamp which is byte[]?
SQL Server's TIMESTAMP datatype has nothing to do with a date and time!
It's just a binary representation of a consecutive number - it's only good for making sure a row hasn't change since it's been read.
In never versions of SQL Server, it's being called RowVersion - since that's really what it is. See the MSDN docs on ROWVERSION:
Is a data type that exposes automatically generated, unique binary numbers within a database. rowversion is generally used as a mechanism
for version-stamping table rows. The
rowversion data type is just an incrementing number and does not
preserve a date or a time. To record a date or time, use a datetime2
data type.
So you cannot convert a SQL Server TIMESTAMP to a date/time - it's just not a date/time.
But if you're saying timestamp but really you mean a DATETIME column - then you can use any of those valid date formats described in the CAST and CONVERT topic in the MSDN help. Those are defined and supported "out of the box" by SQL Server. Anything else is not supported, e.g. you have to do a lot of manual casting and concatenating (not recommended).
The format you're looking for looks a bit like the ODBC canonical (style = 121):
DECLARE #today DATETIME = SYSDATETIME()
SELECT CONVERT(VARCHAR(50), #today, 121)
gives:
2011-11-14 10:29:00.470
SQL Server 2012 will finally have a FORMAT function to do custom formatting......

Casting a double to a DateTime type is off by 2 days?

I store datetime values in the database as sql float type (Converted from an DateTime.OADate) for a myriad of reasons however in certain circumstances it is nice to get a human readable date/time column back from the database. I have found that I can execute the statement
SELECT CAST (timerecorded_utc as DATETIME) FROM tablename
and it will give me the date time string I am looking for but it seems to be off by exactly 2 days. I realize I can just modify the statement (since in time represented as a double 1 day = 1.0) to be
SELECT CAST (timerecorded_utc-2.0 as DATETIME) FROM tablename
BUT I was wondering if this is consistent AND it seems to me there is some reason for the discrepancy that I am missing.
It's because the epochs the dates use are different.
SQL Server's DATETIME uses 01/01/1900 00:00:00 as the epoch, which you can see by running the following query: SELECT CAST(0 AS DATETIME)
OADate is a bit odd, as it could have an epoch of 30/12/1899 00:00:00 or 31/12/1899 00:00:00 depending on whether you believe the Visual Basic or Excel guys, respectively. It would appear that from your two day difference, the .NET version goes with the 30th.
So, epoch off by two days gives two days difference in the outcome when you convert between the two types of date via a raw number.
Epic Epochs... Here is my TSQL solution in SQL Server using the built in
"DateAdd" function:
Select DateAdd(DAY, cast([ENDING DATE] as decimal(10,0)), '12/30/1899')
from YourTable
In my case I was importing a string saved in Excel via C# Core App and uploading to a SQL Server database so my [Ending Date] is a string which I casted as a decimal with no precision as I only needed the actual date, and not the time of day. As #GregBeech mentioned, your base date might be '12/31/1899'.

Categories

Resources