I have a winform client and wcf service that together display historical data. the data stored in the db with utc timestamps. If two clients in different time zones want to look at a most recent day's worth of data based on their local time, can it be possible that they would be looking at different sets of data?
That depends.
If "a day's worth" means "the last 24 hours", then they will both see the same data.
If "a day's worth" means "00:00-23:59 of (current date - 1) in the user's local time zone", then they might see different data.
Example:
event 1 event 2 +---- service request
| | v
|---------------|---------------|- days in Central Europe
--------|-----------------|------- days in the US (Pacific time)
---------------|---------------|-- UTC day boundaries
In case 1, you just go back 24 hours from the time of the service request. It's easy to see that both customers, the one in the US and the one in Europe, will get the same list of events.
In case 2, it's more difficult: With respect to US time, events 1 and 2 happen on the same day. With respect to Europe, both events happen on different days, so the results will be different for the client in the US and the client in Europe.
Without further details the answer is both.
If a day is defined in the local time zone then changed to the UTC format before the request to the db is made they will have different data sets.
if the query is somthing like
select * from dbTable where TimeStamp >= '9/22/2010' and TimeStamp < '9/23/2010'
or
select * from dbTable where TimeStamp >= GetDate() - '24:00:00' and TimeStamp < GetDate()
then they will return the same set of data for both time zones.
It is certainly possible - it is one of the common issues with globalization/localization.
The important thing to do is to convert all requests to UTC time before making the query. If the queries are all UTC-based, then the data returned to the local machines should match, as the UTC times will match.
Related
Consider this scenario with two applications whose data is synced
Application 1 | Application 2
Data from application1 is inserted/updated into application2 based on last modified date of records.
How can we make sure that the last modified date is not dependent on time zones.
I would use a rowversion column in both applications. To decide what to sync you can have a query or a merge statement and in the where clause you can compare the rows table1.RowVersion > table2.RowVersion. (table1 is from application1 and table2 is from application2)
This might help: ToUniversalTime
Convert the times to universal times, these take into account the time zone, daylight savings, etc. If you convert both times before adding the records, you should be able to get the correct last modified date.
I’m having problems with slow queries when returning a fixed number of events starting from a specified earliest datetime, from an index/sourcetype containing tens to hundreds of millions of records. E.g.:
index=test sourcetype=touch_tuio earliest="09/29/2015:00:00:00" | fields _time,host,screen_id,tuio_type
Using the SDK and a maximum limit of 50000 events, because events appear in descending date order and are cropped at 50000, we only get the latest 50000 values which do not start at the requested earliest date. So in this case the resulting date range of the above query is "from": 2015-10-13T12:30:14 "to": 2015-10-13T13:08:41. (where 2015-10-13T13:08:41 is the most recent record in Splunk).
If I use tail, I get the correct values (in ascending date order) starting at the correct earliest datetime, but then the query time is unacceptably long in both C# SDK and dashboard (and will take longer and longer with increasing data). E.g.
index=test sourcetype=touch_tuio earliest="09/29/2015:00:00:00" | fields _time,host,screen_id,tuio_type | tail 50000
In the dashboard, even a query such as below takes a very long time, looking for 10 records starting at a particular date:
index=test sourcetype=touch_tuio earliest="09/29/2015:00:00:00" | fields _time,host,screen_id,tuio_type | tail 10
It seems that Splunk needs to go through EVERY record just to give me 10 records starting at a specific date (09/29/2015:00:00:00). This takes an extreme amount of time.
Does anyone know what I may be doing wrong here?
Is there another way to query a fixed number of events beginning at a selected earliest date, without using tail, or without the increasing search time?
Thank you!
I'm battling with my model for storing dates in a web application used in different time zones. At the moment, all dates are stored in UTC dates. So, lets say I am allowing the user to save a Transaction date, and a field on the screen is "Transaction Date". I default it to DateTime.UTCNow. SQL Server is in the United States, and I am in Australia.
So, what I have done now, is when I present the default transaction date, I add the GMT Offset to the UTC Now date:
var usersCurrentDate = DateTime.UtcNow.AddSeconds(GmtOffset.Value);
Where GmtOffset is a value in seconds, difference from GMT.
This code works - the user get's presented with the current date/time in their zone.
My issue is - how I should store this. Going to the UI, I do the above conversion. Should I then have a converted on the way to the database that converts the date entered by the user, back into UTC date/time?
In that case, I am always sure that all dates in the database are UTC dates. And then when ever I get the data from the database, I need to remember to translate into Local time? So, have a function that takes a Local time (As entered by the user) and converts it to the UTC?
Something like:
public DateTime UserDateToUTC(DateTime userDate)
{
return DateTime.UtcNow.AddSeconds(GmtOffset.Value * -1);
}
Is this an acceptable pattern?
I have an iPhone app, built in Xamarin, that stores objects to a local SQLite database. As it takes a long time to load all of the data, I am loading it 20 records at a time, and loading more as the user scrolls.
I am paging by CreatedDate, and this is stored as an Integer field by its DateTime.Ticks property (the number of 100 nanoseconds since DateTime.MinValue). I always pass the oldest loaded CreatedDate to the read function.
Integer column: 8 bytes signed
DateTime.Ticks: 8 bytes signed
The code to write the CreatedDate is as follows:
var param = cmd.CreateParameter();
param.ParameterName = "#date";
param.Value = myObject.DateCreated.Value.Ticks;
cmd.Parameters.Add(param);
And the code to read the next page is as follows:
cmd.CommandText = "SELECT * FROM MyTable WHERE CreatedDate < " +
(offset.HasValue ? offset.Value.Ticks : DateTime.MaxValue.Ticks) +
" ORDER BY CreatedDate DESC LIMIT " + pageSize;
I can tell that the dates are correct, because my sections in my UITableView are based on their CreatedDates.
So here's all the weird behavior (i.e. my question).
I started off in the simulator. At first, I was using DESC and it was returning the oldest objects. Then I switched it to ASC and it was returning the newest objects. This seems backwards. Then, it magically started working correctly with DESC. Paging began working in the simulator, and still works.
Then I switched to a real iPhone. The DESC and ASC issue showed up again, but this time has not fixed itself (ASC returns the most recent objects). Either way, when I scroll to the bottom of the UITableView, the load gets called again with the correct offset, but it gets no data back (and therefore all paging stops).
Things I know.
All of the data is in the table, because if I SELECT * I get everything with the correct dates back. I have also written all of the dates to the console to be sure.
My newest date is 635194634041230000
My oldest date is 635166796800000000
The offset date that returns no rows is 635193909600000000
Note. I'm OK with constructive criticism on how I'm doing this. I'll upvote if I like it enough. But if you post an answer with that information in it, please also try to address the current issue I'm having.
I'm not in front of a computer, so I can't really debug your code atm, however, I thought I'd link to some generic code I wrote for MonoTouch to dynamically page data from SQLite:
https://github.com/jstedfast/MonoTouch.SQLite/blob/master/MonoTouch.SQLite/SQLiteTableModel.cs
This code uses SQLite-net and not System.Data, but it may still help.
The core logic for paging in and out is in GetItem().
When you created your Database, what datatype did you set for the CreateData column? If you used Integer then Ticks should be ok.
1.2 Date and Time Datatype
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS"). REAL as Julian
day numbers, the number of days since noon in Greenwich on November
24, 4714 B.C. according to the proleptic Gregorian calendar. INTEGER
as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Sadly, I did not post the relevant code. If the record didn't exist yet, I was writing DateTime.Now to the column. So depending on what order the WCF service returned the results (descending or ascending by date), my ordering could be backwards. I'm not quite sure why it was refusing to page, but I'm guessing it had something to do with precision or whether the database was persisting on the device. Both problems are fixed.
I have a linq-to-sql query that groups results by DateTime, something like this:
var TheQuery = from ....
where ....
group a by a.TheDateTime.Date in TheGroups
The problem is that this groups by actual dates that are stored in UTC in the DB. Let's say that I want to group by dates based on the user's timezone. So for instance, if he's in the PST timezone, group by "UTC minus 8 hours", or whatever other offset.
Thanks for your suggestions.
If you're getting all the information back anyway, I'd frankly bring it back to the client and do the grouping in LINQ to Objects. That way you can concentrate on using one platform's date and time API - it's hard enough using one API correctly when it comes to time zones, let alone two.
(As an aside and a plug, it also means you can use whatever .NET API you like for this - such as Noda Time ;)
So I'd put all the filtering and projection you can do before the grouping, then use AsEnumerable() to force the rest of the execution to occur locally.
Don't forget that you almost certainly can't just add or subtract 8 hours, unless the user is really in a time zone which doesn't use daylight saving time (and hasn't historically). Are you also sure that it's appropriate to apply the same time zone to all the data? (It may well - just checking.)