I'm reading a lot about timezone, offset, utc, local time, javascript functions, DST, bacon and I 'm trying to put this all together to build a solid/correct structure for my app.
Suppose my app is something like StackOverflow.
That is how I'm doing ...
The server is in another country, so I set it to UTC 00:00.
I'm storing date as DateTimeOffset.
I'm not storing TimeZoneID.
Date is being sent to the client in this format: 2012-07-19T14:30:00-03:00.
I'm using angular filter to convert it to local time.
I have a few questions about it ...
1º Server TimeZone?
About my server (single server) ... should it be running with a "neutral" UTC (+00:00)? And what if, in the future, we move to a farm where servers run on different locations?
2º What should I store?
Currently, I'm storing just date as DateTimeOffset. I'm reading about saving the TimeZoneID but I'm seeing no use at all for this. Am I missing something?
Or should I store date as DateTimeUtc with a TimeZoneID and manually convert every date with the TimeZone class?
3º How to convert to local time?
Is it safe to convert data on the client? Or date conversions should be always on the server side?
4º About DST.
Using my current approach. Will DST be respected?
One very important thing to understand about date/time is that there is no one right way for doing everything. The common answer "use UTC" is not always applicable. Context is very important, and there are different techniques and approaches based on what the values you are working with are representing. If you could elaborate on what they are used for in your application, I will update my answer accordingly. In the meantime, I'll try to address the specific points you have brought up already:
#1 - Server Time Zone
Keeping your server at UTC is a best practice, and it is what you can expect from cloud providers like Azure or AWS also. But it isn't something that you should be dependent on. Your server should be able to be set to any time zone without it affecting your application. As long as the clock is in sync with an NTP server, choice of time zone should not matter.
So how do you ensure that? Simple, just make sure your application avoids all of the following:
DateTime.Now
DateTimeKind.Local
TimeZone (the entire class)
TimeZoneInfo.Local
DateTime.ToLocalTime()
DateTime.ToUniversalTime() (because it assumes the input is local)
Misc. other methods that assume a local input or output, such as TimeZoneInfo.ConvertTimeToUtc(DateTime) (this particular overload doesn't take a time zone, so it assumes the local time zone)
See also my blog post: The Case Against DateTime.Now.
Note that I didn't include DateTimeOffset.Now in the list. Although it's a little bit of a design smell, it is still "safe" to use.
#2 - What to store
I suggest you read my answer to DateTime vs DateTimeOffset. It should clarify some things. Without regurgitating the whole thing, the main point is that while both represent a point in time accurately, a DateTimeOffset provides perspective, while a UTC DateTime does not.
You also asked when you should store a TimeZoneInfo.Id. There are at least two scenarios where this is required:
If you are recording events in the past or present, and you plan on allowing modifications to the recorded timestamps. You need the time zone to determine what the new offset should be, or how the new input converts back to UTC.
If you are scheduling time out into the future, you will need the time zone as part of the recurrence pattern (even for a single occurrence). See here and here also, (while for other languages, the same principles apply).
Again, the exact answer depends on what exactly the timestamps represent. There is no one ring to rule them all.
#3 - Client Safety
If it's a .NET client, sure you can convert there. But I think you are asking about a JavaScript client browser.
"Safe" is a relative term. If you're asking for exact perfectness, then no. JavaScript isn't safe for that, due to an error in the ECMAScript specification (ES1 through ES5.1. It is being worked on for ES6). You can read more in my blog post: JavaScript Date type is horribly broken.
However, if you are working with relatively current data, and the users of your application are not in a part of the world where time zones are volatile, or you don't require precise results 100% of the time, then you can "safely" use JavaScript to convert to the user's local time zone.
You might avoid some of these issues with libraries that implement the IANA TZDB in JavaScript, such as the ones I list here. But many of them are still dependent on JS Date, so they still have issues. (Side note - I'm working on a JS library that will counter this, but it is not ready to share yet).
Conversions on the server side are a much better choice, as long as you can ask the user for their time zone. Most of the time, I think this is doable.
You might consider asking using a map-based timezone picker, such as this one or this one. Both of which will require you use IANA time zones, which for .NET means using Noda Time, which is a great idea anyway (IMHO).
#4 - Daylight Saving Time
With your current approach, DST will be respected within the definition of the current DST rules for the time zone the user has set for their local browser. (Again, refer to my blog post for why this is the case).
A conversion from any value with an offset (whether -03:00 or Z) that passes through the Date object (which I believe an Angular filter will do), will properly convert to the specific unix timestamp.
The errors that would crop up with DST conversions for prior DST rules are because going from the unix timestamp inside the Date object to the local time zone will always assume that the current DST rule is applicable, even if the time fell into a period that had a different rule.
This actually depends on the actual application you're writing, but the most simple/robust approach IMO is to store/compute all your dates using UTC, and convert it back to the local time zone when you display it to the user.
Related
I'm hesitant to revisit this overly discussed topic, but I created a set of tables to store data and many of the tables included a field named "createdate" designated as "timestamp without time zone". The date/time values fed to these by the code base are always in UTC. It is intended that the UI will be able to control how the data is presented to the user, so some setting would dictate to convert to time zone in the UI.
Some of these timestamps will be for reports, so to display info for the end user. Other times, the values will be used to determine nightly jobs run against the data.
This is a typical is a multi-tenant cloud hosted system with clients across different time zones.
The server should never be moved, although I suppose changing hosting zones is a very remote possibility.
It was written on the .net platform. Not using noda time, just the built in DateTime stuff (for now).
The docs are pretty clear how the timestamp with time zone stores the info:
https://www.postgresql.org/docs/current/datatype-datetime.html
This answer also has good background on the difference in the two main timestamp data types:
https://stackoverflow.com/a/14616640/1905693
This answer also has some good info, but is Java-oriented:
What is the most recommended way to store time in PostgreSQL using Java?
And Josh Berkus has a dated article that was helpful:
https://it.toolbox.com/blogs/josh-berkus/zone-of-misunderstanding-092811
It seems most of these recommend the timestamp with time zone, but in my case, is timestamp without time zone appropriate?
If I did want to rely on pg to do the conversion, would the AT TIME ZONE clause be ok?
From an overall system architecture, is relying on the UI to change presentation a common and reasonable approach? (yeah, this one may be too opinion-flavored for SO's format)
Not a moment
As others said, TIMESTAMP WITHOUT TIME ZONE is the wrong data type.
That type holds only a date with time-of-day, for example noon on January 21st of 2021. But we cannot know if that means noon in Tokyo Japan, noon in Toulouse France, or noon in Toledo Ohio US — three very different moments, several hours apart from one another. So this type cannot represent a moment, is not a specific moment on the timeline.
The TIMESTAMP WITHOUT TIME ZONE type is good for three kinds of use-cases:
Representing multiple moments all known the same in their locality. For example, Acme Corp orders the manager at each factory in Delhi, Düsseldorf, and Detroit, to make an announcement in two days at their local time of noon.
Representing a date and time-of-day where the intended time zone is unknown. I consider this faulty data that should be rejected. But if you insist on writing it to the database, this type would be appropriate.
Booking future appointments where we want to keep the time-of-day even if those pesky politicians change the offset of the time zone(s) in their jurisdiction. These political changes happen surprisingly often. So book an appointment using two columns: TIMESTAMP WITHOUT TIME ZONE in one, and the name of the intended time zone in another. Time zones are named with Continent/Region format such as Africa/Tunis. At runtime, when you need a moment for calendaring, apply the time zone to the date and time to dynamically determine a moment according to the now-current time zone rules. In Noda Time, you would retrieve a LocalDateTime and time zone, to produce a ZonedDateTime for calendaring.
Moments
When you care about moments, specific points on the timeline, use TIMESTAMP WITH TIME ZONE.
In Postgres and many other databases, this type has a bit of a misnomer. The time zone is not actually saved with the date and time. Instead, upon submission to the database, any indicator of time zone or offset-from-UTC is used to adjust to UTC (an offset of zero hours-minutes-seconds). That UTC value is what Postgres writes to the database. And UTC is what Postgres always retrieves from the database.
Beware: Some middleware and tooling has the well-intentioned but very confusing anti-feature of dynamically applying a default time zone to the retrieved value, adjusting from UTC to that time zone. This creates the illusion of that time zone having been saved with the data. But, no, not so. A TIMESTAMP WITH TIME ZONE column stores only UTC values.
Use case examples:
Tracking the moment when a database record was created, modified, or deleted.
Logging for debugging or sysadmin work.
Tracking when a critical contract is signed or comes into effect.
Like you have seen repeatedly in the sources you quote, timestamp with time zone (timestamptz) is the more appropriate choice in all cases where different time zones may be involved.
Internally, storage is the same. But with timestamptz, Postgres already knows where timestamp value applies, while it is just oblivious in this regard with timestamp without time zone (timestamp).
Note that the given time zone is not stored. That's a common misunderstanding. It's only used to compute UTC time. See:
Time zone storage in data type "timestamp with time zone"
AT TIME ZONE is the SQL construct to use to get the corresponding time for a given time zone.
SELECT my_timestamptz AT TIME ZONE 'US/Hawaii';
With timestamp it gets awkward:
SELECT my_timestamp AT TIME ZONE 'UTC' AT TIME ZONE 'US/Hawaii';
Also, when using timestamp any appended time zone information in literals is ignored, which is typically not what you want.
See:
Ignoring time zones altogether in Rails and PostgreSQL
I usually set the timezone of the role I am using?
ALTER ROLE my_db_user IN DATABASE my_database
SET "TimeZone" TO 'UTC';
Seems to help getting the DateTime correctly.
I have a solution in Azure in Ireland. I live in Denmark.
Would it be correct, to save all my times and dates as universal = DateTime.Now.ToUniversalTime() in my database in Ireland?
When I need the danish time, I cannot: universal.ToLocalTime() because I’m still on the server in Ireland. Instead I could write:
danish = TimeZoneInfo.ConvertTimeFromUtc(universal, TimeZoneInfo.FindSystemTimeZoneById("Romance Standard Time"));
But I think it’s a long way.
Alternative 1: I convert date to danish, before I save in my database in Ireland.
Alternative 2: There are one timezone between Denmark and Ireland, so I could: danish = dateTime.AddHours(1), but perhaps it could give errors with the summer and winther time.
A few things:
There's nothing wrong with writing DateTime.Now.ToUniversalTime(), but understand that under the hood that is fetching the current UTC time, converting to local time, then converting back to UTC. Thus, it's both shorter and more efficient to simply call DateTime.UtcNow to get the UTC time directly without conversions.
The server on which your code is running is irrelevant (or should be). Additionally, all Azure instances have their local time zone set to UTC anyway, so you will not see any effects from Ireland's local time zone.
Yes, "Romance Standard Time" is the correct Windows identifier for the local time in Denmark. If you are running on a non-Windows platform, you should use "Europe/Copenhagen" instead. If you are writing for multi-platform usage, then use "Europe/Copenhagen" with my TimeZoneConverter library.
If you just wanted to write shorter code, you might be able to do the following:
DateTime danish = TimeZoneInfo.ConvertTimeBySystemTimeZoneId(universal, "Romance Standard Time");
I say might, because this will only be correct if the DateTime you've got in your universal variable has its Kind property set to DateTimeKind.Utc. If you've set that previously using DateTime.SpecifyKind, then this will work. On the other hand, if you have just retrieved this value from a database, then by default the Kind will by DateTimeKind.Unspecified. The key difference being that ConvertTimeFromUtc treats DateTimeKind.Unspecified as if it were DateTimeKind.Utc, while ConvertTimeBySystemTimeZoneId treats DateTimeKind.Unspecified as if it were DateTimeKind.Local.
In other words, if you don't set the Kind explicitly, it would convert from local time to Danish time. However, since Azure runs its local time zone as UTC, you'd get the same result. Just you might get different results when running elsewhere.
You might consider using DateTimeOffset instead of DateTime. This usually results in clearer code and less errors. (DateTimeKind is not used with DateTimeOffset.) Of course it depends on what exactly you're using this data for, but often it's a good idea.
About your proposed alternatives, I suggest neither. Your original code is better. My thoughts on each:
Alternative 1: I convert date to danish, before I save in my database in Ireland.
It depends on what you're representing.
If this is a unique point in time (the time something happened, usually in the past) - then you should keep it as UTC. That allows conversion to any time zone correctly.
On the other hand, if you are representing the time something is scheduled to happen (in the future), then most often it's the local time that matters most. This is especially important for recurring events in time zones with DST, or events in time zones that might be volatile (where the government makes frequent short-notice changes).
Alternative 2: There are one timezone between Denmark and Ireland, so I could: danish = dateTime.AddHours(1), but perhaps it could give errors with the summer and winter time.
You should not add or subtract time to adjust time zones. After all - you're not talking about a different point in time an hour in the future or an hour in the past.
Also, Denmark and Ireland may be similar today, but that doesn't mean they have always been that way, and it doesn't mean they will necessarily stay that way in the future. Ireland has a different time zone identifiers ("GMT Standard Time" on Windows, "Europe/Dublin" on other platforms).
Use server Utc time to save record in database in order to avoid any time differences when converting datetime from and then converting it back. Once the record use Utc datetime then you can easily convert it to any time zone, so for example: system requires to convert datetime to Denmark local time, and in future it might need to show in Germany datetime
DateTime timeUtc = DateTime.UtcNow;
TimeZoneInfo cstZone = TimeZoneInfo.FindSystemTimeZoneById("Romance Standard Time");
DateTime cstTime = TimeZoneInfo.ConvertTimeFromUtc(timeUtc, cstZone);
I am building a c# calendar application and have stored all the datetimes in Microsoft SQL-Server DateTime2 type. This data type is searchable using operators such as ">",">=" etc..
I have now read more on the subject for example these posts:
Daylight saving time and time zone best practices
How to store repeating dates keeping in mind Daylight Savings Time
Is it always a good idea to store time in UTC or is this the case where storing in local time is better?
I believe I have made an error when using UTC and dealing with different DST values and especially when related to future repeated events.
My current implementation works fine for events, until a repeating series goes over a DST time change.
I believe I will now need to store
local times,
the local timezone
and possibly UTC time
How should I structure my database and what data types should I use to store my data in the database that will support different client timezones and DST values whilst also allowing me to query for matches within specified start and stop datetime ranges?
For repeated events, you definitely need to store the time zone, yes, and I'd store the local date/time. You might also want to store the UTC value of the first occurrence, if that would be useful for comparison purposes. In theory you could just store the UTC occurrence of the first date/time, as that can be unambiguously converted to the local time (if you have the time zone) - but it's likely that you'll only ever need the local time, in which case performing those conversions may be pointless.
You should also consider how you want to handle changes in time zone data - because time zone rules do change, reasonably frequently. (It depends on the country, admittedly.) For example, for efficiency you may want to generate a certain number of occurrences and store the UTC date/time of each occurrence (having worked out what to do with skipped and ambiguous local times due to DST transitions) - but if the time zone data changes, you'll need to perform that generation step again for all repeated events.
To keep it short and sweet I have 2 PC's:
PC 1 has DST switched off
PC 2 has DST switched on.
PC 1 sends a DateTime to PC 2 which it takes and uses to set its time.
All the above I have in place but my questions is that when PC 2 receives the DateTime how can I check to see if it needs DST adjustments?
PC's are UK based.
EDIT:- Bit more detail incase there is confusion.
When PC 2 retrieves the time from PC 1 it will change the system's time to this value but I need to ensure that if a +/- 1 hour is required (DTS) then it is applied before setting the system date and time.
I would recommend using UTC for transport and persistance.
Also take care to not use date time as a critical part of your algorithm - it's just data. Dates and time shift and correct themselves. I've seen apps that crash when the PC corrects it's time :)
Here's another post with more information on the topic:
Daylight saving time and time zone best practices
You didn't show any code, but you tagged your question as c#, so I will answer from that perspective.
If you just need to make sure you're talking about the same moment in time, then use a DateTime with .Kind = DateTimeKind.Utc, such as is obtained from DateTime.UtcNow.
If you actually need to know that PC1 thought it was one time while PC2 thought it was another, and you still want to know that those two things represent the same moment in time, then use a DateTimeOffset. This will include the local date and time of the computer, as well as the offset from UTC that the date and time represent.
With either approach, you should use the ISO8601 format for transport and persistence, so that your meaning is clear regardless of culture or time zone of the observer. In .Net, this is obtained with .ToString("o") from either a DateTime or DateTimeOffset.
A UTC DateTime in ISO8601 format would look like this:
2013-03-06T09:00:00Z
A DateTimeOffset in ISO8601 format would look like this:
2013-03-06T10:00:00+01:00
.Net calls this the Round Trip pattern - because it is designed for the exact purpose you described.
For further clarification on DateTimeOffset - see this post.
How does the conversion to UTC from the standard DateTime format work?
More specifically, if I create a DateTime object in one time zone and then switch to another time zone and run ToUniversalTime() on it, how does it know the conversion was done correctly and that the time is still accurately represented?
There is no implicit timezone attached to a DateTime object. If you run ToUniversalTime() on it, it uses the timezone of the context that the code is running in.
For example, if I create a DateTime from the epoch of 1/1/1970, it gives me the same DateTime object no matter where in the world I am.
If I run ToUniversalTime() on it when I'm running the code in Greenwich, then I get the same time. If I do it while I live in Vancouver, then I get an offset DateTime object of -8 hours.
This is why it's important to store time related information in your database as UTC times when you need to do any kind of date conversion or localization. Consider if your codebase got moved to a server facility in another timezone ;)
Edit: note from Joel's answer - DateTime objects by default are typed as DateTimeKind.Local. If you parse a date and set it as DateTimeKind.Utc, then ToUniversalTime() performs no conversion.
And here's an article on "Best Practices Coding with Date Times", and an article on Converting DateTimes with .Net.
Firstly, it checks whether the Kind of the DateTime is known to be UTC already. If so, it returns the same value.
Otherwise, it's assumed to be a local time - that's local to the computer it's running on, and in particular in the time zone that the computer was using when some private property was first lazily initialized. That means if you change the time zone after your application was started, there's a good chance it will still be using the old one.
The time zone contains enough information to convert a local time to a UTC time or vice versa, although there are times that that's ambiguous or invalid. (There are local times which occur twice, and local times which never occur due to daylight saving time.) The rules for handling these cases are specified in the documentation:
If the date and time instance value is
an ambiguous time, this method assumes
that it is a standard time. (An
ambiguous time is one that can map
either to a standard time or to a
daylight saving time in the local time
zone) If the date and time instance
value is an invalid time, this method
simply subtracts the local time from
the local time zone's UTC offset to
return UTC. (An invalid time is one
that does not exist because of the
application of daylight saving time
adjustment rules.)
The returned value will have a Kind of DateTimeKind.Utc, so if you call ToUniveralTime on that it won't apply the offset again. (This is a vast improvement over .NET 1.1!)
If you want a non-local time zone, you should use TimeZoneInfo which was introduced in .NET 3.5 (there are hacky solutions for earlier versions, but they're not nice). To represent an instant in time, you should consider using DateTimeOffset which was introduced in .NET 2.0SP1, .NET3.0SP1 and .NET 3.5. However, that still doesn't have an actual time zone associated with it - just an offset from UTC. That means you don't know what local time will be one hour later, for example - the DST rules can vary between time zones which happened to use the same offset for that particular instant. TimeZoneInfo is designed to take historical and future rules into account, as opposed to TimeZone which is somewhat simplistic.
Basically the support in .NET 3.5 is a lot better than it was, but still leaves something to be desired for proper calendar arithmetic. Anyone fancy porting Joda Time to .NET? ;)
What #womp said, with the addition that it checks the DateTime's Kind property to see if it might already be a UTC date.
DateTime.ToUniversalTime removes the timezone offset of the local timezone to normalize a DateTime to UTC. If you then use DateTime.ToLocalTime on the normalized value in another timezone, the timezone offset of that timezone will be added to the normalized value for correct representation in that timezone.