I'm hesitant to revisit this overly discussed topic, but I created a set of tables to store data and many of the tables included a field named "createdate" designated as "timestamp without time zone". The date/time values fed to these by the code base are always in UTC. It is intended that the UI will be able to control how the data is presented to the user, so some setting would dictate to convert to time zone in the UI.
Some of these timestamps will be for reports, so to display info for the end user. Other times, the values will be used to determine nightly jobs run against the data.
This is a typical is a multi-tenant cloud hosted system with clients across different time zones.
The server should never be moved, although I suppose changing hosting zones is a very remote possibility.
It was written on the .net platform. Not using noda time, just the built in DateTime stuff (for now).
The docs are pretty clear how the timestamp with time zone stores the info:
https://www.postgresql.org/docs/current/datatype-datetime.html
This answer also has good background on the difference in the two main timestamp data types:
https://stackoverflow.com/a/14616640/1905693
This answer also has some good info, but is Java-oriented:
What is the most recommended way to store time in PostgreSQL using Java?
And Josh Berkus has a dated article that was helpful:
https://it.toolbox.com/blogs/josh-berkus/zone-of-misunderstanding-092811
It seems most of these recommend the timestamp with time zone, but in my case, is timestamp without time zone appropriate?
If I did want to rely on pg to do the conversion, would the AT TIME ZONE clause be ok?
From an overall system architecture, is relying on the UI to change presentation a common and reasonable approach? (yeah, this one may be too opinion-flavored for SO's format)
Not a moment
As others said, TIMESTAMP WITHOUT TIME ZONE is the wrong data type.
That type holds only a date with time-of-day, for example noon on January 21st of 2021. But we cannot know if that means noon in Tokyo Japan, noon in Toulouse France, or noon in Toledo Ohio US — three very different moments, several hours apart from one another. So this type cannot represent a moment, is not a specific moment on the timeline.
The TIMESTAMP WITHOUT TIME ZONE type is good for three kinds of use-cases:
Representing multiple moments all known the same in their locality. For example, Acme Corp orders the manager at each factory in Delhi, Düsseldorf, and Detroit, to make an announcement in two days at their local time of noon.
Representing a date and time-of-day where the intended time zone is unknown. I consider this faulty data that should be rejected. But if you insist on writing it to the database, this type would be appropriate.
Booking future appointments where we want to keep the time-of-day even if those pesky politicians change the offset of the time zone(s) in their jurisdiction. These political changes happen surprisingly often. So book an appointment using two columns: TIMESTAMP WITHOUT TIME ZONE in one, and the name of the intended time zone in another. Time zones are named with Continent/Region format such as Africa/Tunis. At runtime, when you need a moment for calendaring, apply the time zone to the date and time to dynamically determine a moment according to the now-current time zone rules. In Noda Time, you would retrieve a LocalDateTime and time zone, to produce a ZonedDateTime for calendaring.
Moments
When you care about moments, specific points on the timeline, use TIMESTAMP WITH TIME ZONE.
In Postgres and many other databases, this type has a bit of a misnomer. The time zone is not actually saved with the date and time. Instead, upon submission to the database, any indicator of time zone or offset-from-UTC is used to adjust to UTC (an offset of zero hours-minutes-seconds). That UTC value is what Postgres writes to the database. And UTC is what Postgres always retrieves from the database.
Beware: Some middleware and tooling has the well-intentioned but very confusing anti-feature of dynamically applying a default time zone to the retrieved value, adjusting from UTC to that time zone. This creates the illusion of that time zone having been saved with the data. But, no, not so. A TIMESTAMP WITH TIME ZONE column stores only UTC values.
Use case examples:
Tracking the moment when a database record was created, modified, or deleted.
Logging for debugging or sysadmin work.
Tracking when a critical contract is signed or comes into effect.
Like you have seen repeatedly in the sources you quote, timestamp with time zone (timestamptz) is the more appropriate choice in all cases where different time zones may be involved.
Internally, storage is the same. But with timestamptz, Postgres already knows where timestamp value applies, while it is just oblivious in this regard with timestamp without time zone (timestamp).
Note that the given time zone is not stored. That's a common misunderstanding. It's only used to compute UTC time. See:
Time zone storage in data type "timestamp with time zone"
AT TIME ZONE is the SQL construct to use to get the corresponding time for a given time zone.
SELECT my_timestamptz AT TIME ZONE 'US/Hawaii';
With timestamp it gets awkward:
SELECT my_timestamp AT TIME ZONE 'UTC' AT TIME ZONE 'US/Hawaii';
Also, when using timestamp any appended time zone information in literals is ignored, which is typically not what you want.
See:
Ignoring time zones altogether in Rails and PostgreSQL
I usually set the timezone of the role I am using?
ALTER ROLE my_db_user IN DATABASE my_database
SET "TimeZone" TO 'UTC';
Seems to help getting the DateTime correctly.
Related
I am building a system that allows a user to specify his/her working hours. This is stored in the database as:
DayOfWeek: Monday
StartTime: 08:00
EndTime: 17:00
This information is relative to the User A, so based on his time zone. Which the user selects in his/her profile e.g. (UTC+00:00) Dublin, Edinburgh, Lisbon, London
What am I trying to Achieve?
User B currently makes a call to a .NET Web API with a Date e.g. 01 Jan 2019.
I need the API to return User A working hours but in User B time zone.
User B can then book that time with User A by making a second call to the API. In this case the booking is stored in UTC date format.
What I Need?
Can someone please provide a suitable solution for this as both User A and User B can have different Time Zones?
In your comments under the question, you said:
the current solution I have is that I store User A’s working hours as start and end times for each day of the week. I also store the users TimeZone. Currently the API will return the Working Times as a JSON Payload with the workers TimeZone. I then display this data for User B and perform the adjustment with User B TimeZone offset using client side JS. When User B decides to book, the request back to the API is done using User A TimeZone setting and then that date time value is converted and stored in UTC format.
This is mostly good, except when you say that you perform the adjustment using User B's time zone offset.
The problem is that many time zones go through different offsets depending on what date and time you are talking about. For example, if in client-side JS you do something like new Date().getTimezoneOffset(), you are getting the user's current offset. It is not necessarily the same offset that should apply for the date in question. More on this under the heading "Time Zone != Offset" in the timezone tag wiki.
Also, you go on to say that the request back to the API is done in user A's time zone (which is fine), but that you then convert and store the time as UTC. Be careful there - you need to retain the intended appointment time. It's fine to know what the UTC time of a specific event might be, but indeed this can change too.
As an example, consider the latest time zone changes that occurred in Morocco. They were planning to end daylight saving time on October 28th 2018, switching the clocks back from UTC+1 to UTC+0. However, on October 26th 2018, with just two days warning, the government announced that they would stay on UTC+1 permanently and abolish daylight saving time. These sorts of short-notice changes are highly problematic, and has happened around the world many times before. I wrote a blog post about them a couple of years back, which is still highly applicable today.
So, if user A was in Morocco, and you stored an appointment for sometime after the change in UTC, well then since the change didn't happen their appointment would now appear an hour shifted on their local calendar.
In such events, if you had retained the intended local time of the event, then you could also have a process to recompute the UTC time based on what current time zone data you have installed. Short notice changes are super hard, but if there had been say a few months notice, then you'd have time to apply updates and correct the times of the events programmatically. If you only stored the UTC time, then you'd not have that ability.
Another way to think about this is that while UTC always keeps ticking forward consistently (except in some cases around leap seconds), human beings don't think in such terms. If I say that I want to meet you at 10 am in some place, the meeting is aligned to the local time of that place - no matter what changes between the time in that place and UTC occur.
Conversely, scheduling a meeting based on UTC is taking it on blind faith that no changes beyond what are currently predicted will happen. Since we aren't able to peer into the future, we really shouldn't be making such assumptions.
(And again, if you'd like help with code, please show what code you've tried in your question. Generally in .NET, one uses the TimeZoneInfo class for this. Here's an overview on how to use it.)
You can take a look at NodaTime, probably best timezone-related library for .NET
The same is achieved using built-in .Net functionality, but there are some tricky rare issues with it, so it is more reliable for use NodaTime.
We have an application related to aviation, and specifically flights.
Times have to be stored in local, so I chose to use UTC Time + an offset, but then now I realize it is a bad choice:
By storing the timezone as an offset, I lose track of the original timezone and this has implications when dealing with daylight savings.
For example I can store a time in Alpine, UT as UTC time and a -6 offset and also a time in Phoenix, AZ as UTC time and a -6 offset.
But when daylight saving comes, the time will change in Alpine, but not in Phoenix.
So, I need to store the proper timezone and I have seen that there are also different lists with a different syntax, so I am assuming there are different standards.
In C#, what would be the best option to store a local time with the local time zone to make it work with daylight saving changes?
From the discussion in the question's comments, I understand that you are working with flight time schedules - that is, the time a future flight is intended to depart. This is indeed a case where the local time is more important than the UTC time.
Since you have the local time and location of departure (ex: 5:00 PM in Salt Lake City), then you should be storing in your database of scheduled departure times two values:
17:00 - The relevant local time of the departure
SLC - The location where the time is relevant
If this is a specific occurrence of this flight, then you should store the date as well:
2018-06-01T17:00 - The specific relevant local time of the departure
SLC - The location where the local time is relevent
These are the details that are contextually relevant to your business use case. Do not lose sight of them by converting them to UTC.
That said, you might consider storing them as a DateTimeOffset (2018-06-01T17:00-06:00), which makes converting to UTC trivial for a given instance. However there are two problems with this approach:
It cannot work with recurrences, because the offset may change.
Even for a single instance, the offset might change - if the government controlling the time zone decides to change their standard offset or daylight saving time rules before that occurrence takes effect. If you do take a DateTimeOffset approach, or a UTC-based approach, you must be prepared to recalculate future events in the face of such changes. (For more on this, see my blog articles: On the Timing of Time Zone Changes and Time Zone Chaos Inevitable in Egypt. There are countless other examples.)
With regards to the location - because you are working with data that is contextually applicable to the airline industry, I recommend using IATA airport codes like the SLC that I showed above. In other contexts, one might store an IANA time zones identifier like America/Denver, or a Windows time zone identifier like Mountain Standard Time.
You may find my "Airport Time Zones" gist (code and output table) useful for working with IATA airport codes. You'll have to decide how that data will flow through your system. If you are running on Windows and want to use the TimeZoneInfo class to convert times to different time zones, then use the Windows time zone IDs shown there. If you want to use the IANA time zone IDs instead, consider using Noda Time, or you can use my TimeZoneConverter library. There are several different options here, so explore them all carefully and pick ones that make sense to you.
Noda Time would be a great choice, IMHO. Not only would you get excellent time zone support, but you'd also be able to use types like LocalTime or LocalDateTime which align well with the scenarios described.
As I wrote in my comment, do not store local dates. Instead, store datetime values as UTC, and convert to local datetime when you need to display it.
You can use the ConvertTimeFromUtc method of the TimeZoneInfo class for that.
This means you will have to also keep a list of locations and whatever TimeZoneInfo they are associated in - For example,
Jerusalem would be associated with Israel Standard Time,
Rome with W. Europe Standard Time,
Hawaii with Hawaiian Standard Time
and so on. (I'll bet you can find such a list online somewhere.)
Please note that the ConvertTimeFromUtc method handles the daylight savings problem for you as well.
Then you can do something like this to get the local time by location:
DateTime GetLocalDateByCityName(DateTime utc, string cityName)
{
var timeZoneInfoId = GetTimeZoneInfoIdByCityName(string cityName);
return TimeZoneInfo.ConvertTimeFromUtc(utc, TimeZoneInfo.FindSystemTimeZoneById(timeZoneInfoId);
}
And of course, in GetTimeZoneInfoIdByCityName you get the TimeZoneInfoId for the specific city.
You should use TimeZoneInfo.ConvertTime to convert DateTime between time zones if you need to handle Daylight Savings.
TimeZoneInfo.ConvertTime(myDateTime, timeZone);
TimeZoneInfo.ConvertTime(myDateTime, fromTimeZone, toTimeZone);
I am building a c# calendar application and have stored all the datetimes in Microsoft SQL-Server DateTime2 type. This data type is searchable using operators such as ">",">=" etc..
I have now read more on the subject for example these posts:
Daylight saving time and time zone best practices
How to store repeating dates keeping in mind Daylight Savings Time
Is it always a good idea to store time in UTC or is this the case where storing in local time is better?
I believe I have made an error when using UTC and dealing with different DST values and especially when related to future repeated events.
My current implementation works fine for events, until a repeating series goes over a DST time change.
I believe I will now need to store
local times,
the local timezone
and possibly UTC time
How should I structure my database and what data types should I use to store my data in the database that will support different client timezones and DST values whilst also allowing me to query for matches within specified start and stop datetime ranges?
For repeated events, you definitely need to store the time zone, yes, and I'd store the local date/time. You might also want to store the UTC value of the first occurrence, if that would be useful for comparison purposes. In theory you could just store the UTC occurrence of the first date/time, as that can be unambiguously converted to the local time (if you have the time zone) - but it's likely that you'll only ever need the local time, in which case performing those conversions may be pointless.
You should also consider how you want to handle changes in time zone data - because time zone rules do change, reasonably frequently. (It depends on the country, admittedly.) For example, for efficiency you may want to generate a certain number of occurrences and store the UTC date/time of each occurrence (having worked out what to do with skipped and ambiguous local times due to DST transitions) - but if the time zone data changes, you'll need to perform that generation step again for all repeated events.
The back-end is C# and SQL Server.
I save logging activity date times from different time zones in UTC in my DB. I need to populate a globally distributed front end so it localizes the logged UTC time based on where the front end is located, which will be known e.g. PT, ET. It also needs to handle daylight savings time which is where I'm running into difficulty.
As the UTC offset changes based on the time of the year because of daylight savings time, is there a way for me to find out what the UTC offset is in C# given that I know the date/time and the time zone.
is there a way for me to find out what the UTC offset is in C# given that I know the date/time and the time zone
Technically you shouldn't need to find out this information, the underlying system clock/calendar/what-have-you should take care of it. (I'm sure anybody who has worked on date-time logic will attest to the fact that it is non-trivial to say the least.)
Ideally, any time you need to convert a time zone, you have three things:
A DateTime object representing the value to be converted.
A TimeZoneInfo object representing the known current format of the DateTime object (in this case UTC).
A TimeZoneInfo object representing the target time zone.
Given those things, any time you want to display the DateTime to a localized user, you'd use something like TimeZoneInfo.ConvertTime to perform the conversion. This should ideally be done as close to the UI as possible so that backing logic is always consistently performed in UTC.
I'm currently working on a project that requires a user (in their respective time zone) to set a 'cut off' time for certain Notifications that they receive throughout the day. The user will, for instance, say they want no alerts prior to 8:00 AM and no later than 7:30 PM. This user is located in CST (the time zone information for each user is saved) and the 'server' is in EST. The server is where all of the logical decisions occur for which user should be notified, etc.
What would be the best practice to convert the time part only for comparison purposes?
My current thought would be to take the time set by the user, say "7:00 AM", and create a full DateTime object and store that as .ToUniversalTime(). On the server I can convert the DateTime object to Local time and compare the Time portion only.
Does anyone have a 'better' solution or possibly any advice from someone who has tackled a similar issue?
Thanks!
I suggest that you store the minimum time and maximum time as local times, without dates. You will also want to store the users general timezone information, for example "Central Time", not CST or CDT.
When you are ready to make a determination about sending a notification, convert the current UTC time to the user's local timezone and compare the stored time range against the converted time.
Trying to create a full calendar date from stored time information will be problematic in certain cases, specifically for daylight savings changes. By converting from UTC to user's local time for the comparison, that problem will be avoided.