What surprised me the most recently was that POSIX specifies timezone offsets the other way round from the rest of the world, and thus many parsing libraries will interpret something like "2016-1-1 12:00:00 GMT+2" as 14:00 in UTC, instead of 10:00.
It's extra tough to know what to do on that situation if you don't know the provenance of the timestamp, because there are some programs that output timestamps formatted like a POSIX TZ, but with the opposite sign convention.
Indeed. If you just have a random timestamp, no clue what it is. Another reason why you should generate ISO 8601 timestamps if possible, but they are not so nice for humans reading (or even worse, writing) them.
Its a matter of practice, but on the odd occasion, I've wound up accidentally dating a document with a handwritten full ISO 8601 timestamp, usually happens when I have to sign and date something after spending several hours making notes while reading log files.
Another thing to consider is that transitions in offset are not always tied to transitions in DST, and you can (rarely) get transitions in DST that do not involve an offset change. For example, Portugal changed their base zone to Western European Time from Central European Time during a scheduled DST transition. The result was that they switched over to being in "Summer Time" with no change in the clocks.
There are also examples of non-DST related time zone changes of up to 24 hours (mostly from Pacific islands deciding to change what side of the international date line they are on), so in some zones, there are certain entire days that never happened (e.g. the entire day is an "imaginary day") and some that happened "twice".
I hope http://momentjs.com/ is not one of libraries that falls into "Many people write libraries without knowing much about the domain." I have used it many times and it is the best one I have come across.
Timezones conversion is super hard, which is the exact reason I rely on libraries rather than trying to reinvent the wheel.
Saw the title and came here to write about moment. I work on a large code base that had considerable amounts of code related to time conversions in numerous different places, each handled differently with different bugs, and I've slowly been moving things towards just using moment. It's been a huge time saver and has fixed issues everywhere I've ended up using. Probably one of the most useful libraries around for sure!
MomentJS is great and offers pretty much everything you need time manipulation wise. That being said, I've spent many hours debugging problems with moment because many operations mutate the 'moment' instance that you're working with.
If you generate some moments and need to reuse them for multiple calculations, you need to constantly clone them.
I wish there was a moment-like library with good performance that's immutable.
Dealing with timezones and daylight savings time has consumed hundreds of hours of my development time. This seriously should be a standard service of the operating system, not the programming language library.
Just look at all the code to deal with it in D (written by Jonathan Davis):
Except that removing DST won't, if you ever need to calculate a time/date for any time before its (future, hypothetical) removable. That could actually create more bugs.
Probably won't, historical DST switches (and TZ offsets) are already one of the edge cases a TZ library needs to deal with (I'm looking at you, Argentina!).
We didn't have DST in most of Indiana until about 10 years ago. When we switched, most of us were wondering why the hell we were even doing it. It's really not necessary, it confuses people, leads to screwed up schedules when the switch happens, and the fact that there are still a few counties that don't observe it screws things up even more.
Operating systems kind of suck at it. Which is why for Elixir I made Tzdata (https://github.com/lau/tzdata) not rely on any timezone information from the operating systems. A feature I would like from operating systems is to tell me the true time during leap seconds. E.g. 23:59:59, 23:59:60, 00:00:00 Instead of repeating 23:59:59 or just pretending the leap second never happens.
> A feature I would like from operating systems is to tell me the true time during leap seconds. E.g. 23:59:59, 23:59:60, 00:00:00 Instead of repeating 23:59:59 or just pretending the leap second never happens.
In past discussions of leap seconds, it came out that the POSIX standard specifies that no such second as 23:59:60 can exist (by specifying that "one day" consists of exactly 86400 seconds). As I understand things, this is why the erroneous behavior you mention is done.
It is not clear to me why, for the sake of complying with a standard we know to be wrong, we report false information and otherwise do things that we also know are wrong. The obvious approach would seem to be to either change the standard or just ignore it.
Sure, but they usually have more resources than programming language teams. Consider all the programming languages on your machine. Can you trust they all did the timezone, with all its nuances, correctly?
Consider something simpler - the trig functions. I've noticed time after time that many language libraries get it wrong, and this is with decades of development. Errors are usually in the form of precision that is lower than it should be, and poor handling of special values like NaN and infinity.
The situation with that is so erratic that with the D programming language we'd write our own trig implementations rather than use the C library ones.
They might have more resources, but they also have constraints. For instance a tzdata update was just released yesterday. Apple has a lot of resources but I don't see an updated from them today on iOS nor Mac OS X. Elixir tzdata automatically updates to the new version.
Does Apple even have resources that work on tzdata code? Or do they just use open source code?
In particular for Elixir I trust the time zone data from Tzdata more than for any other language, mostly because I know it is designed to have up to date data. (And well, I know the code because I wrote it myself.)
Even if a system is good in many ways it does not matter if the data is not up to date.
I just checked if you can get around it by using unix time, but holy shit, it is poorly designed. Unix' seconds since epoch is not the actual absolute number of seconds since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, it doesn't count leap seconds. I wonder if you can even get a reliable TAI from linux, so you can abandon all this shit and implement proper time handling yourself.
Edit: Seems like we can specify CLOCK_TAI for clk_id on linux now.
My bete noir isn't in the list: GMT is only the UK timezone for about half the year. During the Summer, GMT continues to exist, but the UK time is BST.
But, as the Wikipedia page says "the term GMT should not be used for precise purposes" as it has been superseded by various other standards (e.g. UTC, UT1)
I've encountered English people who made a fuss about the loss of "God's Magnificent Time", but for better or worse GMT is no longer the official name.
The way I get around this in my head is that UK's timezone is "Europe/London". (my definition of "timezone" is "IANA timezone", so while "UTC" or "GMT" are both valid, they aren't what the UK uses.) The "GMT" or "BST" is just a locale-ish thing that gets tacked on to output, much like "EST" or "EDT" for "America/New_York".
You are confusing two things. A time zone (e.g. Europe/London) is a geographical region of places always sharing the same time and it does not change at a given location unless time zones come and go. The zone time (e.g. GMT or BST) on the other hand is the time in use in a given time zone at a given point in time and it commonly changes twice a year between normal time and daylight saving time and back. In short, daylight saving time affects the zone time, not the time zone.
You are right. I have heard that one before too. That one is sort of implicitly covered by this part in the blog post:
"CST is also used for: Cuba Summer Time, China Time, Central Standard Time (Australia). PST is used for Pakistan Standard Time and Pacific Standard Time. If you want a unique identifier for the time zone in the Pacific West of the USA it looks like this: “America/Los_Angeles”."
In this case the name of the time zone "people live in" is "Europe/London", not GMT.
I would add most programmers don't know some time zones have offsets in minutes. In ancient times, some were in seconds.
I live in UTC-04:30 and regularly encounter bugs because of it.
Here in Germany (which once consisted of 100-or-so small states), the introduction of a unified timezone only happened when railroad systems were built, because people noticed for the first time that their clocks were off by a slight amount.
Until the unified timezone happened, timetables would be printed with multiple timezone information. So maybe your train would depart in Dresden at exactly 14:00 Saxonian time, take exactly three hours to drive to Berlin, and arrive there at 17:06 Prussian time.
This is why we should ignore time zones as much as possible. UTC is almost OK, except for the shitshow idiocy of leap seconds. There's no reason for all of us to suffer just so astronomers can use civil time.
But Astronomers don't actually use "Civil Time" they tend to use either UTC, TAI, or for the continuation of record keeping over the last several hundred years... 'Terrestrial Time'...
None of which match up with any Civil Time anywhere on the world, except UTC which does so only periodically for some European countries, and year round in the following countries which resist the idiocy of shifting their clocks around based on the Season: Burkina Faso, Côte d'Ivoire, The Gambia, Ghana, Guinea, Guinea-Bissau, Liberia, Mali, Mauritania, Tomé and Príncipe, Senegal, Leone, Togo, Western Sahara (not technically a country but that's unrelated to the currently customary timezone), Greenland, Iceland, Saint Helena, Ascension Island and Tristan da Cunha
One of the more interesting bugs I've created is in an Elixir program using the (great) Timex[0] library. I realised that for a few particular users, timezones were being inverted when converted (e.g. a user in GMT+5 would behave as if they were in GMT-5 and vice versa). After a lot of time spent proving to myself I was not actually going mad, I opened a Github issue and the library author informed me about one of the stranger quirks of timezones - the Etc/UTC-X timezones[1], which represent areas that don't have an assigned area (usually a continent, like Europe/Berlin).
"The special area of "Etc" is used for some administrative zones, particularly for "Etc/UTC" which represents Coordinated Universal Time. In order to conform with the POSIX style, those zone names beginning with "Etc/GMT" have their sign reversed from the standard ISO 8601 convention. In the "Etc" area, zones west of GMT have a positive sign and those east have a negative sign in their name (e.g "Etc/GMT-14" is 14 hours ahead/east of GMT.)"
Maybe the application only allowed to configure GMT offsets, rather than timezones? I saw an internal application at work that did exactly this, and there was a predictable mess twice a year when DST started or ended.
It bothers me that a minute is not always 60 seconds. For leap seconds why did we choose 23:59:60 instead of 24:00:00? If we had went with the latter we also solve the daylight savings time problem. A day would last until 22:59:59 or 24:59:59 depending on the time of year. It seems like a lot of things would be simpler that way. Does anyone have a better understanding of why things ended up this way?
It's about consistency. 23:59:60 is introducing a special case to the seconds field which you have to account for. 24:00:00 only adds a second to the day where it naturally should go. I don't redefine what a minute is, a minute is always 60 seconds. You are confusing two separate ideas, displaying a time and finding the difference between two times. Lexically you wouldn't say that minute is 1 second long, you would say that the day ends on the first second of the last new minute.
...so what that doesn't matter? Imagine you are an evil genius who has built a rocket to extend earth's orbit by an hour each day. How would you now represent time for a day once we live in that new orbit? Most people would just add an hour so the days end at 24:59:59. What if the rocket didn't extend time by an hour but instead by 59 minutes and 59 seconds. What would you do then? We can repeat this until we get down to one second. Wouldn't it make sense to end up at 24:00:00 instead of 23:59:60 in that situation?
That is one of my all-time favorite youtube videos... This one, and a similar one on unicode[1], are my goto exhibits when I need to illustrate to a non-technical family member how much background complexity is involved in software
I would think the time zone example would go over better than Unicode, because it's less esoteric. Almost everybody deals with time zones at some point; not everybody deals with Unicode.
Would it be too much to have a global movement for uniform time standards? No half-hour timezones, no daylight savings, no leap seconds, just easily predictable time. Calendars are supposed to be convenient, and right now our ancient conventions make it difficult to understand each other (not just computers; think of travelers having to adjust to timezones, "do I need DST or not?"). At least, lets think of the electricity consumed converting date structures.
PS. and while we re at it why not switch time to base-10? And seriously get rid of the imperial units already.
>We already do have a global standard time zone, and everybody who cares already uses it: UTC. There are also more accurate time standards in use for more specialist purposes.
Regarding daylight savings time, Arizona doesn't observe DST but the Navajo Nation region in Arizona does but inside that is a region that doesn't observe DST.
A Book - "TIME – From Earth Rotation to Atomic Physics", by Dennis D. McCarthy and P. Kenneth Seidelmann, (Not a cheap book most places you can find it but it is worth it.)
The book left a permanent impression on me with respect to fundamentally understanding the physicality of time as an aspect of the universe we exist and are attempting to measure every time we record the time something takes to happen.
May I suggest adding "my users are in the same time zone as I am". Automated tests are often written with this implicit assumption too.
I recently fixed a bug for a local hire service I use frequently. I'd made the booking in Seattle, and they were 18 hours late when I got back to Melbourne...
I'll add one: "But surely each place only has one DST each year…"
Nope. Troll Station in Antarctica has four different settings through the year.
Personally since there are only 40 people there I think they should just suck it up and pick two. They are probably causing a lot of software bugs. I know they cost me a couple hours.
Edit: Oh crum, I see Morocco suspends DST during Ramadan, so they have four settings too… and 33 million people. I can't ask that many people to change, it would take too long.
"I can't ask that many people to change, it would take too long."
Made my day.
There is this famous conversation-starter: "If you could enact one law, what would be it?" I always say that I would force the US to ditch imperial units, the Fahrenheit scale, and whatever other wonky specialties they've come up with that just confuse programmers everywhere.
* Time zone (/DST) changes are always announced well in advance.
* Time zone (/DST) changes are always announced in advance.
Yeah, so US announcing DST changes in 2005 that would take effect in 2007 is unusually long compared to most DST changes. A lot of Middle Eastern countries change DST based on the start of Ramadan, which doesn't officially start until someone actually observes the phase of the moon. While you could calculate this from well-known astronomical principles, there ends up being several different ways to handle corner cases which leads to several Islamic calendars in practice. This means that changes can end up being very last minute--a few years ago, the Olsen database wasn't updated for Egypt's DST changes until after they took effect, and there appears to have been a case or two in history where the actual official declaration of the change was, in effect, retroactive.
* 12 AM/12 PM are unambiguous.
* Days, weeks, months, and years all start on consistent patterns.
Well, everything is largely consistent in modern civil time reckoning based on the Gregorian calendar, but it's very definitely not the case if you look back as recently as a few decades ago or think about other times. Julian dates start at noon. Particularly in medieval times, the start of the year might be December 25 or March 25--and the time people switched to January 1 as the consistent start year wasn't necessarily the same as the switch from Julian to Gregorian calendars.
Time zones and DST are fun. When I worked at a healthcare software company, many hospitals chose to have their yearly upgrades done in the fall during the DST transition, because 1) it's confusing to see or even display documentation for events that happened at 2:30AM and the other 2:30AM and 2) it avoids any bugs that might crop up from the doubled hour.
82 comments
[ 4.5 ms ] story [ 183 ms ] threadThere are also examples of non-DST related time zone changes of up to 24 hours (mostly from Pacific islands deciding to change what side of the international date line they are on), so in some zones, there are certain entire days that never happened (e.g. the entire day is an "imaginary day") and some that happened "twice".
Timezones conversion is super hard, which is the exact reason I rely on libraries rather than trying to reinvent the wheel.
Actually this isn't a falsehood, this is true.
https://news.ycombinator.com/item?id=4128208
Just look at all the code to deal with it in D (written by Jonathan Davis):
https://github.com/dlang/phobos/blob/master/std/datetime.d
1.3MB and it's actually raw code. Amazing.
In past discussions of leap seconds, it came out that the POSIX standard specifies that no such second as 23:59:60 can exist (by specifying that "one day" consists of exactly 86400 seconds). As I understand things, this is why the erroneous behavior you mention is done.
It is not clear to me why, for the sake of complying with a standard we know to be wrong, we report false information and otherwise do things that we also know are wrong. The obvious approach would seem to be to either change the standard or just ignore it.
Sure, but they usually have more resources than programming language teams. Consider all the programming languages on your machine. Can you trust they all did the timezone, with all its nuances, correctly?
Consider something simpler - the trig functions. I've noticed time after time that many language libraries get it wrong, and this is with decades of development. Errors are usually in the form of precision that is lower than it should be, and poor handling of special values like NaN and infinity.
The situation with that is so erratic that with the D programming language we'd write our own trig implementations rather than use the C library ones.
Does Apple even have resources that work on tzdata code? Or do they just use open source code?
I wrote more about the issue here: http://www.creativedeletion.com/2015/12/03/timezone-updates-...
In particular for Elixir I trust the time zone data from Tzdata more than for any other language, mostly because I know it is designed to have up to date data. (And well, I know the code because I wrote it myself.)
Even if a system is good in many ways it does not matter if the data is not up to date.
Edit: Seems like we can specify CLOCK_TAI for clk_id on linux now.
I've encountered English people who made a fuss about the loss of "God's Magnificent Time", but for better or worse GMT is no longer the official name.
"CST is also used for: Cuba Summer Time, China Time, Central Standard Time (Australia). PST is used for Pakistan Standard Time and Pacific Standard Time. If you want a unique identifier for the time zone in the Pacific West of the USA it looks like this: “America/Los_Angeles”."
In this case the name of the time zone "people live in" is "Europe/London", not GMT.
eg daylight savings is UTC+13:00 here.
Until the unified timezone happened, timetables would be printed with multiple timezone information. So maybe your train would depart in Dresden at exactly 14:00 Saxonian time, take exactly three hours to drive to Berlin, and arrive there at 17:06 Prussian time.
None of which match up with any Civil Time anywhere on the world, except UTC which does so only periodically for some European countries, and year round in the following countries which resist the idiocy of shifting their clocks around based on the Season: Burkina Faso, Côte d'Ivoire, The Gambia, Ghana, Guinea, Guinea-Bissau, Liberia, Mali, Mauritania, Tomé and Príncipe, Senegal, Leone, Togo, Western Sahara (not technically a country but that's unrelated to the currently customary timezone), Greenland, Iceland, Saint Helena, Ascension Island and Tristan da Cunha
[1] https://bugs.python.org/issue15873
Thankfully, there are third party libraries that do just this (such as iso8601), but its such a silly dependency to need.
"The special area of "Etc" is used for some administrative zones, particularly for "Etc/UTC" which represents Coordinated Universal Time. In order to conform with the POSIX style, those zone names beginning with "Etc/GMT" have their sign reversed from the standard ISO 8601 convention. In the "Etc" area, zones west of GMT have a positive sign and those east have a negative sign in their name (e.g "Etc/GMT-14" is 14 hours ahead/east of GMT.)"
[0] https://github.com/bitwalker/timex [1] https://en.wikipedia.org/wiki/Tz_database#Area
I'm curious: Why where you using e.g. GMT+5 or GMT-5?
Wrong, Russia has combined some timezones into a single one and then has split it back a few years later.
But how does this solve the problem? Sure, 23:59 is now 60 seconds long, but the minute 24:00 is only 1 second long.
[1] https://www.youtube.com/watch?v=MijmeoH9LT4
(Yes, yes, amounts to the same thing. Except when it breaks.)
PS. and while we re at it why not switch time to base-10? And seriously get rid of the imperial units already.
https://en.wikipedia.org/wiki/Swatch_Internet_Time
"So you want to abolish time zones" https://qntm.org/abolish
>We already do have a global standard time zone, and everybody who cares already uses it: UTC. There are also more accurate time standards in use for more specialist purposes.
"You advocate a ________ approach to calendar reform" https://qntm.org/calendar
These links helped convince me that while time zones and calendar systems are very messy, there are some inherent reasons why they are messy.
https://xkcd.com/927/
Two of the best resources on this topic that I've come across are:
A PyCon presentation - "Blame It On Ceasar" https://www.youtube.com/watch?v=GBKqRhn0ekM
A Book - "TIME – From Earth Rotation to Atomic Physics", by Dennis D. McCarthy and P. Kenneth Seidelmann, (Not a cheap book most places you can find it but it is worth it.)
The book left a permanent impression on me with respect to fundamentally understanding the physicality of time as an aspect of the universe we exist and are attempting to measure every time we record the time something takes to happen.
I recently fixed a bug for a local hire service I use frequently. I'd made the booking in Seattle, and they were 18 hours late when I got back to Melbourne...
Nope. Troll Station in Antarctica has four different settings through the year.
Personally since there are only 40 people there I think they should just suck it up and pick two. They are probably causing a lot of software bugs. I know they cost me a couple hours.
Edit: Oh crum, I see Morocco suspends DST during Ramadan, so they have four settings too… and 33 million people. I can't ask that many people to change, it would take too long.
Made my day.
There is this famous conversation-starter: "If you could enact one law, what would be it?" I always say that I would force the US to ditch imperial units, the Fahrenheit scale, and whatever other wonky specialties they've come up with that just confuse programmers everywhere.
Also, let's get rid of DST entirely.
Days don't always start at midnight.
Midnight might not exist--some countries perform their DST transition at midnight, so the clock goes from 11:59 PM to 1:00 AM. Fun stuff!
* Time zone (/DST) changes are always announced well in advance. * Time zone (/DST) changes are always announced in advance.
Yeah, so US announcing DST changes in 2005 that would take effect in 2007 is unusually long compared to most DST changes. A lot of Middle Eastern countries change DST based on the start of Ramadan, which doesn't officially start until someone actually observes the phase of the moon. While you could calculate this from well-known astronomical principles, there ends up being several different ways to handle corner cases which leads to several Islamic calendars in practice. This means that changes can end up being very last minute--a few years ago, the Olsen database wasn't updated for Egypt's DST changes until after they took effect, and there appears to have been a case or two in history where the actual official declaration of the change was, in effect, retroactive.
* 12 AM/12 PM are unambiguous. * Days, weeks, months, and years all start on consistent patterns.
Well, everything is largely consistent in modern civil time reckoning based on the Gregorian calendar, but it's very definitely not the case if you look back as recently as a few decades ago or think about other times. Julian dates start at noon. Particularly in medieval times, the start of the year might be December 25 or March 25--and the time people switched to January 1 as the consistent start year wasn't necessarily the same as the switch from Julian to Gregorian calendars.