Because you can now fully simulate arbitrary times during testing by providing your own implementation of the time provider. If you just take in a timezone as an argument, how will you simulate it being 2041-08-23T21:17:05.023743Z?
True. When you can just make any function be whatever you want whenever you want that sort of abstraction does seem a big much. I was responding to the specific question in reference to the time provider in .net 8.
Time handling seems like one of those cases where a type system could really come to the rescue - times with and without time zones in them shouldn't be the same type, and using one in a place where the other is expected should ideally just give you a compile error.
All timestamps even have a position in space that will experience discrepancies relative to all other timestamps, even if they are UTC, we just usually don't care about it that much.
Since our time is at least all Earth-based, we could do even better than timezone and make timestamps have a lat/long attached for the place in which it was produced. Maybe a linked list for any timestamps that are calculated from that based type.
The social solution is to abolish timezones (mine is of course objectively the right one, other people must deal with breakfast at 9pm wherever they are).
I need a null value for when the timezone is unknown. If a user enters 9:30, they do not want to be bothered with specifying a timezone if they keep everything local to their life.
Right now, I am dealing with this problem that I have historical data recorded in Eastern and Mountain Time. Everything is local and nary a timezone in sight. Data files are mixed, and no obvious way to determine provenance.
I cannot just guess what timezone these data points were collected, so I have to suffice for local time. Which is 100% fine for the users, because that’s exactly how they think about the problem. There are probably no global events against which they need to sync, just interested in the relative differences between events.
It may not matter, but without UTC and offsets, you can't get relative differences between all events accurately because of DST. But it may not matter for your use case.
Well then maybe they should store the epoch alongside the duration value too! Wouldn't want to get confused thinking it was duration since some other epoch!
Here are two counter examples that sprang to mind:
1. I've set my alarm for 8am local time every day. I don't want my alarm clock to go off at 3am just because I'm in a different country
2. If I have an appointment at 10am next November, that's actually a datetime with a locale (given that my appointment is at 10am local time, no matter whether the DST rules change between now and then).
Well, you'll need another type meaning "local time" for #1. That's not a time without time zone, it's an entirely different thing.
Most environments simply can't represent that thing. They never could. With those changes this becomes more obvious, and with some luck people will finally fix this.
(The most absurd case is the "timestamp with local time zone" from SQL, that reads exactly like what you want, but represents a timestamp in a specific time zone, that is discoverable but mostly unknown.)
Well, you seem to have some weird definition for the word "exactly".
"8AM" is exactly time without a timezone. "8AM at local time (local defined by this procedure)" isn't exactly that.
Perhaps you mean something like homomorphic. It is almost perfectly homomorphic. But if you define your types that way, you'll push yourself into the most useless ones you can get quite quickly.
1. That isn’t date data. That’s just storing a local time.
2. That’s why you WANT time zones in this example. It will automatically stay at 10am instead of bouncing around when DST rules change.
I’ve never ever needed timezone-less dates. Even using UTC or timestamps, it’s still UTC.
A date without a time zone is like “10 inches” but without the inches so it’s just “10”. Absolutely meaningless. You start moving just “10” around without the units in your code and then your Mars Orbiter explodes.
That’s not a date-time though. You would not store that using the same type-compatible objects as your regular date-times or that’s how you end up one day blowing up a Mars Orbiter.
Yes, you need both zoned and zoneless versions of dates and times. This is why e.g. java.time has both (and it has LocalDateTime for when you need to say Santa delivers presents at 00:00 on December 25).
Zoneless time is not a mistake. Using it when you meant to use a zoned time is.
Making it easier to use zoneless over zoned is a mistake, though. An even more grave one is making zoneless the default. Unfortunately, that's a mistake that we've made decades ago, and now we have to fix it in a way that deals with all the accumulated mess.
Really, though, this is a recurring lesson in software engineering: don't be in a rush to make something a default just because it's "obvious"; explicit is better than implicit. Strings are another example; we've treated them as arrays of characters for so long that it's still a common default semantics for indexing a string, except now what you often get instead is UTF-8 or UTF-16 code units rather than actual code points. All of which is likely not what you actually need - but what you do need depends so much on the task that it should always be explicit.
Which days are recognized by an administrative region as holidays is an orthogonal concern from timezones. e.g. the US recognizes the 25th but spans multiple timezones.
>2. That’s why you WANT time zones in this example. It will automatically stay at 10am instead of bouncing around when DST rules change.
Respectfully, no - in my scenario, if I used a timezone then I would show up at the wrong time for my appointment: each locale that observes DST has two timezones: one for DST, one for non-DST (e.g. BST and GMT for the UK). If I record a 10am appointment as "10am BST" and my country changes the rules on daylight savings transition date between now and the appointment, then I will show up at the wrong time for my appointment. Whereas if my appointment is recorded as "10am Europe/London" then it's encoding the human reality of my appointment, and can be correctly converted to an Instant in time if the rules change.
It sounds to me from your reply that you are mentally modelling all datetimes as Instants. These are one key aspect of dates+times (and often all a lot of software needs - since it's usually referring to some point in time that has already passed, or a future point in time that is x seconds from some other Instant), but Instants don't cover everything required when working with dates and times.
>I’ve never ever needed timezone-less dates. Even using UTC or timestamps, it’s still UTC.
Try sketching out the design for a calendar application and I bet you'll very quickly run into a whole raft of datetime related logic you might never otherwise encounter. Think particularly about future events: some are fixed instants in time (e.g. a meeting with people across timezones), others aren't (e.g. a dentist appointment in your city - my second example above is an instance where the local time of the appointment is relevant, not the Instant in time that local time referred to when the appointment was made). How do you represent recurring events?
For the classes of problems that you mention, I consider them as separate data types entirely and they shouldn’t ever touch your timezone-aware date-times objects. They should be stored as a composite types of your date, maybe time, and target timezone.
If you can mix timezone-aware objects with non-timezone aware objects like you do in Python, it’s get pretty dangerous because people will just mix them willy nilly and add and subtract the timezone-aware parts and then you get some really subtle date/time bugs.
That’s why I believe either a date-time MUST have a timezone and for other cases, you need a specialty data type for your domain, possibly provided by your date/time library.
> 1. I've set my alarm for 8am local time every day. I don't want my alarm clock to go off at 3am just because I'm in a different country
Without timezone, your clock would actually go off in the middle of the night when you are in a different country. Your clock does not know you change location (no timezone, no location-dependent time), so it will alarm you after exactly 24 hours since the last alarm, which, in another country, can be midnight.
> 2. If I have an appointment at 10am next November, that's actually a datetime with a locale (given that my appointment is at 10am local time, no matter whether the DST rules change between now and then).
If everyone is staying in your city, then that works great (sans catastrophic continental riff or meteorite strike). When at any point you involve more than one geographic location, then the appointment is meaningless without timezones.
Now, maybe it is actually the case that you and your software only care about one geographic location in perpetuity, but just like assuming 32-bit IP addresses, 8-bit ASCII, and length-oblivious arrays are all you need, the teeth marks from its consequential bite tend to be quite deep and sanguine.
> Without timezone, your clock would actually go off in the middle of the night when you are in a different country. Your clock does not know you change location
This does not describe any modern person's experience. Your clock is connected to the internet; it always knows the local time.
You're missing the point. Evaluating the rule requires a timezone, but the rule itself is not defined with reference to a specific timezone (well, the second one is assuming you can use something like PT that changes between PST and PDT throughout the year). To define the first rule requires a zoneless time.
I think ISO 8601 made a big mistake by allowing timezone-less representation[1]. They should have used L for local time so it's explicit, and not allowed any timezone-less representation.
Sure it's technically well-defined as it is, but I think it's too important not to be written out.
In that case you can represent your alarm either as 08:00L if you want to wake up in the morning regardless of where you travel, or 08:00-02 for those cases where you need to stay in sync with home say.
I completely disagree, the local time is the only useful part of ISO 8601.
The big problem here is that it does not use timezones, but UTC offsets. Timezones can and do change their UTC offsets, completely breaking all future timestamps saved in ISO 8601-with-"timezones". "08:00 UTC-02" is ambiguous, "08:00 America/Nuuk" is not.
You don't need both. Named zones can do everything that offsets can, and it's still useful to know the named zone for historical timestamps in many situations.
> After all, if a zone changes offset today it shouldn't change the timestamp I recorded yesterday.
That's not how named timezones work. That behavior would be bad and it would also qualify as ambiguous.
> Half the world experiences a scheduled offset change every time summer time begins or ends.
But they do so because they record the actual offset when the time is recorded. If I send you 2023-10-29T02:15NO, how do you know if it's +01:00 or +02:00? Similarly, if Norway suddenly decided to switch their offset to be 15 minutes back, how will you handle that when reading previously recorded timestamps?
> If I send you 2023-10-29T02:15NO, how do you know if it's +01:00 or +02:00?
I think instead you'd send a timestamp (unix or NTFS or TAI or other) annotated with "Norway", and you would not send the offset.
> Similarly, if Norway suddenly decided to switch their offset to be 15 minutes back, how will you handle that when reading previously recorded timestamps?
Am I that bad at explaining this?
If Norway changes their offset, they will change it as of a specific moment in time. Times before that moment use the old offset, times after that moment use the new offset. The timezone database doesn't just store one or two offsets for a location, it stores every offset that has ever been used.
Checking whether you're before or after 5am on Feb 29th 2024 for a 15 minute change uses the same code as checking whether you're inside any particular year's version of summer time.
> I think instead you'd send a timestamp (unix or NTFS or TAI or other) annotated with "Norway"
This is what I attempted to write with "2023-10-29T02:15NO" (NO being Norway's country code).
> Am I that bad at explaining this?
Could be I'm dense, and it's not my field of expertise.
Let's say Norway has decided to change it's offset from +01:00 to +01:15 at 2024-01-01T01:00:00Z. I have a process that records timestamps, and lets imagine it records them with the suggested named offsets instead of numerical offsets, in the format shown above.
Later next year, you look in the logs and see 2024-01-01T02:07:00NO. How do you know which time this is in UTC?
With numerical offsets this is simple, because it would either be recorded as 2024-01-01T02:07:00+01:00 or 2024-01-01T02:07:00+01:15. But when you just have NO, how to you convert the timestamp to UTC?
A Unix/NTFS/TAI timestamp would not be affected by the offset. Using Unix as the example:
2024-01-01T02:07:00+01:00_NO -> 1704071220_NO
2024-01-01T02:07:00+01:15_NO-> 1704070320_NO
The timestamp doesn't have to be UTC, but it should specifically not be local time. You should be able to convert it to UTC without knowing the offset or time zone.
So, done this way, storing time zone and no offset is unambiguous. And it gives you more information than storing an offset and no time zone.
Thanks, yes that makes sense. I was working under the assumption that ISO 8601 was meant to be human-readable. If you can forego that requirement, then the solution you present would work well indeed.
1 could be better served by an integer representing the minute it should go off anyway, since there is no specific date attached
2 time zones account for DST so that point is moot, but in the context of traveling and still wanting to maintain the same times this is actually something calendar apps often get wrong
No, I can't think of any case where that would be useful (although I do take umbrage with a local datetime being referred to as a 'naive' type - it serves a very specific purpose as a primitive... if it's naive then so is an int, since it does not specify a unit).
I don't know anything about Python's datetime handling (other then to be alarmed by it given what I've seen in this thread). I live in the Java world where the java.time library is one of the best around (after a decade or more of Java having one of the worst date time libraries)
datetime with locale is how timezones are represented in python. What you're asking for there is a datetime with a timezone, not a datetime without a timezone.
At my last job we had types for a `Time.t` (time + date + timezone), a `Date.t` (just a day of the year), and a `Time.Ofday.t` (a time of the day with no timezone or date attached - like 8:00 AM).
This worked really well! You represent (1) with a `Time.Ofday.t`. I suppose if you wanted you could represent (2) with a `Time.Ofday.t` + a `Date.t`, although it kinda seems to me like you want to keep timezone information around if you're dealing with DST changes.
"8am" is a time that inherently has no timezone, and yet we need it. It's not that we don't need timezoneless time - we just need to stop using timezoneless time to represent times with a time zone and vice versa. Most time APIs were simply designed wrong on purpose for "simplicity".
Reminds me of trying Scala and a library where not only were paths a type rather than primitive strings (good, like pathlib in Python), but absolute and relative paths were actually different types.
The way you put it makes it appear very surprising that time is not the extended hello world of type systems. Maybe it's because no environment has ever arrived at an implementation they are actually happy with, before fizzing out at "good enough"? It's clearly a topic where we tend to prefer something rough but shared over something better but custom, so the "good enough" is powerful.
Reading this article prompted me to future-proof a program I maintain for fun that deals with time; it had one use of utcnow, which I fixed.
And then I tripped over a runtime type problem in an unrelated area of the code, despite the code being green under "mypy --strict". (and "100% coverage" from tests, except this particular exception only occured in a "# pragma: no-cover" codepath so it wasn't actually covered)
It turns out that because of some core decisions about how datetime objects work, `datetime.date.today() < datetime.datetime.now()` type-checks but gives a TypeError at runtime. Oops. (cause discussed at length in https://github.com/python/mypy/issues/9015 but without action for 3 years)
One solution is apparently to use `datetype` for type annotations (while continuing to use `datetime` objects at runtime): https://github.com/glyph/DateType
It's not as helpful as you'd think IME - timezoneless date times are suprisingly niche (at least as a useful tool to intentionally choose, rather than an accidental side effect of API choice.)
This would be fantastic. As someone who just occasionally touches Python, I can’t count the number of times I’ve been bitten by bugs caused by unwittingly mixing tz-naive and aware datetimes.
I suspect a lot of financial data code is going to break. I've worked on a number of codebases where the explicit instructions were "everything is UTC, all the time", and enforced at the entry edges of the code. Bringing in timezones was a definite NO for reasons of the unholy mess this would inevitably create:
* India has half-hour timezones
* Countries change their DST rules all the time.
* Conversion between timezones is an utter mess.
As Wes McKinney once said about python datetime: "Welcome to Hell"[0]
You still keep everything in UTC in this new world, it's just the object you use internally now knows that it's UTC and makes it so you can't accidentally do calculations on two naive datetimes which happen to be in different timezones.
You get to have the typing help you enforce that everything is in UTC.
RealLifeLore just had a video about time zones in that area of the world, there's an area between New Zealand and Hawaii where you can go north/south and jump an entire day.
There isn't a perfect geographical width for time zones. So humans pick something to define the boundaries between time zones. And making boundaries on a map is political.
To some extent a compromise between solar time and political and economic realities. For example, the Eastern Time Zone in the USA stretches almost to 90 degrees West, reflecting the East Coast's powerful pull on that part of the country.
A lot of it was GE's fault at the time, specifically. GE wanted facilities in New York, Cincinnati (OH), and Louisville (KY), among others, to all be in the same time zone and had the economic power at the time to lobby the railroads and the cities to make that happen.
Friend told me that Indiana not adopting Daylight Time until quite recently was due to a struggle between broadcasters, who of course wanted to match their networks, and drive in operators, who wanted it to get dark earlier so they could start the shows earlier.
Yeah, Indiana also had an interesting three way battle between Chicago, Louisville, and Indianapolis. A lot of population near Chicago getting Chicago broadcasts (Central), a lot of population near Louisville getting Louisville broadcasts (Eastern), with the state capital Indianapolis interestingly caught up in the middle (physically closer to Chicago, but maybe emotionally more connected to Louisville) which itself as a city eventually after a lot of back and forth settled on Eastern time following Louisville's lead as one of the westernmost cities in the timezone.
Indiana's Daylight Time mistakes were fascinating. It wasn't that the state didn't adopt it, it was that originally the state allowed it to be a per-county decision as timezones have always been in Indiana. At one point in time if you were traveling I-65 which is nearly due north/south between Louisville and Indianapolis you could experience four different timezones (CST, EST, CDT, and EDT) and which ones agreed with each other obviously depended on which month you were traveling. Since Indiana went state-wide Daylight Time and Indianapolis decided on EDT once and for all, all of I-65 today is EDT I believe, but it is still strange to remember the years where that wasn't the case.
(ETA: one of the underappreciated homogenizing factors here has been the modern cellphone. People would get really confused if their cellphones hopped an hour back/forth every so many miles as you passed county lines. In the eras of paper maps and hand-set clock radios in cars that would have mattered a lot less.)
That far south it only varies by a couple of degrees. It does wreak astronomical havoc in Mumbai, though, where the Sun will sometimes be way in the northeast at noon.
because humanity never understood time properly.. so all these facades making it look "simpler" while actualy a lot more complicated.
long time ago there was a special `$ man date` -like page in linux which went into long explaining many "amazing" things about calendar stuff, like whoever feudal in 1553 deciding that certain week was bad and striked it out of his and his country's calendar, or another one that liked certain month and decided to repeat it...
In Europe, a huge driver for standardization of timezones into "reasonable" slots was railway traffic, including cross-border traffic.
Railways are extremely sensitive to exact time and, indeed, the very concept of unified time across the entire region or country only started developing when railways expanded across Europe. Prior to that, individual towns were happy with their own local solar time, but once railway connections were introduced, time irregularities would cause chaos at best and carnage at worst. That led to introduction of unified railway time which developed to timezones as we know them.
Railways aren't as prominent nowadays as they were 100-150 years ago, and countries like Nepal and India don't have extensive, frequently used cross-border railways anyway; any cross-border traffic is sporadic and mostly freight. So there is one fewer reason to cooperate with your neighbors when it comes to time-related issues. Trucks can take weird timezone changes just fine.
Last startup I worked for hired a Nepal-based QA person. There was a bunch of calendaring and daily/weekly charts in the apps, and she found bugs in _everything_.
I make sure to test with Nepal time whenever I'm testing date/time stuff now.
And, of course, there's the (hopefully apocryphal) story of the French initially referring to GMT as 'Paris time minus nine minutes and twenty-one seconds'.
As it happens, this story isn't apocryphal at all! One can readily find the original law, which was enacted on March 9, 1911, and published in the Journal officiel of March 10, 1911 [0]:
> Article unique. — L'heure légale en France et en Algérie est l'heure, temps moyen de Paris, retardée de neuf minutes vingt et une secondes.
The decree which finally replaced it was made on August 9, 1978, and published on August 19, 1978 [1]:
> Art. 2. — Sur l'ensemble du territoire de la République française, le temps légal (ou heure légale) est défini à partir du temps universel coordonné (UTC) établi par le bureau international de l'heure.
When I built a live clock for some new CasparCG based graphics for a major TV program out of singapore some years back, a colleague reviewing it in London tried to trick it with a Nepal offset — apparently they’d run into an issue with the Viz system they used in 2015 when there were a lot of lives from Nepal.
I'd say that wouldn't enforcement be rather simple going forwards since you could just enforce that dt.tzinfo == datetime.timezone.utc, but as far as I know, timezone information checking in Python can be a bit of a pain.
That code runs in the context of humans. Therefore it needs to take humans into account when running.
This is a bit like asking "how do you represent a string length in utf-8?" The answer is, you don’t. It’s almost never a problem in practice.
Just like there is almost never a reason to care about the number of characters in a string, it’s very difficult to imagine that this code comes up in practice. But if it did, the code is simple: for a given timezone, is the current time in utc past 9:30am? If so, the store is open.
Let's say you host a store front for many businesses and it's decided that your customers, who are businesses, some with many stores, want to store a default opening time. Something along the lines of "All Home Depot's open a 6 AM, all Lowes open at 7 AM."
The store property would be "6 AM". Anywhere it’s used, it’s running in the context of humans. When displaying on a website, you don’t need to do any conversions: the human viewing the store knows where it’s located, and they know the current time, so computers don’t need to answer the question of whether it’s open.
If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone.
Indeed. And anyone near a time zone boundary looking at an online map of store locations doesn’t want to see “opens at 6AM” or whatever — they want to know whether the store is open.
Indeed. It's not as bad as mapping address => sales tax (no ZIP Code isn't good enough), but only because it doesn't change as often and tends to follow county boundaries when it doesn't follow state boundaries.
Another good example. Arizona doesn't follow Daylight Savings, but the Navajo Nation, which includes the Four Corners area and is in three states, does observe the change.
However the Hopi reservation, completely surrounded by Navajo territory, is entirely in Arizona, and does not observe DST. So it can get quite complicated and it's not enough to say "Arizona doesn't have Daylight Savings Time".
There's no time zones involved in that at all. Nobody is going to be checking the opening hours of a Home Depot in another timezone and expect to get the date in their local timezone.
However, if I look up the opening time of a Home Depot in another time zone, I expect the result to be in that store's local time! I'll do the conversion myself. If I get my local time instead, I'll still perform the conversion, and get the wrong result.
You could just decide that any time listed anywhere on the internet should include a timezone... but that would be nuts.
I've never visited a brick and mortar store's website to check their opening hours and assumed it would be anything other than in their local timezone.
So maybe individual stores can customize their opening time, but corporate still wishes to store a default. How do you represent the default? How do you handle the DST time changes for individual stores?
I'm trying to poke holes in the "always use UTC and you're good" advice. I have actually encountered this scenario, and I didn't know how to apply that advice.
> I'm trying to poke holes in the "always use UTC and you're good" advice.
There's a much simpler one - recurring meetings/events (in-person or local-ish like within one country) that cross a DST boundary. You always want these to be attached to a local time zone, so it doesn't suddenly happen an hour earlier/later than you intended.
utc is for a specific moment in time (deterministic with great precision for everyday use).
"9:30am local time" may be ambiguous/unknown (the rules may not exist yet)
Using the same programming type for both may cause confusion.
How do you express geographic coordinates? Noon in London and in NY are different moments in time (it may be important if you want to have a remote meeting).
No need for something as brittle as geographic coordinates or addresses. Time Zone names are the right abstraction. A given location very rarely [edit: adopts another] time zone (unless in case of annexations, handovers of territory, etc.), but time zone rules (zone offset from UTC, DST, etc.) change all the time. A consequence of this is that the time zone database of applications should be updated every few months!
"Complicated" then. One would have to consult a map to find out the time zone from geographical coordinates. The time zone is really everything needed to properly perform timezone-aware arithmetic.
Postgres stores this as TIMESTAMP WITHOUT TIME ZONE, sometimes called wall-clock time. IMO this is a good convention -- treating "ideal times" and specific epoch moments as two different types -- and more languages should support this (ideally with a less confusing label).
Reference opening time to local time on each day of the year, and then refer that back to UTC.
Like California is -8 in Standard, -7 in Daylight, so a 9:30 AM opening in the Winter is at 17:30 UTC, in the Summer, 16:30 UTC.
It's a hassle but programmers are supposed to model reality.
As an example of the problems that can arise, I remember when the new Japanese Emperor was to take office upon his father's abdication, there was a big fuss because the Era Name was customarily only revealed after the new Emperor's reign started but all the computer people wanted to be ready in advance. Not sure what finally happened, if it was revealed in advance or not.
What if the date for DST changes tomorrow, but only in some locations? How will you know which stores times need to be adjusted and which don't?
UTC to local time is not a function, it is not somethung that can be computed once and stored for the future. The two are fundamentally different concepts, and can only be related to one another in certain contexts, and only reliably for the past.
> How will you know which stores times need to be adjusted and which don't?
There are a number of libraries for this, usually updated by the unpaid volunteer in Nebraska. Or else your company has staff whose job it is to follow this for any place you operate.
If you record opening times as UTC, well, then you have to change to UTC when the store switches standard <=> daylight.
This is what Fred Brooks called an essential complexity. You can poke the pain from one part of your system to another part, but it's never going away.
The function is complicated. There is no avoiding that.
One Pacific island nation duplicated a calendar day when it switched International Dateline sides to more closely align with Australia instead of Hawaii, reflecting a shift in its economic ties. No idea how they handled it exactly but pretty hairy to model.
The warning is also opt-in; you'll have to run python with PYTHONWARNINGS=always or similar to see it.
Which is scary, because you will be taking the performance hit from generating the warning regardless, which can easily amount to 30% of your total CPU time if you use utcnow/utcfromtimestamp a lot.
Aren’t now, “will be” in a future version. So the can will be kicked down the road as many generations of devs as needed until no one cares any longer.
So, they remove it in 12 months and your code wasn't ready for the change, and people mock you for having not been prepared by updating your code. The Python people have told you to remove it, so nobody will have any sympathy for you if you don't do it. As you demonstrated, people are sticklers for quoting exact wording when it comes to criticizing others.
Has anything in Python been removed in 12 months? TestCase.assertEquals was deprecated in Python 3.2 (February 2011) and was finally removed in Python 3.12 (October 2023). And that was 12 years for just a simple rename from "assertEquals" to "assertEqual".
Stupid decisions don't always get put into practice after they're announced.
I seem to remember reading that remove-if-not is deprecated in Common Lisp, and that this is almost universally viewed as a mistake, but it remains deprecated (and the deprecation is ignored by everyone) just because the standard doesn't change anymore.
For sharing between systems and representing dates to humans debugging a system, the following has worked well …
Planned local events: future date and time string including local time zone
Planned global events: future date and time string in UTC, with UTC timezone
Past events: date and time string in UTC with UTC time zone, unless there is a strong case to be made that these should be localized (see “planned local events” above)
Strings in the format specified by ISO-8601.
Of course, internal representation can be whatever it needs to be for efficiency. For example, maybe your database stores as integer microseconds since an epoch. Fine, but as soon as that leaves the database, ISO-8601 that date.
I like your premise but disagree with your guidance.
You should not convert the integer early, you should actually wait until the very last moment.
Nowadays a timestamp will be marshalled and demashalled by (at the very least) two runtimes until it reaches human eyeballs opening up a wide spectrum of bug potential, difficult to understand code and development slowdowns. Keeping it simple and unambiguous is very important. Time libraries have a habit of coming and going, ints are always there and 99% of the date arithmetic is trivial any way.
Don't use a lib, just pretty print numbers and you'll be fine.
An int is not unambiguous, though. "2023-11-19T14:22:34-03:00" is clearly a timestamp and represents an unambiguous point in time, while a number by itself doesn't mean anything. You have to know that it's a timestamp (probably from context) and you have to know what timezone it's in.
Not in the future it’s not. There’s no way to know the exact epoch time until it has happened. Timestamps “float” and can accommodate local fluctuations or even changes.
Haha. I’m sorry if the terminology is catching you out. It’s not epoch time that floats. Epoch time is an inalienable absolute (at least in theory). However actual cosmological and time as perceived by humans does drift wrt to epoch time. Which of these is right is a philosophical discussion but at the present instant they do indeed converge and so epoch time between now and some time in the early 1960s is “the correctest” from a systems perspective.
But 1700414554 is not unambiguously a date. Could be a count. Or an identifier. It needs metadata to tell you how to interpret it. For this to represent a datetime, the human needs to know that it’s a datetime, and what epoch and timezone to use. The ISO date string removes the ambiguity because the context for interpreting it is within the string.
I never understood this argument. The ISO8601 date string is as much a convention as a unix timestamp is - without context neither are decipherable. If anything, unix timestamp is easier to explain to an alien than a date string. It has a starting point (1970/01/01 00:00 UTC) and it counts seconds from then. Care to explain how an ISO8601 date string is constructed?
Also calculating amount of time between two ISO8601 strings without libraries is nor trivial, or any other operation actually. When dealing with dates, there is only one simple way to work with non-localized times, and it's not ISO-something.
EDIT:
> ...and what epoch and timezone to use.
This is also not correct. Unix timestamp has a well defined epoch and it doesn't deal with timezones (though the epoch itself is, of course, usually defined as 1970/01/01 00:00:00 UTC). You are free to define other timestamps, but unix timestamp is well defined.
Seconds since epoch requires careful and precise handling of datatypes and assumptions. This works great inside a system where that can be managed but for the purpose of exchange between systems the ISO formats contain all relevant information. Either side can use a library to parse the string into their local representation.
Comprehension by extraterrestrial life is a localization problem.
I'm not sure I understand. How is this different from any other quantity with an associated unit? What's the possible failure mode here, interpreting the "created_time_us" field as a distance in km instead of a timedelta in microseconds?
Is the unit seconds or microseconds? How are leap seconds handled? What base type is used? Integer, number, or float? How many bits? What is the precision? What is the CPU architecture? Is it signed? How should overflows be handled?
You can store weight as ohms of resistance on a load cell. But if you want to share that data you need to either normalize the number to a mutually understood unit or provide detailed information about your scale.
Aside from the obvious issue with engineering, physics, etc.
A unix second is allowed to be different from a "real" elapsed second.
The unix day has 86400 "unix seconds" which are actually 86401 real seconds (for days with added leap seconds).
Any application logging real world events against unix time will screw up velocities, force, energy, etc when computing "instrument reading" per "unix second".
It might not matter to most people but it's an issue for surveyors, geophysicists, astrophysicists, engineers, etc.
> Any application logging real world events against unix time will screw up velocities, force, energy, etc when computing "instrument reading" per "unix second".
That is a fair point for situations that depend on short term observations. But, UTC has the same issues there as Unix epoch. I think it is a valid edge case but if you are doing, say, GPS based speed calc I would wager you are already pulling your time reference from a low level source that you can depend on will be running steadily for the X minutes/hours you need it.
Sure, this is what responsible STEM people do for any time lapsed measurements, be those position, nine axis magnetic field recordings, gravity, radiometrics, microwave return, etc - they use an independant epoch based clock source that counts true lapsed time rather than conventional human time.
Point being, it's an area of UnixTime that many overlook - to date, since the 1980s I've had long term multichannel data aquisition running throughout every leap second adjustment which would have had data glitches had the time channel been UnixTime, UTC, etc.
This is the biggest general problem: you can't accurately compare the duration between two date times without having proper context of how the datetimes were stored or generated, especially for future dates. For example, you can't say today what the Unix timestamp will be for 1 January 2030, 10:00:00 AM in Bucharest, Romania. Romania may very possibly change timezone by then. So, if a user sets a meeting for that time, converting to the Unix timestamp according to the current timezone info is wrong. You'll end up alerting them on 1 January 2030 at 09:00:00 AM, or perhaps 08:59:58 AM if some leap seconds get added. Or it may even be entirely different if there is some calendar change for whatever reason by then.
Note that UTC has a similar problem - which is why you actually need a local time string for this type of events.
Correct, neither iso strings mor numbers can get you out of this dilema.
While ISO uses UTC offset and time zone interchangeably it actually is only defined over offsets, not locations.
I've had this problem discussed before. Typically the situation is firing events based on times at specific locations. It is neither straight forward with date libraries nor ist it easy with numeric computation. Sometimes it makes sense to render it down to a specific point in time at storage and not expect the country change timezone overnight. Some times the location itself may be changing randomly (moving objects) but in that case it typically just meant recalculating offsets on movement events.
There's a lot more context baked into an ISO string than a unix timestamp though. Write a regex to find all the strict ISO8601-RFC3339 strings in a directory. Now write one for unix timestamps.
> Also calculating amount of time between two ISO8601 strings without libraries is nor trivial, or any other operation actually.
That's true of any datetime that actual humans use (as opposed to computers, financial systems, sysadmins and programmers).
> Write a regex to find all the strict ISO8601-RFC3339 strings in a directory
That is a weird edge case. I've had this before and the person proposing it hinged on some forensic practice which was based on analyzing human readable data. As I said in my original post, pretty printing ints to ISO timestamps is actually my original opinion.
The good thing about ints is that typically the stdlibs themselves deliver quite good tooling to work with Unix timestamps so in most cases you don't really use a library for the timestamps themselves.
So what? That's true of all formats. How is it relevant to the choice of which format to use at which stage of the process?
> and you have to know what timezone it's in.
You don't.
> clearly a timestamp
Not very helpful. It's easy to set up types in advance. Don't figure out if things look like timestamps at parse time, or you'll be causing the same problems that turn gene names into dates and choke on people named Null.
> Past events: date and time string in UTC with UTC time zone, unless there is a strong case to be made that these should be localized (see “planned local events” above)
I don't think there's any case for storing local time with past events. They happened at a specific date and time, store them in UTC and if needed convert to local on display. Plus, though it's a rare circumstance, for one hour each year DST switches cause a duplicate hour - so local time is ambiguous during that hour.
There is, when you aggregate your data to create graphs such as "My most active hours", you will have a problem if your user moves between timezones, if you did not store the actual local wall time. [0] For those cases, you can store the offset or the denormalized local time along with the UTC version.
The whole of India -- even though it is pretty vast in area -- has a single time-zone country-wide. I am very thankful for this. We already have a lot of other challenges -- including dozens of different languages -- that a single time zone is a relief.
But yes -- the +5:30 offset usually makes for a less-than-ideal factor to plan meetings or follow non local event schedules.
Did I mention we also hate the Summertime / Daylight Saving time adjustments we have to accommodate twice a year when working with western teams -- more adjustment even after most people have already sacrificed on a sane 9-5 schedule to ensure they overlap with EU/US teams for daily work meetings. We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment. Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
If you live closer to the equator it may not make much sense. For the longest time the population of the US was heavily biased to it's more northern populations than it's more southern. In relation to someone that lives in Delhi someone in Maine will see a 4 hour larger swing in day length. If you're talking farther south in India, like Madurai the size of the swing will be even larger.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
If they did that, you'd still be stuck shifting your schedule when they change their working hours.
That's a good thing, and way better than the current state of things. Let the federal government pick summer/winter hours, and everyone can adjust or not as they see fit.
In the most simplest scenario -- once a recurring meeting is fixed at 10:30 AM eastern STANDARD time, say, -- no schedule change is needed all-year-long if we dont tamper with the clock.
It is just that US worker -- due to their own convenience -- decides to show up in their office at 9AM for six months and 10am for other other six months of the year. Recurring meeting does not need to change at all.
The adjustment , if any, is all on US side. And for local season / climate reasons which has nothing to do with the remote team members, and so the remote team members elsewhere in the world dont need to change anything at all.
Just dont tamper with the clock and things will be much more simple.
>> If they did that, you'd still be stuck shifting your schedule when they change their working hours.
No -- we do not want to link our meetings to the changing office hours. We just want a fixed time on a clock fixed clock. Pick a time that works throughout the year for both teams. Like 10.30 AM EST in my example above.
Current the meetings are "fixed time" but on a moving clock -- in reality the meetings shift around but US folks move their clocks to "simulate" for themselves that they are always having their daily meeting "at 10AM" -- they achieve this illusion by inflicting pain on themselves as well as rest of the world. We would love if they can leave us out of this forced dance :)
Most of my life does not involve coordinating across timezones.
You're suggesting that twice every year, we change the times that retailers, offices, government agencies, broadway theaters, restaurants, roads, schools, and so on open and close. Schools alter class schedules and trains modify departure times. Also, we all reset our alarms, because we'll need to wake up at different times anyway.
It makes so much more sense to just change the clocks!
And yes, all of these things would need to change, because they all revolve around the typical work schedule! Parents bring children to school before heading to work. Public transit runs more frequently during rush hour.
There is a discussion to be had about whether we should be shifting our schedules in the first place. But as long as we're shifting schedules, it makes so much more sense to just change the damn clocks!
P.S. Most people don't have white collar jobs! Construction workers, janitors, mechanics and so on don't ever work across timezones.
P.P.S. I'm okay with nixing the time change as long as we don't end up on permanent daylights savings! However, permanent daylights savings is my nightmare.
Or only the people who it actually affects make any such adjustment?
Currently the shop closes at 8pm and it's dark by 4pm; in Summer it closes at 8pm still (on +1 time) and it's dark at what 11pm or later? Explain your point again?
If school starts at 9AM EST for half the year and at 10AM EST for the other half, then people who drop their children before work will be available at 10:30AM for half the year, and will not for the other half.
If there is a certain time that really does work for everyone, then DST doesn't really affect it. At worse, it requires a bit of rescheduling in the calendar if the meeting was set by someone in a DST country. If the meeting was set by someone in India, it will even get automatically adjusted in everyone else's calendar. So DST itself is not actually making things difficult for you, it's only people's habits changing with winter/summer that makes things difficult.
And note, I have direct experience with all of this, working in Romania with colleagues from California and Kolkata with at least three recurring syncups with all three of us per week.
Yes if the meeting host is in a country that does not observe DST/summertime ... then this problem kind of solves itself. Then only those whose local clock dances around the year need to dance around to match :)
Outlook and calendar apps should probably make this a feature: "do not move this meeting for DST/summertime" (even if the host is in US/EU). Make it opt-in initially and a couple of years later make it opt-out (default) choice.
On a lighter note ... recurring meetings tagged to "dynamic time zones" should be allowed only after hosts "agree" to a disclaimer that they have consulted other participants and have their consent :D (a little like recording with two-party consent)
You seem to put very heavy emphasis on the precise recorded time of a recurring meeting in everyone's calendar. In my experience, this is mostly irrelevant: the time of a recurring meeting is often renegotiated when circumstances change for a big enough swathe of participants, such as schedule changes in the more northern/souther latitudes as winter&summer approach.
"We cant imagine why US that has multiple time-zones already in a single country"
Because "The USA" spans from GMT+10 to GMT-10. Your Solar Noon only varies by an hour between Kolkata and Mumbai. Without time zones, the continental U.S. would see a solar noon difference of three and a half hours. And that's not including Alaska, Hawaii, and Guam, which gives us a 20 hour span. You want all those places to be in the same time zone?
Greenland has a single timezone for a 4-hour span. China has a single timezone for a 5-hour span. Having a single timezone for the contiguous US would definitely not be unheard of.
Guam is a bit of an odd duck because it is on the other side of the date line. The US currently spans from UTC-4 in Puerto Rico to UTC+10 in Guam - which is essentially the same as UTC-14. That's a 10-hour span, not a 20-hour one. Still, excluding the minor overseas colonial possessions would probably make the most sense.
Have you been to China? It's time zone make sense for Beijing and the coast but forces the "Chinese" of Xinjiang to do things at unnatural hours for the convenience of the ruling elite.
It's bizarre to suggest that someone in western Alaska should wake up at the same time as someone in Florida because that's what India does.
The goal is that 12:00 is pretty close to noon, which is reasonably possible.
(And even though lining up a time with morning would be significantly more annoying and need to exclude the polar circles, I still wouldn't call it "impossible".)
You know what? Good on them! We should all just use UTC everywhere and adjust business hours accordingly. Get rid of timezones altogether.
If you want to know when the sun rises, sets, or is at its apex just look it up in a table. The former two vary unless you are very close to the equator anyway, and the latter is off by an hour for every country using "daylight saving time" for half the year. Never mind countries that span more than 1/24th of the globe but use a single timezone, and that isn't just China.
Also I don't really see how this is "for the convenience of the ruling elite". I'd be willing to bet money most people in Xinjiang wouldn't have this "problem" in their top ten. Probably not even top hundred. This seems like something you get used to once and then never think about again unless you travel or have a remote meeting.
Perhaps you haven’t heard of the Uyghur genocide by the CCP in Xinjiang. The local population very much does care about their local time zone (and language and culture, other things the Chinese government has stripped away from them). It may be one of the few places where you’ll get a different answer to the question “what time is it” depending on the ethnicity of the person you ask, and, if asked, you might get arrested for answering with the “wrong” time.
> If you want to know when the sun rises, sets, or is at its apex just look it up in a table.
Yes, a table.
A table with time.
A table that divides the world into zones with regard to time.
That definitely abolishes time zones.
> I'd be willing to bet money most people in Xinjiang wouldn't have this "problem" in their top ten.
Yeah, it probably does rank quite a bit below the genocide.
Anyway, time zones solve an important problem: People coordinate with other people close to them, but occasionally need to coordinate with other people far away. How do those far-away people know when the good times to call are? Clocks only work if you have some idea of what times mean in practice to distant people, which is greatly helped along by people setting their clocks to a local time that's known globally.
> A table that divides the world into zones with regard to time.
Think about it more. How sunrise and sunset change by location and by date.
A chart that covers both sunrise and sunset does not naturally have "zones", and any "zones" you try to infer would not resemble time zones at all. You're either looking at big sweeping ellipses, or you're dividing the world into hundreds of small tiles. It's not time zones.
> Have you been to China? It's time zone make sense for Beijing and the coast but forces the "Chinese" of Xinjiang to do things at unnatural hours for the convenience of the ruling elite.
I've been there and it was absolutely fine - a lot better than working in the US with its mess of timezones. Sure, the times in Xinjiang are "wrong" but who cares? I'll take getting up at "11AM" or whatever over having people in different offices think the meeting is an hour earlier or later and not finding out everyone was mixed up until it happens.
If they do things at different times, then they effectively live in a different timezone, but with the downside that’s much harder to know when it is that they are going to do things. With time zones you know that businesses will work roughly from 9-10am to 5-6pm that people will normally go to bed around 10pm and will wake up around 7-8am. You can just apply the conversion and know all of that. If there are no time zones, how do you convert from one time to the other?
> If they do things at different times, then they effectively live in a different timezone, but with the downside that’s much harder to know when it is that they are going to do things.
It's not hard to know when you do things where you live. Some places people drive on the left, some places they drive on the right, people in different places people speak different languages, people in different places get up and go to bed at different times. When you go to Xinjiang as an outsider sure the customary times are a bit unusual but they're far from the only unusual thing about going there.
> With time zones you know that businesses will work roughly from 9-10am to 5-6pm that people will normally go to bed around 10pm and will wake up around 7-8am.
Not necessarily - plenty of ways to be caught out if you make that kind of assumption about a place you're not familiar with (e.g. turning up at a business when it's siesta). When going somewhere unfamiliar you're already best off looking up when your hotel/restaurant/etc. opens, and it's easy to do these days.
Timezones maybe made sense when physically going to a different place was more common than having a phone/video meeting with someone in a different place. But nowadays being able to agree on the same instant in time when you're in two different places is more important and we should standardise.
> It's not hard to know when you do things where you live. Some places people drive on the left, some places they drive on the right, people in different places people speak different languages, people in different places get up and go to bed at different times. When you go to Xinjiang as an outsider sure the customary times are a bit unusual but they're far from the only unusual thing about going there.
Sure, it can work. It's just worse. It'll be harder to adapt to local time if you move there or go there to visit than just changing your clock to match local time. You'll need to convert constantly. What's the upside, though?
> Not necessarily - plenty of ways to be caught out if you make that kind of assumption about a place you're not familiar with (e.g. turning up at a business when it's siesta). When going somewhere unfamiliar you're already best off looking up when your hotel/restaurant/etc. opens, and it's easy to do these days.
Sure, you could still get things wrong, but you're suggesting going from a situation where it mostly works, to a situation where this never works.
>Timezones maybe made sense when physically going to a different place was more common than having a phone/video meeting with someone in a different place. But nowadays being able to agree on the same instant in time when you're in two different places is more important and we should standardise.
Timezones are the way we found to standardise once global commerce became a thing. Timezones give you essentially time and location, so it's easier to figure things out when multiple parties are involved.
> Sure, it can work. It's just worse. It'll be harder to adapt to local time if you move there or go there to visit than just changing your clock to match local time. You'll need to convert constantly.
No it isn't? You don't convert anything, you just do things at the times the locals do them. It's really not hard.
> What's the upside, though?
No changing clocks, no scheduling a meeting at the wrong time because you mixed up the timezones, no calling your parents and accidentally waking them up because it's the middle of the night for them.
> Timezones are the way we found to standardise once global commerce became a thing.
In the distant past each village had its own time; once the railways emerged and it was practical to go from place to place in the same day, we standardised time across decent-sized regions. Now that we can talk to people instantly around the world, it's time to contine that process and standardise time everywhere.
> No it isn't? You don't convert anything, you just do things at the times the locals do them. It's really not hard.
Before you get used to it, when you see a time you'll have no idea what time of the day it is. Is it early? Is it late? Is it during lunch time? You need to convert in your head "11am here means midnight where I come from, so that's actually really late". Very easy to forget and make a mistake.
> No changing clocks, no scheduling a meeting at the wrong time because you mixed up the timezones, no calling your parents and accidentally waking them up because it's the middle of the night for them.
huh? How is using a single time going to help with waking something up because it's the middle of the night for them? I'd say it's more likely. If someone is 6 hours behind you, you'll need to keep in mind that their 10am means what would be your 2am. Even though you both call it 10am, you would definitely not want to call them at 10am. If anything, that's more error prone and confusing.
> Before you get used to it, when you see a time you'll have no idea what time of the day it is. Is it early? Is it late? Is it during lunch time?
That's not a real problem, IME. If someone invites you for lunch, it's going to be at lunch time. If someone wants to schedule a meeting, you have to check your calendar anyway.
> You need to convert in your head "11am here means midnight where I come from, so that's actually really late".
No, you don't. Converting in your head is the wrong approach just as it is for languages. Just get used to when you're going to bed and getting up. (And don't use AM/PM - why would you ever do that? Even within a single timezone it only causes confusion)
> How is using a single time going to help with waking something up because it's the middle of the night for them?
Because you never have to add or subtract a time, which is where most mistakes happen.
> If someone is 6 hours behind you, you'll need to keep in mind that their 10am means what would be your 2am. Even though you both call it 10am, you would definitely not want to call them at 10am.
Right, so you need to know when they go to sleep and when they get up, and not call them when they're asleep. But there's no arithmetic to get wrong, there's no risk of adding six hours instead of subtracting six hours or vice versa.
> The US currently spans from UTC-4 in Puerto Rico to UTC+10 in Guam
No, it spans from UTC-10 in Hawaii to UTC+10 in Guam. And if we include other US territories in the Pacific, like American Samoa, we have to include UTC-11. That's 21 hours.
> Greenland has a single timezone for a 4-hour span
95+% of Greenland's population lives on its west coast. The most common map projection one is likely to come across makes it look like that spans a pretty large longitude range but it actually only covers about 15°, a 1-hour span.
Take a look at a projection that does a better job of representing latitude, such as this [1] azimuthal equidistant projection and you can see that west coast Greenland is a lot more north/south than you might have expected based on the more usual projections.
Here's a map of the towns in Greenland with >300 population [2] showing how much more populated the west coast is than the rest of the country. If you add their populations and the populations of the towns listed in the table but too small to make the map it comes to about 3100 people.
Brazil's stock market does/used to do something along these lines when US & Brazil changed their clocks in opposite direction, to keep the NY time hours of the Brazilian exchange from drifting as far. Given that we all also observed DST at the time, this made things even more complicated for systems.
Having lived in both India and far North in the USA, I'd take half-hour time zones over daylight savings (switches) every time.
Shifting time-zones increases car accidents by about 6-16% for the entire week (studies vary). Just that is a massive enough impact on safety and productivity to discourage the switch. At a personal level, leaving work at 5.30-ish with it being pitch black outside, is among the most depressing things in the world. Even more wasteful is that the average American wakes up at 7.20am. So, they don't gain any daylight but lose an entire hour.
I'm not usually the type to make such accusations, so I apologize in advance for sounding culture-warry. But, switching-time zones is "white person thing". Large percent of the Chinese, Indian & the Middle Eastern population live above the 25th parallel (Florida) and large populations of Korea & Japan live north of the Bay Area without daylight-switching.
Permanent daylight time with later sunrise would be a huge win.
> We cant imagine why US that has multiple time-zones already in a single country
Are you aware the contiguous United States is three times larger than India? Alaska alone is half the size of India and not at the same longitude as the lower 48. Hawaii is even farther west than most of Alaska. The easternmost point in the US is 66 degrees west. The westernmost is 179 degrees west. That’s nearly a third of the globe and I'm not even including territories.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
Is this a serious comment? Europe doesn’t have a central government. The most recent attempt to establish one was extremely unpopular and famously stopped in 1945.
Why should foreign schools and businesses change their hours to accommodate you? A more obvious solution would be to just stop observing DST but that’s not something that will happen unilaterally and will be an especially hard sell at extreme latitudes.
If you want to do business internationally you have to deal with timezones. India isn’t unique in that inconvenience.
People at extreme latitudes should not care much, since for them DST does close to nothing.
The parent's proposal is actually quite sound. It would be a lot more useful to have institutions switch their starting times for part of the year when it makes sense for them, rather than forcing a one-size-fits-all solution on everyone. Countries at different latitudes would be free to switch e.g. school start times at different dates depending on when sunlight shifts too far in one direction, while keeping their clocks synchronised with astronomical time. Heck, Spain basically does that already, since they're on the wrong timezone (geography dictates they should be on British time, but because of events in the '40s, they're on Central European time) - so they do everything one hour "later".
> [...] EU whose countries are often smaller than states in India ... need so much adjustment. Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
This proposal is literally one-size-fits-all.
> It would be a lot more useful to have institutions switch their starting times for part of the year when it makes sense for them, rather than forcing a one-size-fits-all solution on everyone.
This is exactly how it works now. But there are a lot of institutions and they aren't coordinated.
I was not arguing for a single timezone nor surprised that US has multiple timezones. the surprise was that even after multiple timezones, they need so much adjustment twice a year -- which is what I was arguing against. (to be more specific the argument was against how they go about that adjustment -- not that an adjustment is needed which is understandably due to geography and seasons)
you quoted half my sentence which creates an easy strawman.
> A more obvious solution would be to just stop observing DST
yes that is essentially my argument too ...with an extra accommodation for folks who will argue about seasons causing wild changes in sunrise / sunset / daylight.
Because of the comma and ellipsis "need so much adjustment" very much looks like it only applies to "...and EU whose countries are often smaller than states in India."
> A more obvious solution would be to just stop observing DST but that’s not something that will happen unilaterally and will be an especially hard sell at extreme latitudes.
Huh, why would it not happen unilaterally? Different countries already fiddle with their daylight savings times (or absence thereof) unilaterally, and the start of DST is not synchronised across the globe anyway.
So if you have two countries that start DST on different weeks that's already a hassle to deal with, and one of them going off DST altogether doesn't really add any extra hassle in their bilateral dealings.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
For what it's worth, I've (British) never understood that either. Need more daylight hours for farmers to work or whatever, sure, no problem. But.. that's easily resolved as you described? And before widespread centralised time and access to it, surely they just started at sunrise or soon after, whenever the sun happened to rise. I don't get it.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
Or, if you extrapolate this argument further, you could even wonder why the entire world can't run on a single timezone. Like, it can be UTC everywhere.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
Or, if you extrapolate this argument further, you could even wonder why the entire world can't run on a single timezone.
But here's the problem: for example, imagine it's UTC everywhere on Earth. No local timezones.
As per your solution, people in India can go to sleep at 4:30 pm (UTC), wake up at 12:30 am, have lunch at 7:30 am etc.
Likewise, people in San Francisco can go to bed at 6 am, wake up at 2 pm, have lunch at 9 pm etc.
Now, the problem is, when someone wants to setup an international meeting or whatever, nobody would have a clear idea or whether it would be "sleeping time" for any of the participants involved, without having to look it up.
Like, with the current setup, it's kind of universally understood that 3 am local time is sleeping time, hence people would avoid setting up meetings at that time.
But if everywhere on Earth it's the same time such as UTC, I would have no idea if 3 am UTC is your sleeping time or lunch time or whatever. I would have to maintain lookup tables of when the sleep time, lunch time etc would be for every country as per UTC.
Similar problems exist (to a lesser degree) when large countries move to a single timezone.
Yes there are flaws in the existing setup, but moving to a single timezone for large countries would only introduce a new set of challenges.
My comment is being misread as an argument for a single timezone for all of USA.
Kindly re-read.
I was arguing against daylight savings adjustments twice an year. Not against having multiple timezones per country / region. My comment was simply that timezones should be sufficient without the need for DST.
What I said ...
>> We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment.
Simplified ...
>> cant imagine why US, and EU need so much adjustment.
It definitely will break financial firms. I just did a code search in our repos for utcnow and other relevant functions, and the indexer was definitely wheezing pulling in the hits.
Deprecation should not break anything. At some point the code will be removed and the org will have to pin the version until the deprecations are resolved.
> I suspect a lot of financial data code is going to break.
Reminds me of when Numpy removed a lot of types like np.bool a few releases ago, because they were aliases of the standard library types.
Which broke a ton of libraries having Numpy versions pinned to ">=1.0" or even worse, "~=1.0" expecting them to follow the standard practice of incrementing the major version when they break compatibility. It's not terrible because you just pin your project to 1.23 or whatever, but it was very annoying when everything I was using just started breaking.
I wonder if Python plans to do the same with this - the 2 to 3 transition was painful and dragged on for a decade, nobody wants to do that again for 3 to 4. When they remove this function and break back compatibility with old code, is that going to happen in a future 3.x release?
It's worth noting that numpy and scipy have a consistent policy now, although it's not semver. Deprecations will always happen at least 2 minor versions before removal:
So, when using numpy in a package, and you've tested e.g. versions 1.12 to 1.22, you should require 'numpy>=1.12,<1.25', because something deprecated in version 1.23 could be removed in 1.25.
Yes, they're almost certainly going to do it a 3.x release.
I think in some senses these backward compatibility breaks have already happened. I remember that Google got bit by async becoming a keyword in I think Tensorflow.
Generally a lot of old Python 3 code still works today if it doesn't do anything really weird like that, but I do definitely think that the pain of Python 2 to 3 was painful so they don't really want to repeat it.
I think it is probably prudent to treat point releases as major releases at least with regards to Python.
Removing utcnow() doesn't force you to use tz-aware dates though. That's just a leap that the author of this article has made.
If you're working in a system where all dates are utc, then datetime.now() gets you a date in utc and you don't need datetime.utcnow(). The only reason to call utcnow is if you're working on a system where the local timezone isn't utc, so you inherently have to deal with timezones at some level. Which means that using naive timestamps is a bad idea.
datetime.now(tz=timezone.utc) is what you probably want most of the time anyway. You can always manually strip off the time zone component if you really want to use datetimes without associated time zones.
Then accept that UTC is a timezone and just use that everywhere. There are also a lot of things that can’t be easily stored in UTC, like opening hours.
This is only sometimes true. Cases like opening hours and daily batch processing rely on local time. Persisting the values in UTC mean best case you are wrong half of the year and worst case all of it.
6am in my time zone is two different times in UTC. Right now it is 14:00 UTC. But when DST starts it will be 13:00. So what UTC value do you insert into your database for the opening time of the coffee shop? And how do you convert?
Yes, you can store additional information to decipher the meaning of a UTC time but why? Storing locale is good because sometimes you need to know where something is, such as when converting the local time to another zone. But you should avoid creating dependencies between these values because it makes using them individually harder.
The coffee shop opens at 6am. That is an unambiguous fact. The coffee shop is in Seattle, WA. That is another unambiguous fact.
If the time is in UTC then you need to know the date the time was set or store a dtc flag and then build some logic to convert. You can just feed local time+locale into any datetime library and get whatever you need.
UTC is great for a lot of things. Such as recording when something happened. But local time has a place as well.
A store’s opening hours will probably remain 9am-5pm regardless of any time zone changes around it. If you run a bunch of stores across time zones you need to know where each store is and whether it’s summer or winter to work backwards to find out 9-5 in UTC.
The coffee shop in my neighborhood is open from 6am to 2pm Pacific. We observe daylight saving time. The shop hours do not change. So, what UTC value do you store?
Yeah, the stock market opens at what time EST? Including daylight saving time, etc. I agree that any time time goes to a serialized format over a socket or to a database or file it should always be UTC.
Ignoring holidays/weekends: always 9:30AM or (0930) EST. Same with EDT: 9:30AM.
The key here is that the timezone info includes the daylight/standard designation. Or, in the ISO format -4:00 or -5:00 from UTC.
I think you meant to put ET (Eastern Time), which is still 9:30AM, but without a date associated with that time, there's no way to convert that to UTC or other timezones that don't have the same daylight saving time schedule.
UTC (or Unix timestamp, or any other non-localized date-time format) doesn't work for future localized events. It's fundamentally impossible to say what will be the UTC timestamp which corresponds to 9AM in some particular location in the future: the timezone of that location could arbitrarily change between now and then. Countries and smaller areas often experiment with such changes, wars happen and conquerors/liberators change the time etc.
So, when possible, it's best to store (future) human events in a localized datetime string. For things like physical events, the opposite is true of course - you can't say what will be the local date and time when the Sun will hit high noon over Athens 10 years from now, but you can certainly calculate the UTC timestamp for that.
Something like “9AM-5PM EST” can’t be stored in a datetime object even with a timezone slot! Datetimes with a timezone represent specific points in time, not vague “5PM” like concepts.
I feel like every time a discussion on timezones comes up, people compete to come up with different situations where "see, your perfect system can't work, we should give up on timezones entirely".
Here, I'm not even sure what your point is. A datetime object cannot capture a range, no, just like a number can't capture a range of values. But through the magic of having two of them, we can get easily create a data structure containing two datetimes to represent a range. 5pm is not a vague concept, it is the time 5pm, the hour that typically comes after 4pm and before 6pm. If your point is that you can't store only a time in a datetime object, that's true, that's why the standard library also provides the `time` object which represents a time.
Where things might get a bit more complex is if you want to store the time of a recurring event that should occur at the same time on multiple days for a given timezone. In this situation, you can typically use a naive time object to represent a wall clock time, plus the timezone that the user has requested. This way you still have all the raw information to decide when the event will happen in the future. (Note however, that the same time can occur multiple times on the same day, for example during daylight savings changes.)
"every day, from 9AM to 5PM EST" is a thing that can be represented through calendar RRULEs. Usually libraries exist to help with this (Python has one, conveniently called rrule!). This is a thing to describe recurring events, has a spec and everything. So there is a thing.
And you're totally right about `time` existing, it slipped my mind that time has a tzinfo slot. I do think you should still take care and I would avoid having any naive time objects just floating around beyond the serialization layer though.
If “everything is UTC, all the time” is enforced at the entry edges of the code, couldn’t the entry edges be refactored to always specify the UTC timezone?
Financial data deals with cutoff times and those happen in a timezone. Thus particularly in finance timezones are important. You are right though that internally everything should be UTC internally and only when required it gets transposed to the timezone. However many legacy financial systems assume a local time zone and not UTC and they do not expose that information in their data. Happy reconciliation :-).
One edge case to this is I've worked on financial systems where we kept the Asian market systems in JST, while EU/MENA/Americas ran on UTC. As long as you aren't dealing with FX/Futures, markets are generally open no earlier than 7am and no later than 6pm. This worked nicely since it meant that the DATE in each regions system&local clocks agreed.
If the market opens at 9am New York time then that’s not the same as storing 1400 utc. An law could be passed to change the offset on a given day
You have to store time in the relevant geographic time zone. Something occurring at 1200UTC on dec 1st is not the same as something occurring at 1200Europe/London on dec 1st. The two times may coincide today, but if the U.K. government decides to implement daylight saving next week as a fuel rationing measure, then your 1200London event will have to be displayed at 1300UTC, but your 1200UTC event would be at 1200UTC.
1200 London time, so 1100UTC with daylight saving yes.
Point remains, many events are linked to local civil time, which can change - and not just to daylight saving. The Line Islands in Kiribati shifted a whole day in 1994, same with Samoa in 2011. Have an event set for 0800 local on the first Monday of the year in Samoa which you stored as UTC, and your events would have broken. Unless you store the relevant data (it occurs on first monday according to the timezone that Samoa is in), then you will be wrong.
Now of course you can still get problems if a new timezone is created, as time is stored as Europe/London, if Scotland creates its own timezone in the future a meeting in Aberdeen would no longer be in Europe/London but in a new area Europe/Edinburgh, with different behaviour, but that’s far less common than DST changes or other date shifts.
All in all you should store the relevant datetime and zone, not just “normalise” it to UTC.
Using UTC as a universal time standard is stupid because it contains leap-seconds while TAI does not. TAI64* should be the preferred computational monotonic time format.
> I would have made it more clear that these functions work with naive time, maybe by calling them naive_utcnow() and naive_utcfromtimestamp().
> But their names are not the problem here. The specific issue is that some Python date and time functions accept naive timestamps and assume that they represent local time, according to the timezone that is configured on the computer running the code.
Uhhh, no I would not expect local time because as the name entails we're dealing in UTC. Sigh.....
You might not, but the standard library sure does. And that's the exactly the problem! How can functions in the standard library know if the time is local or UTC without any indication on the value itself? (spoiler: they can't!)
As the article author correctly points out, Python should have just thrown when a method is called that requires a TZ with a datetime that doesn’t include one.
Between this, the Unicode circus, and the rejection of PEP582, Python seems more and more like the HOA that isn’t earning their monthly fees IMO.
I don't understand why the functions weren't just made to return non-naive results. You've literally asked for UTC. There's no excuse for allowing the result of that to be interpreted as something that isn't UTC.
Set a separate flag that governs conversion to string to avoid different ISO output for shitty string parsers if you want, but there's no great reason to not just fix the functions so that they continue to work better than before and do the thing that people actually expect them to do, which is produce UTC.
Intentionally naive storage or transfer is irrelevant. That's just serialization. And when you explicitly tell the computer to "deserialize this as UTC" it should fucking do what you told it to and give a zone aware result.
This would be a backwards incompatible change that would cause a lot of issues. For instance, you are not allowed to compare naive and non-naive datetimes, so for instance, `utcnow() > datetime(2023, 11, 19)` would work before, but break following your change.
It is one that can be found via static analysis. That may not be true if behavior is changed. In some cases, this may be "ok" (see: Go's recent loop changes), but in others, it may not.
Yes, but "Unknown function datetime.utcnow" is much easier to diagnose than somewhere in your application, possibly inside a third-party library or maybe even in a different service altogether (because the change traversed serialization and deserilialization) throwing an exception because you can't compare two datetimes.
If this was something used only by your company or a specific project, then I’m absolutely on board. However, this is in a library used in thousands (millions?) of projects. Since it’s potentially a breaking change you can’t just change it and expect everyone to know.
Many people won’t know it’s deprecated until the code fails to run, and having new functions rather than changing the functionality of existing functions makes it much easier to identify what is actually happening.
> I don't understand why the functions weren't just made to return non-naive results. You've literally asked for UTC.
I totally agree with you that it makes no sense this `utcnow()` does not return tz-aware object.
But I think their fix is great in this regard: now you just use `datetime.now(timezone.utc)`. It's more explicit, it returns aware DT object now, and more importantly, it gets rid of a function that I always feel too specific to have anyway (like how we don't have `plus(a, 2)` and `plustwo(a)`).
Except isn't datetime.now() still going to return a tz-naive datetime? I would much rather have a method that requires an explicit tz, or explicitly uses UTC (like utcnow, but with the right return type).
Looking for this mistake in code reviews gets really old.
I got bit by this once. Timezones aware date calculations can be an order of magnitude slower than non tz aware. Consider every time a legislative body changes dst, it creates an entry in the timezone database that needs to be checked when adding or subtracting time from that timezone.
This is one of the reasons why Unix epoch time soldiers on, even though it is totally indecipherable to humans. It can be easily mapped to a timezone-aware type, but performing arithmetic on it is trivial.
After the change that you propose, the timestamps written by this code will jump by an amount equal to the offset of the local timezone this machine runs on (and also lose any DST adjustment they might have had).
Now, let's imagine another independent process is consuming events from the database and compares an event written before this change to one written after.
Python is a popular language, used in production by scheduling applications around the world. The potential downside of this kind of change is measured in billions of dollars. The safe solution is of course to not silently change the API in a breaking way, and instead introduce a new API and get someone to look at why the code stopped working.
On my machine, a naive timestamp from utcnow() and a utc timezone aware datetime object from datetime.now(tz=ZoneInfo('utc')) produces the same timestamp.
datetime.now() does not return utc time (but does return a naive datetime object)
I'm not a Python guy so am not familiar with the culture, but I'd assume many, many programs use the deprecated function, given that it made "the news." It seems not right for backward compatibility to deprecate something many developers are using, simply because the old one wasn't thought out very well, and "developers should switch" to the explicit functions. Especially since, as the author demonstrates, the deprecated one can be easily re-implemented. It seems like a developer inconvenience was added without changing the state of the world. Things like this--I don't know--make me take a development environment less seriously.
In C, we all know we shouldn't use things like strcpy() and gets(), and you'd be a total fool to add them to new code. But for the sake of backward compatibility with programs that aren't going to change, they should be kept in and documentation should nudge people to do it the right way. Maybe that's the Python way of nudging: "This function is deprecated!"
Different language ecosystems have different standards for backwards compatibility, with C (and e.g. Java) being incredibly conservative by modern standards. This is not a good or a bad thing per se, it's just something to consider when picking a tool for the task at hand.
Fwiw, gets() was actually removed in C11 because of it's inherit security problems. :)
But aside from that, I think it's just a different style. Waiting 10 years from deprecation to remove a deprecated thing might be required with C just because of how prevalent C is and how much the developers in that have to think about basically every platform, but removing something on a major release isn't inherently bad, it's just a different tradeoff (being able to move a bit faster and fix some design issues vs. being very very good at backwards compat)
The deprecated function can obviously be reimplemented pretty easily, but so can gets(). With utcnow(), it had some unfortunate effects in that it didn't really do what people thought it did, and because of that, most of the uses were basically bugs to begin with.
(Having experience with .NET for example, you'd think that DateTimeOffset.UtcNow and datetime.utcnow() are semantically similar, but the problem is that utcnow() doesn't actually have a timezone attached to it. Adding one would also be a breaking change, and one that is more subtle.)
> Fwiw, gets() was actually removed in C11 because of it's inherit security problems. :)
It always exists, it's just that the import gets controlled by a macro. Which, hints at a way Python could have solved this same problem without requiring auditing and rewriting code.
One advantage of getting older is that you see a lot of change. In the long run, simple ideas tend to win. And there have been lots of cases where the programming world collectively agreed that some more complex idea was the right way, then a decade later everyone has reverted to the simple idea.
UTC is simple and unambiguous. Time is a number representing a fixed point in time. When displaying it for humans, convert it into human form. When this principle is applied everywhere, it’s hard to make mistakes.
My gut says that deprecating utc as the native time representation will be one of those ideas fervently embraced by a few, ignored by most, and then quietly forgotten about in a decade.
It doesn't seem like they're deprecating "utc as the native time representation." They're just making it harder to get a datetime object that "has no time zone." The message with this change is: Go ahead and keep using UTC, but explicitly specify it, and the object itself will know it's own UTC-ness.
They’re not deprecating UTC. Did you read the link? They’re deprecating functions that read as authoritative, but are actually naive about timezones, which can less to non-UTC results.
You're free to use numbers instead of the datetime module. In fact you don't have to use any part of the standard library if you don't want to. I don't understand your remarks at all.
If you think UTC means that "time is a number" you are very confused though. UTC has no relation with the UNIX timestamps (and the epoch), though they are both ways to represent a specific point in time.
I think the recommendation is to use a datetime with the timezone explicitly set to utc? I think the numbers vs. objects thing seems like a distraction...
You're focusing on representation where you should be thinking about semantics.
A "naive" datetime is the one for which it is not known what the timezone is. The opposite is the one for which it is known. You can absolutely encode UTC timezones as seconds since epoch without any additional information (except that one bit that is needed to distinguish them from "naive" ones, which doesn't have to be a literal bit - it can be just a different data type, for example).
And, yes, it would sure be nice if we didn't have "naive" datetimes at all, and everything was always UTC. But there are too many datetimes recorded in the real world without enough information to recover the timezone and thus the precise UTC time, and they still need to be processed. The mistake was to make this the default behavior - but that is a very old one, dating back to earliest days of computing.
Says you. The python documentation has never said that naive datetimes are UTC. If the language suddenly makes this assumption, then you break existing code. If you don't want to break existing code, your only option is to 1.) deprecate the existing function and 2.) provide an alternative (which is provide an explicit datetime).
No, it is not. It is a datetime for which it is not known whether it is UTC or not. Assuming that it is is only marginally less bad than assuming local time.
>> A "naive" datetime is the one for which it is not known what the timezone is.
> Then it is UTC
No, naiive datetime is: "now" ie a time description that lacks enough information to be able to fix that moment relative to other datetimes. It lacks TZ information.
I happen to physically live at roughly 2.6° west of the Greenwich meridian oh and at 50.9 north of the equator. It is now 23:08 (to one minute precision). I have given you enough information to derive my active time zone, if you consult the TZDB, and hence to convert my given naiive datetime to one that will work for you. If I hadn't included both a northing and an easting (I actually gave you a "westing" but that is simply a negative easting!), you could go very wrong. For example France, Spain etc are directly south of here and they are an hour "ahead" of the UK, right now.
Right now it is: "23:15 UTC+0" I could use GMT+0 instead and that would work too but let's keep it French! That's fair - the Greenwich meridian was picked over the Parisian one (and several others) for 0° east/west for time zone calcs but let's call it UTC instead of CUT which is what the english translation would end up being. I actually prefer UTC - CUT is also a word.
Now, I have not included the date bit of datetime in my examples above. There are multiple naiive interpretations for datetime. Calling them all "UTC" is simply not going to cut it 8)
This misses what the change actually is. Python has two datetimes, one which includes the timezone info and one which doesn't (naive).
There is no assumed timezone for naive datetimes and different software treats it differently. What's being deprecated is methods to create naive datetimes that are implicitly UTC without just attaching that information to the object itself. In the new world everything is still in UTC and you convert to local time at the last minute but now you can be sure that the datetime object you got handed is actually UTC instead of it being a convention that isn't always honored.
Time is two different things. Just making everything UTC until display time is incompatible with that. Sometimes you need to operate on calendar units or clock units as part of business logic.
Scenario 1: You have an event to schedule at 9am local at some future date. You want to convert to UTC. But you can only do that if you know the offset, and the offset for future dates can change or be unknown. How do you resolve this?
Scenario 2: Now the event is "9am every day, indefinitely". How do you represent that as UTC? Do you generate tens of thousands of separate timestamps?
All of this is fine if you are talking about timestamps. But Python's datetime library can do more than that. Basically once you get into future dates that it all starts to fall apart.
Does Go also has a distinction between naive/timezone-aware times? If not don't see why would expect something else. This is an issue because the distinction exists. Note it isn't bad that it does. There're actually arguments in favor of it. It's bad that can freely mix them.
It does, which is great. Note however that Go times can be either wall-time or (wall-time, monotonic time), which have different semantics and are both represented by the same type.
This was an aside and not the point of the article but I can't let it go:
> It is also a best practice to store naive datetimes representing UTC in databases.
It is most definitely NOT a best practice and you SHOULD NOT DO THIS.
If your database supports TIMESTAMP WITH TIME ZONE -- which the "aware" type per the article's terminology -- you should be using it. If your database doesn't, you should get a better one.
Would you care to elaborate? I'm not an expert in this area so I'm probably missing something, but my thought has always been the 'portability' of storing everything in the backend as UTC is more flexible/less complex in the long run, dealing with conversation at the 'last mile' so to speak.
Happy to be corrected here but it's not intuitive to me that dealing with timezones as the 'base unit' makes anything easier.
It's the usual divide between people who actually know how databases work, and hence want to do more stuff there (because it will likely be faster and more robust); and people who don't know (and don't want to know) anything about databases, and hence would rather use them as dumb stores of basic data.
I've been in each camp at one time or another, since being db-agnostic makes sense in some cases (libraries, products) and not in others (websites, services).
It's not an either-or. You can store everything in UTC while also using TIMESTAMP WITH TIME ZONE. The timezone will always be UTC, but that way at least anyone who will be dealing with this data later knows that to be the case, instead of assuming.
Converting from a local date/time with an associated timezone to an UTC time is not a lossless nor unchanging operation. By converting everything to UTC, you're throwing away information that you might need to do the conversion at the 'last mile' later on.
For example, imagine I schedule a dentist appointment for 9AM in my local timezone sometime next summer, and put it in my calendar. Now, if the government decides that we won't do daylight saving times next year, my dentist and I still expect that appointment to happen at 9AM local time. However, if my calendar application had converted it to UTC using the timezone definitions at the moment I created the appointment, it would show in my calendar at 8AM, and I'd be an hour early!
This is not an as hypothetical situation as it seems. To give just two recent examples: the Chilean government decided to postpone the start of DST with less than a month's notice last year, and Morocco shifts the clock an hour during Ramadan, the date of which only becomes certain on the day itself.
Can you explain why? I write web apps, and I typically want to know an instant in time when something occurred and then localize the time for display. Why would I need to store a time zone for that?
Storing UTC for past events is totally fine because it is unambiguous.
However, if you are writing something like a calendar app and storing timestamps for future events, you run into the issue that time zones have the annoying habit of changing.
Let's say you have a user in Berlin, and they want to schedule an event for 2025-06-01 08:00 local time. According to the current timezone definition Berlin will be on CEST at that moment, which is UTC+2, so you store it as 06:00 UTC. Then Europe abandons summer time, and on 2025-06-01 Berlin will now be on CET, or UTC+1. Your app server obviously gets this new definition via its regular OS upgrades.
Your user opens the app, 06:00 UTC gets converted back to local time, and is displayed as 07:00 local time. This is obviously wrong, as they entered it as 08:00 local time!
Storing it as TIMESTAMP WITH TIME ZONE miiiight save you, but if and only if the database stores the timezone as "Europe/Berlin". In practice there's a decent chance it is going to save it as a UTC offset anyways...
If you’re storing time zone info for a transaction in the database instead of storing it as epoch time, you’re probably doing it wrong. Time zone is a user-level presentation layer issue. Keep that junk out of the backend. Imagine writing code to reconcile financial transactions that also had to deal with time zone offsets to figure out if one transaction happened before another. So much extra complexity for no reason. I don’t know where you’ve been doing this or where you got this idea, but it’s most definitely not common or advisable to store TZ info on transactions anywhere I’ve worked in my couple of decades at some very large software companies. TZ info is usually stored with a user account as a preference that they can update, or it’s detected from their browser during presentation time, because that’s when it matters — when you’re showing it to a human. Having machines calculate TZ offsets between database records is asinine, inefficient, and a great way to introduce an entire domain of bugs for absolutely no reason.
> If your database supports TIMESTAMP WITH TIME ZONE -- which the "aware" type per the article's terminology -- you should be using it.
NO!!! IT IS NOT THE SAME AT ALL!
I've been burned by that exact misconception. TIMESTAMP WITH TIME ZONE does NOT store a timezone in the database!
It implicitly converts a timestamp you supply into UTC and DISCARDS the associated timezone information you provided. When you read it back, it converts it to LOCAL time unless you explicitly specify a different conversion. This is a recurring source of confusion and bugs. The timezone a timestamp was actually written in is sometimes VERY important -- and I've personally had to deal with data loss on a project caused by this misconception.
Let me repeat: There is NO WAY to read back what timezone a "TIMESTAMP WITH TIME ZONE" was written with later -- unlike an "aware" datetime in Python -- BECAUSE IT DOES NOT ACTUALLY STORE THE TIME ZONE YOU SUPPLIED! This is a data type that DESTROYS INFORMATION YOU PROVIDED TO THE DATABASE assuming it can be clever later.
I fucking hate that data type SO much.
An "aware" datetime in Python actually tracks a timestamp with a specific timezone explicitly attached to it. It is VERY different!
Documentation for Postgres:
> For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's TimeZone parameter, and is converted to UTC using the offset for the timezone zone.
> When a timestamp with time zone value is output, it is always converted from UTC to the current timezone zone, and displayed as local time in that zone. To see the time in another time zone, either change timezone or use the AT TIME ZONE construct (see Section 9.9.4).
> Date and time objects may be categorized as “aware” or “naive” depending on whether or not they include timezone information.
> With sufficient knowledge of applicable algorithmic and political time adjustments, such as time zone and daylight saving time information, an aware object can locate itself relative to other aware objects. An aware object represents a specific moment in time that is not open to interpretation.
[...]
> For applications requiring aware objects, datetime and time objects have an optional time zone information attribute, tzinfo, that can be set to an instance of a subclass of the abstract tzinfo class. These tzinfo objects capture information about the offset from UTC time, the time zone name, and whether daylight saving time is in effect.
I agree (though less indignantly). TIMESTAMP WITH TIME ZONE basically makes me confused as it is trying to run around guessing how to actively solve a problem I dont actually have, which is, "I don't know how to correctly represent datetimes in my application".
what people usually want, when they are confused about that datatype, is a datatype that stores the timestamp *and* a specific timezone to go along with it, so you get back a timestamp with that same timezone if you did not ask to convert it to another. that would be a bit less efficient from a storage perspective but would likely be more intuitive.
TIMESTAMP WITH TIME ZONE stores the timestamp naively. The type defines a convention for how data is transmitted to and from the datatype.
which is basically exactly how one is supposed to use timezone-naive datetimes. the datetimes are individually "naive" however are meant to always be used within a context that is implicitly non-naive across the full set of datetime objects.
I think naive datetime would be fine, even if it resembles UTC - as long as you don't mix in a serial representation. The module documentation says, "Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, [...]". That definition goes down the drain imho once you call `.timestamp()`, which gives you Unix time as a serial representation (epoch time), which must be referenced against aware datetime, namely 1970-01-01 00:00 UTC. Same trouble vice versa (`.fromtimestamp()`).
Numpy and pandas are their own little island. It's not just dates and time, it's everything.
If you use numpy and pandas, you should also not use Python datetime, generators, most stdlib mathematical functions, the itertools module, random, etc.
It's the first thing you learn if you read any good pandas book, and the first thing I teach in my numpy/pandas trainings.
It has pretty much nothing to do with pendulum.
Basically, half the Python ecosystem is "well, except with numpy/pandas of course".
You know what would help people adopt timezone aware timestamps? Just make now and utcnow return timezone aware timestamps! What's the point creating a naive timestamp when you know the time?
I'm the one arguing against this move, I've already made my thoughts clear that it's a lot of churn, e.g. the PR to remove it from Pip literally caused Pip to have to do a bug release, and has valid use cases, e.g. modelling datetime data types from spreadsheets and databases that do not have timezones.
But I also accept I'm not the one maintaining Python, so I've updated my codebase appropriately (`datetime.datetime.now(datetime.UTC).replace(tzinfo=None)`). Python is not a backwards compatability language, and it's clearly not a strong motivation for most users as I don't see anyone attempting to write a backwards compatibility shim over the standard library, so no point complaining too much.
This is verbose because it is dissuaded. The whole point of this change is to not have users get naive datetime objects, of course it gets more verbose.
`datetime.timezone.utc` is the original way of doing it and should work just fine on a stock Python 3.11 install. `datetime.UTC` is a relatively recent addition, and if you look at the actual type of the object it returns, it's still `datetime.timezone.utc`
Isn't Python adding gradual typing? This is the sort of confusion that becomes apparent when we know and write the types (TZ aware or not) coming out of into functions.
This seems like the blatantly obvious path to me. Perhaps that needs to be dealt with in a major version release tho. TZ naive and TZ datetimes are needed and it’s important to chose the correct one.
I’ve definitely been burned by assuming that “no time zone” means UTC when my python environment assumed I wanted it to mean Pacific time - it’s like the APIs all prefer naive datetimes, and I had to learn a new trick for each one to convince it to do the right thing
> To me it is clear that the Python maintainers behind this deprecation have a problem with naive datetimes and are using this supposed problem as an excuse to cripple them.
I am not a python maintainer, and I also use naive datetimes a lot, and I can absolutely understand the maintainers rationale behind this.
Let me explain.
What's a "naive datetime" suposed to be conceptually? There is no such thing. Datetimes only make sense in the context of a timezone. UTC is just another timezone, with an UTC-Offset of +0
So why do we have this strange thing that may be a UTC timestamp, or a local timestamp, or god knows what in the first place? It makes sense as an implementation detail of the datetime library to have timezones as separate objects, but for consumers of that library? It can cause confusion, and hard to track bugs, when suddenly one of the functions that accept aware or naive is used and spits out something completely unexpected.
I have spent many an hour tracking bugs, where systems spat out complete nonsense timestamps, only to discover that the reason they do that, is a mixed usage of aware and naive dateteime objects.
In my opinion, making explicit use of aware datetime the norm as much as possible, is a good thing. And removing prominently named functions that spit out naive ones, is a good first step.
Let's look at the listed usecases:
> An application may be designed in such a way that all dates and times are in a single timezone that is known in advance.
Okay...so put that TZ info somewhere it's easy to find, like a configuration Singleton, and use it.
> It is also a best practice to store naive datetimes representing UTC in databases.
Okay...read them from, and into datetime objects using UTC as their timezone information.
The bottom line is: There shouldn't be "naive" datetime objects, or at least they shouldn't be a common thing. Conceptually, datetimes always have a timezone attached, they are worthless without one.
I certainly didn't miss that, as I pointed out myself: I use naive datetimes in my codebases. Alot of them. And this change generates quite a bit of work for me.
But I'm okay with that.
The problem with de facto standards is: The fact that something became de facto standard, doesn't make it a better idea. The only thing it means is that it is now even harder to get rid of it.
> It's basically very late to be doing this.
Yes, it is. But not yet impossible. Languages and libraries that wait forever instead of ripping off the bandaid at some point, eventually become so crusted with such early problems, they become problems themselves.
The naive time thing in python really is (was) a nightmare. I once noticed that a db library we use when storing a timestamp in the db would calculate that timestamp from 1970-01-01 00:00:00 at _local time_ instead of utc and had to patch that to make sure we never store a wrong timestamp in our db (luckily all our production servers run on UTC as the local timezone).
Glad that it's finally being addressed at the language level.
I’ve been using Python for going on 14 years now, and I had no idea utcnow() returned a naive date time. I happen to live and deploy to a (mostly) UTC time zone, so i expect a bunch of my code is subtly wrong.
One of my projects added all of the code in PyPI to GitHub, it would be interesting to see how prevalent the use of “utcnow()” is.
> An application may be designed in such a way that all dates and times are in a single timezone that is known in advance. In this case there is no need for individual datetime instances to carry their own timezones, since this uses more memory and processing power for no benefit, since all these timezones would be the same and it would never be necessary to perform timezone math or conversions.
The memory and processing differences between the two seem marginal, at best. Especially when it's at UTC ("plus zero" doesn't take too much processing).
So I'm gonna have to ask for benchmarks on this one.
And even if it was slower one must ask if having a confusing concept like a naïve datetime is worth it. Obviously Python is all about the "programmer productivity at the expense of some performance" trade-off.
Am I correct that the datetime.datetime.now(datetime.timezone.utc) syntax is backwards-compatible all the way to Python 3.2?[1] If so, this seems like a good move. The edge cases like compatibility with data sources that don't indicate the time zone seem like less of an issue than data corruption due to developers not realizing that they were working with "UTC" timestamps that weren't actually UTC.
I'd be more concerned if this was one of those changes where there isn't a good way to write reasonably straightforward code that will run in relatively old versions of Python as well as newer ones.
481 comments
[ 3.4 ms ] story [ 268 ms ] threadThis is 10x better and helps to assure the code can be unit tested with providing the exact date we want in the exact format/timezone
[1] https://blog.ganssle.io/articles/2019/11/utcnow.html
Since our time is at least all Earth-based, we could do even better than timezone and make timestamps have a lat/long attached for the place in which it was produced. Maybe a linked list for any timestamps that are calculated from that based type.
Are we having fun yet?
If we have a call at 9:30 and you drive to the next state now that call is at 10:30 or 8:30 for you.
If you set 9:30 as an alarm to feed your cat every day, fine.
I cannot just guess what timezone these data points were collected, so I have to suffice for local time. Which is 100% fine for the users, because that’s exactly how they think about the problem. There are probably no global events against which they need to sync, just interested in the relative differences between events.
1. I've set my alarm for 8am local time every day. I don't want my alarm clock to go off at 3am just because I'm in a different country
2. If I have an appointment at 10am next November, that's actually a datetime with a locale (given that my appointment is at 10am local time, no matter whether the DST rules change between now and then).
Most environments simply can't represent that thing. They never could. With those changes this becomes more obvious, and with some luck people will finally fix this.
(The most absurd case is the "timestamp with local time zone" from SQL, that reads exactly like what you want, but represents a timestamp in a specific time zone, that is discoverable but mostly unknown.)
"8AM" is exactly time without a timezone. "8AM at local time (local defined by this procedure)" isn't exactly that.
Perhaps you mean something like homomorphic. It is almost perfectly homomorphic. But if you define your types that way, you'll push yourself into the most useless ones you can get quite quickly.
2. That’s why you WANT time zones in this example. It will automatically stay at 10am instead of bouncing around when DST rules change.
I’ve never ever needed timezone-less dates. Even using UTC or timestamps, it’s still UTC.
A date without a time zone is like “10 inches” but without the inches so it’s just “10”. Absolutely meaningless. You start moving just “10” around without the units in your code and then your Mars Orbiter explodes.
Zoneless time is not a mistake. Using it when you meant to use a zoned time is.
Really, though, this is a recurring lesson in software engineering: don't be in a rush to make something a default just because it's "obvious"; explicit is better than implicit. Strings are another example; we've treated them as arrays of characters for so long that it's still a common default semantics for indexing a string, except now what you often get instead is UTF-8 or UTF-16 code units rather than actual code points. All of which is likely not what you actually need - but what you do need depends so much on the task that it should always be explicit.
Python does not. It’s all one object and I think that was a terrible idea.
Respectfully, no - in my scenario, if I used a timezone then I would show up at the wrong time for my appointment: each locale that observes DST has two timezones: one for DST, one for non-DST (e.g. BST and GMT for the UK). If I record a 10am appointment as "10am BST" and my country changes the rules on daylight savings transition date between now and the appointment, then I will show up at the wrong time for my appointment. Whereas if my appointment is recorded as "10am Europe/London" then it's encoding the human reality of my appointment, and can be correctly converted to an Instant in time if the rules change.
It sounds to me from your reply that you are mentally modelling all datetimes as Instants. These are one key aspect of dates+times (and often all a lot of software needs - since it's usually referring to some point in time that has already passed, or a future point in time that is x seconds from some other Instant), but Instants don't cover everything required when working with dates and times.
>I’ve never ever needed timezone-less dates. Even using UTC or timestamps, it’s still UTC.
Try sketching out the design for a calendar application and I bet you'll very quickly run into a whole raft of datetime related logic you might never otherwise encounter. Think particularly about future events: some are fixed instants in time (e.g. a meeting with people across timezones), others aren't (e.g. a dentist appointment in your city - my second example above is an instance where the local time of the appointment is relevant, not the Instant in time that local time referred to when the appointment was made). How do you represent recurring events?
If you can mix timezone-aware objects with non-timezone aware objects like you do in Python, it’s get pretty dangerous because people will just mix them willy nilly and add and subtract the timezone-aware parts and then you get some really subtle date/time bugs.
That’s why I believe either a date-time MUST have a timezone and for other cases, you need a specialty data type for your domain, possibly provided by your date/time library.
Without timezone, your clock would actually go off in the middle of the night when you are in a different country. Your clock does not know you change location (no timezone, no location-dependent time), so it will alarm you after exactly 24 hours since the last alarm, which, in another country, can be midnight.
> 2. If I have an appointment at 10am next November, that's actually a datetime with a locale (given that my appointment is at 10am local time, no matter whether the DST rules change between now and then).
If everyone is staying in your city, then that works great (sans catastrophic continental riff or meteorite strike). When at any point you involve more than one geographic location, then the appointment is meaningless without timezones.
Now, maybe it is actually the case that you and your software only care about one geographic location in perpetuity, but just like assuming 32-bit IP addresses, 8-bit ASCII, and length-oblivious arrays are all you need, the teeth marks from its consequential bite tend to be quite deep and sanguine.
This does not describe any modern person's experience. Your clock is connected to the internet; it always knows the local time.
Your clock would do what your clock is programmed to do. What a weird statement to make about a completely hypothetical tool.
Sure it's technically well-defined as it is, but I think it's too important not to be written out.
In that case you can represent your alarm either as 08:00L if you want to wake up in the morning regardless of where you travel, or 08:00-02 for those cases where you need to stay in sync with home say.
[1]: https://en.wikipedia.org/wiki/ISO_8601#Local_time_(unqualifi...
The big problem here is that it does not use timezones, but UTC offsets. Timezones can and do change their UTC offsets, completely breaking all future timestamps saved in ISO 8601-with-"timezones". "08:00 UTC-02" is ambiguous, "08:00 America/Nuuk" is not.
After all, if a zone changes offset today it shouldn't change the timestamp I recorded yesterday.
> After all, if a zone changes offset today it shouldn't change the timestamp I recorded yesterday.
That's not how named timezones work. That behavior would be bad and it would also qualify as ambiguous.
Half the world experiences a scheduled offset change every time summer time begins or ends. Computers can handle those fine.
But they do so because they record the actual offset when the time is recorded. If I send you 2023-10-29T02:15NO, how do you know if it's +01:00 or +02:00? Similarly, if Norway suddenly decided to switch their offset to be 15 minutes back, how will you handle that when reading previously recorded timestamps?
I think instead you'd send a timestamp (unix or NTFS or TAI or other) annotated with "Norway", and you would not send the offset.
> Similarly, if Norway suddenly decided to switch their offset to be 15 minutes back, how will you handle that when reading previously recorded timestamps?
Am I that bad at explaining this?
If Norway changes their offset, they will change it as of a specific moment in time. Times before that moment use the old offset, times after that moment use the new offset. The timezone database doesn't just store one or two offsets for a location, it stores every offset that has ever been used.
Checking whether you're before or after 5am on Feb 29th 2024 for a 15 minute change uses the same code as checking whether you're inside any particular year's version of summer time.
This is what I attempted to write with "2023-10-29T02:15NO" (NO being Norway's country code).
> Am I that bad at explaining this?
Could be I'm dense, and it's not my field of expertise.
Let's say Norway has decided to change it's offset from +01:00 to +01:15 at 2024-01-01T01:00:00Z. I have a process that records timestamps, and lets imagine it records them with the suggested named offsets instead of numerical offsets, in the format shown above.
Later next year, you look in the logs and see 2024-01-01T02:07:00NO. How do you know which time this is in UTC?
With numerical offsets this is simple, because it would either be recorded as 2024-01-01T02:07:00+01:00 or 2024-01-01T02:07:00+01:15. But when you just have NO, how to you convert the timestamp to UTC?
2024-01-01T02:07:00+01:00_NO -> 1704071220_NO
2024-01-01T02:07:00+01:15_NO-> 1704070320_NO
The timestamp doesn't have to be UTC, but it should specifically not be local time. You should be able to convert it to UTC without knowing the offset or time zone.
So, done this way, storing time zone and no offset is unambiguous. And it gives you more information than storing an offset and no time zone.
2 time zones account for DST so that point is moot, but in the context of traveling and still wanting to maintain the same times this is actually something calendar apps often get wrong
but are there any cases where “now” being naive is useful
I don't know anything about Python's datetime handling (other then to be alarmed by it given what I've seen in this thread). I live in the Java world where the java.time library is one of the best around (after a decade or more of Java having one of the worst date time libraries)
This worked really well! You represent (1) with a `Time.Ofday.t`. I suppose if you wanted you could represent (2) with a `Time.Ofday.t` + a `Date.t`, although it kinda seems to me like you want to keep timezone information around if you're dealing with DST changes.
I think this was it: https://github.com/com-lihaoyi/os-lib
Reading this article prompted me to future-proof a program I maintain for fun that deals with time; it had one use of utcnow, which I fixed.
And then I tripped over a runtime type problem in an unrelated area of the code, despite the code being green under "mypy --strict". (and "100% coverage" from tests, except this particular exception only occured in a "# pragma: no-cover" codepath so it wasn't actually covered)
It turns out that because of some core decisions about how datetime objects work, `datetime.date.today() < datetime.datetime.now()` type-checks but gives a TypeError at runtime. Oops. (cause discussed at length in https://github.com/python/mypy/issues/9015 but without action for 3 years)
One solution is apparently to use `datetype` for type annotations (while continuing to use `datetime` objects at runtime): https://github.com/glyph/DateType
* India has half-hour timezones
* Countries change their DST rules all the time.
* Conversion between timezones is an utter mess.
As Wes McKinney once said about python datetime: "Welcome to Hell"[0]
[0] https://stackoverflow.com/a/13753918
You get to have the typing help you enforce that everything is in UTC.
As in, 02:00+00:00 and 04:00+02:00 are absolutely equivalent representations of the same data.
If you need to know the timezone in which to display a time, you save that timezone separately
If that's messy, consider Nepal's timezone that's on the 45-minute mark (UTC+5:45).
https://en.wikipedia.org/wiki/List_of_UTC_offsets
https://en.wikipedia.org/wiki/Chatham_Islands
Right now is it on +13:45 due to Daylight Saving
https://en.wikipedia.org/wiki/Chatham_Standard_Time_Zone
There isn't a perfect geographical width for time zones. So humans pick something to define the boundaries between time zones. And making boundaries on a map is political.
Way back when, the drive in people won out.
Indiana's Daylight Time mistakes were fascinating. It wasn't that the state didn't adopt it, it was that originally the state allowed it to be a per-county decision as timezones have always been in Indiana. At one point in time if you were traveling I-65 which is nearly due north/south between Louisville and Indianapolis you could experience four different timezones (CST, EST, CDT, and EDT) and which ones agreed with each other obviously depended on which month you were traveling. Since Indiana went state-wide Daylight Time and Indianapolis decided on EDT once and for all, all of I-65 today is EDT I believe, but it is still strange to remember the years where that wasn't the case.
(ETA: one of the underappreciated homogenizing factors here has been the modern cellphone. People would get really confused if their cellphones hopped an hour back/forth every so many miles as you passed county lines. In the eras of paper maps and hand-set clock radios in cars that would have mattered a lot less.)
long time ago there was a special `$ man date` -like page in linux which went into long explaining many "amazing" things about calendar stuff, like whoever feudal in 1553 deciding that certain week was bad and striked it out of his and his country's calendar, or another one that liked certain month and decided to repeat it...
but cannot find it anymore.
Railways are extremely sensitive to exact time and, indeed, the very concept of unified time across the entire region or country only started developing when railways expanded across Europe. Prior to that, individual towns were happy with their own local solar time, but once railway connections were introduced, time irregularities would cause chaos at best and carnage at worst. That led to introduction of unified railway time which developed to timezones as we know them.
Railways aren't as prominent nowadays as they were 100-150 years ago, and countries like Nepal and India don't have extensive, frequently used cross-border railways anyway; any cross-border traffic is sporadic and mostly freight. So there is one fewer reason to cooperate with your neighbors when it comes to time-related issues. Trucks can take weird timezone changes just fine.
Now that the original cause is gone, maybe we can get some new concept going. Timezones seen more terrible than they're worth to me.
I make sure to test with Nepal time whenever I'm testing date/time stuff now.
> Article unique. — L'heure légale en France et en Algérie est l'heure, temps moyen de Paris, retardée de neuf minutes vingt et une secondes.
The decree which finally replaced it was made on August 9, 1978, and published on August 19, 1978 [1]:
> Art. 2. — Sur l'ensemble du territoire de la République française, le temps légal (ou heure légale) est défini à partir du temps universel coordonné (UTC) établi par le bureau international de l'heure.
[0] https://gallica.bnf.fr/ark:/12148/bpt6k2022333z/f2.item
[1] https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000000880128
Obviously it worked fine, as did Chatham Island.
This is a bit like asking "how do you represent a string length in utf-8?" The answer is, you don’t. It’s almost never a problem in practice.
Just like there is almost never a reason to care about the number of characters in a string, it’s very difficult to imagine that this code comes up in practice. But if it did, the code is simple: for a given timezone, is the current time in utc past 9:30am? If so, the store is open.
If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone.
Mapping a location to a time zone seems fraught with peril. I’d store the local time zone explicitly.
Another good example. Arizona doesn't follow Daylight Savings, but the Navajo Nation, which includes the Four Corners area and is in three states, does observe the change.
However the Hopi reservation, completely surrounded by Navajo territory, is entirely in Arizona, and does not observe DST. So it can get quite complicated and it's not enough to say "Arizona doesn't have Daylight Savings Time".
You've never wanted to call a business in another timezone before making the trip out there?
However, if I look up the opening time of a Home Depot in another time zone, I expect the result to be in that store's local time! I'll do the conversion myself. If I get my local time instead, I'll still perform the conversion, and get the wrong result.
You could just decide that any time listed anywhere on the internet should include a timezone... but that would be nuts.
I'm trying to poke holes in the "always use UTC and you're good" advice. I have actually encountered this scenario, and I didn't know how to apply that advice.
open | close | store 6am | 6am | default 7am | 12pm | store1
table stores: name | address store1 | xyx NYC
If you can't find the store in the timetable:
1. Get the timezone of the target store by looking up the address from the table stores
2. Set Hours from the default row in the timetable table on the object with the timezone
There's a much simpler one - recurring meetings/events (in-person or local-ish like within one country) that cross a DST boundary. You always want these to be attached to a local time zone, so it doesn't suddenly happen an hour earlier/later than you intended.
Each store will have an opening time, each will store the local time that the store opens.. just as utc.
"9:30am local time" may be ambiguous/unknown (the rules may not exist yet)
Using the same programming type for both may cause confusion.
How do you express geographic coordinates? Noon in London and in NY are different moments in time (it may be important if you want to have a remote meeting).
Your suggestion to define places by their timezone name is ok (it may work in many cases).
Like California is -8 in Standard, -7 in Daylight, so a 9:30 AM opening in the Winter is at 17:30 UTC, in the Summer, 16:30 UTC.
It's a hassle but programmers are supposed to model reality.
As an example of the problems that can arise, I remember when the new Japanese Emperor was to take office upon his father's abdication, there was a big fuss because the Era Name was customarily only revealed after the new Emperor's reign started but all the computer people wanted to be ready in advance. Not sure what finally happened, if it was revealed in advance or not.
UTC to local time is not a function, it is not somethung that can be computed once and stored for the future. The two are fundamentally different concepts, and can only be related to one another in certain contexts, and only reliably for the past.
There are a number of libraries for this, usually updated by the unpaid volunteer in Nebraska. Or else your company has staff whose job it is to follow this for any place you operate.
If you record opening times as UTC, well, then you have to change to UTC when the store switches standard <=> daylight.
This is what Fred Brooks called an essential complexity. You can poke the pain from one part of your system to another part, but it's never going away.
The function is complicated. There is no avoiding that.
One Pacific island nation duplicated a calendar day when it switched International Dateline sides to more closely align with Australia instead of Hawaii, reflecting a shift in its economic ties. No idea how they handled it exactly but pretty hairy to model.
It’s not a date time.
Which is scary, because you will be taking the performance hit from generating the warning regardless, which can easily amount to 30% of your total CPU time if you use utcnow/utcfromtimestamp a lot.
I seem to remember reading that remove-if-not is deprecated in Common Lisp, and that this is almost universally viewed as a mistake, but it remains deprecated (and the deprecation is ignored by everyone) just because the standard doesn't change anymore.
(It's going to break from the eventual removal)
Planned local events: future date and time string including local time zone
Planned global events: future date and time string in UTC, with UTC timezone
Past events: date and time string in UTC with UTC time zone, unless there is a strong case to be made that these should be localized (see “planned local events” above)
Strings in the format specified by ISO-8601.
Of course, internal representation can be whatever it needs to be for efficiency. For example, maybe your database stores as integer microseconds since an epoch. Fine, but as soon as that leaves the database, ISO-8601 that date.
You should not convert the integer early, you should actually wait until the very last moment.
Nowadays a timestamp will be marshalled and demashalled by (at the very least) two runtimes until it reaches human eyeballs opening up a wide spectrum of bug potential, difficult to understand code and development slowdowns. Keeping it simple and unambiguous is very important. Time libraries have a habit of coming and going, ints are always there and 99% of the date arithmetic is trivial any way.
Don't use a lib, just pretty print numbers and you'll be fine.
You kinda have to guess from context that if it's a 10-digit number, it's probably seconds.
Also calculating amount of time between two ISO8601 strings without libraries is nor trivial, or any other operation actually. When dealing with dates, there is only one simple way to work with non-localized times, and it's not ISO-something.
EDIT:
> ...and what epoch and timezone to use.
This is also not correct. Unix timestamp has a well defined epoch and it doesn't deal with timezones (though the epoch itself is, of course, usually defined as 1970/01/01 00:00:00 UTC). You are free to define other timestamps, but unix timestamp is well defined.
Comprehension by extraterrestrial life is a localization problem.
You can store weight as ohms of resistance on a load cell. But if you want to share that data you need to either normalize the number to a mutually understood unit or provide detailed information about your scale.
Unix time has uniform handling over leap seconds, regardless of the precision.
A unix second is allowed to be different from a "real" elapsed second.
The unix day has 86400 "unix seconds" which are actually 86401 real seconds (for days with added leap seconds).
Any application logging real world events against unix time will screw up velocities, force, energy, etc when computing "instrument reading" per "unix second".
It might not matter to most people but it's an issue for surveyors, geophysicists, astrophysicists, engineers, etc.
That is a fair point for situations that depend on short term observations. But, UTC has the same issues there as Unix epoch. I think it is a valid edge case but if you are doing, say, GPS based speed calc I would wager you are already pulling your time reference from a low level source that you can depend on will be running steadily for the X minutes/hours you need it.
Point being, it's an area of UnixTime that many overlook - to date, since the 1980s I've had long term multichannel data aquisition running throughout every leap second adjustment which would have had data glitches had the time channel been UnixTime, UTC, etc.
Note that UTC has a similar problem - which is why you actually need a local time string for this type of events.
While ISO uses UTC offset and time zone interchangeably it actually is only defined over offsets, not locations.
I've had this problem discussed before. Typically the situation is firing events based on times at specific locations. It is neither straight forward with date libraries nor ist it easy with numeric computation. Sometimes it makes sense to render it down to a specific point in time at storage and not expect the country change timezone overnight. Some times the location itself may be changing randomly (moving objects) but in that case it typically just meant recalculating offsets on movement events.
> Also calculating amount of time between two ISO8601 strings without libraries is nor trivial, or any other operation actually.
That's true of any datetime that actual humans use (as opposed to computers, financial systems, sysadmins and programmers).
That is a weird edge case. I've had this before and the person proposing it hinged on some forensic practice which was based on analyzing human readable data. As I said in my original post, pretty printing ints to ISO timestamps is actually my original opinion.
ISO8601 and friends are literally for formatting dates for the purpose of exchange.
So what? That's true of all formats. How is it relevant to the choice of which format to use at which stage of the process?
> and you have to know what timezone it's in.
You don't.
> clearly a timestamp
Not very helpful. It's easy to set up types in advance. Don't figure out if things look like timestamps at parse time, or you'll be causing the same problems that turn gene names into dates and choke on people named Null.
I don't think there's any case for storing local time with past events. They happened at a specific date and time, store them in UTC and if needed convert to local on display. Plus, though it's a rare circumstance, for one hour each year DST switches cause a duplicate hour - so local time is ambiguous during that hour.
[0] https://blog.nytsoi.net/2022/03/13/utc
Not plural though;
The whole of India -- even though it is pretty vast in area -- has a single time-zone country-wide. I am very thankful for this. We already have a lot of other challenges -- including dozens of different languages -- that a single time zone is a relief.
But yes -- the +5:30 offset usually makes for a less-than-ideal factor to plan meetings or follow non local event schedules.
Did I mention we also hate the Summertime / Daylight Saving time adjustments we have to accommodate twice a year when working with western teams -- more adjustment even after most people have already sacrificed on a sane 9-5 schedule to ensure they overlap with EU/US teams for daily work meetings. We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment. Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
If they did that, you'd still be stuck shifting your schedule when they change their working hours.
In the most simplest scenario -- once a recurring meeting is fixed at 10:30 AM eastern STANDARD time, say, -- no schedule change is needed all-year-long if we dont tamper with the clock.
It is just that US worker -- due to their own convenience -- decides to show up in their office at 9AM for six months and 10am for other other six months of the year. Recurring meeting does not need to change at all.
The adjustment , if any, is all on US side. And for local season / climate reasons which has nothing to do with the remote team members, and so the remote team members elsewhere in the world dont need to change anything at all.
Just dont tamper with the clock and things will be much more simple.
>> If they did that, you'd still be stuck shifting your schedule when they change their working hours.
No -- we do not want to link our meetings to the changing office hours. We just want a fixed time on a clock fixed clock. Pick a time that works throughout the year for both teams. Like 10.30 AM EST in my example above.
Current the meetings are "fixed time" but on a moving clock -- in reality the meetings shift around but US folks move their clocks to "simulate" for themselves that they are always having their daily meeting "at 10AM" -- they achieve this illusion by inflicting pain on themselves as well as rest of the world. We would love if they can leave us out of this forced dance :)
You're suggesting that twice every year, we change the times that retailers, offices, government agencies, broadway theaters, restaurants, roads, schools, and so on open and close. Schools alter class schedules and trains modify departure times. Also, we all reset our alarms, because we'll need to wake up at different times anyway.
It makes so much more sense to just change the clocks!
And yes, all of these things would need to change, because they all revolve around the typical work schedule! Parents bring children to school before heading to work. Public transit runs more frequently during rush hour.
There is a discussion to be had about whether we should be shifting our schedules in the first place. But as long as we're shifting schedules, it makes so much more sense to just change the damn clocks!
P.S. Most people don't have white collar jobs! Construction workers, janitors, mechanics and so on don't ever work across timezones.
P.P.S. I'm okay with nixing the time change as long as we don't end up on permanent daylights savings! However, permanent daylights savings is my nightmare.
Currently the shop closes at 8pm and it's dark by 4pm; in Summer it closes at 8pm still (on +1 time) and it's dark at what 11pm or later? Explain your point again?
If there is a certain time that really does work for everyone, then DST doesn't really affect it. At worse, it requires a bit of rescheduling in the calendar if the meeting was set by someone in a DST country. If the meeting was set by someone in India, it will even get automatically adjusted in everyone else's calendar. So DST itself is not actually making things difficult for you, it's only people's habits changing with winter/summer that makes things difficult.
And note, I have direct experience with all of this, working in Romania with colleagues from California and Kolkata with at least three recurring syncups with all three of us per week.
Outlook and calendar apps should probably make this a feature: "do not move this meeting for DST/summertime" (even if the host is in US/EU). Make it opt-in initially and a couple of years later make it opt-out (default) choice.
On a lighter note ... recurring meetings tagged to "dynamic time zones" should be allowed only after hosts "agree" to a disclaimer that they have consulted other participants and have their consent :D (a little like recording with two-party consent)
Because "The USA" spans from GMT+10 to GMT-10. Your Solar Noon only varies by an hour between Kolkata and Mumbai. Without time zones, the continental U.S. would see a solar noon difference of three and a half hours. And that's not including Alaska, Hawaii, and Guam, which gives us a 20 hour span. You want all those places to be in the same time zone?
Guam is a bit of an odd duck because it is on the other side of the date line. The US currently spans from UTC-4 in Puerto Rico to UTC+10 in Guam - which is essentially the same as UTC-14. That's a 10-hour span, not a 20-hour one. Still, excluding the minor overseas colonial possessions would probably make the most sense.
It's bizarre to suggest that someone in western Alaska should wake up at the same time as someone in Florida because that's what India does.
(And even though lining up a time with morning would be significantly more annoying and need to exclude the polar circles, I still wouldn't call it "impossible".)
If you want to know when the sun rises, sets, or is at its apex just look it up in a table. The former two vary unless you are very close to the equator anyway, and the latter is off by an hour for every country using "daylight saving time" for half the year. Never mind countries that span more than 1/24th of the globe but use a single timezone, and that isn't just China.
Also I don't really see how this is "for the convenience of the ruling elite". I'd be willing to bet money most people in Xinjiang wouldn't have this "problem" in their top ten. Probably not even top hundred. This seems like something you get used to once and then never think about again unless you travel or have a remote meeting.
https://en.m.wikipedia.org/wiki/Xinjiang_Time
Yes, a table.
A table with time.
A table that divides the world into zones with regard to time.
That definitely abolishes time zones.
> I'd be willing to bet money most people in Xinjiang wouldn't have this "problem" in their top ten.
Yeah, it probably does rank quite a bit below the genocide.
Anyway, time zones solve an important problem: People coordinate with other people close to them, but occasionally need to coordinate with other people far away. How do those far-away people know when the good times to call are? Clocks only work if you have some idea of what times mean in practice to distant people, which is greatly helped along by people setting their clocks to a local time that's known globally.
Think about it more. How sunrise and sunset change by location and by date.
A chart that covers both sunrise and sunset does not naturally have "zones", and any "zones" you try to infer would not resemble time zones at all. You're either looking at big sweeping ellipses, or you're dividing the world into hundreds of small tiles. It's not time zones.
I've been there and it was absolutely fine - a lot better than working in the US with its mess of timezones. Sure, the times in Xinjiang are "wrong" but who cares? I'll take getting up at "11AM" or whatever over having people in different offices think the meeting is an hour earlier or later and not finding out everyone was mixed up until it happens.
It's not hard to know when you do things where you live. Some places people drive on the left, some places they drive on the right, people in different places people speak different languages, people in different places get up and go to bed at different times. When you go to Xinjiang as an outsider sure the customary times are a bit unusual but they're far from the only unusual thing about going there.
> With time zones you know that businesses will work roughly from 9-10am to 5-6pm that people will normally go to bed around 10pm and will wake up around 7-8am.
Not necessarily - plenty of ways to be caught out if you make that kind of assumption about a place you're not familiar with (e.g. turning up at a business when it's siesta). When going somewhere unfamiliar you're already best off looking up when your hotel/restaurant/etc. opens, and it's easy to do these days.
Timezones maybe made sense when physically going to a different place was more common than having a phone/video meeting with someone in a different place. But nowadays being able to agree on the same instant in time when you're in two different places is more important and we should standardise.
Sure, it can work. It's just worse. It'll be harder to adapt to local time if you move there or go there to visit than just changing your clock to match local time. You'll need to convert constantly. What's the upside, though?
> Not necessarily - plenty of ways to be caught out if you make that kind of assumption about a place you're not familiar with (e.g. turning up at a business when it's siesta). When going somewhere unfamiliar you're already best off looking up when your hotel/restaurant/etc. opens, and it's easy to do these days.
Sure, you could still get things wrong, but you're suggesting going from a situation where it mostly works, to a situation where this never works.
>Timezones maybe made sense when physically going to a different place was more common than having a phone/video meeting with someone in a different place. But nowadays being able to agree on the same instant in time when you're in two different places is more important and we should standardise.
Timezones are the way we found to standardise once global commerce became a thing. Timezones give you essentially time and location, so it's easier to figure things out when multiple parties are involved.
No it isn't? You don't convert anything, you just do things at the times the locals do them. It's really not hard.
> What's the upside, though?
No changing clocks, no scheduling a meeting at the wrong time because you mixed up the timezones, no calling your parents and accidentally waking them up because it's the middle of the night for them.
> Timezones are the way we found to standardise once global commerce became a thing.
In the distant past each village had its own time; once the railways emerged and it was practical to go from place to place in the same day, we standardised time across decent-sized regions. Now that we can talk to people instantly around the world, it's time to contine that process and standardise time everywhere.
Before you get used to it, when you see a time you'll have no idea what time of the day it is. Is it early? Is it late? Is it during lunch time? You need to convert in your head "11am here means midnight where I come from, so that's actually really late". Very easy to forget and make a mistake.
> No changing clocks, no scheduling a meeting at the wrong time because you mixed up the timezones, no calling your parents and accidentally waking them up because it's the middle of the night for them.
huh? How is using a single time going to help with waking something up because it's the middle of the night for them? I'd say it's more likely. If someone is 6 hours behind you, you'll need to keep in mind that their 10am means what would be your 2am. Even though you both call it 10am, you would definitely not want to call them at 10am. If anything, that's more error prone and confusing.
That's not a real problem, IME. If someone invites you for lunch, it's going to be at lunch time. If someone wants to schedule a meeting, you have to check your calendar anyway.
> You need to convert in your head "11am here means midnight where I come from, so that's actually really late".
No, you don't. Converting in your head is the wrong approach just as it is for languages. Just get used to when you're going to bed and getting up. (And don't use AM/PM - why would you ever do that? Even within a single timezone it only causes confusion)
> How is using a single time going to help with waking something up because it's the middle of the night for them?
Because you never have to add or subtract a time, which is where most mistakes happen.
> If someone is 6 hours behind you, you'll need to keep in mind that their 10am means what would be your 2am. Even though you both call it 10am, you would definitely not want to call them at 10am.
Right, so you need to know when they go to sleep and when they get up, and not call them when they're asleep. But there's no arithmetic to get wrong, there's no risk of adding six hours instead of subtracting six hours or vice versa.
No, it spans from UTC-10 in Hawaii to UTC+10 in Guam. And if we include other US territories in the Pacific, like American Samoa, we have to include UTC-11. That's 21 hours.
95+% of Greenland's population lives on its west coast. The most common map projection one is likely to come across makes it look like that spans a pretty large longitude range but it actually only covers about 15°, a 1-hour span.
Take a look at a projection that does a better job of representing latitude, such as this [1] azimuthal equidistant projection and you can see that west coast Greenland is a lot more north/south than you might have expected based on the more usual projections.
Here's a map of the towns in Greenland with >300 population [2] showing how much more populated the west coast is than the rest of the country. If you add their populations and the populations of the towns listed in the table but too small to make the map it comes to about 3100 people.
[1] https://en.wikipedia.org/wiki/Azimuthal_equidistant_projecti...
[2] https://en.wikipedia.org/wiki/List_of_cities_and_towns_in_Gr...
I was not arguing for a single time zone.
you quoted only half of my sentence and are arguing against a strawman.
Shifting time-zones increases car accidents by about 6-16% for the entire week (studies vary). Just that is a massive enough impact on safety and productivity to discourage the switch. At a personal level, leaving work at 5.30-ish with it being pitch black outside, is among the most depressing things in the world. Even more wasteful is that the average American wakes up at 7.20am. So, they don't gain any daylight but lose an entire hour.
I'm not usually the type to make such accusations, so I apologize in advance for sounding culture-warry. But, switching-time zones is "white person thing". Large percent of the Chinese, Indian & the Middle Eastern population live above the 25th parallel (Florida) and large populations of Korea & Japan live north of the Bay Area without daylight-switching.
Permanent daylight time with later sunrise would be a huge win.
Are you aware the contiguous United States is three times larger than India? Alaska alone is half the size of India and not at the same longitude as the lower 48. Hawaii is even farther west than most of Alaska. The easternmost point in the US is 66 degrees west. The westernmost is 179 degrees west. That’s nearly a third of the globe and I'm not even including territories.
> Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
Is this a serious comment? Europe doesn’t have a central government. The most recent attempt to establish one was extremely unpopular and famously stopped in 1945.
Why should foreign schools and businesses change their hours to accommodate you? A more obvious solution would be to just stop observing DST but that’s not something that will happen unilaterally and will be an especially hard sell at extreme latitudes.
If you want to do business internationally you have to deal with timezones. India isn’t unique in that inconvenience.
Not unilaterally per se, but EU parliament did vote to end it in ... 2021 https://www.europarl.europa.eu/news/en/press-room/20190321IP... . COVID kind of put those plans on the backburner.
People at extreme latitudes should not care much, since for them DST does close to nothing.
The parent's proposal is actually quite sound. It would be a lot more useful to have institutions switch their starting times for part of the year when it makes sense for them, rather than forcing a one-size-fits-all solution on everyone. Countries at different latitudes would be free to switch e.g. school start times at different dates depending on when sunlight shifts too far in one direction, while keeping their clocks synchronised with astronomical time. Heck, Spain basically does that already, since they're on the wrong timezone (geography dictates they should be on British time, but because of events in the '40s, they're on Central European time) - so they do everything one hour "later".
This proposal is literally one-size-fits-all.
> It would be a lot more useful to have institutions switch their starting times for part of the year when it makes sense for them, rather than forcing a one-size-fits-all solution on everyone.
This is exactly how it works now. But there are a lot of institutions and they aren't coordinated.
you quoted half my sentence which creates an easy strawman.
> A more obvious solution would be to just stop observing DST
yes that is essentially my argument too ...with an extra accommodation for folks who will argue about seasons causing wild changes in sunrise / sunset / daylight.
Huh, why would it not happen unilaterally? Different countries already fiddle with their daylight savings times (or absence thereof) unilaterally, and the start of DST is not synchronised across the globe anyway.
So if you have two countries that start DST on different weeks that's already a hassle to deal with, and one of them going off DST altogether doesn't really add any extra hassle in their bilateral dealings.
that's one way to say it
For what it's worth, I've (British) never understood that either. Need more daylight hours for farmers to work or whatever, sure, no problem. But.. that's easily resolved as you described? And before widespread centralised time and access to it, surely they just started at sunrise or soon after, whenever the sun happened to rise. I don't get it.
Or, if you extrapolate this argument further, you could even wonder why the entire world can't run on a single timezone. Like, it can be UTC everywhere.
People in India can go to sleep at
Or, if you extrapolate this argument further, you could even wonder why the entire world can't run on a single timezone.
But here's the problem: for example, imagine it's UTC everywhere on Earth. No local timezones.
As per your solution, people in India can go to sleep at 4:30 pm (UTC), wake up at 12:30 am, have lunch at 7:30 am etc.
Likewise, people in San Francisco can go to bed at 6 am, wake up at 2 pm, have lunch at 9 pm etc.
Now, the problem is, when someone wants to setup an international meeting or whatever, nobody would have a clear idea or whether it would be "sleeping time" for any of the participants involved, without having to look it up.
Like, with the current setup, it's kind of universally understood that 3 am local time is sleeping time, hence people would avoid setting up meetings at that time.
But if everywhere on Earth it's the same time such as UTC, I would have no idea if 3 am UTC is your sleeping time or lunch time or whatever. I would have to maintain lookup tables of when the sleep time, lunch time etc would be for every country as per UTC.
Similar problems exist (to a lesser degree) when large countries move to a single timezone.
Yes there are flaws in the existing setup, but moving to a single timezone for large countries would only introduce a new set of challenges.
Kindly re-read.
I was arguing against daylight savings adjustments twice an year. Not against having multiple timezones per country / region. My comment was simply that timezones should be sufficient without the need for DST.
What I said ...
>> We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment.
Simplified ...
>> cant imagine why US, and EU need so much adjustment.
Pardon any poor punctuation.
Reminds me of when Numpy removed a lot of types like np.bool a few releases ago, because they were aliases of the standard library types.
Which broke a ton of libraries having Numpy versions pinned to ">=1.0" or even worse, "~=1.0" expecting them to follow the standard practice of incrementing the major version when they break compatibility. It's not terrible because you just pin your project to 1.23 or whatever, but it was very annoying when everything I was using just started breaking.
I wonder if Python plans to do the same with this - the 2 to 3 transition was painful and dragged on for a decade, nobody wants to do that again for 3 to 4. When they remove this function and break back compatibility with old code, is that going to happen in a future 3.x release?
https://numpy.org/neps/nep-0023-backwards-compatibility.html
https://numpy.org/doc/stable/dev/depending_on_numpy.html#run...
So, when using numpy in a package, and you've tested e.g. versions 1.12 to 1.22, you should require 'numpy>=1.12,<1.25', because something deprecated in version 1.23 could be removed in 1.25.
I think in some senses these backward compatibility breaks have already happened. I remember that Google got bit by async becoming a keyword in I think Tensorflow.
Generally a lot of old Python 3 code still works today if it doesn't do anything really weird like that, but I do definitely think that the pain of Python 2 to 3 was painful so they don't really want to repeat it.
I think it is probably prudent to treat point releases as major releases at least with regards to Python.
If you're working in a system where all dates are utc, then datetime.now() gets you a date in utc and you don't need datetime.utcnow(). The only reason to call utcnow is if you're working on a system where the local timezone isn't utc, so you inherently have to deal with timezones at some level. Which means that using naive timestamps is a bad idea.
Yes, you can store additional information to decipher the meaning of a UTC time but why? Storing locale is good because sometimes you need to know where something is, such as when converting the local time to another zone. But you should avoid creating dependencies between these values because it makes using them individually harder.
The coffee shop opens at 6am. That is an unambiguous fact. The coffee shop is in Seattle, WA. That is another unambiguous fact.
If the time is in UTC then you need to know the date the time was set or store a dtc flag and then build some logic to convert. You can just feed local time+locale into any datetime library and get whatever you need.
UTC is great for a lot of things. Such as recording when something happened. But local time has a place as well.
The key here is that the timezone info includes the daylight/standard designation. Or, in the ISO format -4:00 or -5:00 from UTC.
I think you meant to put ET (Eastern Time), which is still 9:30AM, but without a date associated with that time, there's no way to convert that to UTC or other timezones that don't have the same daylight saving time schedule.
So, when possible, it's best to store (future) human events in a localized datetime string. For things like physical events, the opposite is true of course - you can't say what will be the local date and time when the Sun will hit high noon over Athens 10 years from now, but you can certainly calculate the UTC timestamp for that.
Here, I'm not even sure what your point is. A datetime object cannot capture a range, no, just like a number can't capture a range of values. But through the magic of having two of them, we can get easily create a data structure containing two datetimes to represent a range. 5pm is not a vague concept, it is the time 5pm, the hour that typically comes after 4pm and before 6pm. If your point is that you can't store only a time in a datetime object, that's true, that's why the standard library also provides the `time` object which represents a time.
Where things might get a bit more complex is if you want to store the time of a recurring event that should occur at the same time on multiple days for a given timezone. In this situation, you can typically use a naive time object to represent a wall clock time, plus the timezone that the user has requested. This way you still have all the raw information to decide when the event will happen in the future. (Note however, that the same time can occur multiple times on the same day, for example during daylight savings changes.)
And you're totally right about `time` existing, it slipped my mind that time has a tzinfo slot. I do think you should still take care and I would avoid having any naive time objects just floating around beyond the serialization layer though.
You have to store time in the relevant geographic time zone. Something occurring at 1200UTC on dec 1st is not the same as something occurring at 1200Europe/London on dec 1st. The two times may coincide today, but if the U.K. government decides to implement daylight saving next week as a fuel rationing measure, then your 1200London event will have to be displayed at 1300UTC, but your 1200UTC event would be at 1200UTC.
Point remains, many events are linked to local civil time, which can change - and not just to daylight saving. The Line Islands in Kiribati shifted a whole day in 1994, same with Samoa in 2011. Have an event set for 0800 local on the first Monday of the year in Samoa which you stored as UTC, and your events would have broken. Unless you store the relevant data (it occurs on first monday according to the timezone that Samoa is in), then you will be wrong.
Now of course you can still get problems if a new timezone is created, as time is stored as Europe/London, if Scotland creates its own timezone in the future a meeting in Aberdeen would no longer be in Europe/London but in a new area Europe/Edinburgh, with different behaviour, but that’s far less common than DST changes or other date shifts.
All in all you should store the relevant datetime and zone, not just “normalise” it to UTC.
> I would have made it more clear that these functions work with naive time, maybe by calling them naive_utcnow() and naive_utcfromtimestamp().
> But their names are not the problem here. The specific issue is that some Python date and time functions accept naive timestamps and assume that they represent local time, according to the timezone that is configured on the computer running the code.
Uhhh, no I would not expect local time because as the name entails we're dealing in UTC. Sigh.....
Between this, the Unicode circus, and the rejection of PEP582, Python seems more and more like the HOA that isn’t earning their monthly fees IMO.
Set a separate flag that governs conversion to string to avoid different ISO output for shitty string parsers if you want, but there's no great reason to not just fix the functions so that they continue to work better than before and do the thing that people actually expect them to do, which is produce UTC.
Intentionally naive storage or transfer is irrelevant. That's just serialization. And when you explicitly tell the computer to "deserialize this as UTC" it should fucking do what you told it to and give a zone aware result.
But I think changing the behaviour and not the name is still a very bad idea.
They screwed up these methods from the start, and removing them is the only way to get rid of the confusion, without introducing subtle breakage.
That's why the suggestion to just keep the functions and change their behavior is naive. It would obviously be very unsafe to do so.
Many people won’t know it’s deprecated until the code fails to run, and having new functions rather than changing the functionality of existing functions makes it much easier to identify what is actually happening.
I totally agree with you that it makes no sense this `utcnow()` does not return tz-aware object.
But I think their fix is great in this regard: now you just use `datetime.now(timezone.utc)`. It's more explicit, it returns aware DT object now, and more importantly, it gets rid of a function that I always feel too specific to have anyway (like how we don't have `plus(a, 2)` and `plustwo(a)`).
Looking for this mistake in code reviews gets really old.
Now, let's imagine another independent process is consuming events from the database and compares an event written before this change to one written after.
Python is a popular language, used in production by scheduling applications around the world. The potential downside of this kind of change is measured in billions of dollars. The safe solution is of course to not silently change the API in a breaking way, and instead introduce a new API and get someone to look at why the code stopped working.
datetime.now() does not return utc time (but does return a naive datetime object)
It is kinda like the 2->3 change with strings
And utcnow is a bad name. But the "fix" is none the better
In C, we all know we shouldn't use things like strcpy() and gets(), and you'd be a total fool to add them to new code. But for the sake of backward compatibility with programs that aren't going to change, they should be kept in and documentation should nudge people to do it the right way. Maybe that's the Python way of nudging: "This function is deprecated!"
But aside from that, I think it's just a different style. Waiting 10 years from deprecation to remove a deprecated thing might be required with C just because of how prevalent C is and how much the developers in that have to think about basically every platform, but removing something on a major release isn't inherently bad, it's just a different tradeoff (being able to move a bit faster and fix some design issues vs. being very very good at backwards compat)
The deprecated function can obviously be reimplemented pretty easily, but so can gets(). With utcnow(), it had some unfortunate effects in that it didn't really do what people thought it did, and because of that, most of the uses were basically bugs to begin with.
(Having experience with .NET for example, you'd think that DateTimeOffset.UtcNow and datetime.utcnow() are semantically similar, but the problem is that utcnow() doesn't actually have a timezone attached to it. Adding one would also be a breaking change, and one that is more subtle.)
It always exists, it's just that the import gets controlled by a macro. Which, hints at a way Python could have solved this same problem without requiring auditing and rewriting code.
UTC is simple and unambiguous. Time is a number representing a fixed point in time. When displaying it for humans, convert it into human form. When this principle is applied everywhere, it’s hard to make mistakes.
My gut says that deprecating utc as the native time representation will be one of those ideas fervently embraced by a few, ignored by most, and then quietly forgotten about in a decade.
This, I think, is the idea that will be quietly ignored in the long run.
Seconds since start of epoch
If you think UTC means that "time is a number" you are very confused though. UTC has no relation with the UNIX timestamps (and the epoch), though they are both ways to represent a specific point in time.
You are reading to much into a simple question.
> UTC has no relation with the UNIX timestamps (and the epoch),
I said nothing about Unix
The epoch in Swift IIRC is in 2000.
The idea of counting time in seconds since an epoch is ubiquitous. I am unsure if it is universal
It was my mistake for trying to engage. Generally people get downvoted for good reason.
Still I'm confused why you're in this thread at all, which is about a change to an API you don't seem to want to use at all.
That is what they said, but it is inaccurate
All time should be UTC until it is displayed
So a number without a timezone is seconds since epoch in UTC
That is "default" not naive
Some Python applications programmers made stupid mistakes so they change the library
Golly. On the face of it Python is a mess. And it is everywhere. Golly
A "naive" datetime is the one for which it is not known what the timezone is. The opposite is the one for which it is known. You can absolutely encode UTC timezones as seconds since epoch without any additional information (except that one bit that is needed to distinguish them from "naive" ones, which doesn't have to be a literal bit - it can be just a different data type, for example).
And, yes, it would sure be nice if we didn't have "naive" datetimes at all, and everything was always UTC. But there are too many datetimes recorded in the real world without enough information to recover the timezone and thus the precise UTC time, and they still need to be processed. The mistake was to make this the default behavior - but that is a very old one, dating back to earliest days of computing.
True. Fair
> A "naive" datetime is the one for which it is not known what the timezone is.
Then it is UTC
That is what UTC is for.
You're absolutely right that it should be, but the problem is that it isn't – Python treats these as belonging to the local time zone.
Says you. The python documentation has never said that naive datetimes are UTC. If the language suddenly makes this assumption, then you break existing code. If you don't want to break existing code, your only option is to 1.) deprecate the existing function and 2.) provide an alternative (which is provide an explicit datetime).
> Then it is UTC
No, naiive datetime is: "now" ie a time description that lacks enough information to be able to fix that moment relative to other datetimes. It lacks TZ information.
I happen to physically live at roughly 2.6° west of the Greenwich meridian oh and at 50.9 north of the equator. It is now 23:08 (to one minute precision). I have given you enough information to derive my active time zone, if you consult the TZDB, and hence to convert my given naiive datetime to one that will work for you. If I hadn't included both a northing and an easting (I actually gave you a "westing" but that is simply a negative easting!), you could go very wrong. For example France, Spain etc are directly south of here and they are an hour "ahead" of the UK, right now.
Right now it is: "23:15 UTC+0" I could use GMT+0 instead and that would work too but let's keep it French! That's fair - the Greenwich meridian was picked over the Parisian one (and several others) for 0° east/west for time zone calcs but let's call it UTC instead of CUT which is what the english translation would end up being. I actually prefer UTC - CUT is also a word.
Now, I have not included the date bit of datetime in my examples above. There are multiple naiive interpretations for datetime. Calling them all "UTC" is simply not going to cut it 8)
I thought the French way was TUC. UTC is the compromise and/or compatibile wit UT1+UT0.
Software sux
Changing the Python libraries to try to fix buggy software strikes me, on the face of it, as a bad idea.
I am sensitive to this having recently worked with ode that stored time as a string, wall clock local, with no timezone.
And made time series data from it
Software sux, because programmers do silly things
Stop it!
There is no assumed timezone for naive datetimes and different software treats it differently. What's being deprecated is methods to create naive datetimes that are implicitly UTC without just attaching that information to the object itself. In the new world everything is still in UTC and you convert to local time at the last minute but now you can be sure that the datetime object you got handed is actually UTC instead of it being a convention that isn't always honored.
And you must do it in UTC
But otherwise use UTC
A local time aware process has to be available for users. Of course
But in the backend it is UTC
Storing time any other way is a world of pain
Scenario 1: You have an event to schedule at 9am local at some future date. You want to convert to UTC. But you can only do that if you know the offset, and the offset for future dates can change or be unknown. How do you resolve this?
Scenario 2: Now the event is "9am every day, indefinitely". How do you represent that as UTC? Do you generate tens of thousands of separate timestamps?
Why would you ever want or need that?
is this English? also I am still not seeing a problem:
Ah, a grammar pedant. Haven't met one in a while.
>also I am still not seeing a problem
Try again setting your timezone to let's say NY and date at 2023-11-04. Oops.
"That is now is 7 PM"
"and want it to run tomorrow"
"Like if simply print"
none of these make sense
> Try again setting your timezone to let's say NY and date at 2023-11-04.
not sure what point youre trying to make, but youre failing:
No wonder since you failed to correctly specify NY timezone in first place. Here's a trivia, it wasn't UTC-5 on that date.
> It is also a best practice to store naive datetimes representing UTC in databases.
It is most definitely NOT a best practice and you SHOULD NOT DO THIS.
If your database supports TIMESTAMP WITH TIME ZONE -- which the "aware" type per the article's terminology -- you should be using it. If your database doesn't, you should get a better one.
Happy to be corrected here but it's not intuitive to me that dealing with timezones as the 'base unit' makes anything easier.
I've been in each camp at one time or another, since being db-agnostic makes sense in some cases (libraries, products) and not in others (websites, services).
For example, imagine I schedule a dentist appointment for 9AM in my local timezone sometime next summer, and put it in my calendar. Now, if the government decides that we won't do daylight saving times next year, my dentist and I still expect that appointment to happen at 9AM local time. However, if my calendar application had converted it to UTC using the timezone definitions at the moment I created the appointment, it would show in my calendar at 8AM, and I'd be an hour early!
This is not an as hypothetical situation as it seems. To give just two recent examples: the Chilean government decided to postpone the start of DST with less than a month's notice last year, and Morocco shifts the clock an hour during Ramadan, the date of which only becomes certain on the day itself.
However, if you are writing something like a calendar app and storing timestamps for future events, you run into the issue that time zones have the annoying habit of changing.
Let's say you have a user in Berlin, and they want to schedule an event for 2025-06-01 08:00 local time. According to the current timezone definition Berlin will be on CEST at that moment, which is UTC+2, so you store it as 06:00 UTC. Then Europe abandons summer time, and on 2025-06-01 Berlin will now be on CET, or UTC+1. Your app server obviously gets this new definition via its regular OS upgrades.
Your user opens the app, 06:00 UTC gets converted back to local time, and is displayed as 07:00 local time. This is obviously wrong, as they entered it as 08:00 local time!
Storing it as TIMESTAMP WITH TIME ZONE miiiight save you, but if and only if the database stores the timezone as "Europe/Berlin". In practice there's a decent chance it is going to save it as a UTC offset anyways...
NO!!! IT IS NOT THE SAME AT ALL!
I've been burned by that exact misconception. TIMESTAMP WITH TIME ZONE does NOT store a timezone in the database!
It implicitly converts a timestamp you supply into UTC and DISCARDS the associated timezone information you provided. When you read it back, it converts it to LOCAL time unless you explicitly specify a different conversion. This is a recurring source of confusion and bugs. The timezone a timestamp was actually written in is sometimes VERY important -- and I've personally had to deal with data loss on a project caused by this misconception.
Let me repeat: There is NO WAY to read back what timezone a "TIMESTAMP WITH TIME ZONE" was written with later -- unlike an "aware" datetime in Python -- BECAUSE IT DOES NOT ACTUALLY STORE THE TIME ZONE YOU SUPPLIED! This is a data type that DESTROYS INFORMATION YOU PROVIDED TO THE DATABASE assuming it can be clever later.
I fucking hate that data type SO much.
An "aware" datetime in Python actually tracks a timestamp with a specific timezone explicitly attached to it. It is VERY different!
Documentation for Postgres:
> For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's TimeZone parameter, and is converted to UTC using the offset for the timezone zone.
> When a timestamp with time zone value is output, it is always converted from UTC to the current timezone zone, and displayed as local time in that zone. To see the time in another time zone, either change timezone or use the AT TIME ZONE construct (see Section 9.9.4).
https://www.postgresql.org/docs/current/datatype-datetime.ht...
Python:
> Date and time objects may be categorized as “aware” or “naive” depending on whether or not they include timezone information.
> With sufficient knowledge of applicable algorithmic and political time adjustments, such as time zone and daylight saving time information, an aware object can locate itself relative to other aware objects. An aware object represents a specific moment in time that is not open to interpretation.
[...]
> For applications requiring aware objects, datetime and time objects have an optional time zone information attribute, tzinfo, that can be set to an instance of a subclass of the abstract tzinfo class. These tzinfo objects capture information about the offset from UTC time, the time zone name, and whether daylight saving time is in effect.
https://docs.python.org/3/library/datetime.html
TIMESTAMP WITH TIME ZONE just sets off my lingering issues from an old project.
All the other posts in this thread misunderstanding it really demonstrate the point about it being a source of recurring confusion though!
which is basically exactly how one is supposed to use timezone-naive datetimes. the datetimes are individually "naive" however are meant to always be used within a context that is implicitly non-naive across the full set of datetime objects.
I'd like to add that if you have any fancy datetime manipulation to do in Python, pendulum (https://pendulum.eustace.io/) is probably your best bet.
It is best until you work with Pandas DataFrame (https://stackoverflow.com/questions/47849342/making-pandas-w...)
If you use numpy and pandas, you should also not use Python datetime, generators, most stdlib mathematical functions, the itertools module, random, etc.
It's the first thing you learn if you read any good pandas book, and the first thing I teach in my numpy/pandas trainings.
It has pretty much nothing to do with pendulum.
Basically, half the Python ecosystem is "well, except with numpy/pandas of course".
I'm the one arguing against this move, I've already made my thoughts clear that it's a lot of churn, e.g. the PR to remove it from Pip literally caused Pip to have to do a bug release, and has valid use cases, e.g. modelling datetime data types from spreadsheets and databases that do not have timezones.
But I also accept I'm not the one maintaining Python, so I've updated my codebase appropriately (`datetime.datetime.now(datetime.UTC).replace(tzinfo=None)`). Python is not a backwards compatability language, and it's clearly not a strong motivation for most users as I don't see anyone attempting to write a backwards compatibility shim over the standard library, so no point complaining too much.
datatime.utcnow() should be the default way of recording & storing time, e.g. in logs or in the database.
All other options are strictly inferior.
There's no reason for `+00:00` visual noise in the UTC-aware datetime.There're many things to complain but this is certainly not one.
Though this should be `datetime.timezone.utc` rather `datetime.UTC` if not mistaken.in any case, just compare
and verbosity is not a virtue in this casehttps://hexdocs.pm/elixir/DateTime.html https://hexdocs.pm/elixir/NaiveDateTime.html
I am not a python maintainer, and I also use naive datetimes a lot, and I can absolutely understand the maintainers rationale behind this.
Let me explain.
What's a "naive datetime" suposed to be conceptually? There is no such thing. Datetimes only make sense in the context of a timezone. UTC is just another timezone, with an UTC-Offset of +0
So why do we have this strange thing that may be a UTC timestamp, or a local timestamp, or god knows what in the first place? It makes sense as an implementation detail of the datetime library to have timezones as separate objects, but for consumers of that library? It can cause confusion, and hard to track bugs, when suddenly one of the functions that accept aware or naive is used and spits out something completely unexpected.
I have spent many an hour tracking bugs, where systems spat out complete nonsense timestamps, only to discover that the reason they do that, is a mixed usage of aware and naive dateteime objects.
In my opinion, making explicit use of aware datetime the norm as much as possible, is a good thing. And removing prominently named functions that spit out naive ones, is a good first step.
Let's look at the listed usecases:
> An application may be designed in such a way that all dates and times are in a single timezone that is known in advance.
Okay...so put that TZ info somewhere it's easy to find, like a configuration Singleton, and use it.
> It is also a best practice to store naive datetimes representing UTC in databases.
Okay...read them from, and into datetime objects using UTC as their timezone information.
The bottom line is: There shouldn't be "naive" datetime objects, or at least they shouldn't be a common thing. Conceptually, datetimes always have a timezone attached, they are worthless without one.
Sometimes conceptual beauty must unfortunately reckon with long-established reality. It's basically very late to be doing this.
But I'm okay with that.
The problem with de facto standards is: The fact that something became de facto standard, doesn't make it a better idea. The only thing it means is that it is now even harder to get rid of it.
> It's basically very late to be doing this.
Yes, it is. But not yet impossible. Languages and libraries that wait forever instead of ripping off the bandaid at some point, eventually become so crusted with such early problems, they become problems themselves.
Glad that it's finally being addressed at the language level.
One of my projects added all of the code in PyPI to GitHub, it would be interesting to see how prevalent the use of “utcnow()” is.
The memory and processing differences between the two seem marginal, at best. Especially when it's at UTC ("plus zero" doesn't take too much processing).
So I'm gonna have to ask for benchmarks on this one.
And even if it was slower one must ask if having a confusing concept like a naïve datetime is worth it. Obviously Python is all about the "programmer productivity at the expense of some performance" trade-off.
I'd be more concerned if this was one of those changes where there isn't a good way to write reasonably straightforward code that will run in relatively old versions of Python as well as newer ones.
[1] https://peps.python.org/pep-0615/