Ask HN: Did you encounter any leap year bugs today?

524 points by sjr1 ↗ HN
After a frantic scramble this morning, our billing team has finished patching a bug which erroneously was charging our monthly subscribers for an extra day.

All test suites are passing now, and SRE has scheduled a postmortem after the QA confirms the fix in 2028.

497 comments

[ 0.36 ms ] story [ 380 ms ] thread
Can't you just backdate some servers to confirm the fix?
That's generally risky to do nowadays because of things like TLS certificate expiration.
In a few hours you'll be able to backdate to "yesterday", with very low chances of hitting cert expiry issues (but I wouldn't be surprised if OP's issue involves components outside of their control or ability to test end-to-end)
I should have written something along the lines of "validity date ranges" instead of expiration: you're much more likely to run into problems where you run into a certificate that was issued in the future relative to when you think is now.
Not directly related to leap year, but a couple weeks ago I set up a script for testing notification behavior that used libfaketime to simulate runs at different times.

I guess this might be less-trivial if you've got a distributed multi-service architecture and perhaps also depend on APIs that aren't under your control in the first place.

Set up a backdated timeserver and use that to sync time
We had test failing that had to be skipped and worked out in the background.
One cleanup script broke because Python doesn’t have a clean way to subtract a year, and if you do now.replace(year=now.year-1), you get a ValueError when now is 2/29.

It’s easy enough to address. There are various StackOverflow posts on such things. Here is one: https://stackoverflow.com/questions/54394327/using-datetime-...

What is your definition of "subtracting a year"? Seems like that's a relatively ambiguous operation without more specification.
Since they mentioned a clean up script, I assume they could easily just use 365 days for that use case.
But then it'll be off by one day for the rest of this year. And someone will notice that they no longer have March 1 2023-March 1 2024 in their chart, but March 2 2023
It's a cleanup script. I bet nobody cares it's off by one day. Also I doubt a cleanup script has a charting function.

Everything is use case dependent. Sometimes the use case is unimportant enough that mistakes are okay.

While that's true, I'm sharing scar tissue here not hypotheticals.

There's often someone out there who interprets such things as a 3 month, or 1 year retention policy and that it means they're entitled to look at the entire range whenever they want.

Can you think of any situation where subtracting a year from today's date is ambiguous when today isn't, well, today?
Yes - on any day, subtracting a year might mean subtracting the average length of a year (which is a bit more than 365 days), or wanting the same day and month number in the previous calendar year, or wanting the same semantic difference ("last Monday of the month in January"), to name a few possible meanings.
Moving bank/festive holidays, first Monday of the year(, first work day of the year not Monday if that's NYD and bank holiday), lunar occasions.

'subtract a year' is imprecise and has many meanings, if what you want is 'same day, same month, previous year' then say that and do that, that's conceptually `date.year -= 1` not `date -= 1 year`, and will have this bug.

You could subtract 365.25 days, but then you're left with a new problem: just because you can amortize a leap day over four years doesn't mean that you get an extra 6 hours each year.
And as I just learnt elsewhere in this thread, it would actually be three four-hundredths less than that anyway, i.e. 365.2425, since only one in four centenaries is a leap year.
It is ambiguous, but one part that's pretty unambiguous is that the result should be a date, not a crash.
Plenty of thought has gone into this. Look at what database date functions do when you ask it to subtract 1 year. I will agree that there is not one answer.
It's not that python doesn't have a clean way to subtract a year, it's that "subtract a year" is imprecise. There's a clean way to subtract 365 days, and there's a clean way to set the year one year earlier. But if you're doing the second thing, is python supposed to silently change to March 1 when you change the year from a leap day? There's no way around handling edge cases.
APy should just figure out what I would have wanted to happen
This has been a factor in contract law for longer than computers have existed. "Subtract a year" or "Add a year" aren't really imprecise, it's just that there are two ways you can interpret it: 365 days, or 12 months. When adding, you get the same result: end of February, the 28th. Subtracting, you get one of March 1st for 365 days and February 28th for 12 months.

Most jurisdictions use the 12 month standard, fwiw. I believe that's the ISO ruling as well, but I wasn't able to confirm that.

Which type of year? Sidereal or solar? Did you account for axial precession?
To be a bit blunt, these questions are entirely irrelevant, since you know the answer as well as I do: the calendar year, the one Western law and international business uses uniformly. It's very clever of you to mention that other concepts of the year exist. Kind of. Would have been more clever if you'd realized that they aren't germane, any more than the Islamic, Jewish, Indian, or Chinese years are.
> There's a clean way to subtract 365 days

What is it? Does it attempt to return a datetime that's the same time as the input but 365 calendar days earlier (ignoring leap seconds? compensating for time zone changes?), or does it subtract 31536000 seconds from the current datetime? Because those aren't necessarily the same and it's ambiguous which one you mean by "subtract 365 days"

Okay I am having really odd undefined behavior in Python in UART communications that were working just fine yesterday... My boss joked it could be a leap year thing but at this point it wouldn't surprise me. Switch over to using Rust and no issues at all
> now.replace(year=now.year-1)

Yeah but this is bad code. Python certainly does have a "clean" way to subtract a year, you subtract a datetime.timedelta object.

timedelta doesn't take "years" as a parameter, for the reasons others have listed here. It's ambiguous what subtracting by a year means, and there's no real sensible default either.
relativedelta will do it:

  from dateutil.relativedelta import relativedelta
  one_year = relativedelta(years=1)
which is why you should use arrow
A fiber cable was cut so we had no internet for the morning, probably not leap year related.
The mcdonalds order waiting thing malfunctioned, displayed de52hg04 instead of 088 and I had to wait a lot longer for my order since it flew under the radar for a while until I spoke to them :)
Just hope the King of Sweden doesn't make another February 30. https://www.timeanddate.com/date/february-30.html
Every time I go to timeanddate.com I feel like I see a link to the Leap Day page (https://www.timeanddate.com/date/leap-day.html) that shows the meme-infamous boyfriend-checking-out-another-girl couple, except she's proposing to him! Obviously this happened earlier that day since they're wearing the same outfits, and I can't help but feel bad for her knowing what's coming but unable to warn of the impending train wreck.
I'll take a February 30 if it means every other month is 30 days too, and we just get the extra days as universal, extended winter holiday where no one can legally be required to work because there is no December 31st or January minus-fourth.
My sister is staying in a hotel and all the keycards stopped working.
What happens in a situation like this? Does the staff issue physical keys? Do the doors even have manual locks? Do the staff walk every body to their room?
About a year ago I stayed in a hotel and the door lock started misbehaving one morning while I was at breakfast. Went to the front desk, got a new card, went back to the room and discovered it still didn't work. After doing that two more times, someone was sent up to the room with a mysterious, palm-sized device with a USB cable hanging off it, which they plugged into a well hidden USB port on the bottom of the lock. The device performed some black magic, and after about 30 seconds a light on the lock changed colour from orange to red and it started functioning correctly again.
This also happened on Leap Day 2020 with Onity locks used in Crown Plaza Hotels. See https://i.imgur.com/GI5A3jW.png. I wonder if it's the same issue never fixed? Can you share with us the hotel chain and/or the lock manufacturer? Thanks.
The Casio F-91W doesn’t account for the year, it showed today’s date as Thursday, March 1st.
Same with my colleagues most recent top model Samsung Galaxy Watch :]
Odd, mine is showing the right date.
Are you sure? My F91W doesn't even ask for year in settings. How would have it known it is a leap year or not.

Some other models (not F91W) does track year.

I was responding to the claim about the Samsung Watch not showing the right date, I unfortunately have never owned the older kinds of "smartwatches" :)
Are they using a third party watch face? My Galaxy Watch 4 has no issues.
I have a custom watch face on my GW4 and it's showing the 29th
Thank you so much for the reminder, that would have screwed me up today :)
I noticed this too and clicked this thread wondering if it would show up! Still an awesome watch.
I just looked at my watch and indeed the date is wrong. Thanks!
One of the largest food store chains in Sweden had their entire card payment go down because someone forgot to handle leap years!
Wikipedia says they are "the second largest retail company in the Nordic countries[citation needed]". Pretty big company anyway and embarrassing for them. Would love to learn more about what the bug was, but I guess they will never say.

https://en.wikipedia.org/wiki/ICA_AB

Tbh I was just too lazy to find its place in the rankings. It sure is a big business anyway
This isn't even one of the weird years!

Is this why we can't have nice things?

> A number of New Zealand petrol pumps stopped working on Thursday due to a "leap year glitch" in payment software, fuel stations and the payment service provider said.

https://www.reuters.com/world/asia-pacific/leap-year-glitch-...

I'm sure it's more complicated than "we didn't think about leap years", but it certainly sounds pretty amateurish.

Old programmer rant: In my day, we fixed the Y2K bug - we went to the future and back several times a day!

In your day people weren't doing everything with string ops in JavaScript.
It does sound amateurish eh? I do most of my business logic inside Microsoft SQL Server Stored Procedure, and MS SQL just takes care of date pretty well.

But then, come to think of it, you don't need to use SQL to have good date logic. C# and Java both have excellent date handing libraries. I don't know about C++, but I'd be surprised if there was not a modern date library for C++, so I am of the opinion there should be no excuses for software not to work on the 29th of Feb.

Yes if you're writing your own date/time routines, dealing with dates as seconds since epoch, or anything like that you're going to get burned.

Use date handling functions and libraries that have been developed by people who know how to do it and have been battle tested.

> but I'd be surprised if there was not a modern date library for C++

The standard library now includes <chrono>. AFAIK: It was mostly written by Howard Hinnant. He now has more date/time libs that expand upon <chrono>: https://github.com/HowardHinnant/date

(comment deleted)
(comment deleted)
Payroll software at work had a one-off planned maintenance day today, presumably to avoid worrying about any bugs.
lucky it didn't fall on a payday eh?
Today is a payday for us, worked great!
I'd suggest not waiting 4 years to publish a postmortem, if you're serious
Yes.

> During the morning on Thursday, no ICA store in Sweden could accept card payments. Instead, you had to use cash, Swish or pay via their app.

> The reason behind the problem was an internal problem in the payment systems at ICA as a result of an extra day in February, leap day.

ICA being the biggest grocery store chain in Sweden

And people in sweden look at me weird when I say I keep some cash around.

At least it wasn't 3 days like when Coop's provider got hacked. They also handle our pay checks at work… makes me feel so safe :D

Heard from a friend in China: the age calculation portion of the app to schedule a marriage certificate had a bug where they subtracted 22 (legal minimum age) from the year, which resulted in 2002-02-29 which doesn't exist. The app intends to compare this against the user's birth date. The error handling code assumes all errors are from the comparison. The app then rejected all marriage certificate appointments by complaining that the users are too young to marry legally.
Haha, that would be quite the appropriate place to put one of those "Please wait and try again" error messages.
"look, today the math just doesn't work out, try tomorrow"
This is often the result of consulting an astrologer in Asia for a marriage date, to be fair.
Maybe some people trying to get away with buying anniversary gifts every 4 years
How do leap day birthdays get handled in general? How is the right age iterated every year?
The most common approach I’ve seen is to alias them to Mar 1st.
Sadly, I can say from experience the American draft does not count the number of birthdays celebrated to figure out if you're eligible to serve.
You got drafted when you were five?
Both Feb 28 and Mar 1 are commonly used to celebrate birthdays.

AFAIK there is no firm convention. Feb is more natural ("My birthday is in February"), Mar is more logical (the 60th days of the year).

Is March more logical? Feb 29 + 1 year = Feb 28.
What’s the significance of 365? Each Earth year has just a smidge under 365.25 days.
365 days is what people commonly think of as a year. So if you're born Feb 29th and celebrate your birthday one year later (whether you call that 365 or 365.25), you land on Feb 28th. Then again, folks born between Jan 1 and Feb 28th of a leap year celebrate their birthdays 366 days later by calendar days.

Anyway, I'm not sure there's any more or less logical date to use and Feb 29th babies seem to choose both about equally:

> “I love when people ask me, ‘Do you celebrate on Feb. 28 or March 1?’” said Raenell Dawn, a co-founder of Honor Society of Leap Year Day Babies. “I get to tell them, ‘Both, because I can.’ But I’m a February baby; I was not born in March.” An informal poll of the society’s members showed about a 50-50 split between the two dates, said Ms. Dawn, who is celebrating her “Sweeter 16” by turning 64 this year.

https://www.nytimes.com/2024/02/28/style/leap-year-explained...

Celebrating on the 28th makes no sense. That is when you celebrate your Birthday Eve every year so your birthday is the next day, whatever that is.
You can make the same argument that March 1st makes no sense, as that is the day of your birthday hangover, the day after whenever your birthday is.
But 1 day is still 24 hours.

If you want to celebrate your birthday at the moment earth reaches the same spot around the sun as when you were born, then we’d all have the same issue - we’d have to celebrate it 6 hours later every year, reseting every 4 years.

February 29 is very much a new day… which simply doesn’t exist on non-leap years.

Think of it this way: if Feb 29 didn't exist that year, it would have been March 1.

If we're being scientific I guess all of us should move our birthday up by 1 day on leap years, but that seems annoying and not sure anyone really cares enough to.

No we should move our birthday celebration up by 6 hours every year. But that would be only relevant if we were celebrating at the exact hour of our birth.
We already don't move them for timezones. I was born at 3am but in France, when I'm in the US I don't celebrate the day before.
By this logic, all birthdays would slowly drift as leap years pass.
Take DOB + n * 365.25 and then pick whatever day that falls in? That way it shouldn’t drift overall. Though, I guess it would imply that what day of the year people celebrate on, would be off by one day on leap years compared to what it is on other years?
This formula doesn't consider how we sometimes skip leap years, and sometimes have leap seconds, so it's vastly imprecise.
So make it more precise. DOB + n * 365.2421.
365.2425
365.2421 is more correct.

Nasa explains:

365 +0.25 - 0.01 + 0.0025 - 0.00025 = 365.24225

https://pumas.nasa.gov/examples/how-many-days-are-year

>Omitting a leap year every 4000 years

Wait, is this right? omitting every 100 and adding every 400 should be enough.

EDIT: the 4000-year rule is just a proposal for now. So 365.2425 is correct.

None of us will live long enough to see the 4000-year rule become a problem. I don't even expect to be around for the next 100-year rule.
I asked JavaScript:

    const { format } = Intl.DateTimeFormat("en-UK", { month: "short", day: "numeric" });

    const date = new Date(2024, 1, 29);
    console.log(format(date));
    // 29 Feb

    date.setYear(2025);
    console.log(format(date));
    // 1 Mar

    const year = 1000 * 60 * 60 * 24 * 365.2425;
    console.log(format(new Date(2024, 1, 29).getTime() + year));
    // 28 Feb
I guess both are logical by some definition of logic.
You're begging the question. [1]

"Feb 29 + 1 year" is verbatim restatement.

---

You can say Feb 29 + 365 days = Feb 28. (And Feb 29 + 730 days = Feb 27.)

---

EDIT:

Note that in the context of birthdays, people use "calendar years," not "unit of time which is ~1 revolution around the sun."

Birthdays aren't celebrated every X million seconds after the moment of birth.

They are celebrated the same day each calendar year -- notwithstanding the fuzzy concept of "same day" for incongruent calendar years. There's no singuar right answer, but that is the core question: "what is the same day next (calendar) year?"

[1] https://en.wikipedia.org/wiki/Begging_the_question

(comment deleted)
(comment deleted)
> Feb 29 + 365 days = Feb 28.

Correct.

> And Feb 29 + 730 days = Feb 27

Incorrect. It's Feb 28 again. In a normal, non-leap year, if you add 365 days then you get back to the same date.

Sorry, typo I meant that Feb 28 + 730 days = Feb 27 (sometimes)
> Birthdays aren't celebrated every X million seconds after the moment of birth.

As a surprise for a friend of mine, I threw him a gigasecond party on the day he turned 1000 million seconds old. Yes he was surprised, and a good time was had by all.

Many people say: Happy turn around the sun! and 365 is not the length of a year is the length of some years so, who knows... I'm against time zones and pro 28 days months and a couple of free days :D
> what is the same day next (calendar) year?"

When I wrote "+ 1 year" I meant year to represent 365 or 365.25 days, not "the same day next year.":

  >>> from datetime import date, timedelta
  >>> year = timedelta(days=365.25)
  >>> date(2024, 2, 29) + year
  datetime.date(2025, 2, 28)
You may have interpreted it as begging the question, but it was not my intent—though I admit my formulation was ambiguous.

> Note that in the context of birthdays, people use "calendar years," not "unit of time which is ~1 revolution around the sun."

You've never heard someone say they've celebrated another trip around the Sun? I literally have a photo from 2007 of my daughter in Montessori celebrating her birthday by holding a globe and walking around a candle representing the Sun. I think most people probably don't really think about whether it's a calendar year or astronomical year because for most people, they are usually equivalent and it doesn't matter.

But that's all beside the point. All I meant to point out is that I disagree that March is more logical. I don't think logic points us to one month or the other. You may feel March is more logical, but I don't accept your priors, so for me March is not more logical than February.

Feb 28 is definitely too early, you haven't lived a whole year yet since your birthday. So March 1 it is. It's not your birthday but you can celebrate having made it another year.
(comment deleted)
Well 'year' can mean two different things though: 'calendar year' or 'solar year'.

Your calendar year birthday would be March 1 since one full calendar year has elapsed.

But Feb 28 is the nearest day to one full solar year, assuming you were born before about 6pm local time on Feb 29.

Two-day long birthday it is! :)

(comment deleted)
Relevant question for driving, voting, marrying, drinking. I'd assume just the date is compared.

If you are born on 29th, on future 28th you are considered "too young", regardless whether a 29th exists or not. On future March 1st you are "old enough" again regardless.

If a 29th exists you are old enough already on that date. Drinking beer in Germany at 16, I guess in some countries at 20 could be relevant cases. For the more common minimum age of 18 for many things, the limit is reached always on March, 1st because a 29th cannot exist.

so … many bullets were dodged today
Opinionated devs are the best devs.

Tell us about your last marriage?, lol

Yes, Hesai LiDAR [1] bug is grounding cars.

[1] https://pandaily.com/hesai-technology-addresses-lidar-produc...

Fascinating - why does a LiDAR involve the date?
For the survalence records

Telling base where, and when, you were

Shouldn't that just be a unix timestamp or something? You'd think resolving that to a date time could be done in the UI instead of in the business logic
Unix time has discrepancies whenever leap seconds occur (several times in my career aquiring geophysical data).

If you're measuring | controlling objects in the physical world (cars, rockets, etc) then you should not use unix time - those glitches will happen and instantaneous computations will go kooky.

https://en.wikipedia.org/wiki/Unix_time#Leap_seconds

But so does resolving to a date. I don't see how resolving to a date which cares about leap days fixes any of that.

You should use a monotonic clock with an arbitrary starting point anyway, unless you need some kind of synchronization between devices, but you probably wouldn't use unixtime there anyway.

> But so does resolving to a date. I don't see how resolving to a date which cares about leap days fixes any of that.

So why bring it up then?

> You should use a monotonic clock with an arbitrary starting point anyway

Sure. We started doing that more than 50 years ago now when broad area geophysical surveying started off.

> unless you need some kind of synchronization between devices,

Can't see the problem - there are ways of syncing base station records against aircraft | boat | vehicle records in post processing .. all the stations, fixed or mobile, use a monotonic epoch based record structure that hold channel data and any sync marks that are broadcast by whatever means - raw GPS time serves well enough for a grain of 1.5 seconds, other marks can be used as required.

I think I'd have used time-since-power-on; your infotainment system could take an RTC offset to get that to human dates.
LiDARs and other sensors typically need to be time-synchronized with the rest of the robotic system. I wouldn't be surprised if the bug is related to that.
(comment deleted)
No, but some of our software writes data to rotating directories named after the date, and while doing some manual debugging on a test system, it started failing to create these directories the first time it rotated on Feb 29 UTC. Turns out it just happened to run out of disk space at that time, but I had myself convinced that it was a leap year bug for over an hour. :)
LOL, a good example of correlation does not equal causation.
It's not. There was no correlation at hand here - this was just a coincidence
I would think it's an edge case of correlation, but still correlation (100% of the time it has had this error, it was on a leap day, 0% of the time it was not)
It might appear as a correlation due to sampling error, but the two variables have no dependence with each other. There is no correlation with it being February 29 and the bug that was described.
I think that's exactly what OP is trying to say when they say 'correlation, not causation'. There is no causal link between 29th February and the bug, but they happened at the same time (were correlated, i.e. had the relationship that they happened at the same time).

This is a fairly typical informal usage of the word correlation in my experience, and while it might not be _technically correct_ (I don't know, I'm not an expert), it's an often enough used idiom.

"Correlation does not equal causation" is a very important and pertinent concept in statistics, and I think we should aim to actually understand the concept instead of just trying to warp the meaning of the words to fit the example

The bug simply was not "a good example of correlation does not equal causation" as any statistician would use the phrase. That's my only point. OP is free to type whatever words he wants into the comment box and press send

anecdote is the singular of date, a correlate is a coincidence.
Python.

    cls = <class 'datetime.datetime'>, data_string = 'Feb 29 04:55:03.687' format = '%b %d %H:%M:%S.%f'
    E       ValueError: day is out of range for month
Interesting... I suppose that is because there is no year? What year does it default to? Can you show your exact line of code?
That doesn’t seem incorrect; given that no year is specified, it seems like it’s evaluating the constraint in the context of an implicit default year. (1970? 0CE?)

The confusing part, to me, is that Python would consider the above string to be parsed into a date in the first place, given that it has no year.

confirmed. and interesting/unexpected! this breaks:

datetime.strptime('Feb 29 13:37:06.942', '%b %d %H:%M:%S.%f')

edit: added code example. import datetime from datetime first obvi

That's because you didn't specify a year.
As mentioned by sibling comments, it's because you're not specifying a year. If you change the day to the 28th you'll see that it defaults to the year 1900:

  >>> datetime.strptime('Feb 28 04:55:03.687', '%b %d %H:%M:%S.%f')
  datetime.datetime(1900, 2, 28, 4, 55, 3, 687000)

  >>> datetime.strptime('Feb 28 13:37:06.942', '%b %d %H:%M:%S.%f') 
  datetime.datetime(1900, 2, 28, 13, 37, 6, 942000)
That makes it weird though, because 1900 was a leap year? I sort of get it, but it's a slightly odd and inconsistent decision.

Edit: no it's not, it's absolutely correct, leap years just aren't as simple as I thought!

1900 was not a leap year... It's 0 mod 4, yes, but it's 0 mod 100 and not 0 mod 400
Yep, thanks, I just wasn't aware of the latter rule at all. Time to re-read Falsehoods perhaps!
Crikey - more helpful to me is the page linked from there - https://learn.microsoft.com/sl-SI/office/troubleshoot/excel/...

> However, there is still a small error that must be accounted for. To eliminate this error, the Gregorian calendar stipulates that a year that is evenly divisible by 100 (for example, 1900) is a leap year only if it is also evenly divisible by 400.

> For this reason, the following years are not leap years:

> 1700, 1800, 1900, 2100, 2200, 2300, 2500, 2600

I had no idea!

"We establish that a bissextile [366th day] shall be inserted every four years (as with the present custom), except in centennial years. So the years 1700, 1800, and 1900 will not be leap years. Assuredly, the year 2000 will have an extra day in it." -- Greg XIII, 1582
My Casio F91W I assumed would know that year YYYY is leap, and would show 29th as Date. No, it showed 1 as in March (it doesn't show months). I had to manually set it back to 28th so that tomorrow it shows correct date.

To be fair, it doesn't ask for year anywhere in settings. It simply doesn't know what year it is.

This one is rather specific, but a game rhythm based Final Fantasy game called Theatrhythm Final Bar Line is simply not allowing people to play today because it has an internal system that awards prizes for specific days and they didn't handle the case of what to do when it's on a leap day. You can boot it up but can't actually play the game as a result.

Not working on the game or anything but found it moderately amusing as someone who owns the game!

That's why they had to release FF7R2 today. It was planned all along /s
Maybe? My paycheck direct deposit didn't show up until almost 7am. Normally it hits right at midnight.