HN Explorer

If you're using YYYY in your JVM service or %G in anything, fix it now

228 points by pinaceae ↗ HN
The single character G instead of Y took down Twitter's API today.

If you're using YYYY in your JVM service or %G in anything, fix it now. You're very likely using the wrong year format.

date Mon Dec 29 00:44:45 EST 2014 date -u "+%G" 2015

ISO 8601 week numbering has 2015 start this week.

74 comments

[ 3.8 ms ] story [ 135 ms ] thread
warrenm
source for this statement?
spatulon
https://twitter.com/jmhodges/status/549430032616017921

Also https://twitter.com/jmhodges/status/549431554879938560 :

"(The correct formats are "yyyy" and "%Y". Yes, this matters and is happening right now. "Week-based years" are a thing.)"

mrweasel
Not much of a source really, it's just some random dudes tweet, with no real explanation. Also it assume (correctly in most case) that %G or YYYY isn't what you where looking for.

If you're going to throw something like this out on the internet, I would like it to be more detailed. Just blasting it out on Twitter is pretty useless.

Reproduction: `date -u "+%G"`, sorry but that does EXACTLY what it's suppose to: [http://pubs.opengroup.org/onlinepubs/009695399/functions/str...].

Depending on your operating system it's even documented. OpenBSD has a pretty good documentation for it, while Mac OS X is somewhat lacking, but to the point where you wouldn't use it without checking what it actually means.[http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man3/...]

Why would you even pick %G if that wasn't what you meant, %Y seems more obvious. Using YYYY I can understand, it's a bit of gotcha that YYYY is one thing and yyyy is something less. It even test correctly more often than not.

dmourati
* Random dude who worked at twitter and cited the people currently working at twitter who found and fixed the problem during the time he tweeted it.
quicksilver03
Just to reformat your commands:

    date
    Mon Dec 29 00:44:45 EST 2014

    date -u "+%G"
    2015
reverend_gonzo
Here's some more information:

The year number of the ISO week very often differs from the Gregorian year number for dates close to 1 January. For example, 29 December 2014 is ISO 2015-W1-1, i.e., it is in year 2015 instead of 2014.

http://en.wikipedia.org/wiki/ISO_week_date

Specifically, you should be using +%Y instead of +%G when passed into date, and 'yyyy' instead of 'YYYY' when used in Java.

pyre
I don't think that %F is what you're looking for:

  ➜  date +%F
  2014-12-29

  ➜  date +%G
  2015

  ➜  date +%Y
  2014
danielweber
I always wondered "who in the world uses that stuff?" when I read date library docs. Now I know.

I also remember at one startup when I asked the payroll guy how many pay periods there were that year, he scoffed "26, it's always 26." Well, not with years having fractionally more than 26 ^H^H^H 52 weeks there aren't.

nirvdrum
Every year has more than 26 weeks ;-) But to your point, I suspect that's why a lot of companies have switched to semi-monthly payroll. It's always 24 pay periods but the pay date itself can shift.
danielweber
English is weird. I meant "it's not true that there are 26 pay periods every year, because years have more than 52 weeks, which means there must be a 'leap pay-period'" every once in a while." But your interpretation was entirely valid based on the way I typed mine.
brlewis
I think the gp was referring to "years having fractionally more than 26 weeks".
paulschreiber
California law requires that you be paid every two weeks.
jlmorton
Nope - It requires that you be paid at least twice per month.

http://www.dir.ca.gov/dlse/faq_paydays.htm

jsprogrammer
My wife works for a UC school and they post-pay on the 1st of each month. I've skimmed the article you've posted and it would seem that such an arrangement is not allowed.

Do you know how they get away with this, or have any other insight?

brianwawok
Schools are sometimes treated more like a government entity and exempt from certain rules and taxes like FICA, right?

That is my only guess...

porpoisemonkey
Teachers are exempt from this rule.

From here: http://www.dir.ca.gov/dlse/Glossary.asp?Button1=P#profession...

professional exemption

A person employed in a professional capacity means any employee who meets all of the following requirements:

Who is licensed or certified by the State of California and is primarily engaged in the practice of one of the following recognized professions: law, medicine, dentistry, optometry, architecture, engineering, teaching, or accounting, or

tlrobinson
Why are those professions exempt?
porpoisemonkey
Honestly I don't know. The objective answer to your question is because that's how exempt professions is defined for the law.

If I had to hazard a guess but I would suspect it's because people in those professions are expected to be salary. Salary is easier to budget out in longer increments as the base pay amounts per pay period typically don't vary. (annual salary / pay periods = pay amount)

IgorPartola
This is exactly why we should adopt a 13 month calendar, with a Year Day as the last day of the year. If it's a leap year, there'd be two consecutive Year Days. These days should be special: they are holidays, so everyone has them off. This immediately solves the payroll problem for most people: just don't pay them for this day (obviously adjusting the numbers such that if you make $60,000/year, you still get that).

For those who work on this day (medical professionals, etc.) the rate is different (being a holiday and all), so they just get paid by the hour, regardless of what they do throughout the year.

Your payroll can now be weekly, biweekly, monthly, or whatever you want: you still get the same amount of money per month. This is nice because you don't get weirdness with things like rent/mortgage payments which are monthly. For example, for those who get paid biweekly, you get two months a year where you get an extra payment. This is silly and leads to more difficult accounting and budgeting.

Monthly is nice, but lots of your expenses are likely one a week-by-week basis: groceries, entertainment, etc.

Weekly is AFAIAC the nicest: it is the easiest to budget because it's as close to the common denominator as possible. However, it is not the LCD, so it still has the weird problem of getting more money in some months vs others.

jxf
It'd be pretty tough to retrofit "a day that doesn't belong to any month" into existing computer systems.
saalweachter
Yes, but ... most existing computer systems are subtly broken for date/time in different ways anyway. Leap days, leap seconds, daylight savings shifts, timezones in general, the above mentioned "year of the week". You'd be substituting a certain number of "this is subtly broken" bugs for a certain number of "this is obviously not implemented correctly" bugs.
jrochkind1
I like the idea of a day that everyone has off, but I'm not sure you realize how drastic a change that would be. NO places of business open, no gas stations, no drug stores, no hospitals, no toll collectors, no trains, plains, or mass transit -- no police or firemen?

Of course, changing the calendar isn't a realistic plan anyway (Americans still won't accept the freaking metric system, that's too much change! and of course there would be costs to having a calendar different than the rest of the world, unless we get EVERYONE to change). But making _everyone_ have off is even less realistic. So payroll systems would still probably need to account for the days.

But I do like the idea of a day or two that literally everyone has off, and would subscribe to your newsletter.

tankenmate
Well Arabic countries traditionally use the Islamic calendar (luna based) rather than the Gregorian calendar (solar based). If memory serves Myanmar (nee Burma) uses a different calendar as well.
jrochkind1
There are a variety of different traditional calendars from different cultures, but I strongly suspect business people in, say, Saudi Arabia, Dubai, or UAE use the same calendar we do. I in fact strongly suspect _everyone_ in those countries do, for their day-to-day non-religious date-related tasks.

Wikipedia confirms "The Islamic calendar is now used primarily for religious purposes, and for official dating of public events and documents in Muslim countries". http://en.wikipedia.org/wiki/Islamic_calendar#Uses

hrasyid
What's your point? They can afford to switch from Islamic calendar to Gregorian, because the majority already use Gregorian. It doesn't mean that it will be easy to make a country switch to a 13-month calendar that nobody else use
wbronitsky
I agree. The solution to a software problem is for sure to change the way the entire world views dates.
IgorPartola
I don't exactly know why I am responding to the comment with the snidest tone... but it's not just a software problem. Everyone has to budget their money, and a 13 month year makes that easier. Businesses spend huge sums of money trying to reconcile their finances because of our current calendar. The amount of human effort wasted on looking up what day of the week something is, is monumental. Aside from inertia, our current calendar doesn't have much going for it. No, I have no delusion that we'd ever switch, but it doesn't mean that staying with the current calendar is actually a good idea.
chrismcb
Right, so instead of having a solvable problem with today's tools, let's throw in a new problem that needs new tools to be solved.
chrismcb
Even in years that have fractionally more than 52 weeks, there is still only 26 pay periods. You only get 27 pay periods when the fractions add up, which may or nit be in a leap year.
smackfu
Yeah, it happens every year, unless the 31st falls on the last day of the week.

Also, the 'Y' character was a relatively late addition to Java, in v1.7. Before that it was an invalid value.

claar 

  Unix date command: Use +%Y instead of +%G
  Java: Use yyyy instead of YYYY
  Python: Use %Y (ISO-8601 year number is not available)
  PHP: Use Y instead of o
Please contribute other languages in replies and I will add them.

(edit: This post had originally suggested the post's title be changed to mention Java, thus vitno's reply below)

vitno
although this post references Java, this doesn't just affect java. This affects anything that uses ISO 8601.
hobbified
Anything at all with strftime: Use %Y instead of %G. It's a C function, but many languages (Perl/Python/Ruby/etc.) expose it through their own datetime libraries.
bbcbasic
.NET: Use yyyy (ISO-8601 year number is not available)
pyre
From strftime(3):

       %G is replaced by a year as a decimal number
          with century. This year is the one that
          contains the greater part of the week (
          Monday as the first day of the week).
s_kilk
I'm puzzled as to why the behaviour of %G would be desirable. Can anyone offer an example?
valevk
It is used to get the year, which a week number belongs to. The current week number (1) belongs to 2015, even though it's 2014. Here, check out this calendar [0]. We are already in week 1 of year 2015. That's why you get the year 2015.

Why is it useful you might ask: when you order a time-series by week number, or if you are grouping by week numbers, you would need the week number of course. But if the data is spanning over two years, you would need also the year for grouping, because you would add up weeks from different years. If you use week number + current year you get errors around the end of some years.

[0] http://www.calendar-365.com/2015-calendar.html

gpvos
Note that these are international (ISO) week numbers. US-American week numbers are different, I'm told.
mrweasel
I image that it would be handy in some forms of accounting software. Some countries are very found of using week numbers, pretty much any government employee in Denmark uses week numbers, rather than actual dates, so you end up needing to known which year the current week belongs to, in this case 2015.
valevk
Exactly. In Germany it is very common to talk in terms of "Kalenderwoche" (calendar week). For planning, accounting, holidays...
fdsfsdaffdsa
yoda time seems not to be affected:

  * Y       year of era (>=0)            year          1996
  * x       weekyear                     year          1996
  * y       year                         year          1996
ddlatham
If you mean Joda time, then "xxxx" is also affected (which is correct behavior):

  public static void main(String[] args) {
    long now = System.currentTimeMillis();
    System.out.println("YYYY: " + DateTimeFormat.forPattern("YYYY").print(now));
    System.out.println("yyyy: " + DateTimeFormat.forPattern("yyyy").print(now));
    System.out.println("xxxx: " + DateTimeFormat.forPattern("xxxx").print(now));	
  }
prints

  YYYY: 2014
  yyyy: 2014
  xxxx: 2015
Dylan16807
Nobody will use xxxx by accident, so it's not 'affected' by any problem/confusion, it's just there doing what the developer intended.
abrodersen
Clearly, Yoda has 900 years of experience dealing with datetime bugs.
smackfu
Y and y are only different for BC dates in the Gregorian calendar, so using the incorrect one generally isn't going to cause problems.
protomyth
Unless of course you read the docs and are using YYYY or %G for a proper use. Gave me a bit of a scare with that headline given one of our scripts uses %G for reporting purposes. I thought there was an error in %G for this year.
raverbashing
And what's an example of proper use of this?

(According to man on Mac OSX for G - "This year is the one that contains the greater part of the week (Monday as the first day of the week)"

mason55
Say you're doing weekly data processing that always needs to be Sun - Sat. You'd want yesterday to be the start of week 1 for 2015. Or any sort of data slicing and dicing that allows viewing by week, you'd want everything from Sun - Weds of this week to roll up to week 1 of 2015.
danielweber
> Say you're doing weekly data processing that always needs to be Sun - Sat

Shouldn't that be Mon-Sun? The Sun-Sat week we are currently in is majority 2014, but the Mon-Sun week that started today is majority 2015.

Sharlin
Yes; ISO weeks are always Mon-Sun.
protomyth
The OpenBSD manual under strftime (referenced in the date man page) shows "%G is replaced by the ISO 8601 year with century as a decimal number." Which leads to http://en.wikipedia.org/wiki/ISO_8601 and http://en.wikipedia.org/wiki/ISO_week_date which says "The ISO 8601 definition for week 01 is the week with the year's first Thursday in it."

Our example is a couple of activity reports we send to the government where we need to have full weeks and they want calendar years and not budget years[1]. I seem to remember payroll for some uses the same scheme (yep, we pay weekly). Plus some internal reporting the the ISO date makes nice since we have a 2 week break which includes Xmas and New Years.

1) These are a bit of odd ducks actually. Most government reporting we do is based on Federal Fiscal years which start Oct 1. I remember in the 90's one foxbase function that did a similar algorithm except centered on Oct 1 and defined the first day of the week as Sunday.

jakobegger
I like to look at weekly sales numbers, so in my SQL query I just group by iso week and iso year.
femto113
They're proper almost any time that week is the fundamental unit of aggregation. I've commonly used these for naming folders (or more recently AWS S3 buckets) for storing log files or periodic data dumps. For example a structure like

  /year/week/date.log
If you don't want uneven sized weeks you want to use the ISO year and week. Here's an example with date, we're already in the first week of 2015:

    % date '+ %G/%V/%F.log'
    2015/01/2014-12-29.log
and 3 days from now still will be:

    % date -v+3d '+ %G/%V/%F.log'
    2015/01/2015-01-01.log
Contrast that with the "non-ISO" year and week:

    % date '+ %Y/%W/%F.log'
    2014/52/2014-12-29.log
in three days that rolls over to a new week

    % date -v+3d '+ %Y/%W/%F.log'
    2015/00/2015-01-01.log
And we'd have 2 folders or buckets containing a less-than-usual number of log files.
jontas
This also seems to affect PHP when using the formatting character 'o', which is PHP's version of the ISO-8601 year number. Using 'Y' instead is returning 2014.

    php > echo date('Y');
    2014
    php > echo date('o');
    2015
teleclimber
Yup. Got bitten by that last year.

When reading the docs [1] it's pretty obvious which one should be used so I am left wondering why I ended up with 'o' in my code. I suspect I copy-pasted something from some random blog post or whatever. Serves me right. :(

[1] http://php.net/manual/en/function.date.php

gred
We ran into this bug in production today (I don't work at Twitter).
fredgrott
interesting little bug...data formats are always fun for interesting bugs like this
[dead] starwarsrebles
umm that is weird all I did was put in g for googleand it brought up this website saying that there was hacking going on I did not want all of this
artenix
Any confirmation if that was the error which took Twitter down yesterday?
Tharkun
Thank you, kind stranger, you just saved my job.