HN Explorer

Apple Developer Website Update

282 points by danielsiders ↗ HN
Email from Apple

Apple Developer Website Update

Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then.

In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon.

222 comments

[ 3.4 ms ] story [ 258 ms ] thread
jamesjyu
Yep, I can confirm I just got this as well.
dfamorato
+1
thrush
Same. It'd be helpful if they'd have a permalink for the email, as I'd imagine a lot of developers (including myself) have/will post to hacker news.
makomk
I figure that's probably why they don't have a link for the e-mail - they don't want their developers posting it publicly.
[deleted]
(comment deleted)
tlongren
"In the spirit of transparency". Right, Apple.
jordanthoms
In the spirit of transparency, we're giving you vague warning that some information might have been accessed _4 days ago_
logicallee
That's not a long time to receive a letter like this. That's as fast as Apple instantly responds to anything, esp. considering the weekend.

And the site was down, so it was clear something was going on.

It is also extremely transparent in the sense that people were wondering this exact thing even earlier today, and now received a response detailing that this is an extremely severe breach, as opposed to something else. What more do you want on a Sunday?

jordanthoms
Well I'm giving them a hard time due to the massive schadenfreude, obviously. Still, this is very vague about what the 'sensitive personal information' is (passwords?), what was encrypted, what was hashed, was it using a proper hashing scheme, etc.

And announcing it just because people have started to speculate is damage control, not taking responsibility.

k-mcgrady
>> "And announcing it just because people have started to speculate is damage control, not taking responsibility."

It's possible it took them a few days to figure out exactly what was taken and waited until they had as much info as possible to make a statement. I doubt Apple would be following blogs during a situation like this with someone making the decision: "oh, people have started to speculate about what's happening - I think we should make a statement."

agent123
Shadenfreude? Do you mean something other than just enjoying their misfortune?

And 'taking responsibility' means solving the problem. They have told us what they are dealing with. What more do you want from them?

tsm
These details are befuddling. "Personal information was encrypted and cannot be accessed". It can't be accessed because it's somehow stored elsewhere, or it can't be accessed because of the encryption? That is, does the intruder currently own my encrypted data?

I'm also disappointed that it took them 72 hours to tell us anything, and that the update doesn't even have a timeline for when the site may be back. "Soon" is meaningless.

tlongren
As I interpreted it, yes, the intruder does have your encrypted data.
[deleted]
(comment deleted)
num3ric
I hope you were using good encryption Apple. This guy cracked 400,000 md5 hashed passwords: http://www.youtube.com/watch?v=0WPny7wk960 He says even with password salts, it could be done.
ptwiggens
Unsalted MD5 has been extremely easy to crack for years. Not sure why the claim would be impressive at this point.
laumars
To be fair, unsalted md5 hashes are only one step above storing passwords in plain text.
MrGando
I could crack your unsalted md5 password with my Gameboy Color.
Watabou
Yeah I'm confused why companies tells us DAYS after something serious happened as opposed to right away. I can understand waiting a day but 3 whole days?! I just don't understand the delay.

It's our data, we should have the right to know what happened to it.

badclient
It can take more than a day to know what happened to your data.
lambda
Yes, but it doesn't take more than a day to know that an intruder had accessed the system in a way that may have compromised your personal information.

Basically, once the problem was serious enough that they felt like they needed to take the site down, I'm pretty sure they knew which machines had been accessed (or at least may have been accessed). They knew that some of those machines had developer's personal information. They could have posted as much up front, rather than waiting 3 days to do so.

ptwiggens
"I'm pretty sure."

No, they likely took the portal down as soon as they knew there was a breach. Highly unlikely they left it up while they investigated, and it takes time to figure out what happened and how much information was taken.

What motivation would there be to wait anyways?

lambda
> No, they likely took the portal down as soon as they knew there was a breach. Highly unlikely they left it up while they investigated, and it takes time to figure out what happened and how much information was taken.

They still haven't said anything about how much had been taken.

My point is they knew how much could have been taken. They knew what machines were at risk; hence taking them down. If those machines that were at risk had sensitive personal information, they should have notified the people affected right away, not three days later.

Taking the site down, with no indication of why, and waiting three days to tell people that their personal information may be at risk (and remember, the possibly compromised information includes credit card numbers, as there are a number of things you need to pay for in your developer account) is just crazy.

You should be upfront and transparent when the breach first occurs. Of course you don't know exactly what has been compromised; but they are still being plenty vague even three days later. If they had posted three days ago what they posted today, it would be a lot more reassuring.

[deleted]
(comment deleted)
larrys
"why companies tells us DAYS after something serious happened"

Companies are people. And all the relevant parties involved in handling this may not be accessible to make a decision as quickly as needs to be done. Or at least quickly enough to satisfy all people.

Do you feel you suffered any harm in particular by the delay of three days?

Watabou
I'm mainly complaining about the delay in telling us anything.

What I would love is an update whenever they suspect an intruder has accessed sensitive information. Many websites like Dropbox and last.fm do have a server status where they tell us if they have any planned maintenance or just general status of the server. Why can't Apple and the rest of the big companies do that?

Also, Apple first said it was just regular maintenance. I'm just confused as to why they said that instead of telling us the truth.

justinsteele
Likely because if you say "we are investigating a possible data leak" and then end with "we discovered it was an undocumented maintenance event by someone on the engineering staff, we have added more detailed logging as well as a better maintenance process so we can be clear about this in the future", many people will think the worst. It's unfortunate.
ajasmin
Apple should have kept us informed from day one.

But, in their defense it may take days to ascertain what exactly happened.

Once a system is compromised it's nearly impossible to trust anything about it. Auditing the logs, and reviewing the code, crypto, and the mix of platforms they're using (see https://news.ycombinator.com/item?id=6078854) in order to understand what data could be accessed and fix all vulnerabilities is not an easy task.

The PlayStation store was not down for such a long time without reason.

nwh
Uh, how does this "encryption" work?

For the website to show these details (and it does, in part, use these details in the interface) it must be able to decrypt these on the web applications side. Ergo the keys for decryption must also be on the server or derived from the users passwords, both of which make the use of encryption a fairly worthless venture.

ED: As another commenter mentioned in an earlier thread, lots of other AppleID facing applications are gone as well ( https://ecommerce.apple.com/ ), so it would be interesting to find out how far this all goes. The websites don't seem that far disconnected from the information in iCloud.

jordanthoms
I think the 'sensitive personal information' is passwords, so the names, addresses etc which are displayed on the site were not encrypted.
[deleted]
(comment deleted)
fleitz
Maybe they phoned for a ransom after breaking in?

Your post is pure speculation and depends heavily on what Apple means by 'sensitive'. I'm guessing that Apple means your CC numbers, certs, shared keys, etc.

Possibly also your support tickets, your bank numbers, etc.

As for how encryption works, I'd suggest Applied Cryptography by Scheiner. I think there's a problem in that book about Bob keeping speculative posts to Alice secret from Eve. After reading that book, I'd suggest applying for a job at Apple to give you first hand knowledge of what they're actually doing and then you could make an informed judgement about what may or may not have been exposed.

nwh
I am aware how crypto works. I was commenting on there being no source for a key in this situation, rendering it a fairly useless venture.
stan_rogers
In a case like this, there would (normally) only be one secret to find. (IVs, or at least the information IVs are derived from, and so forth would be stored with the data.) That doesn't necessarily make it easy to break if the key was securely handled, but it does make it catastrophic if the key is determined.

Apple's email essentially says, "we don't think they have the key, but..." And a complete investigation, along with changes to the system and an opportunity for users to change data as soon as possible under the new system, is the right way to go about it.

wisty
AFAIK, it was a white (or grey) hat hacker. See the comment on the techcrunch article. He nabbed Apple employee details, as proof. But they are probably worried that someone else has also done it.
yulaow
Can it be related to the similar attack on the ubuntu forum? Maybe it was a single group of hackers targeting the servers in which they know a lot of developers have an account
kyro
No reason to be up in arms, folks. They've got the marketing team working on this too.
kabdib
Never underestimate the skill of the 2AM janitorial staff, too.
aaronbrethorst
The marketing team runs the portal.

https://twitter.com/chockenberry/status/358310019537715200

threeseed
Marketing has financial responsibility but engineers still develop and maintain it. And there is lots of cross collaboration with iTunes Store and Apple Online Store teams.

WWDR is all about evangelising the platform to a technical audience. Of course it belongs in marketing and not engineering i.e. it involves road shows, presentations, reach out activities etc. Not everyone that is technical are developers remember.

0x0
Well at least it was "only" the dev center, and not iCloud and iMessage!
Sucker
It's not uncommon for developers to use the same credentials for both their developer account and their iTunes/iCloud account. I do.
0x0
Good scary point (I don't). At least I've pre-emptively disabled Find my Mac to avoid another Wired-like remote wipe. Imagine that being pushed to all ios and mac developers at once!
objclxt
Apple use a centralised credential system. If I was to speculate I would assume what's happened here is the metadata attached to the developer portal (developer contact information, company info, etc) was compromised, not the actual Apple ID. This would explain why Apple are saying no 'sensitive' information (passwords?) was taken.
pdknsk
Hmm so it only takes a few days to "completely overhaul" their developer systems? Not sure I believe this is what they're actually doing. And why haven't they updated their server software before? I know mistakes can never be completely avoided, but this seems slightly amateurish for a company with so much cash.
k-mcgrady
>> "Hmm so it only takes a few days to "completely overhaul" their developer systems?"

They've already been down a few days and haven't said when they'll be back up so it's impossible to know yet how long it'll take.

objclxt
> this seems slightly amateurish for a company with so much cash

I know there are people here who probably have been in the start-up space for all of their working life, but never underestimate how piss poor architecture can be at big companies.

I would place a large amount of money that every single person here who has done a stint at a large corporation has a horror story about terrible, awful architecture, outdated practices, and shoddy insecure software. And yes, that includes companies like Facebook, Google, etc.

stock_toaster 

  > I know there are people here who probably have been in the start-up space for all of their working life, but never underestimate how piss poor architecture can be at big companies.
Systems that are over 5/10/etc years old and are part of a multi-million/billion dollar system even more so.

  > who has done a stint at a large corporation has a horror story about terrible, awful architecture, outdated practices, and shoddy insecure software.
My god that rings so true. ;_;
_delirium
There was some discussion in previous threads about how their systems seem (like many others, to be fair) to be a taped-together mess of old perl scripts and server-side includes and CMSs and who knows what else. So they're probably trying to come back online with a cut-down version that presents a smaller attack surface and contains only components that have been vetted in the last 10 years, which is overall not a bad idea.
toyg
>a cut-down version that presents a smaller attack surface

... or simply a cut-down version they know how to run.

"Mhh, what did that old cron job do, again?"

"I don't know; it was written by Mike, he left 3 years ago."

"oh ok, so not important then."

<12 hours later>

"OMG OUR DB EXPLODED AGAIN! That script was clearly essential! Can you rewrite it??"

"Er, I've looked at it: 2000 LOC of obscure Perl. Gonna take a while..."

threeseed
I've worked on some of those systems. This isn't true.

Most of their web apps are WebObjects/Java apps. Sure some of them would be classed as legacy but it is hardly different from most other enterprise companies. And their systems are completely separate from each other e.g. iTunes, Apple Online Store, ID auth, Developer Portal.

plasma
Is the encryption not good enough (and I mean in general when sites get bcrypt'd passwords stolen, etc) when owners are worried the encrypted data is in the hands of intruders?

As a developer I'd still be concerned if I lost such data when encrypted - so I understand - but what measures can be put in place so that as a developer/site owner you're without uncertainty that the encrypted data will never be encrypted by the attacker (eg, would take trillions of years).

tater
I bet Forstall did it.
0x0
I wonder if the hackers managed to get code signing keys out? Ultimate jailbreak?
RKearney
I highly doubt Apple keeps their master keys anywhere near a public facing web server.
0x0
The dev center seems to be able to autogenerate code signing certificates at least. But maybe those can be revoked via online checks. I wouldn't mind having a wildcard enterprise cert with a 20 year expiration =)
Turing_Machine
Access to an API to get certificates from a different server one at a time is different thing from having the actual private signing key.
0x0
Certainly, but an unrestricted code signing certificate would be quite useful too (until they are revoked)
tcas
Which wouldn't take very long. I'd be amazed if Apple wasn't using a hardware security module to store their intermediate signing keys and logging the keys generated from it.

I wonder where the CRL for the dev certificates is -- we might see an update to it soon.

Turing_Machine
Mmmm... maybe. Okay, let's say you can sign code as anyone, even Apple itself, and create rogue apps.

Now, how do you use that information to compromise iOS devices? You probably won't be able to get it in the App Store, and the iOS devices won't install from anywhere else. You could make an Ad Hoc distribution package, but for that you need to know the UDID of each device and convince your victim to download the rogue app from somewhere other than the App Store.

0x0
You can install .ipa files easily via http and mobilesafari, and enterprise certs are valid for ALL UDIDs :)

So (again, assuming no revocation), you could set up a web based alternative app store, or re-sign cracked apps/games, or just enjoy being able to run code on your own devices (and distribute to others without going through the app store) without maintaining the $99/year subscription, or you could start linking/redirecting unsuspecting web browsing users to install malicious apps (would only need 1 confirm click)

nwh
If you have Apple's code signing keys, you can boot whatever software you want to on an iPhone. This has never been possible outside of Apple, except in the very rare cases of bootrom exploits (see limera1n).
jchimney
I read the comments dismissing apples handling of this. What would you have expected them to do? There is a LOT of forensics going on probably even now trying to get a handle on this. A massive corp isn't going to make an announcement until they have some idea what they're talking about. In my books 4 days is a very quick first announcement from a company of this size.
peterkelly
Good to see some transparency on Apple's part here.

I understand this must be a very challenging situation for them to deal with, and I appreciate the notification. As I'm sure many developers feel, I'd like to know more details, but I'm sure these will come in due course.

peterkelly
Also, let's not jump to conclusions until we have more info. This is clearly a very serious and sensitive incident.
markdown
It's a strange world we live in when every time we're told by a big corp that our personal info was compromised, we're grateful for being told.

This is the worlds most cashed-up corporation. They could buy entire countries, yet they made a conscious choice not to update their server software or hire more competent sys-admins.

There shouldn't be a way for them to gain marketing wins out of this. There should be a law requiring notification when personal information is compromised.

peterkelly
This isn't about marketing. It's about a security breach. And security breaches take time (> 2 days) to properly investigate and report.

It's entirely possible that this is a massive oversight by Apple and they've been extremely negligent in their security policies.

It's equally possible that there's some bug (that either you or I could easily have made the mistake of introducing) that's resulted in this being possible.

Let's calm things down, give it a few days, and then evaluate. Nobody can make an immediate judgement about the exact causes of problems like this. If you're making judgements at this point, you really have no idea whether you're being accurate or not.

And yes, if it turns out to be negligence on Apple's part, I'll be very angry. But let's wait and see.

pdknsk
>> It's about a security breach.

Well, you're very much mistaken. It's not about a security breach at all. If you read carefully, you'll notice it's merely about a "security threat".

:)

stack0v3erfl0w
> And security breaches take time (> 2 days) to properly investigate and report.

We can see that they did their job correctly by the overwhelming amount of details they provided us with.

markdown
My argument was that you were impressed by the press-release. Your opinion of Apple was improved such that you made a post in public expressing your admiration of them for telling you that they'd lost some data that you'd entrusted to them.

This shouldn't happen.

When my 4yr old tells me he did something "wrong" without any prompting (eg. "Dad, I broke your phone"), I'm impressed because he didn't have to out himself, but did so because it was the right thing to do.

Large corporations rarely think in terms of right and wrong... they have a duty to their shareholders, and nobody else. As far as their shareholders are concerned, they shouldn't release damaging information unless not doing so could potentially negatively impact profits down the line. So when Apple tells you they messed up, they're only doing so because they're worried you might find out some other way, which would be worse for them. They aren't doing it out of the kindness of their hearts.

Now if there were a law requiring the disclosure of incidents such as this when personal information is compromised, then Apple wouldn't have a choice in the matter, and they wouldn't be able to fool people like you into thinking they're awesome when they just lost your data through negligence.

> It's entirely possible that this is a massive oversight by Apple and they've been extremely negligent in their security policies.

They just said they'll be updating their software. Why would they do that if they didn't think that that would make the data safer. It's pretty much an admission that they chose not to update the software earlier ie. someone made a decision to use outdated software.

ptwiggens
Your post basically amounts to a conspiracy theory.

"It took them 3 days to tell us something happened. Obviously this means they would have kept it secret if it were at all possible."

It takes time to figure out what happened in a breach. That doesn't mean that Apple is some evil company trying to hide the fact that there was a breach.

markdown
No, I made no mention whatsoever about how long it took them to tell us. Did you even read my comment?

> That doesn't mean that Apple is some evil company trying to hide the fact that there was a breach.

I never said that they were trying to hide anything. Again, did you even read my comment?

> Obviously this means they would have kept it secret if it were at all possible.

Well yes, that is logical. A corporation would keep such a thing secret if they had a guarantee that there was no other way people could find out. There are good people working at Apple, but they are not Apple. A corporation doesn't have morals. It will not damage itself and threaten profits just for fuzzy feelings, any more than it will drop the price of the iPhone 6 to $20 because that would be a good thing for the poor.

jpttsn
> if they had a guarantee that there was no other way people could find out

Do you believe this is unique to corporations? Would "real people" always do the right thing even if they had a guarantee nobody would e able to tell?

markdown
Yes, many people would. I for one.
divtxt
> if it turns out to be negligence on Apple's part, I'll be very angry

I doubt you'll need to get angry - because if it's negligence, you/we won't be told.

fauigerzigerk
There is a law that requires them to notify people immediately. I suppose their excuse for waiting three days will be that they did not know for sure whether any personal data was acquired by the intruder. There's going to be a debate on what they needed to know to "reasonably believe" that data had be accessed. http://info.sen.ca.gov/pub/01-02/bill/sen/sb_1351-1400/sb_13...

(b) Any person or business that maintains computerized data that includes personal information that the person or business does not own shall notify the owner or licensee of the information of any breach of the security of the data immediately following discovery, if the personal information was, or is reasonably believed to have been, acquired by an unauthorized person.

(c) The notification required by this section may be delayed if a law enforcement agency determines that the notification will impede a criminal investigation. The notification required by this section shall be made after the law enforcement agency determines that it will not compromise the investigation.

stack0v3erfl0w
Transparency? After 3 days? Transparency would be telling us right away (with an update as soon as they know more). They should also tell us what kind of info was taken and what does sensitive information mean to them since they don't seem to be sure about that.

> Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed.

Edit: and seriously, what does this "updating our server software, and rebuilding our entire database" mean?

stock_toaster
> Edit: and seriously, what does this "updating our server software, and rebuilding our entire database" mean?

my WAG: paving systems; reinstalling the server OS; updating packages; restoring db from known clean backup; replaying logs/binlogs that are known clean?

That would be my guess.

yapcguy
> "In the spirit of transparency, we want to inform you of the issue."

Ha, what a joke, I can't help laughing at that.

With so many third-party Apple developers drinking the kool-aid, and dreaming of becoming rich, I'm not surprised Apple treat them like fools.

Just yesterday on Twitter, some developers were speculating that the site was taken down to be updated with new SDKs for exciting new features and product lines.

georgemcbay
I'm pretty sure in a fully honest world that would read "In the spirit of not risking running afoul of the security breach statutes in various states we do business in, we want to inform you of the issue".

Apple is great a lot of things but I don't think even their most ardent fans would argue that transparency is one of them.

jpdoctor
> Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed.

So they can't rule out the possibility that sensitive personal information, which cannot be accessed, has been accessed. Got it.

Apparently our intelligence, which cannot be insulted, has been insulted.

jordanthoms
Alternately, inside the reality distortion field developers’ names, mailing addresses, and/or email addresses is not sensitive personal information.
Turing_Machine
Anyone who has my name can find my email address with a simple Google search. My mailing address is on all kinds of public records.
robryan
"Give me a list of all registered developers email addresses" is a little harder though.
ptwiggens
What use is the list without passwords?
robryan
Spamming them with something that may interest iOS developers.
bdcravens
Pretty sure it'd be easy to sell that list for several thousand dollars, over and over.
ptwiggens
How are names, mailing addresses, and email addresses sensitive personal information?

I would imagine that for most of the people signed up, it wouldn't be that hard to track down their name and email just from knowing the name of their app.

CoryG89
Name and e-mail, I'm kinda with you. Everyone who uses my app knowing my current mailing address I look at a little bit different.
VeryVito
One of the understood requirements of publishing an app on the App Store is that developers must provide some means for customers to contact them directly (support page, email address, etc). If you're selling apps on the app store, people can already peddle their wares to your email account.

So yeah, developer's names, addresses and emails are not secrets by any means. Why would anyone buy an app from someone they had no means of identifying?

gurkendoktor
iOS developers != App Store vendors. There are plenty of developers who work on other people's apps for a living.
kristofferR
By "sensitive personal information" they probably just mean passwords and credit card information, not names, email addresses and mailing addresses.
_delirium
Passwords could be hashed, but credit-cards are the big one you have to keep in plaintext. If you want to bill the card without asking for the number to be reentered, there's no way to avoid storing the number and expiration date. PCI does mandate that you keep less than necessary to initiate a new charge, though: you are not allowed to store the 3-digit verification code from the back of the card. Future charges from the same vendor can go through based on the stored information (without re-sending the verification code), but charges from a new vendor would need the code, so this is intended to make it harder for someone who stole the saved information to initiate a new charge. A loophole is that in-person charges do not use the verification code, so someone could use the saved information to fabricate physical cards, and try to use them at stores (the U.S. doesn't typically use either chipped or PIN-protected credit cards, so cloning a card from the number is relatively easy, prevented more or less only by the heuristic fraud-detection algorithms).
kybernetyk
Purchases of developer memberships are handled through Apple's online store. And that is still up.
weaksauce
There also is the bank account information for apple created payments to the developers... that could be part of the compromised items.
conradev
That is in iTunes Connect, completely separate from the developer portal.
[deleted]
(comment deleted)
alexbell
It was down last night though, possibly related?
cmircea
Probably took it down to check and patch.
threeseed
The Apple Online Store is a completely separate system. Different servers, very different codebase.
[deleted]
(comment deleted)
wmboy
> ...but credit-cards are the big one you have to keep in plaintext. If you want to bill the card without asking for the number to be reentered, there's no way to avoid storing the number and expiration date.

Not necessarily, if you're using a payment gateway that supports token billing...

JoshGlazebrook
Yes, but then you're going to start running into the "don't keep all your eggs in one basket" situation. Sure there are companies out there that will store that information for you, so you don't have to worry about living up to the standards of storing it, but what if that company is compromised? You can't just say "oh they should just let someone else deal with storing that information..." SOMEONE is going to have store the actual information in the end.
dools
When it's the bank that issues you the token, the buck stops there. The bank has the card details anyway.
[deleted]
(comment deleted)
X-Istence
Credit cards don't need to be kept in plaintext. I'm a big supporter of having the frontend encrypt the data using a public key. The private key is stored only on backend servers.

So now if something needs to kick off a billing process the frontend sends a signal using a defined service method (preferably something so simple that it is secure) and then the backend goes off and decrypts the data followed by doing the actual processing required.

If the frontend and backend are on two seperate networks, and the frontend is only allowed to talk over TCP/IP port 5930 for example to the backend, now you have reduced your attack surface tremendously while making customer data more secure.

nmcfarl
The last system I worked on that actually stored it’s own CCs (Lately it’s been all tokens, all the time, for me) did roughly this with the added, fun, feature no SSH or remote shells on the backend box.

You sent messages to add or charge our client’s credit cards from the front end - on the ultra-simple protocol, to the 1 (!) open network service on the backend. And that’s all the input it took from the network.

If something more complex was needed someone with much higher permissions than I went to the server room and typed into the terminal. Which really minimized attack surface.

jamesaguilar
Couldn't you also do something like this? Store each user's key, transformed in such a way that you could get it back only if you have the password. Serve the key to the user's session on login (maybe -- depending on how long you store the session, you may want to require password reentry to initiate any charges). Encrypt all sensitive data with the user's key, such that only that logged in user can read it back.

The major drawback would be the same as the benefit. Since you can't know your users' CC numbers, you also can't make recurring charges.

Pipe-dream solution to that -- you should be able to get a token from your payment provider that authorizes you and only you to charge the CC. Should that token leak, you barely even need to revoke it. It can't be used by anyone else, because you need both the token and your company's api secret to charge anything, and even then, all they can do is send (easily refundable) money to your account.

EGreg
Even better - how about simply doing a form of OAuth with the service provider? A token would be authorized for recurring billing or anything else. I think Verified By Visa is an example of something like that...

If such a provider could also SMS you on your chosen # to confirm the purchase then the system would be secure!

jacobwcarlson
> A loophole is that in-person charges do not use the verification code, so someone could use the saved information to fabricate physical cards, and try to use them at stores (the U.S. doesn't typically use either chipped or PIN-protected credit cards, so cloning a card from the number is relatively easy, prevented more or less only by the heuristic fraud-detection algorithms).

This is untrue. The magnetic stripe contains significantly more data than what is printed on the card and much of that Discretionary Data (DD) is used during authorization of 'card-present' transactions.

duskwuff
Specifically, the magnetic stripe contains the CVV1, which is used for card-present transactions. The number on the back is the CVV2, used for card-not-present transactions.
dools
Most banks provide the ability to get a billing token to avoid having to store card details at all. In order to do recurring billing you only need to store the token and the CVV on the back.
jonknee
I would imagine Apple would not trust the ability to bill its customers to a third party. They have touted their collection of CC numbers for quite a few years, it's treated as an asset to the company.
duskwuff
You do not need the CVV for recurring billing. Indeed, card processing standards explicitly forbid you from storing the CVV at all.
[deleted]
(comment deleted)
nayefc
> Future charges from the same vendor can go through based on the stored information (without re-sending the verification code), but charges from a new vendor would need the code, so this is intended to make it harder for someone who stole the saved information to initiate a new charge.

That's not true >"charges from a new vendor would need the code". Online credit card transactions only require a credit card number, expiry month & expiry year. The verification code is optional and is used as a fraud check / deterrent. Payments with an invalid verification code are highly suspicious. Therefore, when Apple (or any merchant) asks for the verification code initially, it passes the initial fraud tests and the card is stored as a "verified card" (or perhaps, only verified cards are stored). Further charges are then most probably legitimate (since it passed the previous fraud check).

hdivider
What I find slightly unnerving is that Apple didn't make this clearer.

If they know that credit card information was not affected, they should say that. E.g. "Sensitive personal information (such as credit card data) was encrypted and cannot be accessed, ..."

It's reasonable to suppose that 'sensitive' includes credit card information, but as it stands it's something we have to interpret.

I'd suggest we all check our credit/debit card statements more often over the coming days, just to be sure. =)

27182818284
>What I find slightly unnerving is that Apple didn't make this clearer.

Apple is not a startup. They were ranked 6th in Fortune 500 for 2013. They are going to be rehearsed, political, and vague with their descriptions. Were you actually expecting them to release a postmortem on their blog with a link to the GitHub repo with the fix?

wfn
Sure, but I don't think it's a bad idea to raise the median level of expected standards. It seems reasonable to ask for clearer reports and some kind of a postmortem.

One of the hopes for the notion of startups searching / optimizing in these kinds of niche spaces (transparency, communication on a more personal / no-bullshit level, whatnot) might be that these kinds of optimizations will hopefully change what is to be expected from IT businesses in general (at least in terms of communication and so on.) One can at least hope..

Someone
If they say X wasn't leaked while it was, they will open themselves for litigation. A company of Apple's size cannot afford to make mistakes there. So, they have to put in weasel words such as "such as".

Also, this isn't a postmortem (yes, we may never see one, it is premature to comment on that _now_)

hdivider
I do see your point. It just seemed odd to me that they didn't take that simple step to clarify what is arguably the most pressing question on everybody's mind: is my CC/bank data safe?

This is most likely just me being too paranoid and literal, of course. =) In general I'm not too disappointed with how they've handled this - it could've been far worse.

bennyg
I'm imagining bank account numbers over CC info/passwords was the sensitive part.
ars
Bank account numbers are not secret. If you write a check to someone they have your account number.
jordanthoms
Well they are in this weird space of being able to be used to pull money from the account though ACH/demand drafts, so they _should_ be secret, but then as you said they are on the bottom of every check. Which is super weird, and not the case in other countries' banking systems.
smegel
I think I would rather someone have my CC number than my home address (which would be the same as my mailing address).
ams6110
You're not in the phone book, then? County property tax database? I mean, your home address is not exactly something you can really hope to keep secret.
jpttsn
Why?
smegel
Credit cards can be cancelled. Home addresses plus other information could be valuable to identity thieves. Maybe it's not that big a deal, but the idea of hackers (criminals?) knowing my name, email and home address seems a bit creepy.
jpttsn
Perhaps if it were a case of a targeted attack I'd agree Identity theft etc doesn't really scale as well as credit card fraud.
[deleted]
(comment deleted)
[deleted]
(comment deleted)
clarky07
names, email adresses, and mailing addresses aren't particularly sensitive. These are all pretty easy to get for most people without hacking anything.
_sabe_
"The intruder had good intent with trying to "secure" our personal information. But despite nothing being hacked, as it was only a 'threat', we still need to tear down and build up the system from scratch again. In the spirit of transparency we've waited 72 hours before giving you this nonsense bullshit. Please note that some (that is all) of you will from now on get regular viagra offerings in cyrillic. Good for you!"
rimantas 

  secure
  verb [ with obj. ]
  
  2 succeed in obtaining (something), especially with difficulty
llamataboot
"First of all, this does not affect iTunes customer accounts—this is a different system and all iTunes customer information is completely safe, Apple told me.

It’s also important to note that the hacker did not get access to any app code or even the servers where the app information was stored. The hacker also did not get access to any credit card information.

The only thing that the hacker could have gotten access to was the names, email addresses and mailing addresses of the developers. At this point, Apple doesn’t know if the hacker even managed to see that information. Worst case, that is all the information they would have seen, according to Apple."

http://www.loopinsight.com/2013/07/21/apple-comments-on-deve...

jpttsn
Apple apparently doesn't agree that, in context of the data they have on you, your name, mailing address or email address qualifies as sensitive.
vmarsy
If the intruder is a patent troll-er, getting developers’ names and mailing addresses can be pretty harmful.
k-mcgrady
Nobody would take that risk. It wouldn't be difficult to figure out how they got all the names and addresses and would seriously backfire.
ptwiggens
How exactly would that be harmful?

If a patent troller wants to find out who is behind an app, they would go through the legal system and use a subpoena.

Literally no reason whatsoever for them to hack a website to get it.

smallsharptools
Until I see an email from Apple myself I will not see this info as credible.
Turing_Machine
Mine just arrived as I was reading this. I'll bet yours will be there soon.
alayne
I got email from Apple about 15 minutes ago with this text. It probably takes some time to send to everyone.
kalleboo
I got the email then came straight to HN to check the discusson.

  Received: by 10.50.11.202 with SMTP id s10csp27972igb;
          Sun, 21 Jul 2013 16:01:44 -0700 (PDT)
  X-Received: by 10.68.172.34 with SMTP id az2mr27321730pbc.201.1374447703980;
          Sun, 21 Jul 2013 16:01:43 -0700 (PDT)
  Return-Path: <developer_bounces@insideapple.apple.com>
  Received: from msbadger0508.apple.com (msbadger0508.apple.com. [17.254.6.162])
          by mx.google.com with ESMTP id yo6si9958126pac.15.2013.07.21.16.01.43
          for <XXX>;
          Sun, 21 Jul 2013 16:01:43 -0700 (PDT)
  Received-SPF: pass (google.com: domain of developer_bounces@insideapple.apple.com designates 17.254.6.162 as permitted sender) client-ip=17.254.6.162;
  Authentication-Results: mx.google.com;
         spf=pass (google.com: domain of developer_bounces@insideapple.apple.com designates 17.254.6.162 as permitted sender) smtp.mail=developer_bounces@insideapple.apple.com;
         dkim=pass header.i=@insideapple.apple.com;
         dmarc=pass (p=REJECT dis=NONE) d=insideapple.apple.com
briandear
Ok, don't think it's credible. The rest of us who've received the email can talk about it in the meantime.
kalleboo
Any idea what "rebuilding our database" means? Reticulating the splines? I hear those go out of alignment sometimes.
Sanddancer
There are lots of moving parts in a database, and lots of places one can hide back doors for later access -- triggers, etc. If you're not sure how hard you got owned, nuking, paving, and auditing is usually the best course of action.
diminoten
Is there any other source that this actually happened besides from a guy posting some text on HN?
hdivider
I also got the same message. It's a real email from Apple. Also, expect to see stories about this on the usual tech sites within minutes.
michaelxia
Thanks Apple! This email was super helpful, now I know exactly whats going on.
__david__
I sense some sarcasm here, but I don't get it. Yesterday the site was just down, now we officially know why and have some sense of a timeline. It seems reasonable enough to me—what more do you want?
michaelxia
Then let me clarify...

1) The site was down since Thursday, not yesterday. 2) You can't "overhaul" and expect to deploy "soon", so wtf are you doing apple? 3) "Soon" is not a timeline, at least not in the real world. 4) What info got owned? What could be effected?

dphase
This may explain some strange occurrences I had yesterday.

Starting at 7am, I received an Apple ID password reset request every 4 hours and 19 minutes, ending last night at midnight.

This Apple ID is also the login for my personal developer account (several years old). My developers IDs used for work never received a password reset request.

ptwiggens
I highly doubt the hackers plan was to get email addresses and try to brute force from there... just doesn't make sense.

If you search Google, people are all the time receiving password reset emails going back years, even repeated ones.

Email addresses are in the clear all the time, and I've never heard of them being considered sensitive before. You should assume everyone has your email address.

thepumpkin1979
`rebuilding our entire database`. So the database was... destroyed...?
tater
I'd let them slide on that if they had the brass to announce it was running on CoreData.
peterkelly
I understand everyone's frustrations with this, and the fact that Apple haven't been immediately clear on exactly what happened. As a developer, I too am alarmed by what has happened.

But these things are complex, and it takes time (i.e. a few days) to fully and properly evaluate what has happened and what information leaks/security breaches have occurred.

Let's give this a reasonable amount of time, and only then pass judgement on their handling of the case.

I don't want to appear like an Apple apologist - and maybe it is a serious fault on their side. But in fairness I do think it's reasonable we give them time to evaluate & respond appropriately.