HN Explorer

Outlook.com DNS records disappeared (outlook.com)

132 points by smcleod ↗ HN
:-( something went wrong Sorry, we can't get that information right now. Please try again later. If the problem continues, contact your helpdesk<p>---<p>dig outlook.com<p>; <<>> DiG 9.8.3-P1 <<>> outlook.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 62938 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0<p>;; QUESTION SECTION: ;outlook.com. IN A<p>;; Query time: 33 msec ;; SERVER: 10.51.20.100#53(10.51.20.100) ;; WHEN: Fri Nov 22 09:37:02 2013 ;; MSG SIZE rcvd: 29

96 comments

[ 3.4 ms ] story [ 133 ms ] thread
shaydoc
unreal, i was editing a document, when it bombed on me, guess i should been editing locally!
mrmondo
I especially loved their status page during the outage:

  current health
	CRM	No issues
	Exchange	No issues
	Identity Service	No issues
	Lync	No issues
	Office 365 Portal	No issues
	Office Subscription	No issues
	Rights Management Service	No issues
	SharePoint	No issues
amaks
That's an executive console.
gamblor956
I have no problem accessing Outlook.com, Office 365, Skydrive, or any other Microsoft website. The problem many be regional...or it may be with your ISP.
jldugger
There's a lot more than one person complaining on twitter. I've seen reports from New Zealand, Kansas, my linode in Dallas, Oregon, etc.
Einherji
Yup. I'm in Iceland and every microsoft service is unreachable, except my regional version of microsoft.com
thisisdallas
It looks like a lot of their other services are down too.

Edit: Also, the Xbox One is launching tonight so if their Live servers are down...it won't be pretty.

joezydeco
Isn't it launching in Europe in just a few minutes?
a3n
Ah, maybe it had something to do with the Xbox One launch. Updates, under pressure, mix and serve.
beat
It'll be interesting to find out the root cause, assuming it ever comes out. Malice or stupidity? Did someone do an oops within the Outlook team (or some other team), or are they under attack?
mrmondo
My guess is someone messed up when making a change to their DNS
beat
DNS problems are at least partly affecting the microsoft.com homepage as well.
AsymetricCom
My guess is they changed DNS so they could keep the service up while cutting off an attacker's route.
manacit
I've heard from people know that an AD GPO was pushed causing firewalls to refuse DNS requests, which would fit in line with what everyone was/is seeing.
than
Down in MN.
kristofferR
Edit: It's back up now.

Yup, it's pretty broken, though it is still up some places.

Outlook.com:

  Grapevine TX, United States (bigguy.gte.net) - Down
  Sacramento CA, United States (CalWeb) - Down	
  Providence RI, United States (Verizon) - Down
  Pasadena CA, United States (Mindspring) - Down
  Mountain View CA, United States (Google) - Up
  Vancouver BC, Canada (Radiant) - Down
  Recife, Brazil (Hotlink Internet) - Down
  London, United Kingdom (Legend Comm) - Down
  Lille, France (Nordnet) - Down
  Merzig Saarland, Germany (Probe Networks) - Up
  Milan, Italy (BT Italy) - Down
  Ankara, Turkey (TTNET) - Down
  St. Petersburg, Russia (Uni of Tech & Design) - Up
  Karachi, Pakistan (Supernet) - Down
  Delhi, India (Tikona Infinet) - Down	
  Bangkok, Thailand (TOT) - Down	
  Petaling Jaya, Malaysia (Clear-Comm) - Down
  Beijing, China (China Unicom) - Down
  Sydney NSW, Australia (Exetel) - Down
  Collingwood VIC, Australia (Pacific Internet) - Down
  Auckland, New Zealand (ICONZ)  - Down
https://www.whatsmydns.net/#A/outlook.com
cclogg
Did something change yet? I'm in Vancouver and have no problem going to outlook.com... that's what is down, right?
ne0phyte
outlook.com, xbox.com, microsoft.com, live.com, ...

Pretty much all sites are down for me (Germany).

natevancouver
It depends on what your local DNS has cached. Also, I’m finding that 1 out of 5 of their DNS servers is still responding from my location (not too far from Vancouver).
mrmondo
Yeah, dropping all your NS records from your domain will do that;

  $ dig NS outlook.com

  ; <<>> DiG 9.9.3-rpz2+rl.13214.22-P2-Debian-1:9.9.3.dfsg.P2-4 <<>> NS outlook.com
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 8538
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0,   ADDITIONAL: 1

  ;; OPT PSEUDOSECTION:
  ; EDNS: version: 0, flags:; udp: 4096
  ;; QUESTION SECTION:
  ;outlook.com.            IN    NS

  ;; Query time: 28 msec
  ;; SERVER: 10.51.20.100#53(10.51.20.100)
  ;; WHEN: Fri Nov 22 09:58:48 EST 2013
  ;; MSG SIZE  rcvd: 40
[deleted]
(comment deleted)
ThomW
Who's scroogled now?

http://www.microsoftstore.com/store/msusa/en_US/cat/Scroogle...

This is so embarrassing.

hsod
this has nothing to do with the scroogled campaing though...
cjfont
Correct, your data being inaccessible isn't the same as being Scroogled (per MS)
wubbfindel
You could argue that it's MORE secure if even you can't access it. But I wouldn't argue that.
oneeyedpigeon
Maybe if Microsoft wasn't wasting its time trying to spread slurs about competitors, it could actually spend that time creating interesting software, building viable new products, and making sure it doesn't f*ck up its own DNS records. Or maybe someone's making that point for them ...
dandelany
Yes, I'm quite sure the marketing team in charge of the Scroogled campaign has the technical chops to prevent a DNS outage.
mistermann
Judging by their revenues, someone must find their software interesting, or at least useful.
niels_olson
They just haven't finished burning through the world's biggest lead.
mistermann
Ah, so they had a huge lead and they are losing it, I see.

Oh wait:

http://www.microsoft.com/en-us/news/press/2013/oct13/10-24fy...

REDMOND, Wash. — October 24, 2013 — Microsoft Corp. today announced revenue of $18.53 billion for the quarter ended September 30, 2013. Gross margin, operating income, net income, and diluted earnings per share for the quarter were $13.42 billion, $6.33 billion, $5.24 billion, and $0.62 per share.

I chuckle at how a lot of anti-MS zealots are absolutely mystified at how non-anti-MS simply can possibly be so delusional.

girvo
Wow. That was a reach.
curiousDog
Yes, ALL the developers (judging by your comment, who are no doubt better coders than you) at Microsoft are working on the new marketing campaign. Jesus christ, viable new products, perhaps they should just start making beer delivery apps on the rails stack, that'll prove themselves right? That 4chan thread the other day was exactly about guys like you.
oneeyedpigeon
Fair point. My original comment was poorly worded; I obviously didn't mean to imply any of the things you're claiming here. I merely (over)reacted out of extreme frustration with a marketing campaign that I can barely believe someone at Microsoft sanctioned. Whatever any of us think of Microsoft (I'm obviously not their biggest fan), you have to admit that this is a low, LOW move, and something that probably never would have happened were Bill Gates still in charge.
stuartd
Hubris -> Atë! (not that any of you youngsters would understand that reference)
pcx66
Wow, didn't expect they were selling merchandise. This is a new low even for Microsoft. Microsoft were never considered to be a company which puts ethics over money, and this just shows that they are jealous. They would do worse if they were in Google's place.

But selling merchandise? Come on, really! Is there at least one person in the world who could use these items and make it seem like they are making a fair point?

criley2
Jealous? To me it shows aggressive brand management. I've never really seen Microsoft as a company above ribbing on competitors. Why is it even a bad thing? Apple did it against PC for like a decade.
pcx66
Apple had a point, that they were cooler. They were saying, "hey, we are different". How does what Microsoft's saying about Google not apply to it? MS doesn't care about your privacy any more than Google. With the Snowden revelations and all, they have no case to make.
criley2
I disagree. Microsoft is a company who makes most of their money selling software, and hardware and services play a small part. They do almost no advertising.

Google is a company who derives about 99% of its revenue directly from its advertising business, of which utilizing user data from free services is the crux of the data-based approach that drives their success.

It seems like false equivalence to me. Same reason why I trust Apple way more than Google with data. Because Apple makes all their money on the hardware, not on the future use of your data for advertising purposes.

meowface
Most aggressive marketing/brand management generally doesn't involve selling products saying "[competitor] is bad and evil!!!"

Commercials sometimes, sure, but not wearable merchandise. I think it makes Microsoft seem quite petty, especially since they're trying to coin a whole new word out of it ("Scroogled").

toomuchtodo
Totally off-topic: Based on your nick, I may know you. Live in Northern Illinois by chance? With a short stint at a place near Oak Brook?
mrmondo
Google's DNS cache will drop them in 250 seconds...
beat
While looking at this, I saw some other temporary DNS fails as well, including HN, Google News, and Twitter. But all came back quickly, except for Outlook/Microsoft domains. (I'm in Minnesota)
pwf
I was unable to resolve any hostnames from my Azure machines from 4:20-4:40 Central time. I'm assuming this is all related.

This about sums up how I feel about Azure: http://i.imgur.com/ESakkWk.png

runn1ng
Wow. Your screenshot is giant.
pwf
Cmd+shift+4 on a retina MBP. Not sure why it does that...
MiguelHudnandez
It's because the metadata saying to render the image @2x is lost when imgur serves the PNG back to our browsers.
shiftpgdn
I got a 404 error on my signup for Azure instead of the "completed order" landing page. Then I got about a dozen phone calls asking me how my experience went with the service stuck in some half setup state.
shanselman
Looks bigger than this. It's ns*.msft.net.
sethhochberg
A friend of mine working at Microsoft mentioned this afternoon the Redmond campus had a full network outage.
Kennethtruyers
Yeps, a lot of Microsoft services are down. I'm in europe and can't reach outlook.com, microsoft.com, windowsazure.com and probably a whole bunch of other services
bluejade
This probably explains an error we were getting on a Node.js app running on Azure websites. Simply creating a new AWS.S3 instance was throwing this: Error: getaddrinfo ENOTFOUND at errnoException (dns.js:37:11) at Object.onanswer [as oncomplete] (dns.js:124:16)
codex
Will negative caching of DNS records prevent a quick recovery? If so, for how long will the negative records be cached?
Caspan
I have my domain at outlook.com and my MX record points to edcb3f68479b498ee412acc8524def.pamx1.hotmail.com and I am getting no return for that. There is a big outage somewhere
coldcode
Damn the guys at the NSA are trying to get on xbox live.