Ask HN: How to enumerate all subdomains of a main domain like foo.com?

41 points by dedalus ↗ HN

35 comments

[ 2.9 ms ] story [ 76.7 ms ] thread
(comment deleted)
Ask the name server: do a AXFR query, as in host -l foo.com.

These days many domains are configured to refuse AXFR queries though. Then there's a misguided but common phenomenon called split-horizon DNS is also common, where you serve different records as answers to the same query based on what the query originator's address is.

> misguided

citation needed.

"Citation needed" comments are useless. If you disagree, state why. Otherwise, you aren't helping anyone and merely adding confusion about one of the base technologies.
I disagree - it’s merely asking for elaboration on a stated opinion.
I'd say it is a very confrontational way of asking for elaboration.
I've thought about this a bit, and decided that doing 'the Wikipedia thing' ('Citation Needed') was minimally confrontational, in that it's kind of a standard, somewhat 'mechanical' approach.

I had no idea that some would consider it 'very confrontational'; that's the exact opposite of my intent.

Speaking transparently, when I write 'Citation Needed', it's usually because I do disagree with the statement in question, but that my disagreement is not sufficiently supported. It also means that I'm open to being corrected with additional information.

I guess the reason it comes off as rude is because it is so short and - for my lack of a better word - rubberstampy.

It is as if some people don't have time to ask politely for sources.

I also think it might trigger the "passive agressive"-detector for some people here. (I think that description has been overused a lot though and don't want to classify it as such.)

This might not be your intention but I wouldn't be very sad to see those words less often here.

And one more thing: using wikipedia as an example for how to behave in society might not be a good idea IMO.

As I noted in another comment, when I reply 'Citation Needed', it's because I weakly disagree with the statement, but that disagreement is insufficiently supported. In many cases, I'll go off and do some independent research at the same time, but mostly I'm requesting that the person provide some additional support for their thought.

It also means that I'm open to being wrong in my disagreement.

So, in short, I don't state disagreement in these cases because the strength of my opinion is too weak to merit it.

It's just counter to the basic internet and DNS design principes and breaks many things in applications working with DNS names and IP addresses. Not to mention the added complexity and twists in troubleshooting when your names are ambiguous. The basic design of DNS is one root and same view for all.
It's refused for security reasons. This was how Valve was hacked in 2003, leading to Half Life 2's leak.

> "I was scanning Valve's network to check for accessible web servers where I thought information about the game might have been held. Valve's network was reasonably secure from the outside, but the weakness was that their name server allowed anonymous AXFRs, which gave me quite a bit of information."

AXFR stands for Asynchronous Full Zone Transfer, a tool used to synchronise backup DNS servers with the same data as the primary server. But it's also a protocol used by hackers to sneak a peek at a website's data. By transferring this data, Gembe was able to discover the names of all the subdomains of ValveSoftware.com.

"In the port scan logs, I found an interesting server which was in Valve's network range from another corporation named Tangis that specialised in wearable computing devices," he says.

"This server had a publically writable web root where I could upload ASP scripts and execute them via the web server. Valve didn't firewall this server from its internal network."

https://www.eurogamer.net/articles/2011-02-21-the-boy-who-st...

I agree it's often motivated by security reasons, but it's of rather questionable effectiveness. Don't put confidential information in the DNS, ever.

Re the Valve case: this could have been also found with a simple ip range port scan, or a bunch of other ways. In the end it was just a server in the network with no access control configured and they happened to spot it in the DNS records first.

"In the port scan logs, I found an interesting server which was in Valve's network range from another corporation named Tangis that specialised in wearable computing devices," he says."

Also, in the modern world, you can see the web servers in a domain in the public cert transparency logs.

> Then there's a misguided but common phenomenon called split-horizon DNS is also common, where you serve different records as answers to the same query based on what the query originator's address is.

Do you consider the whole functionality of views to be misguided, or just predicating the selection of view solely on source address?

I'm not sure if there's a way to handle multi-homed hosts in a more simple/elegant fashion than with views.

Merely for serving distinct "internal" and "external" zones from the same server, it seems like a convenience fraught with pitfalls.

It's all part of the pit you start digging yourself when you start using context dependent (eg rfc1918) addresses. Everything with the IP/Internet architecture was designed to work with globally unique addresseses.
> Everything with the IP/Internet architecture was designed

Sure, but that statement is only true in the past tense and only for a sufficiently-past definition of "everything". One could make a similar complaint about the breakdown of classful routing and subnets, but the Internet has morphed beyond its original design, in response to the realities of how it's been used [1].

Regardless, it seems your objection is to the non-global-uniqueness of private addressing, which existed well before split-dns.

Is there any separate objection you have to split-dns?

[1] The wisdom of each particular decision is debatable, but I, personally, find little interest in exploring alternate-history/what-if scenarios, technical or otherwise.

I don't really see the CIDR analogy - CIDR is in line with the end-to-end architecture and doesn't break applications using IP addresses.

Anyway, consider a basic use case like an application (correctly) caching DNS records for the duration of their time-to-live metadata - and moving between networks that yield different reponses to these queries. With the added twist that different applications cache differently so now you have an incoherent view of the DNS on the same system. Another comment said it won't work with DNSSEC, that's another example stemming from going against the design and basic principles of the DNS.

It's true that there are some reasonable use cases context dependent DNS replies, such CDN's returning a nearby address from a set of geographically replicated content servers, but the common internal/external split-dns setup is fundamentally unsound.

> I don't really see the CIDR analogy

As an analogy, it doesn't directly apply to your particular point, which is why it's an analogy. It would apply to, say, a network engineer.

> an application (correctly) caching DNS records

I'm pretty sure that merely putting the word "correctly" in parentheses doesn't make it so, nor, more to the point, remove the fact of the controversy on that point.

> that different applications cache differently

This is merely an argument in favor of the caching being done by the OS, which would also be aware of a device moving networks. Device mobility (without reboot) is one of those relatively recent uses imposed onto the Internet.

> Another comment said it won't work with DNSSEC, that's another example stemming from going against the design and basic principles of the DNS.

One could lay the blame for incompatibility at the feet of DNSSEC, too, as that technology is also a response to modern needs of the Internet that has little, if anything, to do with any original design or basic principles.

> the common internal/external split-dns setup is fundamentally unsound.

So, again, I still can't tell what it is you're opposed to, on so fundamental a level that you would call it "unsound", with split-dns specifically, separate from the issue of private addressing being used.

Is it any different if two separate nameservers are used instead of split-dns on one?

Perhaps, most importantly, what's the sound, not-misguided, alternative that provides a comparable (maybe even better) experience for the user?

Sorry to butt out, seems this deteriorated into talking past each other and is unlikely to recover.
Well, OK, but I really do wish you would answer my original question as to whether you have an objection to split-dns separate/independent of anything to do with rfc1918.

That's what I've been trying to do all along, which I admit requires talking past whatever point you're trying to make about private addressing and non-uniqueness. I have made a concerted attempt to address those points as they pertain specifically to my question.

Of course, I also hope you'd share your proposed alternative, even for 1918. I doubt anyone here actually enjoys the pain brought about by forced NAT and non-unique IPs (naive VPNs!).

Try fierce pl - there may be newer ways but this has always yielded decent results for me when pentesting. You generally have to do some form of brute force as most DNS servers won't spill their guts these days.
You should use a combination of three tools:

- sublist3r - amass - subfinder

They're all on Github.

Adding a responsive subdomain to any domain you control is trivial and isn't registered anywhere (necessarily). I'm not sure you can achieve this, without additional requirements.
login to cloudflare, add domain, wait for DNS slurp, export full record, delete domain.
Ah, the classical network pentester's problem. There's really no one good way to go about this.

Certificate transparency tools like CTFR (https://github.com/UnaPibaGeek/ctfr) work only if certs are registered.

You could go old school and use a tool like Sublist3r (https://github.com/aboul3la/Sublist3r) or Punter (https://github.com/nethunteros/punter), but ymmv as API endpoints are savvy to these tools and actively work to snub them out.

AXFR queries can be useful if the DNS server allows for it (my experience: 0-15).

Best of luck.

Check Google, Bing, Virustotal, Parse HTTPS Certificates including the metadata (Censys.io is great for specific queries), subdomain bruteforce with a good wordlist, download source code found in Github and regex search for HTTP urls, then parse them.

Now don't do it by hand people have already built tools. I recommend sublist3r https://github.com/aboul3la/Sublist3r, however, grab other subdomain bruteforcer wordlists and append them all together.

Go to https://opendata.rapid7.com/, download the reverse DNS and Forward DNS and grep for your domain. I.E grep "*.mydomain.com" These are amazing.

I will make a note, sometimes if you are looking for servers related to a company specifically people miss ones that aren't in a company's zone file. You need to use a service like Shodan or Censys which regularly scan the internet and index these. It can be a pain to parse through these results but if you are strapped for ideas on getting a foodhold try this. I have found some juicy servers with this in mind.

If you are on a pentest it is completely ok to ask your client for permission to view their zone file/route53 as well. This will save you a lot of time up front.

It's perfectly possible to have a wildcard and respond to every subdomain.

But otherwise just use nslookup/dig/host