Ask HN: How does search engines find domain or IP directories or new domains?

There are feeds of new domain releases. There are regional internet registry stats files detailing all ip disbursements and economies. BGP provides some ground truth as to origins and you can rtt and bgp triangulate to locality pretty well.

Places like Maxmind intermediate finer grained data and rfc9092 exists to make declarative intent.