Ask HN: How can you recursively grep a website?

I want to use a regular expression on a whole domain. What is the easiest way to do that?

3 comments

[ 3.5 ms ] story [ 11.7 ms ] thread

Google's site: operator... or write a crawler to parse all within-site text links, download the site source into files on your machine and then grep.

yep either just use google, or download the site as mentioned and use:

grep myTerm path/to/mysite/

hm looks like ycombi parsed out my asterisks. at the end of the path should've been two.