Ask HN: How to create a Web crawler?

I am creating a website for price comparison. I want to fetch the price of a single product from multiple shopping carts. I have some questions regarding this. 1) What is the best way to fetch price form different websites? 2) Sometime a single product have different name on different shopping carts. How to handle this problem? 3) When I will send multiple requests on a website (shopping cart) using my created web crawler (or web spider) will they block me or can they take a legal action against me? 4) How can I make the process automated to omit manual error.

I will start with 3 shopping carts initially. Most of the shopping carts are not providing any type of API or other type of access to their products. Currently my approach is to tear down the HTML and find required information from HTML.

I want to go with C# & .Net technology because I am good in it (this is what I thought).

Please suggest what is the best way of doing this.