In a Google search, you can enter: site: your_domain
And he will show you the results. You can use the Google Custom Search API for this.
http://code.google.com/apis/customsearch/v1/overview.html
It returns JSON results that you can convert to C # objects using a DataContractSerializer.
You will need to sign up for an API key if you go this route.
Edit
Html Agility Pack, , ,
-