Anglesharp - How do I get ratings and reviews from Yelp?

Question

I want to try my hand and webscraping. I noticed that Anglesharp is pretty good for .Net environment. I try to get a list of all descriptions and ratings from the yelp website and I am not getting any errors or any results. Here is a subset of what html looks like (for more details see https://www.yelp.ca/biz/walmart-toronto-12 "):

<div class="rating-very-large"> <i class="star-img stars_2" title="2.0 star rating"> <img alt="2.0 star rating" class="offscreen" height="303" src="//s3-media4.fl.yelpcdn.com/assets/srv0/yelp_styleguide/c2252a4cd43e/assets/img/stars/stars_map.png" width="84"> </i> <meta itemprop="ratingValue" content="2.0"> </div> <p itemprop="description" lang="en">This Walmart still terrifies me<br><br>Baby things can be found on the back right of the lower level. Godspeed.</p> <div class="rating-very-large"> <i class="star-img stars_1" title="1.0 star rating"> <img alt="1.0 star rating" class="offscreen" height="303" src="//s3-media4.fl.yelpcdn.com/assets/srv0/yelp_styleguide/c2252a4cd43e/assets/img/stars/stars_map.png" width="84"> </i> <meta itemprop="ratingValue" content="1.0"> </div> <p itemprop="description" lang="en">Wow I don&#39;t even know where to begin, </p> 

Here is my request:

 var config = var config = new Configuration().WithJavaScript().WithCss(); var parser = new HtmlParser(config); var document = await BrowsingContext.New(config).OpenAsync("https://www.yelp.ca/biz/walmart-toronto-12"); //Do something with LINQ var descriptionListItemsLinq = document.All.Where(m => m.LocalName == "p" && m.Id.Contains("description")); foreach (var element in descriptionListItemsLinq) { element.Text().Dump(); } 

How to get a list of user reviews (descriptions) and ratings?

Answer 1

I checked the HTML source https://www.yelp.ca/biz/walmart-toronto-12 . As I expected, user reviews are in JSON . You should not use AngleSharp in this scenario.

The photo below is extracted from an HTML source.

and here is an analytic version of JSON :

This is a JSON , and you can deserialize it using Newtonsoft.Json . Just extract the JSON and read what you need from it.