Best way to get specific app data from Blackberry App World (API)

Question

I am collecting statistics about mobile apps using Python, and now I'm looking for the best solution for accessing Blackberry App World data.

So far I have a solution for iOS ( http://www.apple.com/itunes/affiliates/resources/documentation/itunes-store-web-service-search-api.html ) and Android ( https://github.com / liato / android-market-api-py ). The iOS solution uses the API provided by Apple, the Android solution mimics the phone and collects data like a real phone does in a structured way.

Now I can't find a similar solution for the BlackBerry App World, so my question is, what is the best way to go? I can clean the site, but I probably won’t, as my scraper will break if they change their site. Ideally, I would use either the provided API or simulate BlackBerry to access App World data in a more structured way. Any suggestions?

Answer 1

I cleaned up the Blackberry website for a while and still haven't had problems with updates.

Do you use absolute XPaths from the document root to retrieve data? You can make a more reliable scraper using relative XPaths:

//div[@id="priceArea"]/div[@class="contentLic"] 

Answer 2

I scraped the Blackberry website using selenium webdriver and phantomDriver and csquery in .net for a while, and so far there have been no issues with updates.

 //Creating dynamic browser and download the page source code based on apipath by using selenium web driver driver = new PhantomJSDriver(phantomDriverPath); //driver=new ChromeDriver(chromeDriverPath); driver.Url = "https://appworld.blackberry.com/webstore/search/"+<search app name>+"/?lang=en&countrycode=IN"; driver.Navigate(); //Waiting for page loading Thread.Sleep(2000);//2 seconds if (driver.PageSource != null) { //Assigning downloaded page source code to CSQuery CQ dom = CQ.CreateDocument(driver.PageSource); //Waiting for page loading driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30)); //find the elements what ever you want based on the id,class name,tag name string title1 = dom["#topListtopResultsAppTemplateHTML_listItem_0_title"].Text(); } 

Answer 3

I read the Blackberry website using Selenium WebDriver and phantomDriver and CSQuery in .NET , and I have not had a problem with updates so far.

 //Creating dynamic browser and download the page source code //based on apipath by using selenium web driver public IWebDriver driver; driver = new PhantomJSDriver(phantomDriverPath); //driver=new ChromeDriver(chromeDriverPath); driver.Url = "https://appworld.blackberry.com/webstore/search/"+appname+"/lang=en&countrycode=IN"; driver.Navigate(); //Waiting for page loading Thread.Sleep(2000);//2 seconds if (driver.PageSource != null){ //Assigning downloaded page source code to CSQuery CQ dom = CQ.CreateDocument(driver.PageSource); //Waiting for page loading driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30)); //find the elements what ever you want based on the id,class name,tag name string title1 = dom["#topListtopResultsAppTemplateHTML_listItem_0_title"].Text(); } 

Before encoding, download the Selenium WebDriver and phantom driver to your PC (for example, C:\Users\rakesh\Documents\Selenium\PhantomJSDriver ) and install CSQuery in Visual Studio.

Install webdriver:

 Install-Package Selenium.WebDriver 

Install phantomjs:

 Install-Package phantomjs.exe