Geek Answers Handbook

Search Results for github

I need to do a very large search on Github for statistics in my dissertation.

For example, I need to study a large number of Android projects on GitHub, but the site limits the search result to 1000 (for example, https://github.com/search?l=java&q=onCreate&ref=searchresults&type=Code&utf8=%E2%9C%93 ). Also using the Java GitHub API, I tried the org.eclipse.egit.github.core.client.GitHubClient library using the GitHubClient.searchRepositories() method, but even there the number of results is limited.

Does anyone know how to get all the results?

+5
source share
2 answers

The search API will return up to 1000 results for each query (including pagination), as described here:

https://developer.github.com/v3/search/#about-the-search-api

However, there is a neat trick that you could use to get more than 1000 results when doing a repository search. You can divide your search into segments by the date the repositories were created. For example, you can first find repositories created in the first week of October 2013, then the second week, then September, etc.

Since you limit your search to a narrow period, you are likely to get less than 1000 results and therefore be able to get all of them. In case you notice that more than 1000 results are returned for the period, you will have to narrow the period even more so that you can collect all the results.

https://help.github.com/articles/searching-repositories/#search-based-on-when-a-repository-was-created-or-last-updated

You should be able to automate this using the API.

+12
source

If you are looking for all the files in Github with the file name: your-file-name, you can also cut it off using the query: size attribute .

For example, if you search for all files named test.rb on Github, the Github API can return more than 11 million results, but you can only get 1000 of them, because the GitHub search API provides up to 1000 results for each search . URL: https://api.github.com/search/code?q=filename:test.rb+size:1000..1500 could cut your search by changing the size range.

+1
source

More articles:


All Articles