Node.js Twitter API Cursors

Question

I use npm-twit to get subscribers to a specific account.

The Twitter API returns up to 5,000 results from a single GET request.

If the user I request has more than 5,000 followers, the value "next_cursor" with the data is returned.

To get the following 5000 results, I need to re-run the GET function, passing it the next_cursor value as an argument. I just can't figure out how to do this.

I was thinking about a while loop, but I cannot reset the global variable, I think, due to the scope:

var cursor = -1 while ( cursor != 0 ) { T.get('followers/ids', { screen_name: 'twitter' }, function (err, data, response) { // Do stuff here to write data to a file cursor = data["next_cursor"]; }) } 

Obviously, I'm not a JS genius, so any help would be greatly appreciated.

Answer 1

The problem you are facing is related to Node.js asynchronous .

 T.get('followers/ids', { screen_name: 'twitter' }, function getData(err, data, response) { // Do stuff here to write data to a file if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', next_cursor: data['next_cursor'] }, getData); }) } 

Note:

• I gave the name of the internal callback function. This means that we can recursively call it from the inside.
• The loop is replaced by a recursive callback.
• If there is next_cursor data, we call T.get using the same getData function.

Keep in mind that the code here will be executed many times (as many as the following cursors are). Since this is a recursive callback, the order is guaranteed.

---

If you don't like the idea of ​​recursive callbacks, you can avoid this:

• Find out in advance all next_cursor, if possible, and generate queries using the for loop.
• As an alternative, use asynchronous helper modules such as Async (although for training purposes, I would avoid modules if you don't already have a concept).

Answer 2

Consider testing with some 5K + account.

  const T = new Twit(tokens) function getFollowers (screenName, followers = [], cur = -1) { return new Promise((resolve, reject) => { T.get('followers/ids', { screen_name: screenName, cursor: cur, count: 5000 }, (err, data, response) => { if (err) { cur = -1 reject(err) } else { cur = data.next_cursor followers.push(data.ids) if (cur > 0) { return resolve(getFollowers(screenName, followers, cur)) } else { return resolve([].concat(...followers)) } } }) }) } async function getXaqron () { let result = await getFollowers('xaqron') return result } console.log(getXaqron().catch((err) => { console.log(err) // Rate limit exceeded })) 

Answer 3

Fighting this ... Everything seemed to work, but the ['next_cursor'] data has not changed, EVER!

The code should look like this:

 T.get('followers/ids', { screen_name: 'twitter' }, function getData(err, data, response) { // Do stuff here to write data to a file if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', cursor: data['next_cursor'] }, getData); }) } 

The parameter for Twit is not "next_cursor", it's just a "cursor";)