Pagination issue in RESTful API design

Question

I am developing a RESTful API for the mobile application I'm working on. My problem is with large collections containing many items. I understand that it’s good practice to break up a large number of results in a collection.

I read the Facebook GUI API document ( https://developers.facebook.com/docs/graph-api/using-graph-api/v2.2 ), Twitter cursors doc ( https://dev.twitter.com/overview / api / cursoring ), GitHub API doc ( https://developer.github.com/v3/ ) and this post ( best examples of using pagination ).

Consider an example of the /resources collection in my API, which contains 100 elements named resource1 to resource100 and sorted in descending order. This is the answer you will get for a GET request ( GET http://api.path.com/resources?limit=5 ):

 { "_links": { "self": { "href": "/resources?limit=5&page=1" }, "last": { "href": "/resources?limit=5&page=7" }, "next": { "href": "/resources?limit=5&page=2" } }, "_embedded": { "records": [ { resource 100 }, { resource 99 }, { resource 98 }, { resource 97 }, { resource 96 } ] } } 

Now my problem is this scenario:

1st GET /resources with the above contents.

2- After that, something is added to the resource assembly (say, another device adds a new resource for this account). So now I have 101 resources.

3rd GET /resources?limit=5&page=2 , since the original answer will tell me, it will contain the next page of my results. The answer would be:

 { "_links": { "self": { "href": "/history?page=2&limit=5" }, "last": { "href": "/history?page=7&limit=5" }, "next": { "href": "/history?page=3&limit=5" } }, "_embedded": { "records": [ { resource 96 }, { resource 95 }, { resource 94 }, { resource 93 }, { resource 92 } ] } } 

As you can see, resource 96 repeated on both pages (or a similar problem may occur if the resource is deleted in step 2, in which case one resource will be lost).

Since I want to use this in a mobile application and in one list, I have to add the resources of each API call to the one that came before it, so I can have a complete list. But it does bother. Please let me know if you have any suggestion. Thank you in advance.

PS: I considered a timestamp, such as a query string, instead of a cursor-based breakdown, but this would create problems somewhere else for me. (let me know if you need more information about this.)

Answer 1

We just implemented something similar for a mobile application through the REST API. The mobile application passed an additional request parameter, which represents the timestamp in which page elements should be "frozen."

So, your first request will look something like GET /resources?limit=5&page=1&from=2015-01-25T05:10:31.000Z , and then the second page request (after a while) will increase the number of pages, but keep the same timestamp: GET /resources?limit=5&page=2&from=2015-01-25T05:10:31.000Z

It also gives control to the mobile application if it wants to differentiate the soft page (keeping the page 1 request timestamp) from the hard update page (resetting the current timestamp).

Answer 2

Why not just maintain a set of visible resources?

Then, when you process each response, you can check if the resource has already been submitted.