How to paginate in Gremlin

Question

In Tinkerpop 3, how do pagination work? I want to get the first 10 elements of the request, and then the next 10 without loading them all into memory. For example, the query below returns 1,000,000 records. I want to get them 10 by 10 without loading just 1,000,000 at a time.

gV().has("key", value).limit(10) 

Edit

A solution that works through the HttpChannelizer on a Gremlin server would be ideal.

Answer 1

From a functional point of view, a nice Gremlin swap piece would look like this:

 gremlin> gV().hasLabel('person').fold().as('persons','count'). select('persons','count'). by(range(local, 0, 2)). by(count(local)) ==>[persons:[v[1],v[2]],count:4] gremlin> gV().hasLabel('person').fold().as('persons','count'). select('persons','count'). by(range(local, 2, 4)). by(count(local)) ==>[persons:[v[4],v[6]],count:4] 

Thus, you will get the total number of vertices with the result. Unfortunately, the fold() method forces you to read all the vertices that require iterating all of them (i.e., Transfer them to memory).

In this case, iteration of all 100,000 vertices cannot be avoided if you intend to traverse in several separate attempts. For example:

 gremlin> gV().hasLabel('person').range(0,2) ==>v[1] ==>v[2] gremlin> gV().hasLabel('person').range(2,4) ==>v[4] ==>v[6] 

The first statement is the same as if you completed the traversal with limit(2) . In the second round, when only the second two vertices are needed, it is not as if you magically skipped the iterations of the first two, as this is a new round. I don’t know about any implementation of the TinkerPop graph database that will do this efficiently - they all have this behavior.

The only way to make ten vertices at a time without having them all in memory is to use the same Traversal instance as in:

 gremlin> t = gV().hasLabel('person');[] gremlin> t.next(2) ==>v[1] ==>v[2] gremlin> t.next(2) ==>v[4] ==>v[6] 

With this model, you only iterate the vertices once and do not memorize them all at one time.

Some other thoughts on this topic can be found on this blog .