list_folder/continue with an old cursor

[Patrik H.]

What are the rules regarding cursors for use with list_folder/continue?

As a concrete case, say that I do the following sequence of calls: 

1. list_folder => obtaining in response batch "a" of entries and cursor "A"

2. list_folder/continue, supplying cursor "A", and obtaining in response batch "b" of entries and cursor "B"

If there is a need for it, am I at this point allowed to go back ("rewind") and do a call:

3. list_folder/continue, supplying cursor "A", obtaining in response batch "c" of entries and cursor "C"

I did a quick test with the API explorer, and it seemed that no error message was obtained. However, as already indicated by my use of characters above, the batch of results "c" does not seem to exactly match the batch "b", nor does the cursor "C" exactly match the cursor "B". It is fine for me that they do not match, but I am wondering if the above sequence of calls is allowed/supported, i.e. is it ok to use an "old" cursor value, and continue from there as if one had never done call 2?

(The reason one might want to do this is if one is paginating the results, but the following happens: The list_folder/continue call completes, but the app does not succeed in storing the newly received cursor before something bad happens, for example the app crashes, or something else, so it only has available the previous cursor. It would be nice to be able to safely continue without having to start over from scratch.)

If the answer is: "Yes, it is ok", then it would be interesting to know (at least purely out of curiosity): How old cursors can be used? Does the cursor string (which is pretty long) encode everything needed (the relevant timestamp, folder and all settings) so that an arbitrary cursor can be used? Or does Dropbox store (somewhere on the server) the various cursor values as keys to some data that it uses to continue the pagination (in this latter case one would expect that Dropbox does not want to keep around all cursor values ever created)?

Thanks!

Patrik

[Greg-DB · Accepted solution]

Hi Patrik, in general, yes, you can always re-use an old cursor, even if you've already received/used a newer cursor. (In some relatively rare cases cursors will no longer be valid, in which case you'll get the 'reset' error and need to start fresh from /list_folder, but that's not related to using newer cursors.)

If your outlined scenario, the results b and c indeed may not match, and cursors B and C will be different. The /list_folder/continue interface is only guaranteed to return the results needed to get your local state up to date, based on whatever cursor was supplied. This means it won't necessarily return every single event that occurred, but may just return the changes necessary to describe the overall difference in state.

And yes, the cursor string itself encodes all of the information necessary for Dropbox to identify what it needs to respond with for that app/account/cursor. (It's doesn't use timestamps exactly, but rather identifiers with specific meaning on the Dropbox backend.) This will be specific to the account, which is why the cursor length can vary from account to account.

Hope this helps!

[Patrik H.]

Hi Greg,

Thanks for the quick and informative reply!

Concerning your comment that /list_folder/continue may not return every single event, in practice my observations are that it seems (at least at the moment) indeed return all events. So, for example: If a file is created and then immediately deleted, in-between two /list_folder/continue calls, then that file does show up in the results of the second call, even though it is not really necessary to get the local state synched. Is your warning that it may not return all results simply a way to communicate that Dropbox does not want commit to keeping it like that in the future, or do you know of any cases in which, even currently, not all events are included in the results?

Patrik

[Greg-DB]

I believe there are some cases where Dropbox may already omit unnecessary events, but I don't have specific information on that available. (And if not, we indeed may want to make that optimization in the future.) Just make sure you're processing the entries as instructed in the list_folder documentation and you'll get an accurate representation. 

[Patrik H.]

The reason I am curious about this fact is that in my case I am constructing a list of all events in a user's Dropbox account. This is a different use case from trying to ensure that we have an accurate representation of the current state of the Dropbox account (the latter is what the documentation aims to help with, both previously for "delta" and now for "list_folder/continue").

Thus, in my case, what you refer to as "unnecessary events", are in fact not at all unnecessary for my purpose, they are in fact quite important.

The current behavior of the Dropbox API fits my purpose. But if at some point the "optimization" is made to leave out some "unnecessary" events (or indeed if something like that already happens), then there seems to be no way left to ensure that all events are detected. Hence, I would very strongly argue against doing that "optimization". If you can, please communicate this to the team at your end. 🙂

If indeed such an optimization is made, then it might be a good time to introduce an API endpoint that can be used to monitor all events occuring in a Dropbox account.

Thanks again for your always helpful answers!

Patrik

[Greg-DB]

Thanks for the context! I'll send this along to the team.