Forum Discussion

Frank R.15's avatar
Frank R.15
Helpful | Level 5
8 years ago
Solved

API v2 - Long polling

I'm in the process of migrating from the V1 Sync API to the V2 "DIY" API and I would really appreciate some feedback on whether I'm really approaching this correctly, especially with regards to reacting to remote changes and the seemingly insanely heavy network traffic requried by my "solution".

 

I'm using Dropbox to keep my iOS and Mac apps synched, so that both show the same "event logs". On the Mac those logs are stored in "My Folder" folder.

 

On startup, the iOS version (Swift) does a files.listFolder for "My Folder" and subsequent files.listFolderContinue until it has a complete file list, then for each file it does a files.getMetaData and compares the rev (revision) field in the meta with the rev value stored in my local DIY rev-to-file-path database. If the revs are different, it does a files.download for the file and writes to local storage.

 

After all local files are up-to-date,  it does a files.listFolderLongPoll and waits for any updates, if an updates occurs it restarts the entire sync process.

 

All this results, for n log files, in at least n+1 network requests when there's nothing to by synched and 2*n+1 network requests in the worst case. There's a log file for each day of use, so after a year, we have 366 to 731 network requests.. :scream:

 

Question 1: Is this really the most efficient way that this can be done?

 

Question 2: Is there are way of polling only for changes to a specific directory?

 

BTW I really appreciate the great job the support team is doing; even as I'm perplexed by the new API :-)

  • Hi Frank, I'll be happy to help here. 

     

    "Question 1: Is this really the most efficient way that this can be done?"

     

    Reading through your description (thank you for the detailed write up, it really helps), I believe there are a few optimizations to make:

     

    a) "does a files.listFolder" ... "files.listFolderContinue" ... "then for each file it does a files.getMetaData and compares the rev":

     

    If you've already done files.listFolder/files.listFolderContinue, you don't also need to call files.getMetadata for each file. That just returns the same Metadata objects as were already returned in ListFolderResult.entries from files.listFolder/files.listFolderContinue.

     

    Part of the confusion may be that you need to cast the Metadata to e.g., FileMetadata to access FileMetadata.rev though, as shown here.

     

    b) "if an updates occurs it restarts the entire sync process":

     

    You don't need to start totally over. When files.listFolderLongPoll indicates something changed, you should just call back to files.listFolderContinue with the latest cursor you recieved, and it will tell you only about what has changed since then. (You should just continue applying the same steps from the files.listFolder documentation to apply those additional changes to your local state to get up to date.)

     

    "Question 2: Is there are way of polling only for changes to a specific directory?"

     

    No, unfortunately files.listFolderLongPoll will notify of changes in the account even if the changes only occurred outside of the path originally specified in the path set in the original files.listFolder call. We'll consider that a feature request, but I don't know if/when that would be implemented. (It would be non-trivial due to the way these pieces of infrastructure work on the backend.)

     

    Hope this helps, and thanks for the kind words about the support!

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff

    Hi Frank, I'll be happy to help here. 

     

    "Question 1: Is this really the most efficient way that this can be done?"

     

    Reading through your description (thank you for the detailed write up, it really helps), I believe there are a few optimizations to make:

     

    a) "does a files.listFolder" ... "files.listFolderContinue" ... "then for each file it does a files.getMetaData and compares the rev":

     

    If you've already done files.listFolder/files.listFolderContinue, you don't also need to call files.getMetadata for each file. That just returns the same Metadata objects as were already returned in ListFolderResult.entries from files.listFolder/files.listFolderContinue.

     

    Part of the confusion may be that you need to cast the Metadata to e.g., FileMetadata to access FileMetadata.rev though, as shown here.

     

    b) "if an updates occurs it restarts the entire sync process":

     

    You don't need to start totally over. When files.listFolderLongPoll indicates something changed, you should just call back to files.listFolderContinue with the latest cursor you recieved, and it will tell you only about what has changed since then. (You should just continue applying the same steps from the files.listFolder documentation to apply those additional changes to your local state to get up to date.)

     

    "Question 2: Is there are way of polling only for changes to a specific directory?"

     

    No, unfortunately files.listFolderLongPoll will notify of changes in the account even if the changes only occurred outside of the path originally specified in the path set in the original files.listFolder call. We'll consider that a feature request, but I don't know if/when that would be implemented. (It would be non-trivial due to the way these pieces of infrastructure work on the backend.)

     

    Hope this helps, and thanks for the kind words about the support!

    • Frank R.15's avatar
      Frank R.15
      Helpful | Level 5
      Cheers, that's awesome and makes a lot of sense. I did miss the metadata casting and did wonder "why on earth bother sending back metadata at all if the revision isn't in there?"

      I'm getting rid of code left, right and center and getting better performance into the bargain.

      I'm so glad I posted here!

      Cheers,

      Frank

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.5,947 PostsLatest Activity: 4 hours ago
351 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!