Need to see if your shared folder is taking up space on your dropbox 👨‍💻? Find out how to check here.

Forum Discussion

Gags's avatar
Gags
Helpful | Level 6
4 years ago
Solved

Custom Tool - Slow File Upload Speed

Hello Greg-DB    We have developed a tool to help our customers move data from Windows File System to Dropbox.   The upload is done in batches of 8 files using the Dropbox APIs - UploadSessio...
API
  • Gags's avatar
    Gags
    4 years ago

    Hello Greg-DB 

     

    Thank you for the reply !!!

     

    We used the new Dropbox API - UploadSessionStartBatchAsync and this helped in reducing the "too_many_requests" errors.

    We tested 2.2 TB data sample containing 427,002 files for migration using our custom app.
    Last time we faced 26,000 failures which has been reduced to 1,753 in the latest test.

     

    What more can be done to further reduce failures due to "too_many_requests" error response from Dropbox.
    We do wait and retry as shown in the last code snippet.

     

    Thanks,
    Gagan

Gags's avatar
Gags
Helpful | Level 6
4 years ago

Hello Greg-DB 

 

Thank you for the reply !!!

 

We used the new Dropbox API - UploadSessionStartBatchAsync and this helped in reducing the "too_many_requests" errors.

We tested 2.2 TB data sample containing 427,002 files for migration using our custom app.
Last time we faced 26,000 failures which has been reduced to 1,753 in the latest test.

 

What more can be done to further reduce failures due to "too_many_requests" error response from Dropbox.
We do wait and retry as shown in the last code snippet.

 

Thanks,
Gagan

Greg-DB's avatar
Greg-DB
Icon for Dropbox Community Moderator rankDropbox Community Moderator
4 years ago

It sounds like you're using all of the available batch endpoints for this use cases, so the only remaining change that comes to mind would be to increase the chunk size, so that you use fewer requests for sending the file data overall.

 

You can use a chunk size of up to 150 MB, but be aware that increasing the chunk size increases how long each request takes and consequently increases the chances of the network connection failing before it completes. And further, using a larger chunk size means more data needs to be sent on a retry.

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.

The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!