cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
We are making some updates so the Community might be down for a few hours on Monday the 11th of November. Apologies for the inconvenience and thank you for your patience. You can find out more here.

Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Re: Max files Upload using javascript SDK

Max files Upload using javascript SDK

buttflattery
Explorer | Level 4
Go to solution

I can find any relevant section that addresses how many files i can upload in concurrent requests. means if i am using javascript SDK and i trigger 100 files via looping the code that creates a promise for every request. Are there any limits to how many requests i can trigger at a time?

1 Accepted Solution

Accepted Solutions

Greg-DB
Dropbox Staff
Go to solution

You will need to create one "upload session" per file. Each file can be uploaded in multiple pieces or "chunks". Once you've uploaded all of the chunks for each file, you can commit all of the files at once by using the /files/upload_session/finish_batch endpoint. That will help avoid lock contention.

For each single request to upload a chunk of a file, you can technically upload up to 150 MB. In practice, you'll likely want to use a size smaller than that for the sake of reliability. Additionally, using a smaller size means you won't have to re-upload as much if/when a particular request does fail. For instance, in practice you might want to use a chunk size of 12 MB, but you can experiment to see what size works well for your use case.

View solution in original post

6 Replies 6

Greg-DB
Dropbox Staff
Go to solution

There are some limitations to be aware of if you're trying to upload multiple files. I recommend reading the Data Ingress Guide as it covers this in detail:

https://www.dropbox.com/developers/reference/data-ingress-guide

This applies whether or not you're using an official Dropbox SDK.

buttflattery
Explorer | Level 4
Go to solution

hmm that means dropbox recommends that if there are alot of files i should upload all those file in a single sesssion and commit all of them once. And to upload those files i should upload in chunks as i need to manage the upload progress Correct?

Second thing, if the above is correct what is the max size of the chunk that is recommended keeping in mind the requests rate limit?

Greg-DB
Dropbox Staff
Go to solution

You will need to create one "upload session" per file. Each file can be uploaded in multiple pieces or "chunks". Once you've uploaded all of the chunks for each file, you can commit all of the files at once by using the /files/upload_session/finish_batch endpoint. That will help avoid lock contention.

For each single request to upload a chunk of a file, you can technically upload up to 150 MB. In practice, you'll likely want to use a size smaller than that for the sake of reliability. Additionally, using a smaller size means you won't have to re-upload as much if/when a particular request does fail. For instance, in practice you might want to use a chunk size of 12 MB, but you can experiment to see what size works well for your use case.

buttflattery
Explorer | Level 4
Go to solution

One thing more, the "filesUploadSessionFinishBatch" description says 


UploadSessionStartArg.close or UploadSessionAppendArg.close needs to be true for the last upload_session/start or upload_session/append:2 call.

Does this mean, if i have 4 files

- image1.jpg

- image2.jpg

- image3.jpg

- image4.jpg

i should set the "UploadSessionAppendArg.close" to true for the last chunk of every file OR set the "UploadSessionAppendArg.close" for the last chunk of the last file i.e "image4.jpg"? considering i will always upload in chunks.

@Greg-DB 

 

 

Greg-DB
Dropbox Staff
Go to solution

You should set the UploadSessionAppendArg.close to true for the last chunk of every file.

Each file gets its own upload session, and any upload session that you want to finish using filesUploadSessionFinishBatch needs to be "closed" before you can finish it with filesUploadSessionFinishBatch. "Closing" an upload session tells Dropbox that you won't be adding any more data to it.

buttflattery
Explorer | Level 4
Go to solution

perfect thanks alot @Greg-DB  you have been helpfull here other than

"passing the suggestion for a feature request" :stuck_out_tongue_closed_eyes:

Need more support?