Forum Discussion

buttflattery's avatar
buttflattery
Explorer | Level 4
6 years ago
Solved

Max files Upload using javascript SDK

I can find any relevant section that addresses how many files i can upload in concurrent requests. means if i am using javascript SDK and i trigger 100 files via looping the code that creates a promise for every request. Are there any limits to how many requests i can trigger at a time?

  • Greg-DB's avatar
    Greg-DB
    6 years ago

    You will need to create one "upload session" per file. Each file can be uploaded in multiple pieces or "chunks". Once you've uploaded all of the chunks for each file, you can commit all of the files at once by using the /files/upload_session/finish_batch endpoint. That will help avoid lock contention.

    For each single request to upload a chunk of a file, you can technically upload up to 150 MB. In practice, you'll likely want to use a size smaller than that for the sake of reliability. Additionally, using a smaller size means you won't have to re-upload as much if/when a particular request does fail. For instance, in practice you might want to use a chunk size of 12 MB, but you can experiment to see what size works well for your use case.

    • buttflattery's avatar
      buttflattery
      Explorer | Level 4

      hmm that means dropbox recommends that if there are alot of files i should upload all those file in a single sesssion and commit all of them once. And to upload those files i should upload in chunks as i need to manage the upload progress Correct?

      Second thing, if the above is correct what is the max size of the chunk that is recommended keeping in mind the requests rate limit?

      • Greg-DB's avatar
        Greg-DB
        Icon for Dropbox Staff rankDropbox Staff

        You will need to create one "upload session" per file. Each file can be uploaded in multiple pieces or "chunks". Once you've uploaded all of the chunks for each file, you can commit all of the files at once by using the /files/upload_session/finish_batch endpoint. That will help avoid lock contention.

        For each single request to upload a chunk of a file, you can technically upload up to 150 MB. In practice, you'll likely want to use a size smaller than that for the sake of reliability. Additionally, using a smaller size means you won't have to re-upload as much if/when a particular request does fail. For instance, in practice you might want to use a chunk size of 12 MB, but you can experiment to see what size works well for your use case.