Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
I am using Elixir wrapper to DropboxAPI to upload files, which is doing simple upload using `files/upload` route.
def upload(client, path, file, mode \\ "add", autorename \\ true, mute \\ false) do dropbox_headers = %{ :path => path, :mode => mode, :autorename => autorename, :mute => mute } headers = %{ "Dropbox-API-Arg" => Poison.encode!(dropbox_headers), "Content-Type" => "application/octet-stream" } upload_request( client, Application.get_env(:elixir_dropbox, :upload_url), "files/upload", file, headers ) end
But I am having an issue lately while uploading so many files parallelly, I am getting an error as `too many write operations`
I have been reading and looking into https://www.dropbox.com/developers/documentation/http/documentation#files-upload_session-start
But I am not following it totally, What I read from it, I can upload multiple files in one session, When I start a session while uploading one file, I think it works for sending chunks of a single file not, different files.
says, This route helps you commit many files at once into a user's Dropbox.
But how?? How I can send multiple files to Dropbox account not
Also I can send data only through upload session start and finish..
Finish batch never worked for me to upload all data at once.
Each of the /start, /append, and /finish endpoints, but not /finish_batch, can accept file data.
You would need to use /append if there is more data than you can send in the /start and /finish requests.
Upload sessions work by having you send only a portion of the file's data per request.
Exactly how much data you send per request is up to you. For example, a maximum of 8 MB per request is reasonable, but it can vary by use case. So, if you were using a maximum of 8 MB per request, and the file is only 2 MB, you would only need a single /start request to send all of the file data (and wouldn't need to call /append for that particular file). You could then use /finish without sending any more data to commit that file, or use /finish_batch to commit that file (as well as others).
Okay thanks for all your help and replies, right now what I am doing is:
client = ElixirDropbox.Client.new(System.get_env["DROP_BOX_TOKEN"]) {:ok, file_size} = get_file_size(image_path) %{"session_id" => session_id} = ElixirDropbox.Files.UploadSession.start(client, false, image_path) ElixirDropbox.Files.UploadSession.finish(client, session_id, upload_image_path, image_path, file_size) |> handle_upload_response
I still got one error as
"error_summary\": \"too_many_write_operations/\", \"error\": {\"reason\": {\".tag\": \"too_many_write_operations\"
and 5 errors are API timed out.
I have no idea why still there is too many write operations error? where as I am right now first starting the session, then finishing it in a proper way?
but these requests almost goes in parallel, should I need to change something as well? more..
In this code I see you're using the 'finish' method, presumably once per file, to finish the uploads. If you send off multiple of these at the same time, you will cause lock contention.
You should use a single 'finish_batch' to finish multiple file uploads in a single call to avoid this. (Or, only send the multiple 'finish' calls one at a time.)
I uploaded using session start and I got an session_id.
AAAAAAAb0vwx3ypjoH836Q
and then instead of doing append I just finished the batch of this file
curl -X POST https://api.dropboxapi.com/2/files/upload_session/finish_batch \ --header "Authorization: Bearer -SC69zip6zi9bwHgqhYHNYQbrpZ5C7G" \ --header "Content-Type: application/json" \ --data "{\"entries\": [{\"cursor\": {\"session_id\": \"AAAAAAAb0vwx3ypjoH836Q\",\"offset\": 34520},\"commit\": {\"path\": \"/REGULAR/licence\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}}]}"
This gave me results as
{".tag": "async_job_id", "async_job_id": "dbjid:AACg_hehcOljTPYxh6wQhigGQhtc3-eCFneDKK--vzA5V9wBLNi74iNeNqWUQDFOnx1_k-sViaZJpJvXRG0UTlLw"}
and whereas on normal session start and finish I just get the file uploaded moreover when I check batch as
curl -X POST https://api.dropboxapi.com/2/files/upload_session/finish_batch/check \ > --header "Authorization: Bearer -SC69zip6zi9bwHgqhYHNYQbrpZ5C7G" \ > --header "Content-Type: application/json" \ > --data "{\"async_job_id\": \"dbjid:AACg_hehcOljTPYxh6wQhigGQhtc3-eCFneDKK--vzA5V9wBLNi74iNeNqWUQDFOnx1_k-sViaZJpJvXRG0UTlLw\"}"
it returns as
{".tag": "complete", "entries": [{".tag": "failure", "failure": {".tag": "lookup_failed", "lookup_failed": {".tag": "not_closed"}}}]}
So now where I am wrong? the same file is getting uploaded through session start and finish but giving the error in this way?
@ijunaid8989 All of the upload sessions that you wish to finish using /finish_batch need to be "closed" before you call /finish_batch. That message is indicating that the upload session was not closed.
You should close the upload session with the last call you make to upload the final data for that upload session, whether that's on /start or /append, using the "close" parameter.
Thanks for all your replies and I have fixed this issue, my only question is right now, what could be the reason for timeouts from dropbox api?
I have timeout errors a few, while upload session start.
Can you share the code and error output you're getting for these timeouts so we can take a look? Thanks!
Hi there!
If you need more help you can view your support options (expected response time for a ticket is 24 hours), or contact us on X or Facebook.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!