Announcements
Do you have an idea for Dropbox Paper? Share it with us!

Re: questRe: too many write operations when uploading files

Explorer | Level 3

Also I can send data only through upload session start and finish..

 

Finish batch never worked for me to upload all data at once.

Reply
Loading...

Re: questRe: too many write operations when uploading files

Dropboxer

Each of the /start, /append, and /finish endpoints, but not /finish_batch, can accept file data. 

You would need to use /append if there is more data than you can send in the /start and /finish requests.

Upload sessions work by having you send only a portion of the file's data per request. 

Exactly how much data you send per request is up to you. For example, a maximum of 8 MB per request is reasonable, but it can vary by use case. So, if you were using a maximum of 8 MB per request, and the file is only 2 MB, you would only need a single /start request to send all of the file data (and wouldn't need to call /append for that particular file). You could then use /finish without sending any more data to commit that file, or use /finish_batch to commit that file (as well as others).

Reply
Loading...

Re: questRe: too many write operations when uploading files

Explorer | Level 3

Okay thanks for all your help and replies, right now what I am doing is:

 

    client = ElixirDropbox.Client.new(System.get_env["DROP_BOX_TOKEN"])
    {:ok, file_size} = get_file_size(image_path)
    %{"session_id" => session_id} = ElixirDropbox.Files.UploadSession.start(client, false, image_path)
    ElixirDropbox.Files.UploadSession.finish(client, session_id, upload_image_path, image_path, file_size) |> handle_upload_response

I still got one error as

 

"error_summary\": \"too_many_write_operations/\", \"error\": {\"reason\": {\".tag\": \"too_many_write_operations\"

and 5 errors are API timed out. 

 

I have no idea why still there is too many write operations error? where as I am right now first starting the session, then finishing it in a proper way? 

but these requests almost goes in parallel, should I need to change something as well? more..

Reply
Loading...

Re: questRe: too many write operations when uploading files

Dropboxer

In this code I see you're using the 'finish' method, presumably once per file, to finish the uploads. If you send off multiple of these at the same time, you will cause lock contention. 

You should use a single 'finish_batch' to finish multiple file uploads in a single call to avoid this. (Or, only send the multiple 'finish' calls one at a time.)

Reply
Loading...

Re: questRe: too many write operations when uploading files

Explorer | Level 3

I uploaded using session start and I got an session_id. 

AAAAAAAb0vwx3ypjoH836Q

and then instead of doing append I just finished the batch of this file 

 

curl -X POST https://api.dropboxapi.com/2/files/upload_session/finish_batch \
    --header "Authorization: Bearer -SC69zip6zi9bwHgqhYHNYQbrpZ5C7G" \
    --header "Content-Type: application/json" \
    --data "{\"entries\": [{\"cursor\": {\"session_id\": \"AAAAAAAb0vwx3ypjoH836Q\",\"offset\": 34520},\"commit\": {\"path\": \"/REGULAR/licence\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}}]}"

This gave me results as 

 

{".tag": "async_job_id", "async_job_id": "dbjid:AACg_hehcOljTPYxh6wQhigGQhtc3-eCFneDKK--vzA5V9wBLNi74iNeNqWUQDFOnx1_k-sViaZJpJvXRG0UTlLw"}

and whereas on normal session start and finish I just get the file uploaded moreover  when I check batch  as

curl -X POST https://api.dropboxapi.com/2/files/upload_session/finish_batch/check \
>     --header "Authorization: Bearer -SC69zip6zi9bwHgqhYHNYQbrpZ5C7G" \
>     --header "Content-Type: application/json" \
>     --data "{\"async_job_id\": \"dbjid:AACg_hehcOljTPYxh6wQhigGQhtc3-eCFneDKK--vzA5V9wBLNi74iNeNqWUQDFOnx1_k-sViaZJpJvXRG0UTlLw\"}"

it returns as 

 

{".tag": "complete", "entries": [{".tag": "failure", "failure": {".tag": "lookup_failed", "lookup_failed": {".tag": "not_closed"}}}]}

So now where I am wrong? the same file is getting uploaded through session start and finish but giving the error in this way? 

 

Reply
Loading...
Highlighted

Re: questRe: too many write operations when uploading files

Dropboxer

@ijunaid8989 All of the upload sessions that you wish to finish using /finish_batch need to be "closed" before you call /finish_batch. That message is indicating that the upload session was not closed.

You should close the upload session with the last call you make to upload the final data for that upload session, whether that's on /start or /append, using the "close" parameter.

Reply
Loading...

Re: questRe: too many write operations when uploading files

Explorer | Level 3

Thanks for all your replies and I have fixed this issue, my only question is right now, what could be the reason for timeouts from dropbox api? 

 

I have timeout errors a few, while upload session start.

Reply
Loading...

Re: questRe: too many write operations when uploading files

Dropboxer

Can you share the code and error output you're getting for these timeouts so we can take a look? Thanks!

Reply
Loading...
API Support

Have a question? Our Dropbox Community is here to help you find an answer!
You won’t have to wait too long for a reply, but why not learn more about the Community while you’re here.

Not the right discussion for you? Find out what else our Community has to say about API Support.

Or, search our Help Center for even more.