<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: parallel upload failing - &amp;quot;lookup_failed/incorrect_offset&amp;quot; in Dropbox API Support &amp; Feedback</title>
    <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503968#M24936</link>
    <description>&lt;P&gt;Yes, for the offset on the /finish call for a concurrent upload session you should supply the total number of bytes successfully sent for the session previously (i.e., the total length of the file).&lt;/P&gt;</description>
    <pubDate>Thu, 11 Mar 2021 18:21:24 GMT</pubDate>
    <dc:creator>Greg-DB</dc:creator>
    <dc:date>2021-03-11T18:21:24Z</dc:date>
    <item>
      <title>parallel upload failing - "lookup_failed/incorrect_offset"</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503800#M24924</link>
      <description>&lt;P&gt;we need to upload large files (several GiB) to dropbox using the API. i've written a script that uses cURL to do this in parallel. i upload the first segment of 150M in order to obtain the session ID, then upload a number of additional segments in parallel, then close the session once the parallel threads have all finished. this doesn't work. i get the error "&lt;U&gt;&lt;STRONG&gt;&lt;FONT face="courier new,courier"&gt;lookup_failed/incorrect_offset&lt;/FONT&gt;&lt;/STRONG&gt;&lt;/U&gt;" followed by what appears to be an arbitrary number of dots (usually one to three). i'm puzzled by this because i've verified the offsets and they are correct.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;here is some sample debug output from an attempt to upload a 1Gb file using several parallel streams concurrently, with sensitive information redacted :&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;LI-CODE lang="markup"&gt;opening :
curl -x &amp;lt;redacted&amp;gt; -X POST https://content.dropboxapi.com/2/files/upload_session/start -H Content-Type:application/octet-stream -H Authorization:Bearer &amp;lt;redacted&amp;gt; -H Dropbox-API-Arg: {"close":false} --data-binary @-

parallel thread 1 :
curl -x &amp;lt;redacted&amp;gt; -X POST https://content.dropboxapi.com/2/files/upload_session/append_v2 -H Content-Type:application/octet-stream -H Authorization:Bearer &amp;lt;redacted&amp;gt; -H Dropbox-API-Arg: {"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":157286400},"close":false} --data-binary @-

parallel thread 2 :
...Dropbox-API-Arg: '{"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":314572800},"close":false}'

parallel thread 3 :
...Dropbox-API-Arg: '{"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":471859200},"close":false}'

parallel thread 4 :
...Dropbox-API-Arg: '{"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":629145600},"close":false}'

parallel thread 5 :
...Dropbox-API-Arg: '{"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":786432000},"close":false}'

closing :
curl -x &amp;lt;redacted&amp;gt; -X POST https://content.dropboxapi.com/2/files/upload_session/finish -H Content-Type:application/octet-stream -H Authorization:Bearer &amp;lt;redacted&amp;gt; -H Dropbox-API-Arg: {"cursor":{"session_id":"&amp;lt;redacted&amp;gt;","offset":943718400},"commit":{"path":"/par1.mp2","mode":"overwrite","autorename":false,"mute":false}} --data-binary @-&lt;/LI-CODE&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;so it seems the offsets are, indeed, correct. i've also verified that the session id returned after the session is opened (redacted in the dump above) remains unchanged throughout the session.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;is a parallel upload simply not possible with the dropbox API ?&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Thu, 11 Mar 2021 20:09:49 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503800#M24924</guid>
      <dc:creator>xtolchinskii</dc:creator>
      <dc:date>2021-03-11T20:09:49Z</dc:date>
    </item>
    <item>
      <title>Re: parallel upload failing - "lookup_failed/incorrect_offset"</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503915#M24928</link>
      <description>&lt;P&gt;By default, upload sessions only accept the data sequentially, meaning the app needs to only upload the pieces of the file for an upload session one by one in order.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;You can upload in&amp;nbsp;parallel if you want, but you need to set&amp;nbsp;session_type:concurrent and follow a few other constraints. Please refer to &lt;A href="https://www.dropbox.com/developers/documentation/http/documentation#files-upload_session-start" target="_self"&gt;the&amp;nbsp;/2/files/upload_session/start documentation&lt;/A&gt; for information on using that.&lt;/P&gt;</description>
      <pubDate>Thu, 11 Mar 2021 16:27:20 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503915#M24928</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2021-03-11T16:27:20Z</dc:date>
    </item>
    <item>
      <title>Re: parallel upload failing - "lookup_failed/incorrect_offset"</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503944#M24934</link>
      <description>&lt;P&gt;thanks &lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/10"&gt;@Greg-DB&lt;/a&gt;, i missed that in the docs. this looks promising, i'll give it a try.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;one question : since &lt;U&gt;&lt;FONT face="courier new,courier"&gt;/upload_session/finish&lt;/FONT&gt;&lt;/U&gt; in a concurrent upload won't accept file data along with it, what should the offset be in the cursor data for this case ?&amp;nbsp; the total number of bytes already sent with previous calls (i.e. the file size) ?&amp;nbsp; 0 ?&amp;nbsp; ...or maybe it's ignored ?&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Thu, 11 Mar 2021 17:08:58 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503944#M24934</guid>
      <dc:creator>xtolchinskii</dc:creator>
      <dc:date>2021-03-11T17:08:58Z</dc:date>
    </item>
    <item>
      <title>Re: parallel upload failing - "lookup_failed/incorrect_offset"</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503968#M24936</link>
      <description>&lt;P&gt;Yes, for the offset on the /finish call for a concurrent upload session you should supply the total number of bytes successfully sent for the session previously (i.e., the total length of the file).&lt;/P&gt;</description>
      <pubDate>Thu, 11 Mar 2021 18:21:24 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/parallel-upload-failing-quot-lookup-failed-incorrect-offset-quot/m-p/503968#M24936</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2021-03-11T18:21:24Z</dc:date>
    </item>
  </channel>
</rss>

