<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: list_folder_continue -&amp;gt; 504 errors in Dropbox API Support &amp; Feedback</title>
    <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299141#M18175</link>
    <description>&lt;P&gt;Yes, this is likely related to the number of&amp;nbsp;files, but possibly also the&amp;nbsp;amount of file activity. (So, yes, as the number of files goes to infinity, the working limit&amp;nbsp;would go down.)&lt;/P&gt;</description>
    <pubDate>Tue, 25 Sep 2018 16:36:21 GMT</pubDate>
    <dc:creator>Greg-DB</dc:creator>
    <dc:date>2018-09-25T16:36:21Z</dc:date>
    <item>
      <title>list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299023#M18159</link>
      <description>&lt;P&gt;Hello,&lt;/P&gt;&lt;P&gt;I'm using `list_folder_continue` of the Python SDK to recursively traverse (via pagination) a folder with a very large number of files (hundreds of thousands).&amp;nbsp; After a while, I'm running into internal server errors. Here's the log:&lt;/P&gt;&lt;P&gt;2018-09-24 20:09:12,831 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:10:42,843 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-24 20:10:42,845 - dropbox - INFO - HttpError status_code=504: Retrying in 0.7 seconds&lt;BR /&gt;2018-09-24 20:10:43,505 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:12:13,521 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-24 20:12:13,522 - dropbox - INFO - HttpError status_code=504: Retrying in 0.8 seconds&lt;BR /&gt;2018-09-24 20:12:14,320 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:13:44,336 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-24 20:13:44,337 - dropbox - INFO - &lt;A href="https://www.google.com/search?client=ubuntu&amp;amp;channel=fs&amp;amp;q=recusrively+traverse+&amp;amp;ie=utf-8&amp;amp;oe=utf-8HttpError" target="_blank"&gt;https://www.google.com/search?client=ubuntu&amp;amp;channel=fs&amp;amp;q=recusrively+traverse+&amp;amp;ie=utf-8&amp;amp;oe=utf-8HttpError&lt;/A&gt; status_code=504: Retrying in 6.8 seconds&lt;BR /&gt;2018-09-24 20:13:51,148 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:15:21,163 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-24 20:15:21,165 - dropbox - INFO - HttpError status_code=504: Retrying in 9.4 seconds&lt;BR /&gt;2018-09-24 20:15:30,565 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:17:00,580 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-24 20:17:00,582 - dropbox - INFO - HttpError status_code=504: Retrying in 2.0 seconds&lt;BR /&gt;2018-09-24 20:17:02,557 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-24 20:18:32,572 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;/P&gt;&lt;P&gt;Is this attributable to the overall activity in the account (tis a business account with 10 users)?&lt;/P&gt;&lt;P&gt;Many thanks&lt;/P&gt;</description>
      <pubDate>Wed, 29 May 2019 09:10:14 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299023#M18159</guid>
      <dc:creator>deeuu</dc:creator>
      <dc:date>2019-05-29T09:10:14Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299027#M18161</link>
      <description>&lt;P&gt;I should also add that when re-running my script, calling list_folder_continue with the most recent cursor prior to catching the InternalServerError, takes a very long time to get going again, with the same API request attempts:&lt;/P&gt;&lt;P&gt;2018-09-25 08:59:14,422 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-25 09:00:44,445 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-25 09:00:44,446 - dropbox - INFO - HttpError status_code=504: Retrying in 2.0 seconds&lt;BR /&gt;2018-09-25 09:00:46,445 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-25 09:02:16,463 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-25 09:02:16,464 - dropbox - INFO - HttpError status_code=504: Retrying in 3.8 seconds&lt;BR /&gt;2018-09-25 09:02:20,277 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-25 09:03:50,319 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-25 09:03:50,320 - dropbox - INFO - HttpError status_code=504: Retrying in 7.6 seconds&lt;BR /&gt;2018-09-25 09:03:57,961 - dropbox - INFO - Request to files/list_folder/continue&lt;BR /&gt;2018-09-25 09:05:27,982 - urllib3.connectionpool - DEBUG - &lt;A href="https://api.dropboxapi.com:443" target="_blank"&gt;https://api.dropboxapi.com:443&lt;/A&gt; "POST /2/files/list_folder/continue HTTP/1.1" 504 35&lt;BR /&gt;2018-09-25 09:05:27,984 - dropbox - INFO - HttpError status_code=504: Retrying in 0.3 seconds&lt;BR /&gt;2018-09-25 09:05:28,278 - dropbox - INFO - Request to files/list_folder/continue&lt;/P&gt;</description>
      <pubDate>Tue, 25 Sep 2018 08:08:45 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299027#M18161</guid>
      <dc:creator>deeuu</dc:creator>
      <dc:date>2018-09-25T08:08:45Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299107#M18168</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Thanks for the report! This can be related to these operations taking a long time due to a large number of files and/or a large amount of file activity in the affected account(s).&lt;/SPAN&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;SPAN&gt;We'll look into it, but there are a few potential workarounds:&lt;/SPAN&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;SPAN&gt;1) Use the 'limit' parameter on files_list_folder:&lt;/SPAN&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;A href="https://dropbox-sdk-python.readthedocs.io/en/latest/moduledoc.html#dropbox.dropbox.Dropbox.files_list_folder" rel="noreferrer" target="_blank"&gt;https://dropbox-sdk-python.readthedocs.io/en/latest/moduledoc.html#dropbox.dropbox.Dropbox.files_list_folder&lt;/A&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;SPAN&gt;If you specify a smaller value (the default is effectively around 2000), that should help reduce how long each of these calls takes and reduce the likelihood that they will fail. Note that you supply the 'limit' value to&amp;nbsp;&lt;/SPAN&gt;&lt;A href="http://files_list_folder/" rel="noreferrer" target="_blank"&gt;files_list_folder&lt;/A&gt;&amp;nbsp;&lt;SPAN&gt;itself, and it will apply to all results from&amp;nbsp;&lt;/SPAN&gt;&lt;A href="https://dropbox-sdk-python.readthedocs.io/en/latest/moduledoc.html#dropbox.dropbox.Dropbox.files_list_folder_continue" rel="noreferrer" target="_blank"&gt;files_list_folder_continue&lt;/A&gt;&lt;SPAN&gt;&amp;nbsp;using the returned cursor as well.&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;2&lt;SPAN&gt;) If you are using recursive=True,&amp;nbsp;switch to recursive=False when calling&amp;nbsp;&lt;/SPAN&gt;&lt;A href="https://dropbox-sdk-python.readthedocs.io/en/latest/moduledoc.html#dropbox.dropbox.Dropbox.files_list_folder" rel="noreferrer" target="_blank"&gt;files_list_folder&lt;/A&gt;&lt;SPAN&gt;. This means you would need to make a call for each sub-folder you need though.&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Tue, 25 Sep 2018 14:34:04 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299107#M18168</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2018-09-25T14:34:04Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299140#M18174</link>
      <description>&lt;P&gt;Thanks for getting back to me Greg.&lt;/P&gt;&lt;P&gt;I'd rather not have to go with #2 because this implies I can't reliably check for changes in the folder at a later stage using the cursor obtained from `files_list_folder_get_latest_cursor`. i.e. #1 would require me to traverse the entire folder each time I want to check for changes (or require me to store multiple cursors for sub-directories etc.).&lt;/P&gt;&lt;P&gt;I've already been `running &lt;SPAN&gt;files_list_folder`&lt;/SPAN&gt; with `limit` set to 500, and each time the internal server error occurs after about approx 310,000 files/folders. So I can almost predict when the error will occur.&lt;/P&gt;&lt;P&gt;I'll try it again with `limit` set to 50 and report back tomorrow.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Is this issue related to the number of files in a folder, i.e. limit -&amp;gt; 1 as number of files -&amp;gt; infinity? If so, I'll stop using cursors altogether.&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Tue, 25 Sep 2018 16:30:37 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299140#M18174</guid>
      <dc:creator>deeuu</dc:creator>
      <dc:date>2018-09-25T16:30:37Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299141#M18175</link>
      <description>&lt;P&gt;Yes, this is likely related to the number of&amp;nbsp;files, but possibly also the&amp;nbsp;amount of file activity. (So, yes, as the number of files goes to infinity, the working limit&amp;nbsp;would go down.)&lt;/P&gt;</description>
      <pubDate>Tue, 25 Sep 2018 16:36:21 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299141#M18175</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2018-09-25T16:36:21Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299300#M18182</link>
      <description>&lt;P&gt;Hi &lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/10"&gt;@Greg-DB&lt;/a&gt;,&lt;/P&gt;&lt;P&gt;So, after about 270,000 files/folders, I received an Internal Server Error (with `limit` set to 50), and can't seem to get going again with the cursor I stored prior to the collapse.&lt;/P&gt;&lt;P&gt;I guess I have to conclude that cursors aren't reliable for traversing folders with many files and sub-folders.&lt;/P&gt;&lt;P&gt;I'm not even sure its worth re-designing my app to use cursors for checking the state of each sub-directory, so I'll have to go with your #2 each time.&lt;/P&gt;</description>
      <pubDate>Wed, 26 Sep 2018 08:56:53 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299300#M18182</guid>
      <dc:creator>deeuu</dc:creator>
      <dc:date>2018-09-26T08:56:53Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299392#M18187</link>
      <description>&lt;P&gt;Thanks for trying that, and my apologies for the trouble. I'll follow up here when I have any updates on this issue.&lt;/P&gt;</description>
      <pubDate>Wed, 26 Sep 2018 18:21:02 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/299392#M18187</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2018-09-26T18:21:02Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/586714#M27303</link>
      <description>&lt;P&gt;Hello, any updates on this issue ? The issue is not solved, we still face this issue&lt;/P&gt;</description>
      <pubDate>Mon, 28 Mar 2022 12:54:03 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/586714#M27303</guid>
      <dc:creator>NathanA</dc:creator>
      <dc:date>2022-03-28T12:54:03Z</dc:date>
    </item>
    <item>
      <title>Re: list_folder_continue -&gt; 504 errors</title>
      <link>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/587944#M27367</link>
      <description>&lt;P&gt;In addition to the techniques mentioned in this thread, I would also suggest traversing with&amp;nbsp;include_mounted_folders=false.&lt;/P&gt;
&lt;P&gt;Mounted folders refer to folder shares or team folders. With that option, the mount point (shared folder id) is still returned in the output, but not the share's content.&amp;nbsp; You could then traverse the share independently with a different cursor.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;This is particularly helpful for large team folders or scenarios with a lot of large shares.&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;
&lt;P&gt;We have updated our reference guides with best practices since this thread started, &lt;A href="https://www.dropbox.com/developers/reference" target="_self"&gt;check them out here&lt;/A&gt;.&lt;/P&gt;
&lt;P&gt;I don't have a specific update on the 504 error the original poster mentioned - are you seeing the 504 code, or something else?&amp;nbsp; Including a little more detail on the error response you're seeing may help.&lt;/P&gt;</description>
      <pubDate>Sat, 02 Apr 2022 18:25:28 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/list-folder-continue-gt-504-errors/m-p/587944#M27367</guid>
      <dc:creator>kylea</dc:creator>
      <dc:date>2022-04-02T18:25:28Z</dc:date>
    </item>
  </channel>
</rss>

