<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Downloading large amount of files to my web application in Discuss Dropbox Developer &amp; API</title>
    <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725217#M3724</link>
    <description>&lt;P&gt;&lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/1760992"&gt;@daynightdev&lt;/a&gt;&amp;nbsp; and all other interested,&lt;/P&gt;&lt;P&gt;There are many undocumented Dropbox calls. Some of them may be useful in different cases (like the current one). One workaround, I know about, is usage of a call that deals with shared links (unfortunately that is inapplicable to temporary link). To use it, reorganization of files that need to be accessed may be needed - all of them have to be accessible in one folder and a link to this folder should exist or will be created. The workaround implies the Dropbox class update (named here 'DropboxLinkList' and implemented in Python) as follows:&lt;/P&gt;&lt;LI-CODE lang="python"&gt;#!/bin/python3
###############################################################################
# Addition of undocumented link listing functionality to regular Dropbox client
# -----------------------------------------------------------------------------
# file: dropbox_client_link_list.py​
# Author: Здравко
#   www.dropboxforum.com/t5/user/viewprofilepage/user-id/422790
###############################################################################

from dropbox import Dropbox
from dropbox.exceptions import BadInputError, HttpError, ApiError
from urllib.parse import urlparse, parse_qs
import json

class DropboxLinkList(Dropbox):
  """
  Use this class to make requests to the Dropbox API using a user's access
  token. Methods of this class are meant to act on the corresponding user's
  Dropbox. In addition :meth:`sharing_list_shared_link_folder` may be called.
  """
  def sharing_list_shared_link_folder(self, path_link):
    """
    Returns list of links (and other data) pointing to all direct entries
    (files/folders) residing in the pointed folder. When ``path_link`` refers
    to the target using path in the current context existing link is used or
    new one is created if there is no existing.
    
    Route attributes:
      No specific scopes are needed for this method usage,
      but :meth:`sharing_create_shared_link_with_settings` will be needed when
      path is in use to refer target. (i.e. sharing.write) and scope of
      :meth:`sharing_list_shared_links` may be needed too (i.e. sharing.read).
    
    :param str path_link: Either path pointing folder (in current context)
      containing all entries that need to be listed or Dropbox shared link to
      folder which content needs to be listed.
    
    The type of return result is not documented by Dropbox, but usable links
      are referred in '.shared_link_infos[].url' of the response.
    """
    if not path_link.startswith("https://"):
      try:
        metadata = self.sharing_create_shared_link_with_settings(path_link)
      except ApiError as e:
        er = e.error
        if not er.is_shared_link_already_exists():
          raise
        er = er.get_shared_link_already_exists()
        if er != None and er.is_metadata():
          metadata = er.get_metadata()
        else:
          metadata = sharing_list_shared_links(path_link, direct_only=True
                                               ).links[0]
        path_link = metadata.url
    if path_link.startswith("https://www.dropbox.com/sh/"):
      link_type = 's'
    elif path_link.startswith("https://www.dropbox.com/scl/fo/"):
      link_type = 'c'
    else:
      raise BadInputError('', f"Unsupported link type: {path_link}")
    url = urlparse(path_link)
    path = url.path.split('/')
    body = {'t': 'T', 'link_type': link_type}
    if link_type == 's':
      body['link_key'] = path[2]
      body['secure_hash'] = path[3]
      body['sub_path'] = "/".join(path[4:])
    else:
      body['link_key'] = path[3]
      body['secure_hash'] = path[4]
      body['sub_path'] = "/".join(path[5:])
      rlkey = parse_qs(url.query)
      if 'rlkey' not in rlkey:
        raise BadInputError('',
          f"Expected 'rlkey' query param, but is missing: {path_link}")
      body['rlkey'] = rlkey['rlkey'][0]
    url = 'https://www.dropbox.com/list_shared_link_folder_entries'
    headers = {'Content-Type': 'application/x-www-form-urlencoded',
               'Cookie': 't=T'}
    body = "&amp;amp;".join(f"{k}={v}" for (k,v) in body.items())
    r = self._session.post(url, headers=headers, data=body, stream=False,
                           verify=True, timeout=self._timeout)
    if r.status_code != 200:
      raise HttpError('', r.status_code, r.content.decode())
    assert r.headers.get('content-type') == 'application/json', (
                'Expected content-type to be application/json, got %r' %
                r.headers.get('content-type'))
    return json.loads(r.content.decode())&lt;/LI-CODE&gt;&lt;P&gt;As mentioned within the code, existing link of a folder may be listed or directly referring to the folder keeping needed files (with path, id, etc.). Let's say somebody keeps lot of files in a folder with path '/ForListing' that need to be pointed with a link (every one). Instead of looping through all of them and get links one by one (generating lot of API calls), the mentioned folder may be enumerated directly with just few calls (2 or 3 at most):&lt;/P&gt;&lt;LI-CODE lang="python"&gt;import .dropbox_client_link_list

# Proper init here - the same like original Dropbox class
dbx = dropbox_client_link_list.DropboxLinkList(...)
container_folder = '/ForListing​'

...

for lm in dbx.sharing_list_shared_link_folder(container_folder)['shared_link_infos']:
    print(lm['url'])

...&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;The above will list links for all entries (files/folders) inside 'ForListing' folder. They will not be generated - they exist (pre-generated) while files reside within pointed folder. If needed files are hierarchically spread in directory subtree of the pointed folder (not just flat in a single folder), recursive enumeration may be needed, as follows:&lt;/P&gt;&lt;LI-CODE lang="python"&gt;...

def linkList(path_link, prefix=''):
    entries = dbx.sharing_list_shared_link_folder(path_link)
    for lm, tm in itertools.zip_longest(entries['shared_link_infos'], entries['entries']):
        url = lm['url']
        is_dir = tm['is_dir']
        print(f"{'D: ' if is_dir else 'F: '}{prefix}{url}")
        if is_dir:
            linkList(url, prefix + "  ")

linkList(container_folder)&lt;/LI-CODE&gt;&lt;P&gt;Here one more call for each subfolder will be needed. The chance for API call rate overrun to be meet here is negligible. The above code-snippets are for illustration only. The idea is simple enough, so easy enough to be implemented/'translated' into any other programming language. &lt;img class="lia-deferred-image lia-image-emoji" src="https://www.dropboxforum.com/html/@41457EF40051AFF130FDBFE21B496926/emoticons/1f609.png" alt=":winking_face:" title=":winking_face:" /&gt;&lt;/P&gt;&lt;P&gt;Good luck.&lt;/P&gt;</description>
    <pubDate>Sat, 28 Oct 2023 23:39:58 GMT</pubDate>
    <dc:creator>Здравко</dc:creator>
    <dc:date>2023-10-28T23:39:58Z</dc:date>
    <item>
      <title>Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724931#M3716</link>
      <description>&lt;P&gt;Hi,&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;I'm currently using the Dropbox &lt;A href="https://dropbox.github.io/dropbox-sdk-js/Dropbox.html#filesDownload__anchor" target="_self"&gt;download endpoint&lt;/A&gt; in a Vue framework, but I'm experiencing slow loading times for photos due to the large volume I'm downloading. Is there an alternative endpoint, such as batchDownload, that I can use to improve the speed? Or should I consider changing my approach to using the download endpoint?&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Thank you for your assistance.&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 13:03:32 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724931#M3716</guid>
      <dc:creator>daynightdev</dc:creator>
      <dc:date>2023-10-27T13:03:32Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724945#M3717</link>
      <description>&lt;P&gt;If the files are in one or a few folders, you can use &lt;A href="https://dropbox.github.io/dropbox-sdk-js/Dropbox.html#filesDownload__anchor" target="_blank"&gt;filesDownloadZip&lt;/A&gt; to download the entire folder(s) and unzip them locally, instead of downloading each file individually.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;Otherwise, you may want to try running several &lt;A href="https://dropbox.github.io/dropbox-sdk-js/Dropbox.html#filesDownload__anchor" target="_blank"&gt;filesDownload&lt;/A&gt; calls at a time in parallel, if you aren't already.&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 13:45:21 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724945#M3717</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2023-10-27T13:45:21Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724949#M3718</link>
      <description>&lt;P&gt;&lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/1760992"&gt;@daynightdev&lt;/a&gt;, in addition to Greg's proposal, you may considered usage of links (either permanent or temporary) and let client's browser to manage downloads itself. &lt;img class="lia-deferred-image lia-image-emoji" src="https://www.dropboxforum.com/html/@41457EF40051AFF130FDBFE21B496926/emoticons/1f609.png" alt=":winking_face:" title=":winking_face:" /&gt;&lt;/P&gt;&lt;P&gt;Good luck.&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 13:56:24 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/724949#M3718</guid>
      <dc:creator>Здравко</dc:creator>
      <dc:date>2023-10-27T13:56:24Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725001#M3720</link>
      <description>&lt;P&gt;&lt;SPAN&gt;I apologize for not providing the purpose behind my file downloads. I download these files to display them on my web application. I attempted to use '&lt;A href="https://www.dropbox.com/developers/documentation/http/documentation#files-get_temporary_link" target="_self"&gt;GetTemporaryLink&lt;/A&gt;,' but I encountered an error: '429 Too Many Requests.' Can you provide insights on how to resolve this '429 Too Many Requests' issue? Should I consider debouncing or delaying my requests?&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 16:13:16 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725001#M3720</guid>
      <dc:creator>daynightdev</dc:creator>
      <dc:date>2023-10-27T16:13:16Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725009#M3721</link>
      <description>&lt;P&gt;Yes, in such a case you may considered exponential back off If no header denoting proposed delay. Unfortunately, there is not 'bunch' version to get links at once.&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 16:23:03 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725009#M3721</guid>
      <dc:creator>Здравко</dc:creator>
      <dc:date>2023-10-27T16:23:03Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725046#M3723</link>
      <description>&lt;P&gt;&lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/1760992"&gt;@daynightdev&lt;/a&gt;&amp;nbsp;Здравко is correct; when you get a 429, the best practice is to retry the request, respecting the Retry-After header if given in the response, or using an exponential back-off, if not. I recommend referring to the &lt;A href="https://www.dropbox.com/developers/documentation/http/documentation#error-handling" target="_blank" rel="noopener noreferrer"&gt;error documentation&lt;/A&gt; and &lt;A href="https://developers.dropbox.com/error-handling-guide" target="_blank" rel="noopener noreferrer"&gt;Error Handling Guide&lt;/A&gt; for more information.&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;The Dropbox API does not offer a batch version of that method, but I'll pass this along as a feature request. I can't promise if or when that might be implemented though.&lt;/P&gt;</description>
      <pubDate>Fri, 27 Oct 2023 17:32:04 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725046#M3723</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2023-10-27T17:32:04Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725217#M3724</link>
      <description>&lt;P&gt;&lt;a href="https://www.dropboxforum.com/t5/user/viewprofilepage/user-id/1760992"&gt;@daynightdev&lt;/a&gt;&amp;nbsp; and all other interested,&lt;/P&gt;&lt;P&gt;There are many undocumented Dropbox calls. Some of them may be useful in different cases (like the current one). One workaround, I know about, is usage of a call that deals with shared links (unfortunately that is inapplicable to temporary link). To use it, reorganization of files that need to be accessed may be needed - all of them have to be accessible in one folder and a link to this folder should exist or will be created. The workaround implies the Dropbox class update (named here 'DropboxLinkList' and implemented in Python) as follows:&lt;/P&gt;&lt;LI-CODE lang="python"&gt;#!/bin/python3
###############################################################################
# Addition of undocumented link listing functionality to regular Dropbox client
# -----------------------------------------------------------------------------
# file: dropbox_client_link_list.py​
# Author: Здравко
#   www.dropboxforum.com/t5/user/viewprofilepage/user-id/422790
###############################################################################

from dropbox import Dropbox
from dropbox.exceptions import BadInputError, HttpError, ApiError
from urllib.parse import urlparse, parse_qs
import json

class DropboxLinkList(Dropbox):
  """
  Use this class to make requests to the Dropbox API using a user's access
  token. Methods of this class are meant to act on the corresponding user's
  Dropbox. In addition :meth:`sharing_list_shared_link_folder` may be called.
  """
  def sharing_list_shared_link_folder(self, path_link):
    """
    Returns list of links (and other data) pointing to all direct entries
    (files/folders) residing in the pointed folder. When ``path_link`` refers
    to the target using path in the current context existing link is used or
    new one is created if there is no existing.
    
    Route attributes:
      No specific scopes are needed for this method usage,
      but :meth:`sharing_create_shared_link_with_settings` will be needed when
      path is in use to refer target. (i.e. sharing.write) and scope of
      :meth:`sharing_list_shared_links` may be needed too (i.e. sharing.read).
    
    :param str path_link: Either path pointing folder (in current context)
      containing all entries that need to be listed or Dropbox shared link to
      folder which content needs to be listed.
    
    The type of return result is not documented by Dropbox, but usable links
      are referred in '.shared_link_infos[].url' of the response.
    """
    if not path_link.startswith("https://"):
      try:
        metadata = self.sharing_create_shared_link_with_settings(path_link)
      except ApiError as e:
        er = e.error
        if not er.is_shared_link_already_exists():
          raise
        er = er.get_shared_link_already_exists()
        if er != None and er.is_metadata():
          metadata = er.get_metadata()
        else:
          metadata = sharing_list_shared_links(path_link, direct_only=True
                                               ).links[0]
        path_link = metadata.url
    if path_link.startswith("https://www.dropbox.com/sh/"):
      link_type = 's'
    elif path_link.startswith("https://www.dropbox.com/scl/fo/"):
      link_type = 'c'
    else:
      raise BadInputError('', f"Unsupported link type: {path_link}")
    url = urlparse(path_link)
    path = url.path.split('/')
    body = {'t': 'T', 'link_type': link_type}
    if link_type == 's':
      body['link_key'] = path[2]
      body['secure_hash'] = path[3]
      body['sub_path'] = "/".join(path[4:])
    else:
      body['link_key'] = path[3]
      body['secure_hash'] = path[4]
      body['sub_path'] = "/".join(path[5:])
      rlkey = parse_qs(url.query)
      if 'rlkey' not in rlkey:
        raise BadInputError('',
          f"Expected 'rlkey' query param, but is missing: {path_link}")
      body['rlkey'] = rlkey['rlkey'][0]
    url = 'https://www.dropbox.com/list_shared_link_folder_entries'
    headers = {'Content-Type': 'application/x-www-form-urlencoded',
               'Cookie': 't=T'}
    body = "&amp;amp;".join(f"{k}={v}" for (k,v) in body.items())
    r = self._session.post(url, headers=headers, data=body, stream=False,
                           verify=True, timeout=self._timeout)
    if r.status_code != 200:
      raise HttpError('', r.status_code, r.content.decode())
    assert r.headers.get('content-type') == 'application/json', (
                'Expected content-type to be application/json, got %r' %
                r.headers.get('content-type'))
    return json.loads(r.content.decode())&lt;/LI-CODE&gt;&lt;P&gt;As mentioned within the code, existing link of a folder may be listed or directly referring to the folder keeping needed files (with path, id, etc.). Let's say somebody keeps lot of files in a folder with path '/ForListing' that need to be pointed with a link (every one). Instead of looping through all of them and get links one by one (generating lot of API calls), the mentioned folder may be enumerated directly with just few calls (2 or 3 at most):&lt;/P&gt;&lt;LI-CODE lang="python"&gt;import .dropbox_client_link_list

# Proper init here - the same like original Dropbox class
dbx = dropbox_client_link_list.DropboxLinkList(...)
container_folder = '/ForListing​'

...

for lm in dbx.sharing_list_shared_link_folder(container_folder)['shared_link_infos']:
    print(lm['url'])

...&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;The above will list links for all entries (files/folders) inside 'ForListing' folder. They will not be generated - they exist (pre-generated) while files reside within pointed folder. If needed files are hierarchically spread in directory subtree of the pointed folder (not just flat in a single folder), recursive enumeration may be needed, as follows:&lt;/P&gt;&lt;LI-CODE lang="python"&gt;...

def linkList(path_link, prefix=''):
    entries = dbx.sharing_list_shared_link_folder(path_link)
    for lm, tm in itertools.zip_longest(entries['shared_link_infos'], entries['entries']):
        url = lm['url']
        is_dir = tm['is_dir']
        print(f"{'D: ' if is_dir else 'F: '}{prefix}{url}")
        if is_dir:
            linkList(url, prefix + "  ")

linkList(container_folder)&lt;/LI-CODE&gt;&lt;P&gt;Here one more call for each subfolder will be needed. The chance for API call rate overrun to be meet here is negligible. The above code-snippets are for illustration only. The idea is simple enough, so easy enough to be implemented/'translated' into any other programming language. &lt;img class="lia-deferred-image lia-image-emoji" src="https://www.dropboxforum.com/html/@41457EF40051AFF130FDBFE21B496926/emoticons/1f609.png" alt=":winking_face:" title=":winking_face:" /&gt;&lt;/P&gt;&lt;P&gt;Good luck.&lt;/P&gt;</description>
      <pubDate>Sat, 28 Oct 2023 23:39:58 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725217#M3724</guid>
      <dc:creator>Здравко</dc:creator>
      <dc:date>2023-10-28T23:39:58Z</dc:date>
    </item>
    <item>
      <title>Re: Downloading large amount of files to my web application</title>
      <link>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725478#M3725</link>
      <description>&lt;P&gt;Please note that undocumented endpoints are not meant for third party use, and are subject to change without notice.&lt;/P&gt;</description>
      <pubDate>Mon, 30 Oct 2023 15:52:19 GMT</pubDate>
      <guid>https://www.dropboxforum.com/t5/Discuss-Dropbox-Developer-API/Downloading-large-amount-of-files-to-my-web-application/m-p/725478#M3725</guid>
      <dc:creator>Greg-DB</dc:creator>
      <dc:date>2023-10-30T15:52:19Z</dc:date>
    </item>
  </channel>
</rss>

