Dropbox API: Connection broken: IncompleteRead

Thanks for the report. Can you let me know:

• Are you referring to using the /2/files/download_zip endpoint?
• Does it always fail after 2013265920 bytes, or does that number vary?
• Does it always fail after a specific amount of time? If so, how long is that?

I'm facing a similar problem, but in my case I'm merely downloading a very large file (~125 GB). It usually crashes between 58000000000 - 49000000000 bytes read.

Hi denseishin 

I'll be happy to look further into this issue for you, please reply with:

• the name and version number of the platform and SDK/library you are using, if any
• the steps to reproduce the issue, including relevant code snippet(s), but don't
• include any access or refresh token(s)
• the full text of any error or unexpected output

Could you also confirm if requests are failing after a specific amount of time? And if so, how long is that?

• Python 3.8.10 on Ubuntu 20.04 with the Python dropbox package version 11.36.2

code snippet:

   def __init__():
       self.dbx = dropbox.Dropbox(API keys etc. here)
   def download(self, info: dict):
        dbx_path = info["path"]
        self.logger.log(logging.INFO,f"Downloading {dbx_path}")
        noroot = dbx_path.lstrip('/')
        dl_path = self.dl_path / pathlib.Path(noroot)
        dl_path.parent.mkdir(exist_ok=True, parents=True)
        metadata = self.dbx.files_download_to_file(str(dl_path), dbx_path)
        return metadata

INFO:dbx_downloader:Downloading (file name redacted)
INFO:dropbox:Request to files/download
ERROR:modulenameredacted:('Connection broken: IncompleteRead(54876571369 bytes read, 78618595296 more expected)', IncompleteRead(54876571369 bytes read, 78618595296 more expected))

I couldn't test for the amount of time yet

Here's a traceback.

INFO:redacted_module:2025-02-18 16:53:13.153338+00:00: Downloading REDACTED
INFO:dropbox:Request to files/download
ERROR:redacted_module:('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))
Traceback: Traceback (most recent call last):
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 748, in _error_catcher
    yield
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 894, in _raw_read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(55015292051 bytes read, 78479874614 more expected)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 820, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 1060, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 977, in read
    data = self._raw_read(amt)
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 902, in _raw_read
    self._fp.close()
  File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 772, in _error_catcher
    raise ProtocolError(arg, e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/customscript.py", line 104, in _process_msg
    self._work_callback(body)
  File "/path/to/anothercustomscript.py", line 59, in download
    metadata = self.dbx.files_download_to_file(str(dl_path), dbx_path)
  File "/path/to/.venv/lib/python3.8/site-packages/dropbox/base.py", line 1459, in files_download_to_file
    self._save_body_to_file(download_path, r[1])
  File "/path/to/.venv/lib/python3.8/site-packages/dropbox/dropbox_client.py", line 687, in _save_body_to_file
    for c in http_resp.iter_content(chunksize):
  File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 822, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))

It seems to repeat after almost exactly one hour

denseishin,

Thank you for that additional information. If the downloads seem to be failing at about an hour it is most likely due to the server timeout. There isn't an option to delay or prevent that.

The /2/files/download endpoint itself is a Content-Download style endpoint and does support Range requests. Using range requests in this case would be a solution we would recommend to handle/resume incomplete downloads. Keep in mind that even requests with ranges set are subject to any server timeouts, so you will want to continue monitoring any failures and use multiple range requests as needed.

Please keep in mind that I'm using the official Python SDK for Dropbox. Does that mean that it is not suitable for large files?

Hi denseishin,

Our official Python SDK and download endpoint are suitable for handling large files. However, depending on factors such as network conditions and server response times, long-running downloads may sometimes encounter timeouts.

The option to use range requests was suggested to help mitigate this—range requests allow downloading files in smaller parts rather than in a single request. This can help reduce the likelihood of timeouts while still enabling successful file retrieval. That said, you should still be able to download large files without issue under normal conditions.