We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
Dropbox API: Connection broken: IncompleteRead
Hi everyone,
I am using the API to zip and download folders each with 1000 .jpegs in them (approx 4GB - 5GB). Historically this has been no issue. In the last week however, I have started receiving...
Here's a traceback.
INFO:redacted_module:2025-02-18 16:53:13.153338+00:00: Downloading REDACTED
INFO:dropbox:Request to files/download
ERROR:redacted_module:('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))
Traceback: Traceback (most recent call last):
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 748, in _error_catcher
yield
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 894, in _raw_read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(55015292051 bytes read, 78479874614 more expected)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 820, in generate
yield from self.raw.stream(chunk_size, decode_content=True)
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 1060, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 977, in read
data = self._raw_read(amt)
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 902, in _raw_read
self._fp.close()
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 772, in _error_catcher
raise ProtocolError(arg, e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/path/to/customscript.py", line 104, in _process_msg
self._work_callback(body)
File "/path/to/anothercustomscript.py", line 59, in download
metadata = self.dbx.files_download_to_file(str(dl_path), dbx_path)
File "/path/to/.venv/lib/python3.8/site-packages/dropbox/base.py", line 1459, in files_download_to_file
self._save_body_to_file(download_path, r[1])
File "/path/to/.venv/lib/python3.8/site-packages/dropbox/dropbox_client.py", line 687, in _save_body_to_file
for c in http_resp.iter_content(chunksize):
File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 822, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))It seems to repeat after almost exactly one hour
DB-Des
Dropbox Community Moderator
Dropbox Community ModeratorThank you for that additional information. If the downloads seem to be failing at about an hour it is most likely due to the server timeout. There isn't an option to delay or prevent that.
The /2/files/download endpoint itself is a Content-Download style endpoint and does support Range requests. Using range requests in this case would be a solution we would recommend to handle/resume incomplete downloads. Keep in mind that even requests with ranges set are subject to any server timeouts, so you will want to continue monitoring any failures and use multiple range requests as needed.
Please keep in mind that I'm using the official Python SDK for Dropbox. Does that mean that it is not suitable for large files?
- DB-Des
Hi denseishin,
Our official Python SDK and download endpoint are suitable for handling large files. However, depending on factors such as network conditions and server response times, long-running downloads may sometimes encounter timeouts.
The option to use range requests was suggested to help mitigate this—range requests allow downloading files in smaller parts rather than in a single request. This can help reduce the likelihood of timeouts while still enabling successful file retrieval. That said, you should still be able to download large files without issue under normal conditions.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
