We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
mikemcrae25
2 years agoNew member | Level 2
Dropbox API: Connection broken: IncompleteRead
Hi everyone,
I am using the API to zip and download folders each with 1000 .jpegs in them (approx 4GB - 5GB). Historically this has been no issue. In the last week however, I have started receiving...
denseishin
11 months agoNew member | Level 1
I'm facing a similar problem, but in my case I'm merely downloading a very large file (~125 GB). It usually crashes between 58000000000 - 49000000000 bytes read.
DB-Des
Dropbox Community Moderator
11 months agoHi denseishin
I'll be happy to look further into this issue for you, please reply with:
- the name and version number of the platform and SDK/library you are using, if any
- the steps to reproduce the issue, including relevant code snippet(s), but don't
- include any access or refresh token(s)
- the full text of any error or unexpected output
Could you also confirm if requests are failing after a specific amount of time? And if so, how long is that?
- denseishin10 months agoNew member | Level 1
- Python 3.8.10 on Ubuntu 20.04 with the Python dropbox package version 11.36.2
code snippet:
def __init__(): self.dbx = dropbox.Dropbox(API keys etc. here) def download(self, info: dict): dbx_path = info["path"] self.logger.log(logging.INFO,f"Downloading {dbx_path}") noroot = dbx_path.lstrip('/') dl_path = self.dl_path / pathlib.Path(noroot) dl_path.parent.mkdir(exist_ok=True, parents=True) metadata = self.dbx.files_download_to_file(str(dl_path), dbx_path) return metadataINFO:dbx_downloader:Downloading (file name redacted)
INFO:dropbox:Request to files/download
ERROR:modulenameredacted:('Connection broken: IncompleteRead(54876571369 bytes read, 78618595296 more expected)', IncompleteRead(54876571369 bytes read, 78618595296 more expected))I couldn't test for the amount of time yet
- denseishin10 months agoNew member | Level 1
Here's a traceback.
INFO:redacted_module:2025-02-18 16:53:13.153338+00:00: Downloading REDACTED INFO:dropbox:Request to files/download ERROR:redacted_module:('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected)) Traceback: Traceback (most recent call last): File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 748, in _error_catcher yield File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 894, in _raw_read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) urllib3.exceptions.IncompleteRead: IncompleteRead(55015292051 bytes read, 78479874614 more expected) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 820, in generate yield from self.raw.stream(chunk_size, decode_content=True) File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 1060, in stream data = self.read(amt=amt, decode_content=decode_content) File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 977, in read data = self._raw_read(amt) File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 902, in _raw_read self._fp.close() File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/path/to/.venv/lib/python3.8/site-packages/urllib3/response.py", line 772, in _error_catcher raise ProtocolError(arg, e) from e urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/path/to/customscript.py", line 104, in _process_msg self._work_callback(body) File "/path/to/anothercustomscript.py", line 59, in download metadata = self.dbx.files_download_to_file(str(dl_path), dbx_path) File "/path/to/.venv/lib/python3.8/site-packages/dropbox/base.py", line 1459, in files_download_to_file self._save_body_to_file(download_path, r[1]) File "/path/to/.venv/lib/python3.8/site-packages/dropbox/dropbox_client.py", line 687, in _save_body_to_file for c in http_resp.iter_content(chunksize): File "/path/to/.venv/lib/python3.8/site-packages/requests/models.py", line 822, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(55015292051 bytes read, 78479874614 more expected)', IncompleteRead(55015292051 bytes read, 78479874614 more expected))It seems to repeat after almost exactly one hour
- DB-Des10 months ago
Dropbox Community Moderator
Thank you for that additional information. If the downloads seem to be failing at about an hour it is most likely due to the server timeout. There isn't an option to delay or prevent that.
The /2/files/download endpoint itself is a Content-Download style endpoint and does support Range requests. Using range requests in this case would be a solution we would recommend to handle/resume incomplete downloads. Keep in mind that even requests with ranges set are subject to any server timeouts, so you will want to continue monitoring any failures and use multiple range requests as needed.
- denseishin10 months agoNew member | Level 1
Please keep in mind that I'm using the official Python SDK for Dropbox. Does that mean that it is not suitable for large files?
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.
If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!