"The response ended prematurely" after one hour of a file download

[neilgd]

I'm trying to download a large (80Gb) file from the Dropbox servers.

After *exactly* one hour, I get an exception indicating that the Dropbox servers have dropped the connection (System.IO.IOException: The response ended prematurely. in .NET)

I have tried using both the .NET HttpClient (for a shared link from dropboxusercontent.com) and the Dropbox API.

Any ideas? Thanks!

[Greg-DB · Accepted solution]

If these are failing after an hour, that would be due to the server timeout. There isn't an option to delay or prevent that.

Content-download style endpoints, such as /2/files/download, support "range requests" though. I recommend using range requests to continue downloading the rest of the data if a connection fails, and repeat as needed for up to one hour per connection.

Here's a basic example showing how to do that:

HttpClient client = new HttpClient();

HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "https://content.dropboxapi.com/2/files/download");
request.Headers.Add("Authorization", "Bearer " + accessToken);
request.Headers.Add("Dropbox-API-Arg", "{\"path\":\"/test.txt\"}");  // in real code, use a JSON library to build this
request.Headers.Add("Range", "bytes=5-");  // this requests the rest of the file after the first 5 bytes

HttpResponseMessage response = await client.SendAsync(request);
Stream stream = response.Content.ReadAsStream();  // read from stream as desired

For instance, in this example, this would be helpful if the app previously only downloaded the first 5 bytes of the file, and needs the rest. In your actual use, you would programmatically set the byte range as needed based on what you successfully downloaded so far of course.

[Greg-DB · Accepted solution]

If these are failing after an hour, that would be due to the server timeout. There isn't an option to delay or prevent that.

Content-download style endpoints, such as /2/files/download, support "range requests" though. I recommend using range requests to continue downloading the rest of the data if a connection fails, and repeat as needed for up to one hour per connection.

Here's a basic example showing how to do that:

HttpClient client = new HttpClient();

HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "https://content.dropboxapi.com/2/files/download");
request.Headers.Add("Authorization", "Bearer " + accessToken);
request.Headers.Add("Dropbox-API-Arg", "{\"path\":\"/test.txt\"}");  // in real code, use a JSON library to build this
request.Headers.Add("Range", "bytes=5-");  // this requests the rest of the file after the first 5 bytes

HttpResponseMessage response = await client.SendAsync(request);
Stream stream = response.Content.ReadAsStream();  // read from stream as desired

For instance, in this example, this would be helpful if the app previously only downloaded the first 5 bytes of the file, and needs the rest. In your actual use, you would programmatically set the byte range as needed based on what you successfully downloaded so far of course.

[neilgd]

Thank you! That works a treat.