GET request to /files/download fails with "The network connection was lost"

Hi,

The Joplin note taking app uses Dropbox's HTTP API  to upload and download files. Since a few days ago, some of our its requests are failing with "The network connection was lost".

The failing request downloads files from Dropbox using a GET request to content.dropboxapi.com/2/files/download:

fetch(
    "https://content.dropboxapi.com/2/files/download",
    {
        "headers": {
            "Dropbox-API-Arg": "{\"path\":\"/info.json\"}",
            "Authorization": "Bearer " + API_TOKEN,
            "Content-Type": "application/octet-stream"
        },
        "method":"GET"
    }
)

Sometimes, the above fetch does work. However, I can reproduce the failure consistently by sending the above request twice, but only in some environments:
 - Running code from the Safari development tools, inspecting a WebView embedded in Joplin.
 - Running code from the Chrome development tools, connected to React Native Hermes.

I can't reproduce this when using the same code in a NodeJS REPL or the Firefox JavaScript console.

The examples in the Dropbox documentation send POST rather than GET requests to files/download. Switching to a POST request does seem to fix the issue on Android. However, it still fails on iOS.

Another post on this forum suggests that fetch on iOS doesn't support sending POST requests with an empty body. Dropbox doesn't seem to allow a non-empty body for this request.

Is this forum the correct place to be asking for help with this issue? Is there a way to continue using a GET request to /files/download to download files?

Notes:
- The original version of this post was marked as spam, perhaps due to links to Joplin's source code and issue tracker. These links have been removed and can be provided on request.

- Joplin uses React Native 0.71. The issue still happens after upgrading to React Native 0.73.8.

I have the opposite issue!
I also have a ReactNative app, and In the last few days (since April 30), I have seen this error occur only for my iOS users and only on the https://content.dropboxapi.com/2/files/download… endpoint.

Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost."

But I use POST for it. Once I change it to GET it seems to fix the issue on iOS.

WDYT?

While we can't offer insight on the client platforms/libraries in particular, this is the right place for help with the Dropbox API itself.

For reference, the Dropbox does use POST for all API calls by default, but "content-download" style endpoints, such as /2/files/download, do additionally support GET. And it's correct that the endpoint does not support POST requests with a non-empty body.

I just tried it out, and confirmed that the API is still allowing both GET and POST for this endpoint in my testing:

curl -v -X POST https://content.dropboxapi.com/2/files/download \
    --header "Authorization: Bearer <ACCESS_TOKEN>" \
    --header "Dropbox-API-Arg: {\"path\":\"/test.txt\"}" \
    --header "Content-Type: application/octet-stream"

> POST /2/files/download HTTP/2
> Host: content.dropboxapi.com
> User-Agent: curl/8.4.0
> Accept: */*
> Authorization: Bearer <ACCESS_TOKEN>
> Dropbox-API-Arg: {"path":"/test.txt"}
> Content-Type: application/octet-stream
>
< HTTP/2 200
< content-type: application/octet-stream
< accept-ranges: bytes
< cache-control: no-cache
< dropbox-api-result: {"name": "test.txt", "path_lower": "/test.txt", "path_display": "/test.txt", "id": "id:25N5ksooX-sAAAAAAAQYrQ", "client_modified": "2024-03-11T15:24:40Z", "server_modified": "2024-03-11T15:24:40Z", "rev": "6136423edf446021eccc7", "size": 27, "is_downloadable": true, "content_hash": "cceac37da5288a6b6716737310c14305c9b16a6e0ff0abce4a306c3de3a59f75"}
< etag: W/"6136423edf446021eccc7"
< original-content-length: 27
< x-server-response-time: 375
< date: Mon, 06 May 2024 15:09:05 GMT
< server: envoy
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-robots-tag: noindex, nofollow, noimageindex
< content-length: 27
< x-dropbox-response-origin: far_remote
< x-dropbox-request-id: d0eb597d801d4203819d86c9b65b42a1
<
* Connection #0 to host content.dropboxapi.com left intact
This is the test file data.

curl -v -X GET https://content.dropboxapi.com/2/files/download \
    --header "Authorization: Bearer <ACCESS_TOKEN>" \
    --header "Dropbox-API-Arg: {\"path\":\"/test.txt\"}" \
    --header "Content-Type: application/octet-stream"

> GET /2/files/download HTTP/2
> Host: content.dropboxapi.com
> User-Agent: curl/8.4.0
> Accept: */*
> Authorization: Bearer <ACCESS_TOKEN>
> Dropbox-API-Arg: {"path":"/test.txt"}
> Content-Type: application/octet-stream
>
< HTTP/2 200
< content-type: application/octet-stream
< accept-ranges: bytes
< cache-control: no-cache
< content-disposition: attachment; filename=unspecified
< content-security-policy: sandbox
< dropbox-api-result: {"name": "test.txt", "path_lower": "/test.txt", "path_display": "/test.txt", "id": "id:25N5ksooX-sAAAAAAAQYrQ", "client_modified": "2024-03-11T15:24:40Z", "server_modified": "2024-03-11T15:24:40Z", "rev": "6136423edf446021eccc7", "size": 27, "is_downloadable": true, "content_hash": "cceac37da5288a6b6716737310c14305c9b16a6e0ff0abce4a306c3de3a59f75"}
< etag: W/"6136423edf446021eccc7"
< original-content-length: 27
< vary: Dropbox-API-Arg, Authorization
< x-content-security-policy: sandbox
< x-content-type-options: nosniff
< x-server-response-time: 340
< x-webkit-csp: sandbox
< date: Mon, 06 May 2024 15:09:29 GMT
< server: envoy
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-robots-tag: noindex, nofollow, noimageindex
< content-length: 27
< x-dropbox-response-origin: far_remote
< x-dropbox-request-id: 1eaaff94c0f84f2f91cf96059913ee87
<
* Connection #0 to host content.dropboxapi.com left intact
This is the test file data.

I included the "Content-Type" request header to match your example, but it's not actually required for this. Also, I tried the GET request several times in a row without issue.

So, it does sound like this issue is environment-specific. We'd be happy to look into it further on our side to see if there's something we can do about it from the Dropbox server side, but it's unclear what the issue is exactly so far. Is there a way you could enable some more verbose output (similar to -v for curl above) to gather more information about how exactly the request is failing in the affected environments?

Thank you for checking!

> I just tried it out, and confirmed that the API is still allowing both GET and POST for this endpoint in my testing

Locally, I only experience the issue when sending a GET request for the same file two or more times. The first request is successful, but subsequent requests fail, until another file is requested. 

> We'd be happy to look into it further on our side to see if there's something we can do about it from the Dropbox server side, but it's unclear what the issue is exactly so far. Is there a way you could enable some more verbose output (similar to -v for curl above) to gather more information about how exactly the request is failing in the affected environments?

I'm not sure how to do this for React Native. As commented above, I was able to reproduce part of the issue in Safari/iOS. I'm attaching screenshots of the "network" tab for a failing and passing GET request in this environment.

Edit: It may be necessary to click on the image above to view it with greater resolution.
Note that I can't reproduce the POST issue in Safari.

Thanks! It looks like there's no response received. In cases like that, we'd need to refer to the network client error to see why that failed (e.g., DNS issue, timeout, etc.). It doesn't look like there's much there though, unfortunately. I do see the "TypeError: Load failed". Is there any additional information there if you expand those two dropdown arrows next to that? And is there anything in the "Preview" tab?

The "Preview" tab shows an error message:

The dropdown by "TypeError: Load failed" similarly doesn't show additional information.

The right-click menu for the failed request has options that might be helpful:

"Copy HTTP request":

GET /2/files/download
Authorization: Bearer TOKEN_HERE
Content-Type: application/octet-stream
Origin: https://js-draw.web.app
Referer: https://js-draw.web.app/
Accept: */*
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15
Dropbox-API-Arg: {"path":"/info.json"}

"Copy HTTP response":

NaN null

"Copy as cURL":

curl 'https://content.dropboxapi.com/2/files/download' \
-X 'GET' \
-H 'Authorization: Bearer [[token]]' \
-H 'Content-Type: application/octet-stream' \
-H 'Origin: https://js-draw.web.app' \
-H 'Referer: https://js-draw.web.app/' \
-H 'Accept: */*' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Sec-Fetch-Site: cross-site' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15' \
-H 'Dropbox-API-Arg: {"path":"/info.json"}'

Interestingly, the CURL request is successful when run multiple times in the MacOS terminal app.

I'm now trying to see if I can log network requests made by the iOS simulator (relevant Apple documentation).

Thanks for sharing that. Yes, unfortunately it doesn't look like there's anything else useful there.

Just to check, I also tried the curl command a few times but that also worked fine for me.

Logging the failing network requests themselves sounds like a good next step, so let us know if you're able to get anything from that.

> Logging the failing network requests themselves sounds like a good next step, so let us know if you're able to get anything from that.

default	15:15:31.519737-0700	Joplin	CFNetwork Diagnostics [3:272077] 15:15:31.519 {
HTTPCookieStorage::copyCookiesForURL: <CFHTTPCookieStorage 0xredacted [0xredacted]>
                         Request URL: https://content.dropboxapi.com/<redacted>
                    MainDocument URL: null
} [3:272077]
default	15:15:31.519624-0700	Joplin	CFNetwork Diagnostics [3:272076] 15:15:31.519 {
AddCookies Continue: request GET https://content.dropboxapi.com/<redacted>
       HTTPProtocol: Task: 0xredacted
} [3:272076]
default	15:15:31.519507-0700	Joplin	CFNetwork Diagnostics [3:272075] 15:15:31.519 {
Vary Validation: (null)
     Evaluation: PASS
        Request: https://content.dropboxapi.com/<redacted>
     VaryHeader: Dropbox-API-Arg, Authorization
} [3:272075]
default	15:15:31.519387-0700	Joplin	CFNetwork Diagnostics [3:272074] 15:15:31.519 {
LoaderWhatToDo: (null)
       Request: https://content.dropboxapi.com/<redacted>
   CachePolicy: 0
      WhatToDo: originload
   CreateToNow: 0.00093s
} [3:272074]
default	15:15:31.519254-0700	Joplin	CFNetwork Diagnostics [3:272073] 15:15:31.519 {
Vary Validation: (null)
     Evaluation: PASS
        Request: https://content.dropboxapi.com/<redacted>
     VaryHeader: Dropbox-API-Arg, Authorization
} [3:272073]
default	15:15:14.050982-0700	Joplin	Task <redacted-task-id>.<10> resuming, timeouts(0.0, 604800.0) QOS(0x19) Voucher (null)
default	15:15:14.050704-0700	Joplin	CFNetwork Diagnostics [3:271985] 15:15:14.050 {
HTTPCookieStorage::copyCookiesForURL: <CFHTTPCookieStorage 0xredacted [0xredacted]>
                         Request URL: https://content.dropboxapi.com/<redacted>
                    MainDocument URL: null
} [3:271985]
error	15:15:10.042740-0700	Joplin	Task <C49036A5-617D-4742-B3E3-58641BD95148>.<9> finished with error [-1005] Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=57, NSUnderlyingError=0x600000c78060 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0xredacted [0xredacted]>{length = 16, capacity = 16, bytes = 0xredacted}, _kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <redacted-task-id>.<9>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <redacted-task-id>.<9>"
), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://content.dropboxapi.com/2/files/download, NSErrorFailingURLKey=https://content.dropboxapi.com/2/files/download, _kCFStreamErrorDomainKey=1}
default	15:15:10.042500-0700	Joplin	Task <redacted-task-id>.<9> summary for task failure {transaction_duration_ms=442, response_status=200, connection=8, reused=1, reused_after_ms=9620, request_start_ms=1, request_duration_ms=0, response_start_ms=440, response_duration_ms=0, request_bytes=179, response_bytes=96, cache_hit=true}
default	15:15:10.042383-0700	Joplin	CFNetwork Diagnostics [3:271970] 15:15:10.042 {
           Did Fail: (null)
             Loader: request GET https://content.dropboxapi.com/<redacted>
              Error: Error Domain=kCFErrorDomainCFNetwork Code=-1005 UserInfo={NSErrorPeerAddressKey=<private>, _kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1}
init to origin load: 0.000844955s
         total time: 0.442753s
        total bytes: 0
} [3:271970]

Many of the `<redacted>` sections above were removed by XCode or iOS.

Additionally, I notice that the above log includes "cache_hit=true".

Yes, unfortunately it's still not very clear with this output.

I do see that it has "response_status=200", which doesn't seem to match the other output we've seen (where there doesn't seem to be a response), but that may be because as you noted it says "cache_hit=true", so perhaps it's actually referring to a cached response from a previous call. That may imply an issue with how the platform's caching works. I'm not aware of a mistake with how the Dropbox API servers currently handle any caching headers, but if there seems to be an issue with that please let me know what exactly seems to cause an issue for your platform and I can raise that with the team.

Also, I notice there are some other error codes "_kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1". I'm not familiar with those, and we can't provide support for the platform/network client itself as it's not made by Dropbox, but do you know what those particular codes may indicate?

> Also, I notice there are some other error codes "_kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1". I'm not familiar with those, and we can't provide support for the platform/network client itself as it's not made by Dropbox, but do you know what those particular codes may indicate?

I don't know what the `_kCF...` error codes mean and didn't find references to these particular error codes with a web search or in the Apple documentation.

> I do see that it has "response_status=200", which doesn't seem to match the other output we've seen (where there doesn't seem to be a response), but that may be because as you noted it says "cache_hit=true", so perhaps it's actually referring to a cached response from a previous call.

It's strange that this is a cache hit — the original response contains `Cache-Control: no-cache`. It would be interesting to compare the current headers with those from a few weeks ago to see if this, or other cache-related headers changed.

We merged a workaround that, on request failure, sends a request for a different file before retrying the original request. Even though failure doesn't occur for most `files/download` GET requests, the extra API requests required by this workaround aren't ideal.

Forcing iOS to avoid the cache may be a better solution. As such, I've also tried:
• Passing `Cache-Control: no-cache` as an additional request header. (Doesn't work -- same "network request failed" error).
• Adding a new header and adding/referencing it in the `Vary` header (see this StackOverflow discussion). (Doesn't work -- same "network request failed" error).

• Retrying with `?` appended to the Dropbox `files/download` URL on failure. (Doesn't work -- same "network request failed" error).

• Retrying with `?retry` appended to the Dropbox `files/download` URL on failure. This results in `Error (400): Error in URL parameters: Bad query field: 'retry'`.