Connection timeouts

[joe_walker]

Hi,

From approximately 22:30 UTC on 27 Oct until present we have been getting timeouts when trying to connect to the Dropbox API:

cURL error 7: Failed to connect to content.dropboxapi.com port 443: Connection timed out (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)

Traceroute to content.dropbox.com is as follows:

traceroute to content.dropboxapi.com (162.125.83.8), 30 hops max, 60 byte packets
 1  vzsrv03.vps.apdcsy1.digitalpacific.com.au (101.0.66.201)  0.051 ms  0.031 ms  0.030 ms
 2  vlan-1046.dsw01-pod01.apdcsy1.as55803.net.au (101.0.126.101)  2.174 ms  2.179 ms  2.137 ms
 3  ae2-0.cor01.apdcsy1.as55803.net.au (101.0.125.121)  0.593 ms ae2-0.cor02.apdcsy1.as55803.net.au (101.0.125.113)  0.631 ms  0.587 ms
 4  ae0-0.bdr01.apdcsy1.as55803.net.au (101.0.127.113)  1.213 ms ae1-0.bdr01.apdcsy1.as55803.net.au (101.0.127.121)  1.165 ms ae0-0.bdr01.apdcsy1.as55803.net.au (101.0.127.113)  1.123 ms
 5  lt-2-2-0-10.bdr01.eqsy3.as55803.net.au (101.0.126.124)  0.490 ms  0.438 ms  0.413 ms
 6  ten-0-2-0-7-101.bdr01.syd03.nsw.vocus.net.au (49.255.198.124)  0.881 ms  1.021 ms  1.044 ms
 7  BE-107.cor01.syd11.nsw.VOCUS.net.au (114.31.192.80)  0.752 ms BE-107.cor02.syd04.nsw.VOCUS.net.au (114.31.192.82)  0.754 ms  1.165 ms
 8  BE-100.bdr02.syd03.nsw.VOCUS.net.au (114.31.192.39)  1.086 ms  1.072 ms  1.076 ms
 9  * * *
10  po1-syd2-ea01.net.dropbox.com (162.125.86.189)  0.940 ms  0.955 ms  0.963 ms
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

[Greg-DB]

Thanks for the report! We'll check on this. For reference, are you still seeing this now?

[joe_walker]

Hi Greg,

Yes - the connection still times out and the traceroute result is still the same as above.

[Greg-DB]

Thanks! We're looking into this, but we haven't been able to find a network issue for that network route. 

To try to narrow this down:

• Are there multiple users of your app, and if so, are multiple users affected, or just yourself?
• Which API endpoints in particular are you using, and which are affected?
• Is this affecting all of the attempts to connect to content.dropboxapi.com, or only some of them? If it's only some of them, about what percent are affected?
• Is there anything on your network connection, e.g., a proxy, firewall, anti-virus, etc. what may be interfering with HTTPS connections to content.dropboxapi.com?
• Can you share the information on https://dropbox-debug.com from an affected machine?

[joe_walker]

We are the only users of the app.

Specifically the call to the "files/get_metadata" times out, however all attempts to connect to content.dropboxapi.com time out.

I do not know of anything on the network which would be interfering with the connection, and do not know of anything that changed around the time the problem started occurring. The server's host (Digital Pacific) has not reported that they are aware of any network issues, though I have submitted a ticket to double check.

As the app is running on a server without a web browser, is there any other way to get the debug information provided by dropbox-debug.com?

[Greg-DB]

Calls to files/get_metadata would be made to api.dropboxapi.com (not content.dropboxapi.com). There are other endpoints on api.dropboxapi.com as well though, such as files/list_folder. Are you attempting any other calls to api.dropboxapi.com, and if so, are those not failing?

Please do let me know if you hear anything from Digital Pacific.

And no, the debug page uses JavaScript to run the tests, so you'd need to use a browser (e.g., you can't just curl the page), so unless you can use your server as a proxy, or something to that effect, we'll need to investigate without that information.

[joe_walker]

Thanks Greg. Sorry, I was mistaken about the specific call it's timing out on.

Here's an example of a curl command failing on a content.dropboxapi.com endpoint:

$ curl -X POST https://content.dropboxapi.com/2/files/upload --header "Authorization: Bearer <token>" --header "Dropbox-API-Arg: {\"path\": \"/test.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" --header "Content-Type: application/octet-stream" --data-binary @test
curl: (7) Failed to connect to 2620:100:6033:8::a27d:5308: Network is unreachable

Interestingly, the error message here points to an IPV6 address. From that server, `ping6 ipv6.google.com` outputs `Connect: Network is unreachable`, so I wonder if the issue is that IPV6 is not enabled on that server.

Should we need IPV6 enabled to properly use the API?

[Greg-DB]

I see, thanks.

It looks like that's a different error message now though. For reference, can you let me know exactly which endpoint(s) and host(s) are currently returning which error message(s) for you? 

And no, IPv6 isn't required. Both v4 and v6 should work. I just tried it myself and confirmed that https://content.dropboxapi.com/2/files/upload is usable via both.

[joe_walker]

I believe that the difference in the error message is due to the message in my initial report coming from libcurl being used by a PHP application as opposed to command line curl.

Calls to both files/list_folder and file_requests/get returned the expected result. Therefore it seems api.dropboxapi.com is working fine for me.

Attempting to use the /files/upload endpoint returns the error mentioned previously:

$ curl -X POST https://content.dropboxapi.com/2/files/upload     --header "Authorization: Bearer <token>"     --header "Dropbox-API-Arg: {\"path\": \"/test.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}"     --header "Content-Type: application/octet-stream"     --data-binary @test
curl: (7) Failed to connect to 2620:100:6033:8::a27d:5308: Network is unreachable

As does a call to /files/download:

$ curl -X POST https://content.dropboxapi.com/2/files/download     --header "Authorization: Bearer <token>"     --header "Dropbox-API-Arg: {\"path\": \"/test.txt\"}"
curl: (7) Failed to connect to 2620:100:6033:8::a27d:5308: Network is unreachable

So it seems that the problem is related to API calls on content.dropboxapi.com.

[Greg-DB]

Thank you, that helps clarify the scope of the issue. We still haven't found anything on our side that should be causing this, but I'll check with the network team again.

By the way, to clarify your earlier note of "Should we need IPV6 enabled to properly use the API?", do you mean that the IPv6 stack is not currently enabled on your own server? The Dropbox API servers accept both, but the sample attempts you showed do indicate that your client is attempting to use IPv6. If IPv6 is not enabled on your server but your client is trying to use it for some reason, perhaps that's causing the issue. If that's the case, you may want to try the '-4' option to force curl to use IPv4, or check the verbose '-v' output of curl when accessing api.dropboxapi.com as a comparison.