Video URLs return 403 error

[ephraimt]

Recently, video links which we get from Dropbox (iOS/Android) Chooser return either a 403 error, or they return 200 but never load.

We are using link type: DbxChooser.ResultType.DIRECT_LINK / DBChooserLinkTypeDirect so the links are in this format: https://dl.dropboxusercontent.com/1/view/...

Image links work correctly.

Did something change in the API?

Thank you.

[Greg-DB]

Thanks for the report. I'm not aware of something that should have affected this, but I can look into it. I could use some more information though. Could you let me know:

• what kind(s) of video file type(s) are affected? (e.g., .mov, .mp4, etc.)
• when did this start occurring, to the best of your knowledge?
• when you get a 403 error, is that occurring within 4 hours of the link's creation?
• when you get a 403 error, are you getting any response body?
• when you get a 200 but say they "never load", how are exactly are you attempting to load them?

By the way, please note that the iOS and Android Chooser are now considered deprecated. Please refer to the documentation for more information.

[ephraimt]

Hi Greg,

Thank you for the quick reply!

Before answering your questions, let me explain our normal flow:

1. Users of our React Native mobile app choose a file from Dropbox.

2. We get the direct link and immediately use it as the URL input for AWS MediaConvert.

3. This worked fine for the past 9 months or so. 

Your questions helped me eliminate some possible issues, so I realize now that the 403 error and the 200 not loading were my local problem and are therefore irrelevant.

So the actual problem is this:

Since some time after Jan. 24 2022, when MediaConvert accesses one of these Dropbox links, it reports:

3450 HTTP Server Error

You specified an HTTP URL for an input file, but the HTTP server returned an error or failed. Contact the team that is responsible for maintaining the HTTP server that hosts your file.

https://docs.aws.amazon.com/mediaconvert/latest/ug/mediaconvert_error_codes.html

Here are the AWS MediaConvert requirements for input URLS:

• All input files must be publicly readable.

• The HTTP(S) server must not require authentication.

• The HTTP(S) server must accept both HEAD and range GET requests.

• The URL that you specify can't include parameters.

If your HTTP(S) input uses redirection, it must follow these restrictions:

• You can redirect only once from the URL that you provide as your input. You can't redirect to a URL that, in turn, contains a redirect.

• The HTTP(S) status response code from the initial server must be either 301 or 302.

• The HTTP(S) response from the initial server must use its Location headers to provide the URL that it's redirecting MediaConvert to.

Is it possible that Dropbox recently added multiple redirection or made some other change?

Can you check your server logs to find what the exact error is? An example link that failed about an hour ago is:

https://dl.dropboxusercontent.com/1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4

Lastly, here are the answers to your questions:

• The problem occurs with both .mp4 and .mov files. 
• The most recent link created that worked was on Jan. 24, and the first link that failed was on Jan. 31. (We did not create any links in between those dates.)
• I got the 403 error by pasting the URL into a browser, but based on your question, I now realize that it was because those links were old links that I was using to debug the issue. So I understand that those links were invalid and so the 403 case is not relevant.
• Regarding the 200 response that fails, that was also when pasting the Dropbox link into a browser, and seems to be working now, so we should also remove that 200 response discussion from the investigation. (Perhaps was a local network issue on my machine.)
• I am aware that Dropbox Chooser is deprecated, but unfortunately there does not seem to be a ready solution for React Native which is not based on Chooser. (Other than building it from scratch.)

Thank you so much for your help!

[Greg-DB]

Thanks for the detailed reply! While I can't rule out some other change (I'll see if I can track anything down), but the current links do appear to comply with the requirements listed here (and don't use redirects).

For reference, I manually tried out a few requests on the sample link you provided to confirm these requirements:

# this performs a HEAD request:
curl -v -I "https://dl.dropboxusercontent.com/1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4" -o out

# > HEAD /1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4 HTTP/2
# > Host: dl.dropboxusercontent.com
# > User-Agent: curl/7.64.1
# > Accept: */*
# >
# * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
# < HTTP/2 200
# < accept-ranges: bytes
# < cache-control: max-age=60
# < content-disposition: inline; filename="PXL_20220201_085252956.mp4"; filename*=UTF-8''PXL_20220201_085252956.mp4
# < content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
# < content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
# < etag: 1643705622862534n
# < pragma: public
# < set-cookie:  uc_session=...; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
# < x-content-type-options: nosniff
# < x-server-response-time: 156
# < content-type: application/json
# < accept-encoding: identity,gzip
# < date: Tue, 01 Feb 2022 21:52:06 GMT
# < server: envoy
# < strict-transport-security: max-age=31536000; includeSubDomains; preload
# < x-robots-tag: noindex, nofollow, noimageindex
# < content-length: 365070
# < vary: Accept-Encoding
# < x-dropbox-response-origin: far_remote
# < x-dropbox-request-id: 9e5c9cedbb624e859530c0849a3da45b
# <
#   0  356k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
# * Connection #0 to host dl.dropboxusercontent.com left intact
# * Closing connection 0


# this performs a GET request:
curl -v -X GET "https://dl.dropboxusercontent.com/1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4" -o out

# > GET /1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4 HTTP/2
# > Host: dl.dropboxusercontent.com
# > User-Agent: curl/7.64.1
# > Accept: */*
# >
# * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
#   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
# < HTTP/2 200
# < accept-ranges: bytes
# < cache-control: max-age=60
# < content-disposition: inline; filename="PXL_20220201_085252956.mp4"; filename*=UTF-8''PXL_20220201_085252956.mp4
# < content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
# < content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
# < etag: 1643705622862534n
# < pragma: public
# < set-cookie:  uc_session=...; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
# < x-content-type-options: nosniff
# < x-server-response-time: 157
# < content-type: video/mp4
# < accept-encoding: identity,gzip
# < date: Tue, 01 Feb 2022 21:52:42 GMT
# < server: envoy
# < strict-transport-security: max-age=31536000; includeSubDomains; preload
# < x-robots-tag: noindex, nofollow, noimageindex
# < content-length: 365070
# < x-dropbox-response-origin: far_remote
# < x-dropbox-request-id: 3c65634fb10446a2bf2a6b995aa8f426
# <
# { [15688 bytes data]
# 100  356k  100  356k    0     0   502k      0 --:--:-- --:--:-- --:--:--  502k
# * Connection #0 to host dl.dropboxusercontent.com left intact
# * Closing connection 0


# this performs a GET request for a particular Range:
curl -v -X GET "https://dl.dropboxusercontent.com/1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4" -H "Range:bytes=0-1" -o out

# > GET /1/view/j2n5x2q7licm5l2/ParsempoVideos/PXL_20220201_085252956.mp4 HTTP/2
# > Host: dl.dropboxusercontent.com
# > User-Agent: curl/7.64.1
# > Accept: */*
# > Range:bytes=0-1
# >
# * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
# < HTTP/2 206
# < accept-ranges: bytes
# < cache-control: max-age=60
# < content-disposition: inline; filename="PXL_20220201_085252956.mp4"; filename*=UTF-8''PXL_20220201_085252956.mp4
# < content-range: bytes 0-1/365070
# < content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
# < content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
# < etag: 1643705622862534n
# < pragma: public
# < set-cookie:  uc_session=...; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
# < x-content-type-options: nosniff
# < content-type: video/mp4
# < accept-encoding: identity,gzip
# < date: Tue, 01 Feb 2022 21:52:52 GMT
# < server: envoy
# < strict-transport-security: max-age=31536000; includeSubDomains; preload
# < x-robots-tag: noindex, nofollow, noimageindex
# < content-length: 2
# < x-dropbox-response-origin: far_remote
# < x-dropbox-request-id: 580ed15d1101449db87072fb130b46ca
# <
# { [2 bytes data]
# 100     2  100     2    0     0      3      0 --:--:-- --:--:-- --:--:--     3
# * Connection #0 to host dl.dropboxusercontent.com left intact
# * Closing connection 0

From the output here, it appears all of these requirements are met. The link is publicly accessible without any authentication or parameters and supports HEAD, GET, and GET with Range, all returning 200 (or 206 as applicable for Range), without any redirects.

Unfortunately, without knowing more specifically what I'm looking for, I don't have a way to look up any potential errors here. Does MediaConvert offer a way to get any additional information regarding an error accessing the links, or whatever failed exactly? For instance, if you can get the text of a failed response showing the request and response headers like above, that would be helpful.

Otherwise, might there be any other requirements not listed here?

[Greg-DB]

I looked into this further, and I can report that we did make a change to how we serve these links on HTTP/1.1 in particular. When calling with HTTP/1.1, we are now using "Transfer-Encoding: chunked" and no longer return "Content-Length". HTTP clients should handle this automatically, but it sounds like yours may not support it. Can you check if that is a requirement?

We do also support HTTP/2 on these though, where we do return "Content-Length" (e.g., as seen in my sample requests which happened to be using HTTP/2.) Can you see if you can have it use HTTP/2 instead?

[ephraimt]

Awesome, that is great information!

1. Was that change deployed between Jan. 24 and Jan 31? (So we can be confident this is the cause of the issue.)

2. Were there any other changes made that we should consider also?

2. I will check with AWS but I imagine getting them to deploy any change on their side will take a long time.

3. Do you have any suggestion for a workaround so that we could get the old format response?

Thanks so much.

[Greg-DB]

1. Yes, the timeline lines up for this.

2. I don't have any other changes to report on this.

3. Thanks for the note.

4. This isn't configurable by any sort of option on the API unfortunately; this is something that standard HTTP clients should just automatically handle. The only workarounds I can recommend would be to have the client support "Transfer-Encoding: chunked" without "Content-Length" on HTTP/1.1, or upgrade to HTTP/2.

[Greg-DB]

Update: in order to temporarily accommodate clients that don’t properly support automatically handling “Transfer-Encoding: chunked”, we’re temporarily rolling back this change, so that these links will no longer use “Transfer-Encoding: chunked” and will instead return “Content-Length” on HTTP/1.1. We will begin rolling that out starting around 2/17. That will be in place until around 3/1. At that point, we will begin using “Transfer-Encoding: chunked” and no longer returning “Content-Length” on HTTP/1.1 again.

Going forward, please ensure that your clients are able to automatically handle both chunked encoding and non-chunked encoding automatically.

[Greg-DB]

Update: the team has been able to complete some further updates to our infrastructure to be able to support the previous non-chunked behavior going forward indefinitely. That means that we plan to continue returning Content-Length (and not 'Transfer-Encoding: chunked') in the future and will not be reverting this as previously planned. (Regardless, for HTTP compatibility in general, we still recommend you make sure your HTTP clients support both types.) Hope this helps!

[ephraimt]

Thank you for the update.

Are the changes you discussed already deployed?

I tested now and there is still an error when accessing the files.