Since 31/01/22 all the download files become corrupted.

[kamzata]

I own a Magento 1.9.4.2 store which has only downloadable products in it. Once a customer buy a digital product, Magento sends an email with the download link generated by Magento itself from the Dropbox one. This is worked since 2013 from 31/01/22. Suddenly, it doesn't work anymore. All the download files, past and present, become corrupted once downloaded!

This is a big issue for me. What's happened?

I also tried to restore my store from a 20/01/22 backup, but nothing's changed. It still doesn't work!

Dropbox, what's happened?

______________________

Edit:

I checked the file types:

• Downloaded from Magento link [Corrupted]: application/octet-stream; charset=binary
• Downloaded from Dropbox link [Ok]: application/zip; charset=binary

[kamzata]

I checked the http response header and it says:

content-disposition: attachment; filename="testfile.zip"
content-type: application/zip

but, as I wrote, once downloaded the file type become application/octet-stream and the file is corrupted. Any hints?

[kamzata]

Here's the curl output from the Magento server:

root@magento /tmp# curl -I https://dl.dropboxusercontent.com/s/v2d3vm174biluk1/filetest.zip?dl=1

HTTP/2 200
accept-ranges: bytes
cache-control: max-age=60
content-disposition: attachment; filename="filetest.zip"; filename*=UTF-8''filetest.zip
content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
etag: 233d
pragma: public
set-cookie:  uc_session=uIn423ZW179jClyoN4r0YtVWgKogBVoLNoWNzXmy0qQPyWJZverGltTFPC2eLXUh; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
x-content-type-options: nosniff
x-server-response-time: 220
content-type: application/json
accept-encoding: identity,gzip
date: Tue, 08 Feb 2022 12:45:20 GMT
server: envoy
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-robots-tag: noindex, nofollow, noimageindex
content-length: 94851266
vary: Accept-Encoding
x-dropbox-response-origin: far_remote
x-dropbox-request-id: b57ed34ca29b4648a4e2e055339b8edb

[Greg-DB]

Can you clarify what you mean when you say the file is corrupted? What is in the downloaded file? For example, is it empty, or missing data, longer than expected, etc?

Also, note that the "-I" option on curl only performs a HEAD request, meaning it only requests the headers, so it won't return the actual file data to inspect.

And it looks like the shared link you posted has been revoked/deleted. Do you have a current sample you can share?

Finally, note that the "dl" parameter is meant for use on "www.dropbox.com/s/...", not "dl.dropboxusercontent.com/s/...". You can find documentation for that here. You should use "www.dropbox.com/s/..." with that instead of accessing "dl.dropboxusercontent.com/s/..." directly.

[kamzata]

Thanks for your reply. When I say the file is corrupted I mean the zip file you download cannot be extracted because "the archive is damage".

This some concrete example. This is a zip test file which contains a pdf. If I try to download the file from this url ( https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip?dl=1 ) from my browser I can extract it without any issue. If I try to do the same from my server it results corrupted.

This is the curl output from my server (check the wrong "content-type") [Result: Corrupted]:

 nginx  ⌁ root  /etc/nginx  curl -I https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip?dl=1

HTTP/2 200
accept-ranges: bytes
cache-control: max-age=60
content-disposition: attachment; filename="testfile.zip"; filename*=UTF-8''testfile.zip
content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
etag: 1644342873714211d
pragma: public
set-cookie:  uc_session=BDnnO07ayQJM8sbiAijB34ctfMey60W0C4YcU6nxwEKs5vcZ8gYtQtZyJh1cATqI; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
x-content-type-options: nosniff
x-server-response-time: 204
content-type: application/json
accept-encoding: identity,gzip
date: Tue, 08 Feb 2022 18:05:15 GMT
server: envoy
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-robots-tag: noindex, nofollow, noimageindex
content-length: 11867
vary: Accept-Encoding
x-dropbox-response-origin: far_remote
x-dropbox-request-id: 0f3550a667b246b4884a9781edbd423f

It's always worked until 31/01/22. No server update has been made since 2 months.

[kamzata]

This is the output from the CLI on my local PC [Result: Corrupted]:

❯ curl -I https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip\?dl\=1

HTTP/2 200
accept-ranges: bytes
cache-control: max-age=60
content-disposition: attachment; filename="testfile.zip"; filename*=UTF-8''testfile.zip
content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
etag: 1644342873714211d
pragma: public
set-cookie:  uc_session=DF8IA7wiq9ZfMojfS1xdMRH0fX5Om1C8LHZNfRu5E2gHMtL6Mf6ihX6JenJzZHlR; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
x-content-type-options: nosniff
x-server-response-time: 238
content-type: application/json
accept-encoding: identity,gzip
date: Tue, 08 Feb 2022 18:16:52 GMT
server: envoy
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-robots-tag: noindex, nofollow, noimageindex
content-length: 11867
vary: Accept-Encoding
x-dropbox-response-origin: far_remote
x-dropbox-request-id: 967f401abcab4713a0d4466207bf5c39

This is the output from my Chrome browser [Result: OK]:

[kamzata]

Actually doing from CLI on my local PC:

wget https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip\?dl\=1

I obtain:

testfile.zip?dl=1

and if I do:

❯ unzip testfile.zip\?dl=1
Archive:  testfile.zip?dl=1
  inflating: testfile.pdf
  inflating: __MACOSX/._testfile.pdf

It works.

But if I do the same thing from my server:

wget https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip\?dl\=1

I obtain (please, note the quotes added on the filename):

'testfile.zip?dl=1'

 and if I do:

unzip 'testfile.zip?dl=1.1'
Archive:  testfile.zip?dl=1.1
  inflating: testfile.pdf
  inflating: __MACOSX/._testfile.pdf

It works. 

[Greg-DB]

Can you try inspecting the corrupt download to see what the issue is specifically? For example, compare it to the original file to see why it does not appear to be a valid zip file.

Also, note that curl's "-I" doesn't actually download the file, so that may not be a useful test. Here's how you should actually do this with curl, also using the corrected hostname:

curl -L "https://www.dropbox.com/s/y33dnhk2rgquju4/testfile.zip?dl=1" -o out.zip

This would download the linked file to a local file "out.zip" (via "-o"), telling curl to follow redirects (via "-L"). I just gave that a try and the downloaded file works fine for me.

Though, this only tests this using curl, which may not be representative of exactly what the client in the app is doing. (Likewise for wget.) This may require some more troubleshooting in the app itself. So, if you're not the developer/programmer of "Magento", you should reach out to them first.

[kamzata]

testfile.zip [Original]:

11867 byte

application/zip; charset=binary

testfile.zip [Corrupted]:

11880 byte

application/octet-stream; charset=binary

I tried to unzip the corrupted file from the terminal (unzip testfile.zip) and it worked! If I try to use 7zip, Keka, Winzip or some other GUI utility they say "Unzip error: unknown format".

If I use "www.dropbox.com" rather than "dl.dropboxusercontent.com" Magento just download an almost empty testfile.zip?dl=1 file.

[Greg-DB]

Thanks for the additional information.

So, it looks like the "corrupted" download has an extra 13 bytes of data, which some zip utilities tolerate and which some don't. Based on this, it looks like this is because your client isn't properly decoding the HTTP message. We recently updated the server back-end for these links to use "Transfer-Encoding: chunked" (and accordingly not return the "Content-Length" response header) on HTTP/1.1. HTTP clients are expected to handle and decode this automatically. It looks like yours isn't doing so though, and instead is incorrectly saving the raw response body, including some additional HTTP chunk data, as the file. This is something the developer of the app will need to correct.

Also, since it downloads an almost empty file when using the www.dropbox.com link, it sounds like Magento probably isn't following redirects when downloading. You may want to ask the developer to support that as well.