Forum Discussion

Explorer | Level 3

4 years ago

Since 31/01/22 all the download files become corrupted.

I own a Magento 1.9.4.2 store which has only downloadable products in it. Once a customer buy a digital product, Magento sends an email with the download link generated by Magento itself from the Dro...

API

kamzata

Explorer | Level 3

4 years ago

Thanks for your reply. When I say the file is corrupted I mean the zip file you download cannot be extracted because "the archive is damage".

This some concrete example. This is a zip test file which contains a pdf. If I try to download the file from this url ( https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip?dl=1 ) from my browser I can extract it without any issue. If I try to do the same from my server it results corrupted.

This is the curl output from my server (check the wrong "content-type") [Result: Corrupted]:

 nginx  ⌁ root  /etc/nginx  curl -I https://dl.dropboxusercontent.com/s/y33dnhk2rgquju4/testfile.zip?dl=1

HTTP/2 200
accept-ranges: bytes
cache-control: max-age=60
content-disposition: attachment; filename="testfile.zip"; filename*=UTF-8''testfile.zip
content-security-policy: report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent ; sandbox allow-forms allow-scripts allow-top-navigation allow-popups
content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
etag: 1644342873714211d
pragma: public
set-cookie:  uc_session=BDnnO07ayQJM8sbiAijB34ctfMey60W0C4YcU6nxwEKs5vcZ8gYtQtZyJh1cATqI; Domain=dropboxusercontent.com; HttpOnly; Path=/; Secure
x-content-type-options: nosniff
x-server-response-time: 204
content-type: application/json
accept-encoding: identity,gzip
date: Tue, 08 Feb 2022 18:05:15 GMT
server: envoy
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-robots-tag: noindex, nofollow, noimageindex
content-length: 11867
vary: Accept-Encoding
x-dropbox-response-origin: far_remote
x-dropbox-request-id: 0f3550a667b246b4884a9781edbd423f

It's always worked until 31/01/22. No server update has been made since 2 months.

Greg-DB

Dropbox Community Moderator

4 years ago

Can you try inspecting the corrupt download to see what the issue is specifically? For example, compare it to the original file to see why it does not appear to be a valid zip file.

Also, note that curl's "-I" doesn't actually download the file, so that may not be a useful test. Here's how you should actually do this with curl, also using the corrected hostname:

curl -L "https://www.dropbox.com/s/y33dnhk2rgquju4/testfile.zip?dl=1" -o out.zip

This would download the linked file to a local file "out.zip" (via "-o"), telling curl to follow redirects (via "-L"). I just gave that a try and the downloaded file works fine for me.

Though, this only tests this using curl, which may not be representative of exactly what the client in the app is doing. (Likewise for wget.) This may require some more troubleshooting in the app itself. So, if you're not the developer/programmer of "Magento", you should reach out to them first.

kamzata

Explorer | Level 3

4 years ago

testfile.zip [Original]:

11867 byte

application/zip; charset=binary

testfile.zip [Corrupted]:

11880 byte

application/octet-stream; charset=binary

I tried to unzip the corrupted file from the terminal (unzip testfile.zip) and it worked! If I try to use 7zip, Keka, Winzip or some other GUI utility they say "Unzip error: unknown format".

If I use "www.dropbox.com" rather than "dl.dropboxusercontent.com" Magento just download an almost empty testfile.zip?dl=1 file.

Greg-DB

Dropbox Community Moderator

4 years ago

Thanks for the additional information.

So, it looks like the "corrupted" download has an extra 13 bytes of data, which some zip utilities tolerate and which some don't. Based on this, it looks like this is because your client isn't properly decoding the HTTP message. We recently updated the server back-end for these links to use "Transfer-Encoding: chunked" (and accordingly not return the "Content-Length" response header) on HTTP/1.1. HTTP clients are expected to handle and decode this automatically. It looks like yours isn't doing so though, and instead is incorrectly saving the raw response body, including some additional HTTP chunk data, as the file. This is something the developer of the app will need to correct.

Also, since it downloads an almost empty file when using the www.dropbox.com link, it sounds like Magento probably isn't following redirects when downloading. You may want to ask the developer to support that as well.

kamzata

Explorer | Level 3

4 years ago

That could be the cause. Unfortunately, I'm using Magento 1 which is not more supported from Magento devs so I need to do this on my own.

However... you said "HTTP clients are expected to handle and decode this automatically...". How the HTTP client should handle it if it doesn't do automatically?

This is the Magento helper:

<?php
/**
 * Magento
 *
 * NOTICE OF LICENSE
 *
 * This source file is subject to the Open Software License (OSL 3.0)
 * that is bundled with this package in the file LICENSE.txt.
 * It is also available through the world-wide-web at this URL:
 * http://opensource.org/licenses/osl-3.0.php
 * If you did not receive a copy of the license and are unable to
 * obtain it through the world-wide-web, please send an email
 * to license@magento.com so we can send you a copy immediately.
 *
 * DISCLAIMER
 *
 * Do not edit or add to this file if you wish to upgrade Magento to newer
 * versions in the future. If you wish to customize Magento for your
 * needs please refer to http://www.magento.com for more information.
 *
 * @category    Mage
 * @package     Mage_Downloadable
 * @copyright  Copyright (c) 2006-2019 Magento, Inc. (http://www.magento.com)
 * @license    http://opensource.org/licenses/osl-3.0.php  Open Software License (OSL 3.0)
 */

/**
 * Downloadable Products Download Helper
 *
 * @category    Mage
 * @package     Mage_Downloadable
 * @author      Magento Core Team <core@magentocommerce.com>
 */
class Mage_Downloadable_Helper_Download extends Mage_Core_Helper_Abstract
{
    const LINK_TYPE_URL         = 'url';
    const LINK_TYPE_FILE        = 'file';

    const XML_PATH_CONTENT_DISPOSITION  = 'catalog/downloadable/content_disposition';

    /**
     * Type of link
     *
     *  string
     */
    protected $_linkType        = self::LINK_TYPE_FILE;

    /**
     * Resource file
     *
     *  string
     */
    protected $_resourceFile    = null;

    /**
     * Resource open handle
     *
     *  resource
     */
    protected $_handle          = null;

    /**
     * Remote server headers
     *
     *  array
     */
    protected $_urlHeaders      = array();

    /**
     * MIME Content-type for a file
     *
     *  string
     */
    protected $_contentType     = 'application/octet-stream';

    /**
     * File name
     *
     *  string
     */
    protected $_fileName        = 'download';

    /**
     * Retrieve Resource file handle (socket, file pointer etc)
     *
     * @return resource
     */
    protected function _getHandle()
    {
        if (!$this->_resourceFile) {
            Mage::throwException(Mage::helper('downloadable')->__('Please set resource file and link type.'));
        }

        if (is_null($this->_handle)) {
            if ($this->_linkType == self::LINK_TYPE_URL) {

                /**
                 * Validate URL
                 */
                $urlProp = parse_url($this->_resourceFile);
                if (!isset($urlProp['scheme'])
                    || strtolower($urlProp['scheme'] != 'http') && strtolower($urlProp['scheme'] != 'https')) {
                    Mage::throwException(Mage::helper('downloadable')->__('Invalid download URL scheme.'));
                }
                if (!isset($urlProp['host'])) {
                    Mage::throwException(Mage::helper('downloadable')->__('Invalid download URL host.'));
                }
                switch ($urlProp['scheme']) {
                    case 'https':
                        $scheme = 'ssl://';
                        $port = 443;
                        break;
                    case 'http':
                    default:
                        $scheme = '';
                        $port = 80;
                }
                $hostname = $scheme . $urlProp['host'];

                if (isset($urlProp['port'])) {
                    $port = (int)$urlProp['port'];
                }

                $path = '/';
                if (isset($urlProp['path'])) {
                    $path = $urlProp['path'];
                }
                $query = '';
                if (isset($urlProp['query'])) {
                    $query = '?' . $urlProp['query'];
                }

                try {
                    $this->_handle = fsockopen($hostname, $port, $errno, $errstr);
                }
                catch (Exception $e) {
                    throw $e;
                }

                if ($this->_handle === false) {
                    Mage::throwException(Mage::helper('downloadable')->__('Cannot connect to remote host, error: %s.', $errstr));
                }

                $headers = 'GET ' . $path . $query . ' HTTP/1.0' . "\r\n"
                    . 'Host: ' . $urlProp['host'] . "\r\n"
                    . 'User-Agent: Magento ver/' . Mage::getVersion() . "\r\n"
                    . 'Connection: close' . "\r\n"
                    . "\r\n";
                fwrite($this->_handle, $headers);

                while (!feof($this->_handle)) {
                    $str = fgets($this->_handle, 1024);
                    if ($str == "\r\n") {
                        break;
                    }
                    $match = array();
                    if (preg_match('#^([^:]+): (.*)\s+$#', $str, $match)) {
                        $k = strtolower($match[1]);
                        if ($k == 'set-cookie') {
                            continue;
                        }
                        else {
                            $this->_urlHeaders[$k] = trim($match[2]);
                        }
                    }
                    elseif (preg_match('#^HTTP/[0-9\.]+ (\d+) (.*)\s$#', $str, $match)) {
                        $this->_urlHeaders['code'] = $match[1];
                        $this->_urlHeaders['code-string'] = trim($match[2]);
                    }
                }

                if (!isset($this->_urlHeaders['code']) || $this->_urlHeaders['code'] != 200) {
                    Mage::throwException(Mage::helper('downloadable')->__('An error occurred while getting the requested content. Please contact the store owner.'));
                }
            }
            elseif ($this->_linkType == self::LINK_TYPE_FILE) {
                $this->_handle = new Varien_Io_File();
                if (!is_file($this->_resourceFile)) {
                    Mage::helper('core/file_storage_database')->saveFileToFilesystem($this->_resourceFile);
                }
                $this->_handle->open(array('path'=>Mage::getBaseDir('var')));
                if (!$this->_handle->fileExists($this->_resourceFile, true)) {
                    Mage::throwException(Mage::helper('downloadable')->__('The file does not exist.'));
                }
                $this->_handle->streamOpen($this->_resourceFile, 'r');
            }
            else {
                Mage::throwException(Mage::helper('downloadable')->__('Invalid download link type.'));
            }
        }
        return $this->_handle;
    }

    /**
     * Retrieve file size in bytes
     */
    public function getFilesize()
    {
        $handle = $this->_getHandle();
        if ($this->_linkType == self::LINK_TYPE_FILE) {
            return $handle->streamStat('size');
        }
        elseif ($this->_linkType == self::LINK_TYPE_URL) {
            if (isset($this->_urlHeaders['content-length'])) {
                return $this->_urlHeaders['content-length'];
            }
        }
        return null;
    }

    public function getContentType()
    {
        $handle = $this->_getHandle();
        if ($this->_linkType == self::LINK_TYPE_FILE) {
            if (function_exists('mime_content_type') && ($contentType = mime_content_type($this->_resourceFile))) {
                return $contentType;
            } else {
                return Mage::helper('downloadable/file')->getFileType($this->_resourceFile);
            }
        }
        elseif ($this->_linkType == self::LINK_TYPE_URL) {
            if (isset($this->_urlHeaders['content-type'])) {
                $contentType = explode('; ', $this->_urlHeaders['content-type']);
                return $contentType[0];
            }
        }
        return $this->_contentType;
    }

    public function getFilename()
    {
        $handle = $this->_getHandle();
        if ($this->_linkType == self::LINK_TYPE_FILE) {
            return pathinfo($this->_resourceFile, PATHINFO_BASENAME);
        }
        elseif ($this->_linkType == self::LINK_TYPE_URL) {
            if (isset($this->_urlHeaders['content-disposition'])) {
                $contentDisposition = explode('; ', $this->_urlHeaders['content-disposition']);
                if (!empty($contentDisposition[1]) && strpos($contentDisposition[1], 'filename=') !== false) {
                    return substr($contentDisposition[1], 9);
                }
            }
            if ($fileName = @pathinfo($this->_resourceFile, PATHINFO_BASENAME)) {
                return $fileName;
            }
        }
        return $this->_fileName;
    }

    /**
     * Set resource file for download
     *
     * @param string $resourceFile
     * @param string $linkType
     * @return Mage_Downloadable_Helper_Download
     */
    public function setResource($resourceFile, $linkType = self::LINK_TYPE_FILE)
    {
        if (self::LINK_TYPE_FILE == $linkType) {
            //check LFI protection
            /**  $helper Mage_Core_Helper_Data */
            $helper = Mage::helper('core');
            $helper->checkLfiProtection($resourceFile);
        }

        $this->_resourceFile    = $resourceFile;
        $this->_linkType        = $linkType;

        return $this;
    }

    /**
     * Retrieve Http Request Object
     *
     * @return Mage_Core_Controller_Request_Http
     */
    public function getHttpRequest()
    {
        return Mage::app()->getFrontController()->getRequest();
    }

    /**
     * Retrieve Http Response Object
     *
     * @return Mage_Core_Controller_Response_Http
     */
    public function getHttpResponse()
    {
        return Mage::app()->getFrontController()->getResponse();
    }

    public function output()
    {
        $handle = $this->_getHandle();
        if ($this->_linkType == self::LINK_TYPE_FILE) {
            while ($buffer = $handle->streamRead()) {
                print $buffer;
            }
        }
        elseif ($this->_linkType == self::LINK_TYPE_URL) {
            while (!feof($handle)) {
                print fgets($handle, 1024);
            }
        }
    }

    /**
     * Use Content-Disposition: attachment
     *
     * @param mixed $store
     * @return bool
     */
    public function getContentDisposition($store = null)
    {
        return Mage::getStoreConfig(self::XML_PATH_CONTENT_DISPOSITION, $store);
    }
}

and this is the Controller:

<?php
/**
 * Magento
 *
 * NOTICE OF LICENSE
 *
 * This source file is subject to the Open Software License (OSL 3.0)
 * that is bundled with this package in the file LICENSE.txt.
 * It is also available through the world-wide-web at this URL:
 * http://opensource.org/licenses/osl-3.0.php
 * If you did not receive a copy of the license and are unable to
 * obtain it through the world-wide-web, please send an email
 * to license@magento.com so we can send you a copy immediately.
 *
 * DISCLAIMER
 *
 * Do not edit or add to this file if you wish to upgrade Magento to newer
 * versions in the future. If you wish to customize Magento for your
 * needs please refer to http://www.magento.com for more information.
 *
 * @category    Mage
 * @package     Mage_Downloadable
 * @copyright  Copyright (c) 2006-2019 Magento, Inc. (http://www.magento.com)
 * @license    http://opensource.org/licenses/osl-3.0.php  Open Software License (OSL 3.0)
 */

/**
 * Download controller
 *
 * @category    Mage
 * @package     Mage_Downloadable
 * @author      Magento Core Team <core@magentocommerce.com>
 */
class Mage_Downloadable_DownloadController extends Mage_Core_Controller_Front_Action
{

    /**
     * Return core session object
     *
     * @return Mage_Core_Model_Session
     */
    protected function _getSession()
    {
        return Mage::getSingleton('core/session');
    }

    /**
     * Return customer session object
     *
     * @return Mage_Customer_Model_Session
     */
    protected function _getCustomerSession()
    {
        return Mage::getSingleton('customer/session');
    }

    protected function _processDownload($resource, $resourceType)
    {
        $helper = Mage::helper('downloadable/download');
        /*  $helper Mage_Downloadable_Helper_Download */

        $helper->setResource($resource, $resourceType);

        $fileName       = $helper->getFilename();
        $contentType    = $helper->getContentType();

        $this->getResponse()
            ->setHttpResponseCode(200)
            ->setHeader('Pragma', 'public', true)
            ->setHeader('Cache-Control', 'must-revalidate, post-check=0, pre-check=0', true)
            ->setHeader('Content-type', $contentType, true);

        if ($fileSize = $helper->getFilesize()) {
            $this->getResponse()
                ->setHeader('Content-Length', $fileSize);
        }

        if ($contentDisposition = $helper->getContentDisposition()) {
            $this->getResponse()
                ->setHeader('Content-Disposition', $contentDisposition . '; filename='.$fileName);
        }

        $this->getResponse()
            ->clearBody();
        $this->getResponse()
            ->sendHeaders();

        session_write_close();
        $helper->output();
    }

    /**
     * Download sample action
     *
     */
    public function sampleAction()
    {
        $sampleId = $this->getRequest()->getParam('sample_id', 0);
        $sample = Mage::getModel('downloadable/sample')->load($sampleId);
        if (
            $sample->getId()
            && Mage::helper('catalog/product')
                ->getProduct((int) $sample->getProductId(), Mage::app()->getStore()->getId(), 'id')
                ->isAvailable()
        ) {
            $resource = '';
            $resourceType = '';
            if ($sample->getSampleType() == Mage_Downloadable_Helper_Download::LINK_TYPE_URL) {
                $resource = $sample->getSampleUrl();
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_URL;
            } elseif ($sample->getSampleType() == Mage_Downloadable_Helper_Download::LINK_TYPE_FILE) {
                $resource = Mage::helper('downloadable/file')->getFilePath(
                    Mage_Downloadable_Model_Sample::getBasePath(), $sample->getSampleFile()
                );
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_FILE;
            }
            try {
                $this->_processDownload($resource, $resourceType);
                exit(0);
            } catch (Mage_Core_Exception $e) {
                $this->_getSession()->addError(Mage::helper('downloadable')->__('Sorry, there was an error getting requested content. Please contact the store owner.'));
            }
        }
        return $this->_redirectReferer();
    }

    /**
     * Download link's sample action
     *
     */
    public function linkSampleAction()
    {
        $linkId = $this->getRequest()->getParam('link_id', 0);
        $link = Mage::getModel('downloadable/link')->load($linkId);
        if (
            $link->getId()
            && Mage::helper('catalog/product')
                ->getProduct((int) $link->getProductId(), Mage::app()->getStore()->getId(), 'id')
                ->isAvailable()
        ) {
            $resource = '';
            $resourceType = '';
            if ($link->getSampleType() == Mage_Downloadable_Helper_Download::LINK_TYPE_URL) {
                $resource = $link->getSampleUrl();
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_URL;
            } elseif ($link->getSampleType() == Mage_Downloadable_Helper_Download::LINK_TYPE_FILE) {
                $resource = Mage::helper('downloadable/file')->getFilePath(
                    Mage_Downloadable_Model_Link::getBaseSamplePath(), $link->getSampleFile()
                );
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_FILE;
            }
            try {
                $this->_processDownload($resource, $resourceType);
                exit(0);
            } catch (Mage_Core_Exception $e) {
                $this->_getCustomerSession()->addError(Mage::helper('downloadable')->__('Sorry, there was an error getting requested content. Please contact the store owner.'));
            }
        }
        return $this->_redirectReferer();
    }

    /**
     * Download link action
     */
    public function linkAction()
    {
        $id = $this->getRequest()->getParam('id', 0);
        $linkPurchasedItem = Mage::getModel('downloadable/link_purchased_item')->load($id, 'link_hash');
        if (! $linkPurchasedItem->getId() ) {
            $this->_getCustomerSession()->addNotice(Mage::helper('downloadable')->__("Requested link does not exist."));
            return $this->_redirect('*/customer/products');
        }
        if (!Mage::helper('downloadable')->getIsShareable($linkPurchasedItem)) {
            $customerId = $this->_getCustomerSession()->getCustomerId();
            if (!$customerId) {
                $product = Mage::getModel('catalog/product')->load($linkPurchasedItem->getProductId());
                if ($product->getId()) {
                    $notice = Mage::helper('downloadable')->__('Please log in to download your product or purchase <a href="%s">%s</a>.', $product->getProductUrl(), $product->getName());
                } else {
                    $notice = Mage::helper('downloadable')->__('Please log in to download your product.');
                }
                $this->_getCustomerSession()->addNotice($notice);
                $this->_getCustomerSession()->authenticate($this);
                $this->_getCustomerSession()->setBeforeAuthUrl(Mage::getUrl('downloadable/customer/products/'),
                    array('_secure' => true)
                );
                return ;
            }
            $linkPurchased = Mage::getModel('downloadable/link_purchased')->load($linkPurchasedItem->getPurchasedId());
            if ($linkPurchased->getCustomerId() != $customerId) {
                $this->_getCustomerSession()->addNotice(Mage::helper('downloadable')->__("Requested link does not exist."));
                return $this->_redirect('*/customer/products');
            }
        }
        $downloadsLeft = $linkPurchasedItem->getNumberOfDownloadsBought()
            - $linkPurchasedItem->getNumberOfDownloadsUsed();

        $status = $linkPurchasedItem->getStatus();
        if ($status == Mage_Downloadable_Model_Link_Purchased_Item::LINK_STATUS_AVAILABLE
            && ($downloadsLeft || $linkPurchasedItem->getNumberOfDownloadsBought() == 0)
        ) {
            $resource = '';
            $resourceType = '';
            if ($linkPurchasedItem->getLinkType() == Mage_Downloadable_Helper_Download::LINK_TYPE_URL) {
                $resource = $linkPurchasedItem->getLinkUrl();
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_URL;
            } elseif ($linkPurchasedItem->getLinkType() == Mage_Downloadable_Helper_Download::LINK_TYPE_FILE) {
                $resource = Mage::helper('downloadable/file')->getFilePath(
                    Mage_Downloadable_Model_Link::getBasePath(), $linkPurchasedItem->getLinkFile()
                );
                $resourceType = Mage_Downloadable_Helper_Download::LINK_TYPE_FILE;
            }
            try {
                $this->_processDownload($resource, $resourceType);
                $linkPurchasedItem->setNumberOfDownloadsUsed($linkPurchasedItem->getNumberOfDownloadsUsed() + 1);

                if ($linkPurchasedItem->getNumberOfDownloadsBought() != 0 && !($downloadsLeft - 1)) {
                    $linkPurchasedItem->setStatus(Mage_Downloadable_Model_Link_Purchased_Item::LINK_STATUS_EXPIRED);
                }
                $linkPurchasedItem->save();
                exit(0);
            }
            catch (Exception $e) {
                $this->_getCustomerSession()->addError(
                    Mage::helper('downloadable')->__('An error occurred while getting the requested content. Please contact the store owner.')
                );
            }
        } elseif ($status == Mage_Downloadable_Model_Link_Purchased_Item::LINK_STATUS_EXPIRED) {
            $this->_getCustomerSession()->addNotice(Mage::helper('downloadable')->__('The link has expired.'));
        } elseif ($status == Mage_Downloadable_Model_Link_Purchased_Item::LINK_STATUS_PENDING
            || $status == Mage_Downloadable_Model_Link_Purchased_Item::LINK_STATUS_PAYMENT_REVIEW
        ) {
            $this->_getCustomerSession()->addNotice(Mage::helper('downloadable')->__('The link is not available.'));
        } else {
            $this->_getCustomerSession()->addError(
                Mage::helper('downloadable')->__('An error occurred while getting the requested content. Please contact the store owner.')
            );
        }
        return $this->_redirect('*/customer/products');
    }

}

So, you said it should not get anymore the file size because you set "Transfer-Encoding: chunked". How should it handle without it?

About Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!

Forum Discussion

Since 31/01/22 all the download files become corrupted.

About Dropbox API Support & Feedback

Related Content

All files downloaded from Dropbox are corrupt

Re: Since 31/01/22 all the download files become corrupted.

I downloaded my files and got a corrupt file

Download button is unresponsive when trying to download password-protected files from the website

OTA for CC3200 Corrupted Download file.

Recent Discussions

/oauth2/authorize (code flow) redirects to http://www.dropbox.com/home instead of redirect_uri

Add id and .tag fields to DeletedMetadata in the API

Shared Link - Download in .NET

Dropbox API server root certificate changes for .Net

GetCurrentAccountAsync API Failing for some Team Members while using Impersonation