downloading a large file using python v2 API

[Olaf B.2]

Dear Dropboxers,

would it be possible to see an example for large file download, equivalent to https://www.dropboxforum.com/hc/en-us/community/posts/205544836-python-upload-big-file-example for the upload?

Thanks.

[MarceloC]

Hello!

This post is a bit old, but was the incremental download implemented in the v2 Python SDK?

I'm having troubles to manage large accounts (many files and large files) so I'm developing some tools using Python. Downloading large files (>20Gb) using the desktop application takes ages and have no control, and even using the navigator there are many interruptions or abortions, so the idea is to have total control about exactly what is being downloaded, and to be able to restart from the last sucessful chunk as needed.

I'm already being able to upload large files using files_upload_session_start/append/finish methods.

Thanks and regards.

[Greg-DB]

@MarceloC No, unfortunately support for Range requests still hasn't been implemented in the Dropbox API v2 Python SDK. 

[MarceloC]

Hi Greg,

Thank you for your reply!

Although I'm just starting with Python and still have no much knowledge of it's libraries or objects structure, I tried to implement your suggestion and got to have the partial download working!

It is just an workaround and very FAR from a proper or elegant solution, but if anyone wants just to make it work, this is what I did... (use under your own responsability!)

As I could not even find from where the request_json_string method (where the header is set) is called to try to include some proper parameters, I just included 2 global variables in the dropbox unit, and created a method to set them from the main program.

In the beginning of the file dropbox.py (backup it before changing!!) I added the global (in class context) variables: (may be inserted in the first lines of the file)

Download_Range_Start=-1
Download_Range_End=-1

Just before the request_json_string method definition (line 389 at the original dropbox.py file) I added the method to set the variables:

    def set_download_range(self,Start,End):
        global Download_Range_Start, Download_Range_End
        Download_Range_Start=Start
        Download_Range_End=End

And, inside the request_json_string method included two lines (the 3rd and 4th lines below - line 433 at the original dropbox.py file):

        ...
        elif route_style == self._ROUTE_STYLE_DOWNLOAD:
            headers['Dropbox-API-Arg'] = request_json_arg
            if (Download_Range_Start>=0)and(Download_Range_End>=Download_Range_Start):
                headers['Range'] = 'bytes='+str(Download_Range_Start)+'-'+str(Download_Range_End)
            stream = True
        elif route_style == self._ROUTE_STYLE_UPLOAD:
        ...

To use just put in your main program before downloading the file: (dbx is the main dropbox object)

    dbx.set_download_range(5,10)

This will make the download bring from the 6th to the 11th byte from the file (it is 0 based).

As it is using global variables, be careful because this change will affect all downloads done after it, and may persist if the library is retained in the same process (and possibly may affect other threads in the same process, so not recomended for multi-thread apps).

To return to the original condition (no range) use:

    dbx.set_download_range(-1,-1)

The Start position must be inside the file, to prevent an exception. The End position may be after the file end.

This approach is different than the one for upload, based in a session (appending chunks) controlled by Dropbox server. In this case it is up to you to create and control a "session" in the client side and merge the chunks.

Well, if someone wants to improve and do a more elegant implementation adding parameters to the download methods, please be welcome.

*I only tested with files_download_to_file method, but probably it will work with files_download as well.