Programmatically download a shared file using C++ Curl

[Makhlouf]

Hi,

I developped an aplication in C++ that runs on Windows. I am trying to add a feauture in the program to download a shared file from my dropbox folder. I would like the downloading to go in background in similar way to windows API URLDownloadToFile(). The windows function downloads files from the web. But when I use the dropbox file link, the function downloads an html file instead of my file. I tried to use CURL library but got stuck with the access_token. I created an application access_token as described by Dropbox developper guide. When I use this access_token I receive authorization faliure error message. I wonder if the job I am trying to do is actually doable, and whether my approach is the right one. Where should I go or what should I do to get a shared file downloaded with my windows application. 

Thanks for your help

[Greg-DB]

You can download a file from a shared link directly without using an access token, like you tried initially. Note that by default however, Dropbox shared links point to an HTML preview page, not the file data itself. You can modify the link to point to the file data though, as described here:

https://help.dropbox.com/files-folders/share/force-download

[Makhlouf]

Thanks for your answer. I read this article and tried it but did not work. Here is my observations:

1- If I leave the shared link as it is, curl will download an html file which when opened in a browser I get the below screen.

2- If I chose to skip sign in, I can not download teh file.

3- If I modified the shared link with dl=1 instead of dl=0, I receive nothing.

4- Windows function URLDownloadToFile() download an html file using the original link also. However, it can not be open in the browser. It seems it has error message but I can not read html to find out the problem.

I am not quite sure what I am doing wrong. Please advise.

[Greg-DB]

I'm not sure I follow. Can you share your code and whatever error/output you're getting when you try to download from a shared link with dl=1? Thanks in advance! 

[Makhlouf]

Below is the code I use:

#include "Stdafx.h"
#include "curl.h"
#include "Download.h"

static const char* FileURL[] = {
"https://www.dropbox.com/s/9thz78bwfeykqra/version.dat?dl=0" ,
"https://www.dropbox.com/s/9thz78bwfeykqra/version.dat?dl=1"
} ;

static size_t write_data (void *ptr,size_t size,size_t nmemb,void* stream)
{
size_t written = fwrite (ptr,size,nmemb,(FILE * ) stream) ;
return written ;
}

int DownloadFile ()
{
static char TargetURL[] = "E:\\Download\\DownloadURL\\Test.dat" ;
static char TargetCURL[] = "E:\\Download\\DownloadCURL\\Test.dat" ;

// Download the file using windows function into DownloadURL folder
URLDownloadToFile (nullptr,FileURL[1],TargetURL,0,nullptr) ;

// Download the file using curl library into DownloadCURL folder
if (auto curl = curl_easy_init ()) {
auto fp = fopen (TargetCURL,"wb") ;
curl_easy_setopt (curl,CURLOPT_URL,FileURL[1]) ;
curl_easy_setopt (curl,CURLOPT_WRITEFUNCTION,write_data) ;
curl_easy_setopt (curl,CURLOPT_FOLLOWLOCATION,1) ;
curl_easy_setopt (curl,CURLOPT_FAILONERROR,1) ;
curl_easy_setopt (curl,CURLOPT_WRITEDATA,fp) ;
curl_easy_setopt (curl,CURLOPT_HTTPPROXYTUNNEL,1) ;

/* Perform the request, res will get the return code */
auto res = curl_easy_perform (curl) ;

fclose (fp) ;
curl_easy_cleanup (curl) ;
} /* if (auto curl = curl_easy_init ()) */

return true ;

} /* Download */

The original link is FileURL[0] and the modified link is FileURL[1]

The downloaded file with Curl library, and original link, is an html the opens a web page for downloading the file. If I typed the shared link into the address bar in my browser, I get same page.

When I download the modified link, both windows function and Curl library download nothing. Curl function curl_easy_perform () returns : (6) CURL_COULDNT_RESOLVE_HOST.

would you like to inspect the html files I get with original link, Because they may have some information that I can not read.

Thank for your help. 

[Greg-DB]

There are a few things that I notice that you may want to address:

• You should get rid of the "dl=0" URL, since that's not expected to work. (It is supposed to return an HTML page, which isn't what you want.)
• You're attempting to use both 'URLDownloadToFile' and 'curl', but you only need one. You should probably just decide on one and remove the other. (Personally, I'd just use 'curl' since it's more configurable.)
• You're setting 'CURLOPT_HTTPPROXYTUNNEL', which tells curl to use a proxy defined by 'CURLOPT_PROXY', but you're not setting 'CURLOPT_PROXY' that I can see. You should probably get rid of 'CURLOPT_HTTPPROXYTUNNEL', or properly set 'CURLOPT_PROXY'.
• The 'CURL_COULDNT_RESOLVE_HOST' error seems to indicate a DNS issue looking up the Dropbox domain, but the Dropbox domains are currently resolving properly, so you may need to check on any issues with your DNS server. It's also possible this is related to your proxy configuration, as noted above.
• Your linked "version.dat" file appears to just contain 56 bytes that don't print out anything visible. If you're trying to print that out as text, that might be the "nothing" you're referring to. You may want to try this with a different file that contains some printable text first for clarity.

[Makhlouf]

Dear Greg,

Thank you for your kind support. Here is what I did:

1- I got rid of "dl=0" URL.

2- I got ride of URLDownladToFile(). I will stick to curl library.

3- I got ride of curl_easy_setopt (curl,CURLOPT_HTTPPROXYTUNNEL,1) ;

4- I am not using any proxy. 

5- I still receive (6) CURL_COULDNT_RESOLVE_HOST error.

6- When I say receive nothing, I mean the downloaded file size is zero byte.

7- I uploaded the cpp file and tried to downloaded it (1.45 KB). The downloaded file size is still zero byte.

Below is the new code with above changes:

#include "Stdafx.h"
#include "curl.h"
#include "Download.h"

static const char* FileURL =
"https://www.dropbox.com/s/4cn3neh4omhhzr5/Download.cpp?dl=1" ;

static size_t write_data (void *ptr,size_t size,size_t nmemb,void* stream)
{
size_t written = fwrite (ptr,size,nmemb,(FILE * ) stream) ;
return written ;
}

int DownloadFile ()
{
static char TargetFile[] = "E:\\Download\\DownloadCURL\\Download.cpp" ;

// Download the file using curl library into DownloadCURL folder
if (auto curl = curl_easy_init ()) {
auto fp = fopen (TargetFile,"wb") ;
curl_easy_setopt (curl,CURLOPT_URL,FileURL) ;
curl_easy_setopt (curl,CURLOPT_WRITEFUNCTION,write_data) ;
curl_easy_setopt (curl,CURLOPT_FOLLOWLOCATION,1) ;
curl_easy_setopt (curl,CURLOPT_FAILONERROR,1) ;
curl_easy_setopt (curl,CURLOPT_WRITEDATA,fp) ;

/* Perform the request, res will get the return code */
auto res = curl_easy_perform (curl) ;

fclose (fp) ;
curl_easy_cleanup (curl) ;
} /* if (auto curl = curl_easy_init ()) */

return true ;

} /* Download */

If everything else is correct, could it be that my curl library is not functioning properly?

I downloaded the source code for the library (version 7-67.0). There were many options to apply during the compilation. I once built it with OpenSSL support. Then removed the OpenSSL support following a post I read about Windows / OpenSSL compatibility.

So any suggestion regarding building the library will be much appreciated.

Thanks again for your help

[Greg-DB]

Thanks, in that case the empty file isn't surprising since the call is failing, with the 'CURL_COULDNT_RESOLVE_HOST' error. I did just try it myself, and the Dropbox domain is resolving properly for me.

That error indicates a DNS issue, so you'll need to troubleshoot your DNS configuration/server. I can't offer support for your DNS server though, or curl or OpenSSL, as those are made by third parties.

[Здравко]

Hi @Makhlouf,

Did you try: https://www.dropbox.com/s/9thz78bwfeykqra/version.dat?raw=1 in your code?

[Makhlouf]

Dear Greg,
Thanks a lot for you help. I will investigate the curl library myself. Glad to hear it worked for you. This gives me hope to do it myself as well.