Forum Discussion

hramosvz's avatar
hramosvz
Explorer | Level 4
5 years ago

Problems downloading big size files (20 GB) from the API .NET

Hello.

 

I'm trying to download some files from a couple of folders in dropbox, but when I try to download a couple of files that I have of about 20 gb, something strange happens. It appears that the file is downloaded correctly, but the file size in my local computer is about 13 gb, so the file it's corrupted. with 5gb files or lower this doesn't happen. this is the code.

 

dropboxFileInfo = (FileMetadata)item;

using (var response = await dbx.Files.DownloadAsync(dropboxFileInfo.PathLower.ToString()))
{
Console.WriteLine("Descargando " + dropboxFileInfo.Name.ToString() + "...");
using (FileStream fileStream = File.Create(localname))
{

(await response.GetContentAsStreamAsync()).CopyTo(fileStream);
var local = Convert.ToInt64(fileStream.Length);
var cloud = Convert.ToInt64(dropboxFileInfo.Size);

if (local == cloud)
{

Console.WriteLine("Descargado");
}
else
{
Console.WriteLine("Error al descargar");

}

 

 

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff

    Does the actual downloaded size vary, or is it the same each time? If it's the same, please let me know what it is for reference.

     

    Also, how long does the (incomplete) download take? I believe a single connection is only valid for 90 minutes, so that could be the cause if it takes that long. (Though the HTTPS client is expected to throw an error in that case when it sees the response is incomplete, but that may not be occurring unfortunately.)

    • hramosvz's avatar
      hramosvz
      Explorer | Level 4

      It varies. I did some more testing and it also happens with smaller files. I was downloading a file of 3.48 GB but just 1.75 GB were downloaded. This happened in exaclty 15 minutes without any error showing. This the complete method code.

       

       

            private async Task<int> Run()
              {
                  DropboxCertHelper.InitializeCertPinning();
      
                  
      
                  // Specify socket level timeout which decides maximum waiting time when no bytes are
                  // received by the socket.
                  var httpClient = new System.Net.Http.HttpClient(new WebRequestHandler { ReadWriteTimeout = 100 * 10000000 })
                  {
                      // Specify request level timeout which decides maximum time that can be spent on
                      // download/upload files.
                      Timeout = TimeSpan.FromHours(10)
                  };
      
                  try
                  {
                      var config = new DropboxClientConfig("SubidaBacks")
                      {
                          HttpClient = httpClient
                      };
      
                      var dbx = new DropboxClient("************", config);
      
                      var nbackups = 0;
                      var ListaBaks = new List<string>();
                     
      
                      using (dbx)
                      {
      
      
                          var full = await dbx.Users.GetCurrentAccountAsync();
      
      
                          foreach (var folder in Folders.Split(';'))
                          {
                              var listOfFolders = await dbx.Files.ListFolderAsync(folder, true);
      
                              FolderMetadata dropboxFolderInfo = new FolderMetadata();
                              FileMetadata dropboxFileInfo = new FileMetadata();
                              Int64 local, cloud;
      
      
      
                              foreach (var item in listOfFolders.Entries)
                              {
                                  try
                                  {
                                      if (item.IsFile)
                                      {
      
                                          foreach (var fecha in Dates())
                                          {
      
                                              if (item.Name.ToString().Contains(fecha))
                                              {
      
                                                  var localname = localFilePath + "\\" + item.Name;
      
                                                  if (!File.Exists(localname))
                                                  {
                                                      dropboxFileInfo = (FileMetadata)item;
                                                      var response = await dbx.Files.DownloadAsync(dropboxFileInfo.PathLower.ToString());
                                                      ulong fileSize = response.Response.Size;
                                                      const int bufferSize = 1024 * 1024;
      
                                                      var buffer = new byte[bufferSize];
      
                                                      Console.WriteLine("Descargando " + dropboxFileInfo.Name.ToString() + "...");
      
                                                      using (var stream = await response.GetContentAsStreamAsync())
                                                      {
                                                          using (var file = new FileStream(localname, FileMode.OpenOrCreate))
                                                          {
                                                              var length = stream.Read(buffer, 0, bufferSize);
      
                                                              while (length > 0)
                                                              {
                                                                  file.Write(buffer, 0, length);
                                                                  var percentage = 100 * (ulong)file.Length / fileSize;
                                                                  // Update progress bar with the percentage.
                                                                  // progressBar.Value = (int)percentage
                                                                  Console.WriteLine(percentage.ToString()+"% - "+ dropboxFileInfo.Name.ToString() +" - " + DateTime.Now.ToString());
      
                                                                  length = stream.Read(buffer, 0, bufferSize);
                                                              }
      
                                                               local = Convert.ToInt64(file.Length);
                                                               cloud = Convert.ToInt64(dropboxFileInfo.Size);
      
                                                             
      
                                                          }
      
      
                                                      }
      
                                                      if (local == cloud)
                                                      {
                                                          ListaBaks.Append(item.PathLower.ToString());
                                                          nbackups = nbackups + 1;
                                                          Console.WriteLine("Descargado");
                                                          
                                                      }
                                                      else
                                                      {
                                                          File.Delete(localname);
                                                          Console.WriteLine("Error al descargar");
      
                                                      }
      
                                                      
                                                  }
                                              }
                                          }
                                      }
                                  }
                                  catch (Exception ex)
                                  {
                                      Console.WriteLine(ex.Message);
                                  }
                              }
                          }
      
      
                      }
      
                      // Correo.EnvioCorreo(nbackups.ToString(), string.Join("\n", ListaBaks));
      
                  }
                  catch (HttpException e)
                  {
                      Console.WriteLine("Exception reported from RPC layer");
                      Console.WriteLine("    Status code: {0}", e.StatusCode);
                      Console.WriteLine("    Message    : {0}", e.Message);
                      if (e.RequestUri != null)
                      {
                          Console.WriteLine("    Request uri: {0}", e.RequestUri);
                      }
                  }
      
                  return 0;
              }

       

      • Greg-DB's avatar
        Greg-DB
        Icon for Dropbox Staff rankDropbox Staff

        Thanks for following up and confirming the size varies.

         

        Also, just to clarify your last message, where you mentioned it failed after exactly 15 minutes, does it always fails after exactly 15 minutes (or did you mean just this last attempt in particular)?

         

        And you mentioned there was no error, but does it get to your "Console.WriteLine("Error al descargar");" line, or does it seem to stop somewhere before that?

         

        I'm trying to reproduce this, but it seems to be working fine for me, even when downloading takes longer than 15 minutes.

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.5,945 PostsLatest Activity: 19 minutes ago
351 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!