CSharp SDK await dbx.Files.UploadAsync(...) object gets disposed before upload

Steve: thanks for the tip, from which I presume there's no capacity in the SDK to upload large files.

Qiming: If you have any further information on how to easily do this, I'd be grateful.  

As this is for a series of Continuous Integration projects (Automated Builds), I'll first see if the Google Drive SDK or OneDrive SDK can handle large file uploads in the interim, I have a next Monday delivery window on this proof of concept.

However I'd be very interested to be supplied with some boilerplate code for this, CI + Dropbox is one example of Dropbox use where each and every instance of it's use is virtually guaranteed of having to undertake a paid subscription because of CI typically keeps the accumulated results.

Thanks, and Kind regards,

Anthony

"Steve: thanks for the tip, from which I presume there's no capacity in the SDK to upload large files."

I'm not sure that you mean. That's precisely what those methods I linked to are for.

Steve, my mistake, when you said .Net SDK I thought you were referring to the .Net HttpClient API & etc, instead of the Dropbox SDK, I'll take a look, that's great news indeed as Dropbox is our first choice, thanks again!

Qiming: If you have any boilerplate code for this, I'd appreciate it!

Kind regards,

Anthony

Below is example code about how to do chunk upload.

 private async Task Upload(string file)
 {
 const string TargetPath = "/test.zip";
 const int ChunkSize = 4096 * 1024;
 using (var fileStream = File.Open(file, FileMode.Open))
 {
 if (fileStream.Length <= ChunkSize)
 {
 await this.client.Files.UploadAsync(TargetPath, body: fileStream);
 }
 else
 {
 Console.WriteLine("Start chunk uploading");
 await this.ChunkUpload(TargetPath, fileStream, ChunkSize);
 }
 } 
 }

 private async Task ChunkUpload(String path, FileStream stream, int chunkSize)
 {
 int numChunks = (int)Math.Ceiling((double)stream.Length / chunkSize);
 byte[] buffer = new byte[chunkSize];
 string sessionId = null;
 for (var idx = 0; idx < numChunks; idx++)
 {
 var byteRead = stream.Read(buffer, 0, chunkSize);

 using (var memSream = new MemoryStream(buffer, 0, byteRead))
 {
 if (idx == 0)
 {
 var result = await this.client.Files.UploadSessionStartAsync(memSream);
 sessionId = result.SessionId;
 }
 else
 {
 var cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));

 if (idx == numChunks - 1)
 {
 await this.client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memSream);
 }
 else
 {
 await this.client.Files.UploadSessionAppendAsync(cursor, memSream);
 }
 }
 }
 }
 }

Qiming, thank you very much indeed, once I get all of this finished I'll blog it as payback, once again, thanks!

Kind regards,

Anthony

Qiming, it works like a dream, and thanks so much, I'll create a Nuget package soon that specifically targets CI with built in logging.

Kind regards,

Anthony

Thanks Qiming. 

Cheers Qiming,

I was wondering if there was a way to pass a value from the upload chunks to the view so the user can see how long is left of the upload.

The user presses a submit button and  this calls an actionResult method in the controller which gets the file and uploads to dropbox using above method.

Currently have this in the Upload Chunks method.

decimal complete = chunkSize * idx;
decimal fileSize = stream.Length;
decimal percentageComplete = (complete / fileSize) * 100;

Wondering if there was any way to get the percentageComplete value to display in the view? 
Thanks

Hi Nathan,

I think this is not sdk level thing. What you need is a way to get progress for some long running action in MVC application. I think with SignalR you may be able to do that.

Thanks Qiming, I managed to get it to work using signarR, was using this guide mostly

http://stackoverflow.com/questions/30711074/using-signalr-hub-from-mvc-action

Here's my code if anyone else wants to do it

In my controllor create action 

Snippet

var dropBoxService = new DropboxService(uploadFile, fileName);
                    var task = Task.Run((Func<Task>)dropBoxService.Upload);
                    task.Wait();

Then In my DropboxService Class

Snippet

public class DropboxService
    {
        private readonly HttpPostedFileBase _file;
        private readonly string _fileName;
 
        public DropboxService(HttpPostedFileBase file, string fileName)
        {
            this._file = file;
            this._fileName = fileName;
        }
 
        public async Task Upload()
        {
            // Specify socket level timeout which decides maximum waiting time when on bytes are
            // received by the socket.
 
            var httpClient = new HttpClient(new WebRequestHandler { ReadWriteTimeout = 10 * 1000 })
            {
                // Specify request level timeout which decides maximum time taht can be spent on
                // download/upload files.
                Timeout = TimeSpan.FromMinutes(20)
            };
 
            var dbx = new DropboxClient(ConfigurationManager.AppSettings["DropBoxAccessToken"], httpClient: httpClient);
            using (dbx)
            {
                const int chunkSize = 1024 * 1024;
 
                const string folder = "/CandidateTasks";
 
                using (var mem = _file.InputStream)
                {
                    if (mem.Length <= chunkSize)
                    {
                        await dbx.Files.UploadAsync(
                            folder + "/" + _fileName,
                            WriteMode.Overwrite.Instance,
                            body: mem);
                    }
                    else
                    {
                        Console.WriteLine("Start chunk uploading");
                        var path = folder + "/" + _fileName;
                        await ChunkUpload(path, mem, chunkSize, dbx);
                    }
                }
            }
        }
 
        private static async Task ChunkUpload(string path, Stream stream, int chunkSize, DropboxClient client)
        {
            var numChunks = (int)Math.Ceiling((double)stream.Length / chunkSize);
            var buffer = new byte[chunkSize];
            string sessionId = null;
            // Initialize Hub context
            var hubContext = GlobalHost.ConnectionManager.GetHubContext<ProgressHub>();
            hubContext.Clients.All.sendMessage("Initalizing...", 0);
            for (var idx = 0; idx < numChunks; idx++)
            {
                var byteRead = stream.Read(buffer, 0, chunkSize);
                Debug.WriteLine("Idx " + idx);
                using (var memSream = new MemoryStream(buffer, 0, byteRead))
                {
                    if (idx == 0)
                    {
                        var result = await client.Files.UploadSessionStartAsync(memSream);
                        sessionId = result.SessionId;
                    }
                    else
                    {
                        var cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));
                        decimal complete = chunkSize * idx;
                        decimal fileSize = stream.Length;
                        var percentageComplete = (complete / fileSize) * 100;
 
                        // Indicate file is being uploaded
                        hubContext.Clients.All.sendMessage("Uploading To Dropbox: ", (int)percentageComplete);
 
                        if (idx == numChunks - 1)
                        {
                            await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memSream);
                        }
                        else
                        {
                            await client.Files.UploadSessionAppendAsync(cursor, memSream);
                        }
                    }
                }
            }
            hubContext.Clients.All.sendMessage("Attachment Uploaded ", 100);
        }

In my view I have

Snippet

@using (Html.BeginForm("Create", "CandidateTask", FormMethod.Post, new { enctype = "multipart/form-data", id = "form1" }))
{ 
blah blah
Snippet

<input name="uploadFile" type="file" accept=".jpg, .jpeg, .png, .gif,
                  .doc, .pdf, .ppt, .odt, .xls, .xlsx, .zip, .rar,
                  .mp3, .m4a, .ogg, .wav,
                  .mov, .mpg, .ogv, .3gp, .3g2, .wmv, .mp4" />
           <div class="alert alert-warning">
               <strong>Warning!</strong> Uploaded attachments must only...
           </div>
           <br />
 
           <input type="submit" id="button1" value="Create New Task" class="btn btn-success" />
           <span id="msg" style="color:red;" />


Then I have the progress bar
Snippet

<div class="col-lg-12">
    <div id="status"></div>
</div>
 
<div class="col-lg-12">
    <div class="progress">
        <div class="progress-bar" role="progressbar" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100" style="min-width: 20px;">
            0%
        </div>
    </div>
</div>

The script I use in the view is
Snippet

<!-- Progress Indicator Column -->
<script type="text/javascript" language="javascript">
 
    $(document).ready(function () {
        $('.progress').hide();
        $('#button1').on('click', function () {
            $('#form1').submit();
        })
 
        $('#form1').on('submit', function (e) {
            var thisForm = this;
            e.preventDefault();
 
            $.ajax({
                url: '/CandidateTask/Create',
                type: "POST",
 
                data: new FormData(this),
                processData: false,
                contentType: false,
                success: function (data) {
                    console.log("done");
                    var success = data.success;
                    var errorMsg = data.errorMsg;
                    if (success === true) {
                        var targetUrl = '/CandidateTask/Index';
                        $(thisForm).load(targetUrl);
                    } else {
                        //alert(data.errorMsg);
                        $("#msg").html(data.errorMsg);
                    }
 
                    //    alert("errorsdsdf" + data.errorMessage + data.success);
                }
 
            });
 
            // initialize the connection to the server
            var progressNotifier = $.connection.progressHub;
 
            // client-side sendMessage function that will be called from the server-side
            progressNotifier.client.sendMessage = function (message, count) {
                // update progress
                UpdateProgress(message, count);
            };
 
            // establish the connection to the server and start server-side operation
            $.connection.hub.start().done(function () {
                // call the method CallLongOperation defined in the Hub
                progressNotifier.server.callLongOperation();
            });
        });
    });
 
    function UpdateProgress(message, count) {
 
        // get status div
        var status = $("#status");
 
        // set message
        status.html(message);
 
        // get progress bar
        if (count > 0) {
            $('.progress').show();
        }
 
        $('.progress-bar').css('width', count + '%').attr('aria-valuenow', count);
        $('.progress-bar').html(count + '%');
 
    }
</script>

And in my shared layout 
Snippet

@Styles.Render("~/Content/css")
    @Scripts.Render("~/bundles/jquery")
    @Scripts.Render("~/bundles/bootstrap")
    @Scripts.Render("~/bundles/modernizr")
    @Scripts.Render("~/bundles/signalr")
    @Scripts.Render("/signalr/hubs")
    <script src="http://malsup.github.com/jquery.form.js"></script>
    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
    <script src="~/Scripts/jquery.signalR-2.2.0.js" type="text/javascript"></script>
 
   
    @RenderSection("scripts", false)

I use the same startup and progress Hub as in the stack overflow example.

last of all in my controller I return true/false result back
Snippet

var result = new { success = false, errorMsg = "ERRORRR" };
          return Json(result, JsonRequestBehavior.AllowGet);

Bit messy but may help some people so thought I'd share