cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Want to learn some quick and useful tips to make your day easier? Check out how Calvin uses Replay to get feedback from other teams at Dropbox here.

Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

.Net SDK on UWP download stream sends empty buffers on read

.Net SDK on UWP download stream sends empty buffers on read

Opuscope
Helpful | Level 5
Go to solution

Hi Dropbox dev community!

 

We're having a bit of a problem with the .Net SDK on our UWP app : we're using response.GetContentAsStreamAsync() after getting the path of a file on a dropbox account, for progression measurement and having it non blocking for our UI (context : we're on a Hololens inside a Unity app). We used to go for response.GetContentAsByteArrayAsync() before but on bigger files, the hololens memory just doesn't hold up and usually crashes the entire download task. So we went for a stream for progressively downloading and saving the file locally. This is our current code :

 

 

ulong fileSize = response.Response.Size;
int bufferSize = 4 * 1024 * 1024;
var buffer = new byte[bufferSize];
ulong downloadProgress = 0;
using (var stream = await response.GetContentAsStreamAsync())
{
    int byteReadLength = stream.Read(buffer, 0, bufferSize);
    while (byteReadLength > 0)
    {
        using (var outputStream = fileStream.GetOutputStreamAt(downloadProgress))
        {
            using (var writer = new DataWriter(outputStream))
            {
                writer.WriteBytes(buffer);
                await writer.StoreAsync();
                await outputStream.FlushAsync();
            }
        }
        downloadProgress += (ulong)buffer.Length;

        if (downloadProgress > fileSize)
            break;
        if (downloadProgress + (ulong)bufferSize > fileSize)
        {
            bufferSize = (int)(fileSize - downloadProgress);
        }
        byteReadLength = stream.Read(buffer, 0, bufferSize);
    }
}

First things first : on usage, this code seems to work, the streaming seems to happen chunk by chunk of 4mb and the save to file does create an adequately sized result on the hard drive. Keep in mind this runs on a Hololens, the main reason we force the file save during the stream read loop is because we do not have the memory capacity to fully load big (300mb+) files into RAM.

 

But... when trying to open the downloaded file locally, may it be a video, a (really big) text file or a picture, it's always corrupted. And further analysis of the file indicates that :

  • the first 4mb chunk went through correctly
  • the rest is just 0x00 bytes with occasional nonsensical clutter every 100mbs or so

So, we kinda have a few questions:

  • are we doing something wrong?
  • why is there no EOF on stream? We had to do the downloadProgress check stuff to check we weren't getting more data than was expected, because we managed to fill up a Hololens' entire HDD with one download that never ended

Small remark : yes, we know the SDK is not really compatible with Unity, but at this point in the app, the context is full UWP, completely separate from the Unity main thread. Also, it works with the GetContentAsByteArrayAsync() method, only that the Hololens doesn't like getting blasted with so much live memory usage.

 

thank you 🙂

 

1 Accepted Solution

Accepted Solutions

Greg-DB
Dropbox Staff
Go to solution

We looked into this, and we believe the issue here is due to this possible behavior from Stream.Read:

 

 

Return Value
Type: System.Int32
The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.

 

That means that byteReadLength in the code snippet here won't always necessarilly be a whole 4 MB. The being the case, it looks like you'll need to make two changes:

- when you write the data to the file, only write as much as byteReadLength

- when you increment your downloadProgress, only add as much as byteReadLength

View solution in original post

2 Replies 2

Greg-DB
Dropbox Staff
Go to solution

We looked into this, and we believe the issue here is due to this possible behavior from Stream.Read:

 

 

Return Value
Type: System.Int32
The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.

 

That means that byteReadLength in the code snippet here won't always necessarilly be a whole 4 MB. The being the case, it looks like you'll need to make two changes:

- when you write the data to the file, only write as much as byteReadLength

- when you increment your downloadProgress, only add as much as byteReadLength

Opuscope
Helpful | Level 5
Go to solution

it works much better indeed! Thank you!

Need more support?
Who's talking

Top contributors to this post

  • User avatar
    Opuscope Helpful | Level 5
  • User avatar
    Greg-DB Dropbox Staff
What do Dropbox user levels mean?