One month down in 2025: How are your resolutions coming along? Check out how to get back on track here.
Forum Discussion
DS6
8 years agoExplorer | Level 4
Dropbox chunks upload slow & unpredictable
I am trying to upload files larger than 150 MB using Dropbox API v2 on iOS. Problem is upload is sometimes slow, sometimes it hangs in the middle for several seconds, sometimes it stalls in the middle forever without failing or invoking the failure block. Using chunk sizes less than 50 MB makes it more reliable but chunk size like 100 MB throws all the issues. This happens during tests so I believe in production users might see more issues.
Here is the sample code I use for uploading. I am also looking for a way to continue these uploads in background. Please help.
#define DB_CHUNK_SIZE (50*1024.0*1024.0)
//Dropbox specific
@property DBUploadTask<DBFILESUploadSessionStartResult *, DBNilObject *> * uploadTask;
@property DBUploadTask<DBNilObject *, DBFILESUploadSessionLookupError *> * continueTask;
@property DBUploadTask<DBFILESFileMetadata *, DBFILESUploadSessionFinishError *> *finishTask;
@property unsigned long long offset;
@property NSFileHandle *fileHandle;
@property NSNumber *fileSize;
@property NSString *sessionId;
DBUserClient *client = [DBClientsManager authorizedClient]; _fileHandle = [NSFileHandle fileHandleForReadingAtPath:[url path]]; Float64 chunkSize = DB_CHUNK_SIZE;//50 MB chunk size NSNumber *fileSizeNumber = [fileAttributes objectForKey:NSFileSize]; _fileSize = fileSizeNumber; NSData *data = [_fileHandle readDataOfLength:chunkSize]; LibraryViewController *__weak wSelf = self; _uploadTask = [client.filesRoutes uploadSessionStartData:data]; [_uploadTask setResponseBlock:^(DBFILESUploadSessionStartResult * _Nullable result, DBNilObject * _Nullable routeError, DBRequestError * _Nullable networkError) { if (result) { NSLog(@"%@\n", result); wSelf.sessionId = result.sessionId; DBFILESUploadSessionCursor *cursor = [[DBFILESUploadSessionCursor alloc] initWithSessionId:result.sessionId offset:[NSNumber numberWithUnsignedLongLong:wSelf.fileHandle.offsetInFile]]; [wSelf uploadNextChunk:cursor atURL:url]; } else { NSLog(@"%@\n%@\n", routeError, networkError); [wSelf.fileHandle closeFile]; dispatch_async(dispatch_get_main_queue(), ^{ [wSelf cancelUpload:nil]; }); } }]; [_uploadTask setProgressBlock:^(int64_t bytesWritten, int64_t totalBytesWritten, int64_t totalBytesExpectedToWrite) { // NSLog(@"\n%lld\n%lld\n%lld\n", bytesWritten, totalBytesWritten, totalBytesExpectedToWrite); wSelf.progress.progress = totalBytesWritten*1.f/wSelf.fileSize.longLongValue; }]; - (void)uploadNextChunk:(DBFILESUploadSessionCursor *)cursor atURL:(NSURL *)url { NSData *data; DBUserClient *client = [DBClientsManager authorizedClient]; unsigned long long offset = _fileHandle.offsetInFile; if (_fileSize.longLongValue - _fileHandle.offsetInFile > DB_CHUNK_SIZE) { data = [_fileHandle readDataOfLength:DB_CHUNK_SIZE]; } else { data = [_fileHandle readDataOfLength:(_fileSize.longLongValue - _fileHandle.offsetInFile)]; } LibraryViewController *__weak weakSelf = self; NSString *path = [NSString stringWithFormat:@"/%@", [[url path] lastPathComponent]]; if (data.length < DB_CHUNK_SIZE) { DBFILESCommitInfo *info = [[DBFILESCommitInfo alloc] initWithPath:path]; _finishTask = [client.filesRoutes uploadSessionFinishData:cursor commit:info inputData:data]; [_finishTask setResponseBlock:^(DBFILESFileMetadata * _Nullable result, DBFILESUploadSessionFinishError * _Nullable routeError, DBRequestError * _Nullable networkError) { if (result) { NSLog(@"Finished uploading: %@", result); } else { NSLog(@"finishError last chunk: %@", routeError); NSLog(@"error: %@", networkError); } dispatch_async(dispatch_get_main_queue(), ^{ [weakSelf cancelUpload:nil]; }); }]; return; } _continueTask = [client.filesRoutes uploadSessionAppendV2Data:cursor inputData:data]; [_continueTask setResponseBlock:^(DBNilObject * _Nullable result, DBFILESUploadSessionLookupError * _Nullable routeError, DBRequestError * _Nullable networkError) { if (result) { NSLog(@"Result, uploading chunk %@", result); DBFILESUploadSessionCursor *cursor = [[DBFILESUploadSessionCursor alloc] initWithSessionId:weakSelf.sessionId offset:[NSNumber numberWithUnsignedLongLong:weakSelf.fileHandle.offsetInFile]]; [weakSelf uploadNextChunk:cursor atURL:url]; } else { NSLog(@"finishError: %@", routeError); NSLog(@"error: %@", networkError); [weakSelf.fileHandle closeFile]; weakSelf.fileHandle = nil; [weakSelf cancelUpload:nil]; } }]; [_continueTask setProgressBlock:^(int64_t bytesWritten, int64_t totalBytesWritten, int64_t totalBytesExpectedToWrite) { // NSLog(@"Continue: bytesWritten = %lld, total bytes written %lld", bytesWritten, totalBytesWritten); dispatch_async(dispatch_get_main_queue(), ^{ weakSelf.progress.progress = ((totalBytesWritten + offset)*1.f/weakSelf.fileSize.longLongValue); }); }]; }
- Greg-DB
Dropbox Staff
It sounds like you're running in to general network reliability issues. While the API endpoints can technically accept up to 150 MB per request, we generally don't reccomend doing that much, as the connections can become unreliable due to various other factors.
Using a much smaller chunk size is usually preferable, e.g., 8 MB. I recommend trying a few different sizes to see what's optimal for your scenario.- DS6Explorer | Level 4
I will try to conduct test with 8 MB chunks, but I see the issue with chunked approach is it doesn't work in background on iOS. Is there any approach to get the upload working in background?
- Greg-DB
Dropbox Staff
There's some information about background tasks here:
https://github.com/dropbox/dropbox-sdk-obj-c#note-about-background-sessions
I don't believe there's a great solution though, as that applies to individual upload requests. Wrapping the upload session calls in a background task will get you a couple of minutes, but it won't fail gracefully.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.5,941 PostsLatest Activity: 20 hours ago
If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!