HTTP API V2 download range request: strange behaviour

[Vladimir K.2]

Hi! I'll appreciate any help. Thank you!

I'm trying to download file by chunks.

URL: https://content.dropboxapi.com/2/files/download

File size: 1200 bytes

First time (after creation in dropbox) I read it with 3 GET requests, which contain corresponding header fields:

Range: bytes=0-499

Response comes with first 500 bytes of data

Range: bytes=500-999

Response comes with next 500 bytes of data

Range: bytes=1000-1199

Response comes with last 200 bytes of data

Everything is OK.

But when I try to download this file again, something goes wrong:

Range: bytes=0-499

Response comes with last 200 bytes of data
Content-range field in response header: 1000-1199/1200 

Range: bytes=500-999

Response comes with last 200 bytes of data
Content-range field in response header: 1000-1199/1200

Range: bytes=1000-1199

Response comes with last 200 bytes of data
Content-range field in response header: 1000-1199/1200

What's the problem? Can I do something with this?

P.S. Actually file size is around 1.2 MB, and chunk size is 512 kB.

[Greg-DB]

I can't seem to reproduce this behavior. Can you share the raw requests and responses so we can look into it? Just be sure to redact the access token. Thanks in advance! 

[Vladimir K.2]

Here are they (for smaller file):

GET https://content.dropboxapi.com/2/files/download?arg=%7B%22path%22:%20%22/test2.jpg%22%7D
{
 Authorization = "Bearer TOKEN";
 Range = "bytes=0-524287";
}

206
}
 "Accept-Ranges" = bytes;
 Connection = "keep-alive";
 "Content-Disposition" = "attachment; filename=unspecified";
 "Content-Length" = 524288;
 "Content-Range" = "bytes 0-524287/833408";
 "Content-Security-Policy" = sandbox;
 "Content-Type" = "application/octet-stream";
 Date = "Wed, 13 Apr 2016 20:38:59 GMT";
 "Dropbox-API-Result" = "{\"name\": \"test2.jpg\", \"path_lower\": \"/test2.jpg\", \"path_display\": \"/test2.jpg\", \"id\": \"id:hF-8wvxIrdAAAAAAAAAAVA\", \"client_modified\": \"2016-04-13T13:05:13Z\", \"server_modified\": \"2016-04-13T14:21:42Z\", \"rev\": \"10b645fbff27\", \"size\": 833408, \"media_info\": {\".tag\": \"metadata\", \"metadata\": {\".tag\": \"photo\", \"dimensions\": {\"height\": 1000, \"width\": 1000}}}}";
 Etag = "W/\"10b645fbff27\";
 "Original-Content-Length" = 524288;
 Server = nginx;
 Vary = "Dropbox-API-Arg, Authorization";
 "X-Content-Security-Policy" = sandbox;
 "X-Content-Type-Options" = nosniff;
 "X-Dropbox-Request-Id" = 64ce59cda4729419d32a1cca8148e020;
 "X-Robots-Tag" = "noindex, nofollow, noimageindex";
 "X-WebKit-CSP" = sandbox;
}

GET https://content.dropboxapi.com/2/files/download?arg=%7B%22path%22:%20%22/test2.jpg%22%7D
{
 Authorization = "Bearer TOKEN";
 Range = "bytes=524288-1048575";
}

206
{
 "Accept-Ranges" = bytes;
 Connection = "keep-alive";
 "Content-Disposition" = "attachment; filename=unspecified";
 "Content-Length" = 309120;
 "Content-Range" = "bytes 524288-833407/833408";
 "Content-Security-Policy" = sandbox;
 "Content-Type" = "application/octet-stream";
 Date = "Wed, 13 Apr 2016 20:40:32 GMT";
 "Dropbox-API-Result" = "{\"name\": \"test2.jpg\", \"path_lower\": \"/test2.jpg\", \"path_display\": \"/test2.jpg\", \"id\": \"id:hF-8wvxIrdAAAAAAAAAAVA\", \"client_modified\": \"2016-04-13T13:05:13Z\", \"server_modified\": \"2016-04-13T14:21:42Z\", \"rev\": \"10b645fbff27\", \"size\": 833408, \"media_info\": {\".tag\": \"metadata\", \"metadata\": {\".tag\": \"photo\", \"dimensions\": {\"height\": 1000, \"width\": 1000}}}}";
 Etag = "W/\"10b645fbff27\";
 "Original-Content-Length" = 309120;
 Server = nginx;
 Vary = "Dropbox-API-Arg, Authorization";
 "X-Content-Security-Policy" = sandbox;
 "X-Content-Type-Options" = nosniff;
 "X-Dropbox-Request-Id" = a45b71c7f42234a7271bed7df03b06c3;
 "X-Robots-Tag" = "noindex, nofollow, noimageindex";
 "X-WebKit-CSP" = sandbox;
}

-----------------------------------------------------------

GET https://content.dropboxapi.com/2/files/download?arg=%7B%22path%22:%20%22/test2.jpg%22%7D
{
 Authorization = "Bearer TOKEN";
 Range = "bytes=0-524287";
}

206
{
 "Accept-Ranges" = bytes;
 Connection = "keep-alive";
 "Content-Disposition" = "attachment; filename=unspecified";
 "Content-Length" = 309120;
 "Content-Range" = "bytes 524288-833407/833408";
 "Content-Security-Policy" = sandbox;
 "Content-Type" = "application/octet-stream";
 Date = "Wed, 13 Apr 2016 20:44:04 GMT";
 "Dropbox-API-Result" = "{\"name\": \"test2.jpg\", \"path_lower\": \"/test2.jpg\", \"path_display\": \"/test2.jpg\", \"id\": \"id:hF-8wvxIrdAAAAAAAAAAVA\", \"client_modified\": \"2016-04-13T13:05:13Z\", \"server_modified\": \"2016-04-13T14:21:42Z\", \"rev\": \"10b645fbff27\", \"size\": 833408, \"media_info\": {\".tag\": \"metadata\", \"metadata\": {\".tag\": \"photo\", \"dimensions\": {\"height\": 1000, \"width\": 1000}}}}";
 Etag = "W/\"10b645fbff27\";
 "Original-Content-Length" = 309120;
 Server = nginx;
 Vary = "Dropbox-API-Arg, Authorization";
 "X-Content-Security-Policy" = sandbox;
 "X-Content-Type-Options" = nosniff;
 "X-Dropbox-Request-Id" = 5538c11467902b81a6e7353a1d1768ed;
 "X-Robots-Tag" = "noindex, nofollow, noimageindex";
 "X-WebKit-CSP" = sandbox;
}

GET https://content.dropboxapi.com/2/files/download?arg=%7B%22path%22:%20%22/test2.jpg%22%7D 
{
 Authorization = "Bearer TOKEN";
 Range = "bytes=309120-833407";
}

206
{
 "Accept-Ranges" = bytes;
 Connection = "keep-alive";
 "Content-Disposition" = "attachment; filename=unspecified";
 "Content-Length" = 309120;
 "Content-Range" = "bytes 524288-833407/833408";
 "Content-Security-Policy" = sandbox;
 "Content-Type" = "application/octet-stream";
 Date = "Wed, 13 Apr 2016 20:46:00 GMT";
 "Dropbox-API-Result" = "{\"name\": \"test2.jpg\", \"path_lower\": \"/test2.jpg\", \"path_display\": \"/test2.jpg\", \"id\": \"id:hF-8wvxIrdAAAAAAAAAAVA\", \"client_modified\": \"2016-04-13T13:05:13Z\", \"server_modified\": \"2016-04-13T14:21:42Z\", \"rev\": \"10b645fbff27\", \"size\": 833408, \"media_info\": {\".tag\": \"metadata\", \"metadata\": {\".tag\": \"photo\", \"dimensions\": {\"height\": 1000, \"width\": 1000}}}}";
 Etag = "W/\"10b645fbff27\";
 "Original-Content-Length" = 309120;
 Server = nginx;
 Vary = "Dropbox-API-Arg, Authorization";
 "X-Content-Security-Policy" = sandbox;
 "X-Content-Type-Options" = nosniff;
 "X-Dropbox-Request-Id" = 4fc663ac6e844df8e9874e252f002373;
 "X-Robots-Tag" = "noindex, nofollow, noimageindex";
 "X-WebKit-CSP" = sandbox;
}

[Greg-DB]

Thanks! It looks like these aren't the actual raw requests though. How are you printing out the request headers you're sending? Can you print out the raw requests, and double check that you're sending the exact "Range" header that you're intending to? 

[Vladimir K.2]

I don't know how to get raw requests, because they are generated by the following obj-c code.

+ (CloudHTTPResponse*) sendDownloadRequestWithToken:(NSString*)token path:(NSString*)path offset:(UInt64)offset andLength:(UInt64)length {
    NSString *url = @"https://content.dropboxapi.com/2/files/download";
    NSString *json = [NSString stringWithFormat:@"{\"path\": \"%@\"}", path];
    NSURL *urlWithArgs = [NSURL URLWithString:
                          [NSString stringWithFormat:@"%@?arg=%@", url, [json stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]]];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:urlWithArgs];
    [request setHTTPMethod:@"GET"];
    [request setValue:[NSString stringWithFormat:@"Bearer %@", token] forHTTPHeaderField:@"Authorization"];
    [request setValue:[NSString stringWithFormat:@"bytes=%llu-%llu", offset, offset+length-1] forHTTPHeaderField:@"Range"];
    return [self sendRequestSynchronously:request];
}

+ (CloudHTTPResponse*) sendRequestSynchronously:(NSURLRequest*)request {
    NSLog(@"%@", [request URL]);
    NSLog(@"%@", [request HTTPMethod]);
    NSLog(@"%@", [request allHTTPHeaderFields]);
    dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);
    NSURLSession *session = [NSURLSession sharedSession];
    __block CloudHTTPResponse *httpResponse = nil;
    NSURLSessionDataTask *task = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        httpResponse = [[CloudHTTPResponse alloc] initWithHeaderFields:[(NSHTTPURLResponse *)response allHeaderFields]
statusCode:(int)[(NSHTTPURLResponse *)response statusCode]
data:data
andError:error];
        dispatch_semaphore_signal(semaphore);
    }];
    [task resume];
    dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
    NSLog(@"%i", httpResponse.statusCode);
    NSLog(@"%@", httpResponse.headerFields);
    return httpResponse;
}

As you can see, headers are printed right before sending the request and after receiving the response.

[Vladimir K.2]

Hmmm... Now it seems to me, that I have a problem with the shared NSURLSession and caching. I'll check it and report here soon.

[Vladimir K.2]

Hooray! Local caching was the reason. All I needed to do is to create custom NSURLSession with the default configuration and change its caching policy to NSURLRequestReloadIgnoringLocalCacheData. Now everything works fine. Thanks for trying to help me 🙂

NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
configuration.requestCachePolicy = NSURLRequestReloadIgnoringLocalCacheData;
NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration];