cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Are you interested in hearing how one of our Community members uses Dropbox for sailing trips? Read all about it here.

Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Http 504 Gateway Timeout when trying to get upto 1000 or more files in c#

Http 504 Gateway Timeout when trying to get upto 1000 or more files in c#

aksg
New member | Level 2

I am trying to fetch a folder using C# dropbox api and I am paginating results as well but still I get timeout on it as it contains more than 1000 files.

 

Here is my pagination code:

 

var children = await GetDropboxClient().Files.ListFolderAsync(dbxItem.PathLower == DROPBOX_ROOT_PATH ? string.Empty : dbxItem.PathLower);
						
while (children.HasMore)
{
      children = await GetDropboxClient().Files.ListFolderContinueAsync(children.Cursor);				
}
return children.Entries;

 

9 Replies 9

Mark
Super User II
*moves to API area*

 


:penguin::penguin: - :penguin: - :penguin: - :penguin:


Heart Did this post help you? If so please mark it for some Kudos below. 


:white_check_mark: Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.


:arrows_counterclockwise: Did this post not resolve your issue? If so please give us some more information so we can try and help - please remember we cannot see over your shoulder so be as descriptive as possible! 


 

Greg-DB
Dropbox Staff
Thanks for the report. To help us look into this, please share:

- the full error output
- the relevant cursor(s)
- the affected user IDs

Thanks in advance!

aksg
New member | Level 2

I want to attach a fiddler trace but I don't see an option to upload other than image files. How can I do that?

aksg
New member | Level 2

But here the call: 

 

POST https://connectoreastsftest.sharefileconnect.com/dropbox/v3/Items(e797f2aKVXZaWW5kkUdydma2IxQEFpXbp-UXmRunpCbrWhibpOiaGBoa6iYamKTpphgbW6YmW1iYGiWaKmSUlBQUW-nrJ-fn5aUml-QXFRvrFafpJhZklqQWl-gl5-fqAwA_)/Children?%24select=Owner%2FId%2CCreator%2FId%2CId%2CCreationDate%2CCreatorNameShort%2CDescription%2CFileName%2CName%2CFileCount%2CFileSizeBytes%2CHasMultipleVersions%2CPreviewStatus%2CState%2CVirusStatus%2CUri%2Curl%2CESignatureDocument%2CExpirationDate%2CMaxPreviewSize%2CStreamID%2CFileLockInfo%2CInfo%2CInfo%2FCanDeleteCurrentItem%2CProperties%2CProperties%2F%2A%2CInfo%2FCanDownload%2CInfo%2FCanSend%2CInfo%2FCanManagePermissions%2CInfo%2FCanAddNode%2CDlpInfo%2CDlpInfo%2FStatus%2CHasPendingAsyncOp%2CFavorite%2CFavorite%2F%2A%2CBundle%2CBundle%2F%2A&%24expand=ESignature%2CFileLockInfo%2CDlpInfo%2CInfo%2CFavorite%2CFavorite%2FItem%2CBundle%2COwner%2CCreator&includeDeleted=false&orderingMode=UseFolderOptions HTTP/1.1
Host: connectoreastsftest.sharefileconnect.com
Connection: keep-alive
Content-Length: 2
Accept: application/json, text/plain, */*
Origin: https://connectors3.sharefiletest.com
X-SF-App: ShareFileWeb
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
X-HTTP-Method-Override: GET
Content-Type: application/json
Referer: https://connectors3.sharefiletest.com/app/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8,hi;q=0.6
Cookie: DbxConnector_AuthId=477b3691becf491d9139fdeae81f4104
{}

And response: 

HTTP/1.1 504 GATEWAY_TIMEOUT
Content-Length: 0
Connection: keep-alive
 

 

aksg
New member | Level 2

But here is the request that fails: 

 

POST https://connectoreastsftest.sharefileconnect.com/dropbox/v3/Items(e797f2aKVXZaWW5kkUdydma2IxQEFpXbp-UXmRunpCbrWhibpOiaGBoa6iYamKTpphgbW6YmW1iYGiWaKmSUlBQUW-nrJ-fn5aUml-QXFRvrFafpJhZklqQWl-gl5-fqAwA_)/Children?%24select=Owner%2FId%2CCreator%2FId%2CId%2CCreationDate%2CCreatorNameShort%2CDescription%2CFileName%2CName%2CFileCount%2CFileSizeBytes%2CHasMultipleVersions%2CPreviewStatus%2CState%2CVirusStatus%2CUri%2Curl%2CESignatureDocument%2CExpirationDate%2CMaxPreviewSize%2CStreamID%2CFileLockInfo%2CInfo%2CInfo%2FCanDeleteCurrentItem%2CProperties%2CProperties%2F%2A%2CInfo%2FCanDownload%2CInfo%2FCanSend%2CInfo%2FCanManagePermissions%2CInfo%2FCanAddNode%2CDlpInfo%2CDlpInfo%2FStatus%2CHasPendingAsyncOp%2CFavorite%2CFavorite%2F%2A%2CBundle%2CBundle%2F%2A&%24expand=ESignature%2CFileLockInfo%2CDlpInfo%2CInfo%2CFavorite%2CFavorite%2FItem%2CBundle%2COwner%2CCreator&includeDeleted=false&orderingMode=UseFolderOptions HTTP/1.1
Host: connectoreastsftest.sharefileconnect.com
Connection: keep-alive
Content-Length: 2
Accept: application/json, text/plain, */*
Origin: https://connectors3.sharefiletest.com
X-SF-App: ShareFileWeb
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
X-HTTP-Method-Override: GET
Content-Type: application/json
Referer: https://connectors3.sharefiletest.com/app/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8,hi;q=0.6
Cookie: DbxConnector_AuthId=477b3691becf491d9139fdeae81f4104
{}

Greg-DB
Dropbox Staff
That seems to be the POST to your own site. Can you share the information I requested for the Dropbox API call itself? Thanks in advance!

aksg
New member | Level 2

@Greg :I think I found the issue. Our servers hosted on AWS EC2 were timing out as the default idle time was 300s. What should be the idle connection time set to avoid timing out for files upto 1000?


@Greg-DB wrote:
That seems to be the POST to your own site. Can you share the information I requested for the Dropbox API call itself? Thanks in advance!

 

aksg
New member | Level 2

Also, if I had paginated results using the code mentioned initially in my post, then shouldn't it get the results in batches instead of getting all at once?  I think that's what is happening and its timing out eventually.

Greg-DB
Dropbox Staff
How long each call takes, as well as how long it takes to list everything, will vary from account to account, e.g., based on the number of files/activity, so I can't offer a guarantee on how much time it would take.

Each call to a ListFolderAsync or ListFolderContinueAsync method makes one call to the Dropbox API, with each call returning one page of results. If you have that entire code snippet, including the while loop, blocking the HTTP response for your own page, that can cause your own HTTP call to time out, since the number of Dropbox API calls/pages is unbounded. (I.e., per the above, if the user has many files, there will be many calls to make). I don't have full context on your app of course, but it sounds like perhaps you want to only return one page at a time per HTTP call to your app itself, or perhaps move this into an asynchronous code path, retrieved via AJAX call or some other mechanism.
Need more support?
Who's talking

Top contributors to this post

  • User avatar
    Greg-DB Dropbox Staff
  • User avatar
    aksg New member | Level 2
What do Dropbox user levels mean?