We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
Http 504 Gateway Timeout when trying to get upto 1000 or more files in c#
I am trying to fetch a folder using C# dropbox api and I am paginating results as well but still I get timeout on it as it contains more than 1000 files.
Here is my pagination code:
var children = await GetDropboxClient().Files.ListFolderAsync(dbxItem.PathLower == DROPBOX_ROOT_PATH ? string.Empty : dbxItem.PathLower);
while (children.HasMore)
{
children = await GetDropboxClient().Files.ListFolderContinueAsync(children.Cursor);
}
return children.Entries;
9 Replies
- Mark
Super User II
*moves to API area* - Greg-DB
Dropbox Community Moderator
Thanks for the report. To help us look into this, please share:
- the full error output
- the relevant cursor(s)
- the affected user IDs
Thanks in advance! I want to attach a fiddler trace but I don't see an option to upload other than image files. How can I do that?
But here the call:
POST https://connectoreastsftest.sharefileconnect.com/dropbox/v3/Items(e797f2aKVXZaWW5kkUdydma2IxQEFpXbp-UXmRunpCbrWhibpOiaGBoa6iYamKTpphgbW6YmW1iYGiWaKmSUlBQUW-nrJ-fn5aUml-QXFRvrFafpJhZklqQWl-gl5-fqAwA_)/Children?%24select=Owner%2FId%2CCreator%2FId%2CId%2CCreationDate%2CCreatorNameShort%2CDescription%2CFileName%2CName%2CFileCount%2CFileSizeBytes%2CHasMultipleVersions%2CPreviewStatus%2CState%2CVirusStatus%2CUri%2Curl%2CESignatureDocument%2CExpirationDate%2CMaxPreviewSize%2CStreamID%2CFileLockInfo%2CInfo%2CInfo%2FCanDeleteCurrentItem%2CProperties%2CProperties%2F%2A%2CInfo%2FCanDownload%2CInfo%2FCanSend%2CInfo%2FCanManagePermissions%2CInfo%2FCanAddNode%2CDlpInfo%2CDlpInfo%2FStatus%2CHasPendingAsyncOp%2CFavorite%2CFavorite%2F%2A%2CBundle%2CBundle%2F%2A&%24expand=ESignature%2CFileLockInfo%2CDlpInfo%2CInfo%2CFavorite%2CFavorite%2FItem%2CBundle%2COwner%2CCreator&includeDeleted=false&orderingMode=UseFolderOptions HTTP/1.1 Host: connectoreastsftest.sharefileconnect.com Connection: keep-alive Content-Length: 2 Accept: application/json, text/plain, */* Origin: https://connectors3.sharefiletest.com X-SF-App: ShareFileWeb User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36 X-HTTP-Method-Override: GET Content-Type: application/json Referer: https://connectors3.sharefiletest.com/app/ Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.8,hi;q=0.6 Cookie: DbxConnector_AuthId=477b3691becf491d9139fdeae81f4104 {}And response:
HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive
But here is the request that fails:
POST https://connectoreastsftest.sharefileconnect.com/dropbox/v3/Items(e797f2aKVXZaWW5kkUdydma2IxQEFpXbp-UXmRunpCbrWhibpOiaGBoa6iYamKTpphgbW6YmW1iYGiWaKmSUlBQUW-nrJ-fn5aUml-QXFRvrFafpJhZklqQWl-gl5-fqAwA_)/Children?%24select=Owner%2FId%2CCreator%2FId%2CId%2CCreationDate%2CCreatorNameShort%2CDescription%2CFileName%2CName%2CFileCount%2CFileSizeBytes%2CHasMultipleVersions%2CPreviewStatus%2CState%2CVirusStatus%2CUri%2Curl%2CESignatureDocument%2CExpirationDate%2CMaxPreviewSize%2CStreamID%2CFileLockInfo%2CInfo%2CInfo%2FCanDeleteCurrentItem%2CProperties%2CProperties%2F%2A%2CInfo%2FCanDownload%2CInfo%2FCanSend%2CInfo%2FCanManagePermissions%2CInfo%2FCanAddNode%2CDlpInfo%2CDlpInfo%2FStatus%2CHasPendingAsyncOp%2CFavorite%2CFavorite%2F%2A%2CBundle%2CBundle%2F%2A&%24expand=ESignature%2CFileLockInfo%2CDlpInfo%2CInfo%2CFavorite%2CFavorite%2FItem%2CBundle%2COwner%2CCreator&includeDeleted=false&orderingMode=UseFolderOptions HTTP/1.1 Host: connectoreastsftest.sharefileconnect.com Connection: keep-alive Content-Length: 2 Accept: application/json, text/plain, */* Origin: https://connectors3.sharefiletest.com X-SF-App: ShareFileWeb User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36 X-HTTP-Method-Override: GET Content-Type: application/json Referer: https://connectors3.sharefiletest.com/app/ Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.8,hi;q=0.6 Cookie: DbxConnector_AuthId=477b3691becf491d9139fdeae81f4104 {}- Greg-DBThat seems to be the POST to your own site. Can you share the information I requested for the Dropbox API call itself? Thanks in advance!
Greg :I think I found the issue. Our servers hosted on AWS EC2 were timing out as the default idle time was 300s. What should be the idle connection time set to avoid timing out for files upto 1000?
Greg-DB wrote:
That seems to be the POST to your own site. Can you share the information I requested for the Dropbox API call itself? Thanks in advance!Also, if I had paginated results using the code mentioned initially in my post, then shouldn't it get the results in batches instead of getting all at once? I think that's what is happening and its timing out eventually.
- Greg-DBHow long each call takes, as well as how long it takes to list everything, will vary from account to account, e.g., based on the number of files/activity, so I can't offer a guarantee on how much time it would take.
Each call to a ListFolderAsync or ListFolderContinueAsync method makes one call to the Dropbox API, with each call returning one page of results. If you have that entire code snippet, including the while loop, blocking the HTTP response for your own page, that can cause your own HTTP call to time out, since the number of Dropbox API calls/pages is unbounded. (I.e., per the above, if the user has many files, there will be many calls to make). I don't have full context on your app of course, but it sounds like perhaps you want to only return one page at a time per HTTP call to your app itself, or perhaps move this into an asynchronous code path, retrieved via AJAX call or some other mechanism.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
