We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
Engg
3 years agoExplorer | Level 3
System.Threading.Tasks.TaskCanceledException: A task was canceled.
We have 500 machine which upload file at same time. We are getting RateLimitException. We are doing a retry after waiting rateLimitException.ErrorResponse.RetryAfter seconds. to test this. I came ...
- 2 years ago
In general modifying the folder should not affect probability for exception appearing, but cannot prevent it. Some decrement may happen if different files/folders appear in different namespaces (namespaces are locked independently, so less likely compete), but better don't rely on.
Again, try to schedule the uploads in non overlap way and handle the possible exception in proper way. Example of such exception handling (again just example and far from optimal) follows:
private readonly int retryLimit = 20; ... var backoff = 0; var rand = new Random(); var commitInfo = new CommitInfo(filePath.Substring(filePath.LastIndexOf('/')), WriteMode.Overwrite.Instance, false); for (var i = 0; i < retryLimit && !string.IsNullOrEmpty(sessionId); ++i) { if (backoff != 0) { Console.WriteLine("Ограничение при завършване на файл {0}. Нов опит след {1} милисекунди...", commitInfo.Path, backoff); Thread.Sleep(backoff); } lastCursor = new UploadSessionCursor(sessionId, (ulong)fileStream.Length); memStream = new MemoryStream(buffer, 0, 0); try { await dbx.Files.UploadSessionFinishAsync(lastCursor, commitInfo, body: memStream); sessionId = string.Empty; } catch (RateLimitException rateLimit) { backoff = (int)(backoff * 1.1); backoff = Math.Max(backoff, rateLimit.RetryAfter * 1000); backoff = Math.Max(backoff, 1000); backoff += rand.Next(500); } } if (string.IsNullOrEmpty(sessionId)) { Console.WriteLine("Качването на {0} приключи \ud83d\ude09.", commitInfo.Path); } else { Console.WriteLine("Неуспешно завършване на файл {0} след {1} опита \ud83d\ude15!", commitInfo.Path, retryLimit); }The above just shows how my advices (from previous page) can be implemented. Here sessionId represents a session ready for finishing (i.e. content uploaded already).
Здравко
3 years agoLegendary | Level 20
Hm..🤔 I thought I was clear, but it seems not enough.
Engg wrote:...
List<Task> tasks = new List<Task>();foreach (var item in dbLocalFilesPath.Select((value, i) => new { i, value }))
{string dbLocalPath = item.value;
int index = item.i;
Logger.Instance.LogInfo($"dbLocalPath: {dbLocalPath}");
tasks.Add(Task.Run(async () =>...
As can be seen here you are starting multiple uploads at the same time (all files at the particular machine). Right? All right up to here. Let see what's next:
Engg wrote:...
FileMetadata fileMetadata = await files.UploadAsync(path, clientModified: clientModified, body: body);...
await client.Files.UploadSessionFinishAsync(cursor: cursor, commit: new CommitInfo(remotePath), body: memStream);
...
In all cases, for any file size, you are using uploads that directly try modify Dropbox account folder tree!!! Dropbox doesn't let this to happen! Every change in the folder tree is strictly serialized and if many waiting query appear, some of them receive rate limit error (read this limit as a limiting change rate, not exactly the upload rate). That's where your issue is coming from. Is it more clear now? What if you upload using sessions regardless file size and instead of using finish at the every file end (something that triggers the change) you use batch version of the finish (just one change - here I assume you have no more than 1000 files per machine that need upload)? 🧐 Where will limiting error happen almost sure and where wouldn't? 😉
PS: In addition you can en-schedule your machines uploads in a slightly different time - let's say 10 seconds time steps one after other.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.
If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!