We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
Local folder structure and online structure is out of sync
tl;dr - I moved a bunch of stuff and started a large upload yesterday. My local DB stopped syncing with the cloud (and seems to be in a connecting/failed sync loop). If I reinstall dropbox, will it ...
For those playing along at home (or finding this in the future); Dropbox appears to be exclusively single threaded for Indexing and has, in 10 hours, indexed roughly 175,000 of the 500,000 files I uploaded. During this time, network activity (even with the 300GB pending upload) has been trivial (in the 5kB/s range. most of which is receive rather than send) so it seems that the processes are serialized. At this rate, 18 more hours before I can upload anything or see any of the the file structure mirrored throughout my devices.
Also, files which have been uploaded from other clients are not yet downloading on my machine, so it's convenient that I took the week off and don't need to collaborate with anyone over DropBox!
Daphne
Dropbox Community Moderator
Dropbox Community ModeratorThanks for updating us here jordan T.9!
I'm glad to hear that the indexing is currently progressing and you'll see the upload start after this process has finished.
If you're seeing a slight degradation in the performance of the desktop app, it's most likely just due to the number of files you have syncing in your Dropbox folder. While the soft limit is around 300,000 files, this highly depends on your OS specifications too so your device might be able to handle a larger number before you see this.
However, if you'd like to try and speed up the uploading process when that gets started, I can suggest checking that your bandwidth settings for the desktop app aren't limited.
I hope this helps - Cheers!
Even with the PC running 24/7, the online directory structure still has not synchronized 90 hours later. The network connections are not limited (not that it matters, DB is using a trivial amount of bandwidth). It continues to index and fail (tray icon turns gray, notifications area becomes empty), then restart.
I'm in the process of re-installing dropbox, but am not hopeful.
Edit: On re-install I got a message (email) that I had just deleted 215,000 files. This is not promising.
--
I've noticed that there is no tecnihcal information on the Dropbox site - or in any help files - about the 300k file limit. This is fairly critical information, especially given how easy it is to exceed this limit in a 2TB box - I have 950k files+folders in just 1.4TB of combined personal and consulting files. It must make Dropbox for Business awful to manage. I'm going to attempt some pruning in hopes of getting this process moving. and hope that I don't end up with data loss.
- Daphne
Hey jordan T.9,
Thanks for getting back to me here with your current process.
If you were to click the Dropbox icon, are you seeing any error messages within the sync status at all?
With issues in the desktop app performance where there are a large number of files syncing, we generally suggest using selective sync to alleviate some of the load on the application. As I mentioned though, the soft limit does depend highly on your current OS and specifications as some computer systems can handle more than others.
Finally, about the email that you received, if you just want to check on that event, I can suggest checking your Events page to see this event. However, as Dropbox sees moves as an addition to the new location and deletion from the original location, I believe that the email may have stemmed from syncing the moves you intially made.
Keep me posted on how it goes - Thanks!
Okay, the syncing kerfuffle is nearly at an end. After endless failures and restarts, the system is somewhat back in sync. Again, for those finding this in the future:
1) Do NOT unlink your account and re-link it if you have files folders out of sync. The re-link process will treat any two folders which were out of sync as unique data sets. The result is that you will have files in both places (the old one, referenced in the Cloud version and the new one, referenced on your local drive). The conflict will result in a s**t-ton of file-duplicates whic you will have to manually de-duplicate (likely my next two days of work). It will also increase your DB volume - both total bytes and total files.
2) If you should be foolish enough to put over 330,000 files on dropbox, never let a Windows PC see the entire database. If you have to move large qualities of files around, do it on the web interface. As much as it sucks, the Dropbox app is unable to efficiently process massive moves and will crash. Repeatedly.
AFAICT - and you can correct me if I'm wrong, Daphne - the Windows app is 32 bit and cannot access datasets in excess of 4GB in size. Dropbox reserves somewhere on the order of 10-12kB of space for file information and tracking. Once you hit 330k files, you've hit the 4GB limit and the app fails. The reason Dropbox is cagey about this is that there the OSX app is 64 bit and Linux has both 32 and 64 bit options, meaning that Windows is the only OS, despite the large numbers of users, which only has a 32 bit app. As a result, even a professional workstation running windows (I'm running a 16 core Xeon with 48GB RAM) cannot process any more files than a single core celeron with 4GB of RAM. And, since the sync process appears to be single-threaded under windows, it can't process appreciably faster either.
I'll end this just by saying that Dropbox is good at warning/preventing you from exceeding your disk quota (2GB/1TB/2TB), but has no ability in the UI to prevent high file-count choking. I would request that Dropbox consider re-working the ability to address large filesets but, in the meantime, add a warning about moving or adding total file counts over 330k. At 2TB space, that's 6MB per file, and it's easy to have a large number of files smaller than that in a typical user account.
