We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
Local folder structure and online structure is out of sync
tl;dr - I moved a bunch of stuff and started a large upload yesterday. My local DB stopped syncing with the cloud (and seems to be in a connecting/failed sync loop). If I reinstall dropbox, will it ...
Daphne
Dropbox Community Moderator
Dropbox Community ModeratorThanks for updating us here jordan T.9!
I'm glad to hear that the indexing is currently progressing and you'll see the upload start after this process has finished.
If you're seeing a slight degradation in the performance of the desktop app, it's most likely just due to the number of files you have syncing in your Dropbox folder. While the soft limit is around 300,000 files, this highly depends on your OS specifications too so your device might be able to handle a larger number before you see this.
However, if you'd like to try and speed up the uploading process when that gets started, I can suggest checking that your bandwidth settings for the desktop app aren't limited.
I hope this helps - Cheers!
Even with the PC running 24/7, the online directory structure still has not synchronized 90 hours later. The network connections are not limited (not that it matters, DB is using a trivial amount of bandwidth). It continues to index and fail (tray icon turns gray, notifications area becomes empty), then restart.
I'm in the process of re-installing dropbox, but am not hopeful.
Edit: On re-install I got a message (email) that I had just deleted 215,000 files. This is not promising.
--
I've noticed that there is no tecnihcal information on the Dropbox site - or in any help files - about the 300k file limit. This is fairly critical information, especially given how easy it is to exceed this limit in a 2TB box - I have 950k files+folders in just 1.4TB of combined personal and consulting files. It must make Dropbox for Business awful to manage. I'm going to attempt some pruning in hopes of getting this process moving. and hope that I don't end up with data loss.
- Daphne
Hey jordan T.9,
Thanks for getting back to me here with your current process.
If you were to click the Dropbox icon, are you seeing any error messages within the sync status at all?
With issues in the desktop app performance where there are a large number of files syncing, we generally suggest using selective sync to alleviate some of the load on the application. As I mentioned though, the soft limit does depend highly on your current OS and specifications as some computer systems can handle more than others.
Finally, about the email that you received, if you just want to check on that event, I can suggest checking your Events page to see this event. However, as Dropbox sees moves as an addition to the new location and deletion from the original location, I believe that the email may have stemmed from syncing the moves you intially made.
Keep me posted on how it goes - Thanks!
Okay, the syncing kerfuffle is nearly at an end. After endless failures and restarts, the system is somewhat back in sync. Again, for those finding this in the future:
1) Do NOT unlink your account and re-link it if you have files folders out of sync. The re-link process will treat any two folders which were out of sync as unique data sets. The result is that you will have files in both places (the old one, referenced in the Cloud version and the new one, referenced on your local drive). The conflict will result in a s**t-ton of file-duplicates whic you will have to manually de-duplicate (likely my next two days of work). It will also increase your DB volume - both total bytes and total files.
2) If you should be foolish enough to put over 330,000 files on dropbox, never let a Windows PC see the entire database. If you have to move large qualities of files around, do it on the web interface. As much as it sucks, the Dropbox app is unable to efficiently process massive moves and will crash. Repeatedly.
AFAICT - and you can correct me if I'm wrong, Daphne - the Windows app is 32 bit and cannot access datasets in excess of 4GB in size. Dropbox reserves somewhere on the order of 10-12kB of space for file information and tracking. Once you hit 330k files, you've hit the 4GB limit and the app fails. The reason Dropbox is cagey about this is that there the OSX app is 64 bit and Linux has both 32 and 64 bit options, meaning that Windows is the only OS, despite the large numbers of users, which only has a 32 bit app. As a result, even a professional workstation running windows (I'm running a 16 core Xeon with 48GB RAM) cannot process any more files than a single core celeron with 4GB of RAM. And, since the sync process appears to be single-threaded under windows, it can't process appreciably faster either.
I'll end this just by saying that Dropbox is good at warning/preventing you from exceeding your disk quota (2GB/1TB/2TB), but has no ability in the UI to prevent high file-count choking. I would request that Dropbox consider re-working the ability to address large filesets but, in the meantime, add a warning about moving or adding total file counts over 330k. At 2TB space, that's 6MB per file, and it's easy to have a large number of files smaller than that in a typical user account.
- Daphne
Hey jordan T.9,
Thanks for updating me here and giving such a great amount of detail into your findings! I'm sure that this will be helpful to others who might have a similar issue you experienced.
What you mentioned about syncing a directory larger than 300,000 files is indeed helpful too as long as we keep in mind that this is a soft limit. Also, as you know, the post you created seperately for requesting a way for the desktop app to notify you when you reach this many files has been passed along to our dev team too.
Again, thanks for taking the time to provide all this info and I hope you have a great start to the week - Thanks!
