cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Want to learn some quick and useful tips to make your day easier? Check out how Calvin uses Replay to get feedback from other teams at Dropbox here.

Delete, edit, and organize

Solve issues with deleting, editing, and organizing files and folders in your Dropbox account with support from the Dropbox Community.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Local folder structure and online structure is out of sync

Local folder structure and online structure is out of sync

jordan T.9
Helpful | Level 5

tl;dr - I moved a bunch of stuff and started a large upload yesterday. My local DB stopped syncing with the cloud (and seems to be in a connecting/failed sync loop).  If I reinstall dropbox, will it delete my local files which haven't been uploaded yet?

---

Dropbox has been pretty bulletproof for me for several years, but I'm having an odd issue now that I'm moving more data to dropbox. 

Brief timeline:

2 weeks ago I moved ~500GB of files (guessing 300,000 files in 1000 directories) into a top level driectory on my dropbox, called "/Transfer".  I wanted to have them upload while I was away on vacation (idle internet connection).  Worked like a charm.

16 hours ago I moved those files/directories into the places they should be in my folder tree (i.e. out of /Transfer and into /Files/Images, /Files/Music, /Files/Documents, etc.), which already existed but had only 50,000 files or so) and I also re-organized the existing folders a bit. 

There was one folder - a shared folder, call it Games - which couldn't be moved.  Windows (10, pro, 1809) claimed it was in use/open and would not move it. It was the only(?) shared folder I attempted to move, so I just let it be.

I then added another 300GB to upload into a new folder. The sync bubble on the Dropbox tray icon was illuminated, so I let it do it's thing overnight. 

---

This morning I decided to check on the folders and try to move that stubborn Games (shared) folder it again. Same issue (folder is open, cannot move). So I decided to attempt the move on Dropbox.com, thinking it wouldn't be limited by a random file lock on my PC. When I opened my diretory tree on Dropbox.com, all of the folders I moved yesterday on my local machine were in their original location (still in /Transfer). [Insert a moment of panic]

In checking the Dropbox tray icon, yesterday/this morning it showed the blue syncing icon.  If I hover over the tray icon it shows [Dropbox 77.4.131 / Checking for changes...]. If I left click on the icon, there is no activity, and when I click away the Dropbox icon turns gray (inactive) and hovering brings up [Dropbox 77.4.131 / Connecting... /Checking for changes...].

Something is broken, but the Dropbox UI is pretty opaque.

 

1) Is it safe to re-install Dropbox and should I expect dropbox to push the coud configuration back to me, or will Dropbox attempt to take my PCs local versions and reconfigure the cloud?

 

2) is there any chance that the local files I have put into my local Dropbox folder will be deleted?

 

 

11 Replies 11

Daphne
Dropbox Staff
Hey there @jordan T.9,
 
You should be safe to go ahead with reinstalling the desktop app. When you start up the desktop app again and the app “checks for changes” it will detect where the most recent change was made (desktop folder or website) and sync those changes accordingly.
 
Since you made the move of the folders from the desktop app, and I'm assuming no subsequent changes to the folders via the website, the move made on the desktop Dropbox folder will sync to your account online.
 
In terms of troubleshooting, if you haven’t already I might even suggest a simple reboot of your computer to refresh the desktop app and see if it will be able to progress past the “checking for changes”. 
 
Keep me posted on your progress - Thanks!

Daphne
Community Moderator @ Dropbox
dropbox.com/support


Heart Did this post help you? If so, please give it a Like below.
:arrows_counterclockwise: Still stuck? Ask me a question!
:pushpin: Tips & Tricks Find new ways to stay in flow or share your tips on how you work smarter with Dropbox.

jordan T.9
Helpful | Level 5

Thanks for the reply. Seeing the DB app drop out and restart multiple times I went ahead and rebooted the machine (just for completeness: Dell Precision 2x8 Core Xeon / 48GB / 2TB SSD / local folders located at c:/dropbox on primary NTFS drive). 

Right clicking on the Dropbox tray icon now indicates recent activity. There is little/no write activity is noted in the last 24 hours, so I'm going to assume it's scanning. I've got ~960k files (no symlinks, no active database files like .pst/.ost), so it will take some time.  Syncback takes a good 40-60 minutes to scan prior to backup, so I'll just let it run for a while and check back later today. 

If it chokes or hangs I'll try a re-install of DB; hopefully it was just a hiccup. 

Daphne
Dropbox Staff

Hey @jordan T.9,

It's good to hear that the app seems to be updating now with recent activity, hopefully you'll see it progressing soon!

I'll be here ready and waiting should you run into any issues - Cheers!


Daphne
Community Moderator @ Dropbox
dropbox.com/support


Heart Did this post help you? If so, please give it a Like below.
:arrows_counterclockwise: Still stuck? Ask me a question!
:pushpin: Tips & Tricks Find new ways to stay in flow or share your tips on how you work smarter with Dropbox.

jordan T.9
Helpful | Level 5

For those playing along at home (or finding this in the future); Dropbox appears to be exclusively single threaded for Indexing and has, in 10 hours, indexed roughly 175,000 of the 500,000 files I uploaded. During this time, network activity (even with the 300GB pending upload) has been trivial (in the 5kB/s range. most of which is receive rather than send) so it seems that the processes are serialized.  At this rate, 18 more hours before I can upload anything or see any of the the file structure mirrored throughout my devices. 

Also, files which have been uploaded from other clients are not yet downloading on my machine, so it's convenient that I took the week off and don't need to collaborate with anyone over DropBox!

Daphne
Dropbox Staff

Thanks for updating us here @jordan T.9!

I'm glad to hear that the indexing is currently progressing and you'll see the upload start after this process has finished.

If you're seeing a slight degradation in the performance of the desktop app, it's most likely just due to the number of files you have syncing in your Dropbox folder. While the soft limit is around 300,000 files, this highly depends on your OS specifications too so your device might be able to handle a larger number before you see this.

However, if you'd like to try and speed up the uploading process when that gets started, I can suggest checking that your bandwidth settings for the desktop app aren't limited.

I hope this helps - Cheers!


Daphne
Community Moderator @ Dropbox
dropbox.com/support


Heart Did this post help you? If so, please give it a Like below.
:arrows_counterclockwise: Still stuck? Ask me a question!
:pushpin: Tips & Tricks Find new ways to stay in flow or share your tips on how you work smarter with Dropbox.

jordan T.9
Helpful | Level 5

Even with the PC running 24/7, the online directory structure still has not synchronized 90 hours later.  The network connections are not limited (not that it matters, DB is using a trivial amount of bandwidth). It continues to index and fail (tray icon turns gray, notifications area becomes empty), then restart. 

I'm in the process of re-installing dropbox, but am not hopeful. 

Edit: On re-install I got a message (email) that I had just deleted 215,000 files. This is not promising. 

--

I've noticed that there is no tecnihcal information on the Dropbox site - or in any help files - about the 300k file limit. This is fairly critical information, especially given how easy it is to exceed this limit in a 2TB box - I have 950k files+folders in just 1.4TB of combined personal and consulting files. It must make Dropbox for Business awful to manage. I'm going to attempt some pruning in hopes of getting this process moving.  and hope that I don't end up with data loss. 

Daphne
Dropbox Staff

Hey @jordan T.9,

Thanks for getting back to me here with your current process.

If you were to click the Dropbox icon, are you seeing any error messages within the sync status at all?

With issues in the desktop app performance where there are a large number of files syncing, we generally suggest using selective sync to alleviate some of the load on the application. As I mentioned though, the soft limit does depend highly on your current OS and specifications as some computer systems can handle more than others.

Finally, about the email that you received, if you just want to check on that event, I can suggest checking your Events page to see this event. However, as Dropbox sees moves as an addition to the new location and deletion from the original location, I believe that the email may have stemmed from syncing the moves you intially made.

Keep me posted on how it goes - Thanks!


Daphne
Community Moderator @ Dropbox
dropbox.com/support


Heart Did this post help you? If so, please give it a Like below.
:arrows_counterclockwise: Still stuck? Ask me a question!
:pushpin: Tips & Tricks Find new ways to stay in flow or share your tips on how you work smarter with Dropbox.

jordan T.9
Helpful | Level 5

Okay, the syncing kerfuffle is nearly at an end. After endless failures and restarts, the system is somewhat back in sync. Again, for those finding this in the future:

1) Do NOT unlink your account and re-link it if you have files folders out of sync. The re-link process will treat any two folders which were out of sync as unique data sets. The result is that you will have files in both places (the old one, referenced in the Cloud version and the new one, referenced on your local drive).  The conflict will result in a s**t-ton of file-duplicates whic you will have to manually de-duplicate (likely my next two days of work). It will also increase your DB volume - both total bytes and total files. 

2) If you should be foolish enough to put over 330,000 files on dropbox, never let a Windows PC see the entire database. If you have to move large qualities of files around, do it on the web interface. As much as it sucks, the Dropbox app is unable to efficiently process massive moves and will crash. Repeatedly. 

AFAICT - and you can correct me if I'm wrong, Daphne - the Windows app is 32 bit and cannot access datasets in excess of 4GB in size. Dropbox reserves somewhere on the order of 10-12kB of space for file information and tracking. Once you hit 330k files, you've hit the 4GB limit and the app fails.  The reason Dropbox is cagey about this is that there the OSX app is 64 bit and Linux has both 32 and 64 bit options, meaning that Windows is the only OS, despite the large numbers of users, which only has a 32 bit app.  As a result, even a professional workstation running windows (I'm running a 16 core Xeon with 48GB RAM) cannot process any more files than a single core celeron with 4GB of RAM. And, since the sync process appears to be single-threaded under windows, it can't process appreciably faster either. 

 

I'll end this just by saying that Dropbox is good at warning/preventing you from exceeding your disk quota (2GB/1TB/2TB), but has no ability in the UI to prevent high file-count choking. I would request that Dropbox consider re-working the ability to address large filesets but, in the meantime, add a warning about moving or adding total file counts over 330k. At 2TB space, that's 6MB per file, and it's easy to have a large number of files smaller than that in a typical user account. 

Daphne
Dropbox Staff

Hey @jordan T.9,

Thanks for updating me here and giving such a great amount of detail into your findings! I'm sure that this will be helpful to others who might have a similar issue you experienced.

What you mentioned about syncing a directory larger than 300,000 files is indeed helpful too as long as we keep in mind that this is a soft limit. Also, as you know, the post you created seperately for requesting a way for the desktop app to notify you when you reach this many files has been passed along to our dev team too.

Again, thanks for taking the time to provide all this info and I hope you have a great start to the week - Thanks!


Daphne
Community Moderator @ Dropbox
dropbox.com/support


Heart Did this post help you? If so, please give it a Like below.
:arrows_counterclockwise: Still stuck? Ask me a question!
:pushpin: Tips & Tricks Find new ways to stay in flow or share your tips on how you work smarter with Dropbox.

Need more support?
Who's talking

Top contributors to this post

  • User avatar
    Daphne Dropbox Staff
  • User avatar
    Здравко Legendary | Level 20
  • User avatar
    jordan T.9 Helpful | Level 5
What do Dropbox user levels mean?