Dropbox re-syncs 10,000+ files after every reboot

[andyyy]

Dropbox re-syncs 10,000+ files after every reboot - files that I haven't touched, and it always looks to be the same directories affected, not the whole Dropbox.

This also seems to happen after I pause Dropbox for even 10 minutes, work on a handful of files (like literally 5 files), then un-pause - it needs to re-sync thousands of files.

Any suggestiongs as to how to fix/troubleshoot?

[Elixir]

Hi @andyyy, Is it possible that those files are being used/modified by another program (including antivirus programs)? If the files are not being modified, Dropbox won't sync them. Can you please check what are those files that are constantly or so often syncing? It sounds like you have put some application or system files/folder in your Dropbox folder, and whenever that application or system is running they are being modified. 

[andyyy]

Hi Elixir,

Thanks for you reply. I would have thought that if the file had changed, it would say 'Uploading' in the Dropbox status bar - but it just says 'Syncing'. I also checked the Created, Modified, and Accessed attributes on a file that 'needed' re-syncing, and these had not changed after the reboot, so not sure why they would need to sync again? 

The problem folders look to be where I store my projects - code, etc. The node_modules folder for each project can contain upwards of 40,000+ files. Does Dropbox need to re-sync / re-index all these files on a reboot? Even if they haven't changed?

Is there a way to tell Dropbox to permanently ignore certain folder names? (I realise this can manually be done using Selective Sync).

[Elixir · Accepted solution]

I am not sure what types of files you have, and how they are being modified, but I have seen posts where people had problems using Github and Dropbox together. In principle, if a file is not modified, it does not require to be re-indexed/re-uploaded by Dropbox. Note that even if you are not making any changes to those files, but some other program may be changing some meta information (like time/date stamp etc.) on those files. Are there symlinks present in your Dropbox directory? FYI, Dropbox does not support symlinks. Also, note that if you have more than 300,000 files, the performance of your Desktop app may degrade. I think it is better to move your working project directory out of Dropbox, and make manual backup (by copying, also you can reduce the file number by creating single zip of the whole folder having 40k+ files) in Dropbox if that is a viable option for you. 

[andyyy]

Hi Elixir,

I don't have any symlinks, and I have around 60,000 files all up. But I think you're right about the Git thing - it was only directories with .git folders there were re-syncing. I'm running Windows Subsystem for Linux, so I'm thinking after a reboot the Linux/Git side was maybe 'touching' these files causing them to re-sync. After deleting the .git folders, the re-syncing after reboot stopped.

So, atleast I know why it's happening now... Thanks for your help.

[andyyy]

Hi there,

It looks like the propblem is still happening - I just rebooted after having not changed any files, and Dropbox is stuck 'Syncing' thousands of files. I don't have any .git folders, and only one project/folder with node_modules - this node_modules folder is the only one 'Syncing'.

If I perform a Windows search on that folder and select only files modified 'today', no results are found. Can you tell me how Dropbox decides if the file has changed or not? Because Windows is saying that it hasn't changed.

I only have 111,091 files selectively synced to my PC (there are much more in my online Dropbox). The problematic folder has 29,940 files in 4990 folders - far less than the 300,000 file limit recommendation. Could the problem be that Dropbox is struggling to traverse the node_modules folder structure?

[Jane]

Hey @andyyy, apologies for the tardy reply. Are you still facing this issue? 

[andyyy]

Hey Jane, yes, issues still apparent as described.

[Jane]

Thanks for your swift update here Andy (@andyyy) & I hope you have a wonderful October ahead! :umbrella:

As you &  Elixir have tried some troubleshooting on this thread, let me have a closer look on your setup through our Official channel. In order to investigate having all our tools available, I’ve made sure to reach out to you directly via email on my end. 

Whenever you find some time, please check your inbox for my email & I’ll make sure to promptly get back in touch with you asap. Thanks for reaching out on our Community & I’ll be awaiting your next reply!

[andyyy]

To those that stumble upon this - I didn't realise I had symlinks in my node_modules folders, and it is most likely these causing the issues - as Elixir mentioned, symlinks aren't officially supported by Dropbox, and I can't do without the symlinks, so we are not troubleshooting any further.