cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Update: Find information on Dropbox support during COVID-19 here
Close
cancel
Showing results for 
Search instead for 
Did you mean: 

Unnecessarily high cpu usage when sync a folder deselected with selected sync

dylan-chong
Helpful | Level 5

I am experiencing high CPU usage when there is a large folder (lots and lots of files) deselected by selected sync and it is changed. In particular, node_modules as a folder that triggers this problem, and is very annoying because I have many of these folders in my dropbox (all deselected by selective sync)

These are some steps on how to reproduce the problem (using the terminal).

cd ~/Dropbox
git clone https://github.com/dylan-chong/prioritask # actually or any project with a large node_modules folder
cd prioritask
npm install # and wait for finish before proceeding. Ignore any warnings, it is fine as long as it creates a massive node_modules folder

# now open up the dropbox selective sync settings and ignore ~/Dropbox/prioritask/node_modules

# Now lets wait a few minutes for the dropbox sync to finish

# now trigger the dropbox scanning:
rm -rf node_modules # only if it exists
npm install
# now dropbox will go crazy because the node_modules folder was created, even though it is not selected in selective sync! See below

Directly after doing the above instructions, open the activity monitor and look at the information for the dropbox process under 'open files and ports'. I see that, for a reasonable amount of time dropbox spends time scanning the ignored node_modules directory. As can be seen in the screenshot (which is looking very near the bottom of the open files and ports section), dropbox is accessing the node_modules subfolders, and watching over time, it accesses many of the subfolders - obviously a waste of time and cpu+battery power since dropbox already knows that it is deselected by selective sync

 Screen Shot 2018-09-05 at 7.42.36 PM.png

 

 

This problem is very important to me because when dropbox ramps up CPU, not enough CPU power is available to use Dragon dictation software on my computer - I often have to wait 5 or more seconds to see the results of dictation instead of the usual one second, which causes a massive productivity loss. And these dropbox synchronisation is last for quite a long time on a dual core CPU. The only choice is I have are unreasonable - type and aggravate my RSI, uninstall drop box, spend several thousand dollars on a more powerful computer just so dropbox can sync...

Are there any dropbox staff that can report this problem to the developers and get it fixed? 

11 Replies 11

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

Dropboxer
Dropboxer
Hi Dylan, if I understand the code correctly, you’re letting the code compile itself to generate the node_modules folder, then deselecting that folder in the Dropbox desktop application.
 
Afterwards, you delete the node_modules folder completely, and then run the npm program again.
 
It then recreates the node_modules folder, which is when Dropbox uses CPU to sync the files.
 
If so, then it’s possible that the app needs to still generate information for that folder.
 
The Dropbox desktop application can only selectively sync folders that already exist on the Dropbox site. When the node_modules folder was created, you were able to unsync it in the app.
 
However, once the folder was deleted using rm -rf, and then recreated with npm install, since the app knew it was deleted from the computer, it would still need to create the sync icons for the folder and files within, confirming that it was selectively 'unsynced'.
 
Depending on the number of files that were generated, this can take some CPU time as you discovered. 
 
Have you tried pausing syncing when using Dragon Dictation to see if this helps? If you’re using the software and saving the info in Dropbox, then naturally it won’t sync during this time.

Jay
Community Moderator @ Dropbox
https://dropbox.com/support


Heart Did this post help you? If so please give it a Like below. 
 Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.
 Still stuck? Ask me a question! (
Questions asked in the community will likely receive an answer within 4 hours!)

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

dylan-chong
Helpful | Level 5

Interesting, I have had a further play around with dropbox after your response.

I tried creating an empty node_modules folder, and letting that synchronise up dropbox. I then disabled the folder using selective sync. I then ran that npm install again, but I received the same results with dropbox chewing through my CPU - and could observe using activity monitor that the node modules folder that I disabled the selected sync for was being traversed by dropbox.

Interestingly, when I delete all the files in the node modules folder, dropbox does not go crazy.

Also, it would be rather infeasible to disable the dropbox think while I am using the dictation software, because I use the dictation software to control my computer basically 100% of the time. By the way, just so it is clear, I dictation software is not related to triggering the dropbox synchronisation, as I can trigger the dropbox synchronisation by following the instructions I posted above without using my dictation software

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

Dropboxer
Dropboxer
Apologies, I thought you only use the dictation software when editing documents, not for controlling the entire machine. At least it would have helped when doing docs to prevent slowdown when the desktop app registers the new unsynced files from node_modules.
 
Most likely though, symlinks would be the issue, since GitHub can generate them dynamically, causing this CPU usage.
 
Dropbox doesn’t support symlinks within the Dropbox folder for the above reason, as well as RAM usage, general performance decrease, and even data loss.
 
Could you follow the instructions on step 7 here to detect if there are any symlinks in your Dropbox account? The article itself might be beneficial to help you on this matter.

Jay
Community Moderator @ Dropbox
https://dropbox.com/support


Heart Did this post help you? If so please give it a Like below. 
 Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.
 Still stuck? Ask me a question! (
Questions asked in the community will likely receive an answer within 4 hours!)

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

dylan-chong
Helpful | Level 5
> find ~/Dropbox -type l -exec ls -lah {} \;
lrwxr-xr-x  1 Dylan  staff    27B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/command-line-args -> ../command-line-args/bin.js
lrwxr-xr-x  1 Dylan  staff    19B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/user-home -> ../user-home/cli.js
lrwxr-xr-x  1 Dylan  staff    19B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/atob -> ../atob/bin/atob.js
lrwxr-xr-x  1 Dylan  staff    19B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/gulp -> ../gulp/bin/gulp.js
lrwxr-xr-x  1 Dylan  staff    19B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/strip-bom -> ../strip-bom/cli.js
lrwxr-xr-x  1 Dylan  staff    20B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/semver -> ../semver/bin/semver
lrwxr-xr-x  1 Dylan  staff    23B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/color-support -> ../color-support/bin.js
lrwxr-xr-x  1 Dylan  staff    18B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/which -> ../which/bin/which
lrwxr-xr-x  1 Dylan  staff    20B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/.bin/mkdirp -> ../mkdirp/bin/cmd.js
lrwxr-xr-x  1 Dylan  staff    20B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/needle -> ../needle/bin/needle
lrwxr-xr-x  1 Dylan  staff    16B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/rimraf -> ../rimraf/bin.js
lrwxr-xr-x  1 Dylan  staff    33B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/detect-libc -> ../detect-libc/bin/detect-libc.js
lrwxr-xr-x  1 Dylan  staff    20B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/semver -> ../semver/bin/semver
lrwxr-xr-x  1 Dylan  staff    12B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/rc -> ../rc/cli.js
lrwxr-xr-x  1 Dylan  staff    20B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/mkdirp -> ../mkdirp/bin/cmd.js
lrwxr-xr-x  1 Dylan  staff    32B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/node-pre-gyp -> ../node-pre-gyp/bin/node-pre-gyp
lrwxr-xr-x  1 Dylan  staff    19B 19 Jun 12:18 /Users/Dylan/Dropbox/Programming/GitHub/itunes-applescripts-no-dev/node_modules/fsevents/node_modules/.bin/nopt -> ../nopt/bin/nopt.js
lrwxr-xr-x  1 Dylan  staff    19B 17 May  2017 /Users/Dylan/Dropbox/Programming/GitHub/aenea-setup/aenea/server/osx/aenea -> ../../client/aenea/
lrwxr-xr-x  1 Dylan  staff    19B 17 May  2017 /Users/Dylan/Dropbox/Programming/GitHub/aenea-setup/aenea/server/linux_x11/aenea -> ../../client/aenea/
lrwxr-xr-x  1 Dylan  staff    19B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/user-home -> ../user-home/cli.js
lrwxr-xr-x  1 Dylan  staff    31B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/not-in-publish -> ../in-publish/not-in-publish.js
lrwxr-xr-x  1 Dylan  staff    19B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/atob -> ../atob/bin/atob.js
lrwxr-xr-x  1 Dylan  staff    16B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/rimraf -> ../rimraf/bin.js
lrwxr-xr-x  1 Dylan  staff    19B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/gulp -> ../gulp/bin/gulp.js
lrwxr-xr-x  1 Dylan  staff    43B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/livereload -> ../gulp-server-livereload/bin/livereload.js
lrwxr-xr-x  1 Dylan  staff    23B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/sshpk-conv -> ../sshpk/bin/sshpk-conv
lrwxr-xr-x  1 Dylan  staff    27B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/sassgraph -> ../sass-graph/bin/sassgraph
lrwxr-xr-x  1 Dylan  staff    27B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/in-publish -> ../in-publish/in-publish.js
lrwxr-xr-x  1 Dylan  staff    22B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/strip-indent -> ../strip-indent/cli.js
lrwxr-xr-x  1 Dylan  staff    19B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/strip-bom -> ../strip-bom/cli.js
lrwxr-xr-x  1 Dylan  staff    20B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/semver -> ../semver/bin/semver
lrwxr-xr-x  1 Dylan  staff    25B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/sshpk-verify -> ../sshpk/bin/sshpk-verify
lrwxr-xr-x  1 Dylan  staff    31B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/not-in-install -> ../in-publish/not-in-install.js
lrwxr-xr-x  1 Dylan  staff    27B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/node-gyp -> ../node-gyp/bin/node-gyp.js
lrwxr-xr-x  1 Dylan  staff    26B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/node-sass -> ../node-sass/bin/node-sass
lrwxr-xr-x  1 Dylan  staff    23B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/sshpk-sign -> ../sshpk/bin/sshpk-sign
lrwxr-xr-x  1 Dylan  staff    27B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/in-install -> ../in-publish/in-install.js
lrwxr-xr-x  1 Dylan  staff    16B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/uuid -> ../uuid/bin/uuid
lrwxr-xr-x  1 Dylan  staff    23B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/color-support -> ../color-support/bin.js
lrwxr-xr-x  1 Dylan  staff    18B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/which -> ../which/bin/which
lrwxr-xr-x  1 Dylan  staff    20B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/mkdirp -> ../mkdirp/bin/cmd.js
lrwxr-xr-x  1 Dylan  staff    19B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/.bin/nopt -> ../nopt/bin/nopt.js
lrwxr-xr-x  1 Dylan  staff    20B  8 Aug 21:43 /Users/Dylan/Dropbox/Programming/GitHub/mana-music-centre-website/node_modules/node-gyp/node_modules/.bin/semver -> ../semver/bin/semver
lrwxr-xr-x  1 Dylan  staff    79B 22 Apr 20:44 /Users/Dylan/Dropbox/School/comp307/assignment-2/.Python -> /usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Python
lrwxr-xr-x  1 Dylan  staff    80B 21 Jul 22:15 /Users/Dylan/Dropbox/School/comp309/assignment-1/.Python -> /usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/Python

So what it looks like as there are lots of symbolic links from the node modules directory, but only to other files in the node modules directory itself (and not other directories, so there shouldn't be a performance hit because of these symbolic links)

the last two symbolic links to point to outside the drop box folder. They are created by python virtual environments, so wouldn't make sense to delete them. However, I don't need them right now, so I deleted them and observed no performance difference when running through my instructions

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

Dropboxer
Dropboxer
Any symlinks affect performance, since the desktop app needs to navigate to those folders to attempt to resolve them.
 
If you remove the first set of symlinks you mentioned referring to within the node_modules section, does performance improve?

Jay
Community Moderator @ Dropbox
https://dropbox.com/support


Heart Did this post help you? If so please give it a Like below. 
 Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.
 Still stuck? Ask me a question! (
Questions asked in the community will likely receive an answer within 4 hours!)

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

dylan-chong
Helpful | Level 5

I removed all of the symbolic links that were inside of a non-modules directory because they can be easily regenerated. After doing that, removing all of the files inside a node modules directory, and then running npm install against all causes drop box to go crazy on the cpu

 

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

Dropboxer
Dropboxer
Thanks for the reply, would it be okay if we reach out to you at your forum profile email to investigate this matter further?

Jay
Community Moderator @ Dropbox
https://dropbox.com/support


Heart Did this post help you? If so please give it a Like below. 
 Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.
 Still stuck? Ask me a question! (
Questions asked in the community will likely receive an answer within 4 hours!)

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

dylan-chong
Helpful | Level 5
Yup, sure

Re: Unnecessarily high cpu usage when sync a folder deselected with selected sync

doublesharp
Explorer | Level 4

Any resolution here? I have the same problem, but on OSX. Dropbox regularly runs at 100% CPU for *long* periods of time, I'm guessing related to node_modules. Why can't Dropbox just implement something like .dropboxignore and be done with it? This solution works fine for git, docker, lint, and more. No need to resolve symlinks in a folder matched by a glob in this file... 

Work Smarter with Dropbox

The way we work is changing. Share and discover new ways to work smarter with Dropbox in our community.

Sound good? Let's get started.
Who's talking

Top contributors to this post

What do Dropbox user levels mean?
Need more support?