I have a system that runs on top of a set of Dropbox-synced folders, allowing a team to independently contribute data and files. Periodically, the system then runs analytics and further processing on those folders.
One thing I've found is that we get intermittent access errors when reading through large numbers of these files as part of processing them. I suspect the issue is related to bad timing - we just happen to try to read it as Dropbox is accessing it or its folder and we therefore get an access error.
In trying to think about how to address this, there are two options that we came up with quickly:
- Always try multiple times if the first read fails.
- Turn of Dropbox sync when we're doing our processing.
The shortcoming of the first approach is that we run a variety of analytics and algorithms (this is our development area for a team of algorithm developers, and separately we're constantly generating dashboards and reports on the data itself), and we would have to get everyone to start using accesses designed to make multiple attempts if the first one fails.
The shortcoming of the second is that to the best of my knowledge I would have to manually pause the Dropbox sync, then allow the processing to run, the manually un-pause the sync. Because all of our bigger analytics and algorithms runs are scheduled to happen at night, this would be an extremely inconvenient approach.
I'm wondering if Dropbox has an existing mechanism by which I could have it pause syncing at a certain time each day, and then resume syncing at a subsequent time. For example, I might want it to pause syncing at 2 am (we won't be uploading new data at that point, and the algorithms are scheduled to run at this time) and then resume syncing at 4 am (giving enough time for the various algs and analytics to complete).
I'm wondering if Dropbox has an existing mechanism by which I could have it pause syncing at a certain time each day, and then resume syncing at a subsequent time.
No, it doesn't, but you can use any scheduling application that is capable of starting and terminating an executable; for instance, the scheduling service that is built-in to Windows.
I would not even bother to use windows scheduler, instead add a process kill on Dropbox.exe to the front of the analytics run script and a process launch on Dropbox.exe at the end. That way the shut down and restart of DB runs in sync with the analytics no matter what time its run.
I also want to automate or schedule the sync on/off setting so that an upload is only running over night. In my case, I am uploading single large files (over 10GB).
Will this brute force hack of terminating and restarting the desktop program correctly resume the sync or will it restart from the beginning? I'm on OS X.
It's impossible for me to just observe the behavior because I have no way of verifying now much of the file has been uploaded at any given time, and the "time remaining" displayed in the desktop app is completely un-reliable probably because the upload is being limited by my ISP ... which is why I want to only turn it on over night because it's going to take multiple days.
Really wish dropbox would add the ability to schedule or automate when syncs can happen in the desktop app, even just exposing the right hook for the OS X automator or a bash script would be great.
Obviously the majority of the API deals directly with Dropbox servers. Is there no API call to pause (and unpause) the windows app's activity on the local machine?
I'm interested in writing a small app to control sync based on WiFi SSIIDs (since in Australia, mobile/cell data plans can get very expensive, very fast, when traveling around a bit). Killing the systray app seems a bit brutal, and removes the convenience of readily clicking into the Dropbox folder area...
The way we work is changing. Share and discover new ways to work smarter with Dropbox in our community.Sound good? Let's get started.
For more info on available support options, see this article.
If you found the answer to your question, please 'like' the post to say thanks to the user!