cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Want to know all about tools for creatives that work seamlessly with Dropbox, like Adobe? Well, check it out right here!

Dropbox files & folders

Get in sync with the Dropbox Community. Our members can answer all your questions on Dropbox files and folders. Join a discussion or start your own today.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

How can I prevent very long syncing time of files in large folders?

How can I prevent very long syncing time of files in large folders?

Akiva
Explorer | Level 4

I have numerous folders in Dropbox, and the largest one is my Outlook Backup of office emails over many years, which is about 40GB. Occasionally, like today, I have to go in and save a new email, and the resulting syncing and indexing can take literally hours, which really slows down my PC. How can I prevent this very long syncing time of files in large folders?

1 Accepted Solution

Accepted Solutions

Re: How can I prevent very long syncing time of files in large folders?

Rich
Super User II

@Akiva wrote:

So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.


I do exactly what you do; open a separate PST file and manually move messages to it. This file is not open in Outlook all the time; only as needed. While doing this and storing the file in Dropbox, there's nothing you can do about the sync times. Here's why ...

 

Dropbox works by taking your files and breaking them into smaller chunks (4Mb). It then creates a hash for each chunk and syncs those chunks to the Dropbox servers. If you edit a file, Dropbox again breaks the saved file into chunks, hashes them and then compares the hash of each chunk to those stored online. Any chunk that is different is synced. This makes it so you can take a large file, say 100Mb, edit it and only sync a few Mb because Dropbox only syncs the portion that has changed.

 

For that to work, the majority of the file needs to remain unchanged and then Dropbox can sync just the changed bits. If a program re-writes the entire file during the save process, or a large enough portion of it, then every 4Mb chunk of that file will likely be different from those saved online, so Dropbox will need to sync each and every one, resulting in the full file being re-synced and re-indexed.

 

I don't know how Outlook saves a PST file but I do know that enough of the file appears to change for Dropbox to have to re-index the entire file. To demonstrate this I just did a binary comparison of two PST files where the only difference between them was one email message. Basically, I exited Outlook and made a copy of the PST file. I then opened Outlook, added one new message to the file, then exited Outlook and made another copy of the PST file. I then did a binary compare of these two files to show the differences between them. The files are displayed side by side. Everything you see in red is different from the file on the opposite side. That's a lot of change for just one new message being added to the PST.

 

If your PST files are large, it's going to take time to index and sync. Most of mine are in the 4-5Gb range, and each is just a calendar year of mail, so they sync fairly quickly. If you're just dumping everything into a single PST file, perhaps you should consider splitting them in a similar manner.

View solution in original post

6 Replies 6

Re: How can I prevent very long syncing time of files in large folders?

Mark
Super User II

Simply put... dont put them in Dropbox or get a faster internet connection.

 

In fact I cannot strongly recommend enough that you do NOT sync your Outlook folder in Dropbox - there is a very real and high risk of data corruption. Outlook doesnt like other programs accessing its data files while its open so if Dropbox tries to sync it could cause a loss, or, if you open it on another device before the syncing has finished you will get a a conflicted copy and have to manually fix it - no software will be able to do that for you.


 


:penguin::penguin: - :penguin: - :penguin: - :penguin:


Heart Did this post help you? If so please mark it for some Kudos below. 


:white_check_mark: Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.


:arrows_counterclockwise: Did this post not resolve your issue? If so please give us some more information so we can try and help - please remember we cannot see over your shoulder so be as descriptive as possible! 


 

Re: How can I prevent very long syncing time of files in large folders?

Akiva
Explorer | Level 4

Thanks for responding, but let me explain in a bit more detail. I am not syncing Outlook. Yes, I know that's a problem. What I do is open another Outlook data file within Outlook - the Outlook backup file that's in Dropbox - and save my new email in the job folder. I then close the second Outlook (backup) file. So I'm back where I started, except for the updated email saved in the Outlook backup file in Dropbox. It's then that the long long syncing starts, for a simple additional email of a couple of MBs.

 

Does that explain my problem better?

Re: How can I prevent very long syncing time of files in large folders?

Mark
Super User II

Personally... I'd still not be doing it - Outlook at Dropbox (in fact any cloud solution dont play well)!

 

Dropbox actually only syncs the 4mb changes of a file, but, as Outlook probably updates huge chunks just being opened it has to resync all of those 4mb blocks.


 


:penguin::penguin: - :penguin: - :penguin: - :penguin:


Heart Did this post help you? If so please mark it for some Kudos below. 


:white_check_mark: Did this post fix your issue/answer your question? If so please press the 'Accept as Solution' button to help others find it.


:arrows_counterclockwise: Did this post not resolve your issue? If so please give us some more information so we can try and help - please remember we cannot see over your shoulder so be as descriptive as possible! 


 

Re: How can I prevent very long syncing time of files in large folders?

Akiva
Explorer | Level 4

So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.

Re: How can I prevent very long syncing time of files in large folders?

Rich
Super User II

@Akiva wrote:

So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.


I do exactly what you do; open a separate PST file and manually move messages to it. This file is not open in Outlook all the time; only as needed. While doing this and storing the file in Dropbox, there's nothing you can do about the sync times. Here's why ...

 

Dropbox works by taking your files and breaking them into smaller chunks (4Mb). It then creates a hash for each chunk and syncs those chunks to the Dropbox servers. If you edit a file, Dropbox again breaks the saved file into chunks, hashes them and then compares the hash of each chunk to those stored online. Any chunk that is different is synced. This makes it so you can take a large file, say 100Mb, edit it and only sync a few Mb because Dropbox only syncs the portion that has changed.

 

For that to work, the majority of the file needs to remain unchanged and then Dropbox can sync just the changed bits. If a program re-writes the entire file during the save process, or a large enough portion of it, then every 4Mb chunk of that file will likely be different from those saved online, so Dropbox will need to sync each and every one, resulting in the full file being re-synced and re-indexed.

 

I don't know how Outlook saves a PST file but I do know that enough of the file appears to change for Dropbox to have to re-index the entire file. To demonstrate this I just did a binary comparison of two PST files where the only difference between them was one email message. Basically, I exited Outlook and made a copy of the PST file. I then opened Outlook, added one new message to the file, then exited Outlook and made another copy of the PST file. I then did a binary compare of these two files to show the differences between them. The files are displayed side by side. Everything you see in red is different from the file on the opposite side. That's a lot of change for just one new message being added to the PST.

 

If your PST files are large, it's going to take time to index and sync. Most of mine are in the 4-5Gb range, and each is just a calendar year of mail, so they sync fairly quickly. If you're just dumping everything into a single PST file, perhaps you should consider splitting them in a similar manner.

View solution in original post

Re: How can I prevent very long syncing time of files in large folders?

Akiva
Explorer | Level 4

What a great and detailed reply. And thanks for doing the research. Guess I have no choice but to split it into smaller files, but the problem then will be that instead of showing 14 job files, I will have to show  about 350 job files every time I open Dropbox. But it's definitely better than the current sitch.
So thanks again.

Poll
Do you work or study in the creative industry? If so, which area?
If you use Adobe, don't forget to check out our latest integrations update here.
Who's talking

Top contributors to this post

  • User avatar
    Akiva Explorer | Level 4
  • User avatar
    Rich Super User II
  • User avatar
    Mark Super User II
What do Dropbox user levels mean?
Need more support?