The support forum

Deltas folder is growing massive in size

soylent :

Feb 27, 2015

I noticed that my "\Bvkup\engine\backup-0002\deltas" folder is over 34 GB. What can I do to safely shrink these? Does it keep growing forever?

Alex Pankratov :

Feb 27, 2015

You can simply delete it.

Each file there contains block hashes for one of the files at the source.  When you delete /deltas/, all changed files will be re-copied in full on the next run, the app will rebuild the hash files -- but only for files that actually changed -- and it will then use delta copying for them on subsequent runs.

Second option is that you can move this folder elsewhere if the disk space is at premium where it is at by default. Let me know and I can explain how to do that.

--

Tangentially related - I am planning to add support for auto-trimming the /deltas/ folder. That is, to remove hashes for files that are not or rarely modified. This is what you would've used in your case to shrink /deltas/ down.

Froggie :

Feb 27, 2015

Soylent, none of mine have ever gotten more than 10s of megabytes so there is definitely an issue there...

Alex Pankratov :

Feb 27, 2015



Also, the default block size for delta copying is 32KB and each block is reduced to 20 bytes of hash, meaning that /deltas/ size will roughly be 0.06% of the backup size.

soylent :

Feb 27, 2015

I'm using Bvkup to automatically copy the backup files created by my Veeam backup program. I set Bvkup to continuously watch that folder and copy over any new full backups (ignoring the incrementals) to another storage location. The fulls are 800 GB each.

I wonder if the continuous backup is what's causing the huge deltas? When Veeam is writing the 800 GB file, Bvkup is monitoring that, right? Even though I have Bvkup set to delay 2 hours after it detects any new changes?
Is there a better way to set this up?

soylent :

Feb 27, 2015

Also I just realized that Bvkup shows the destination snapshot as 77 TB. Maybe I shouldn't be using delta changes for this job, since the file doesn't get modified after it's written.

Basically I need it to watch for any new .VBK files. When it sees one appear, wait about 2 hours, then copy it over.

Alex Pankratov :

Feb 27, 2015

When Veeam is writing the 800 GB file, Bvkup is monitoring that, right? Even though I have Bvkup set to delay 2 hours after it detects any new changes?


Yes, certainly. The app gets a nudge from Windows whenever the file is created or updated. It then just raises internal flag, waits for the specified pause and then runs the backup.

Is there a better way to set this up?


I experimented with some code to detect "live" files, i.e files that are getting modified *during* the backup. Unfortunately, there's no way to detect such files reliably without parsing the NTFS journal, and this is not something that is currently supported by the app. So what you have set up is in fact the optimal way of handling it for the moment.

Maybe I shouldn't be using delta changes for this job, since the file doesn't get modified after it's written.


Yep, you absolutely not need to use delta copying. For one, it won't populate /deltas/, and for two it will actually be easier on the CPU as the app will able just to ferry data from A to B without needing to compute hashes.

Alex Pankratov :

Feb 27, 2015


* Just keep in mind that if/when you switch from "delta copying" to "copy files in full", you will need to delete the /deltas/ folder by hand.

soylent :

Mar 03, 2015

Great, thanks for the prompt reply.

Niloc :

Mar 10, 2015

Hi Alex - please forgive a Bvckup newbie, but like the comments above, I've found that my  Bvckup Beta2 v 65 has produced 24,445 files & 3,056 folders in my C: drive "\Bvkup\engine\backup-0002\deltas" folder.

Please could you explain how to move that folder elsewhere? I have 3 data/personal partitions that I could use, if only I knew how!

Do those files/folders only get produced when using Delta copying? If yes, then I assume that if I use non-delta copying, this will take much longer?

Thank you in anticipation of your help, as well as thanks for a superb backup tool.

Kind regards, Niloc

Alex Pankratov :

Mar 10, 2015

Niloc, for the the beta versions you have two options.

You can either move the whole config directory elsewhere and launch the app with "-c ..." command line parameter [1] or you can potentially move /deltas to some other place and then symlink the original /deltas folder to it. With latter there are some caveats (for one, NTFS is required, for two, creating symlinks requires admin privileges, etc.).

For versions starting with Release 70 it's also possible to create a file called redirect.ini in /deltas/ folder and put a new location in it (just as a first line).

[1] https://bvckup2.com/support/forum/topic/43

Phoenix :

Mar 23, 2018

I think the folder simply needs to be limited in size.  I can't offload this elsewhere either.  Mine is 1.6 GB and I don't think that is reasonable.  Perhaps set a limit at 200 MB per entry as a default.

Alex Pankratov :

Mar 23, 2018



Have you seen the description of how delta copying works?
https://bvckup2.com/support/forum/topic/739

In the "Details" section it mentions that the size of the delta state (which is what the /deltas/ folder is exclusively comprised of ) is about 0.06% of the size of files that are copied with delta copying.

You want to reduce the size of /deltas/ - you need to reduce the amount of files that are copies with delta copying, or switch delta copying off completely.

There are two size thresholds that are used to decided if a file should be copied with delta copying or not:

1. Files over 32 MB are always copied with delta copying
2. Files over 2 MB are copied with delta copying if they've been modified within last 30 days

32 MB, 2 MB and 30 days are configurable. If you are up to changing them for your setup, let me know and I'll explain the details.

However, what you can do in a pinch is just nuke /deltas/ folder. On the next backup run the program will recopy all modified files in full and this will rebuild the /deltas/ entries for files that are getting modified *now*.

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Blog / RSS
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy