The support forum

Delta copy not working

eggshen :

May 17, 2018

I am backing up music files (.mp3 and .flac) to a NAS from the PC's local HDD.  Every time a song is played the tags are updated to reflect the playcount.  Even though this represents only a miniscule change to the file, Bvckup2 copies the file in full everytime.  If I listen to a lot of music between backups this results in hundreds of MBs of files being copied just because a very small change in the metatdata.

Is this behavior expected?  

Alex Pankratov :

May 17, 2018



Delta copying is used only for files over certain size, so it's likely that individual mp3s fall under this threshold. You can however decrease it:

1. Right-click on the backup job in main window
    Select Open Location > Configuration and Logging

2. Use Notepad to create a file called override.ini and put the following
    line in it:

        conf.copying.delta.min_file_size         65536

3. Save the ini, exit Notepad, restart Bvckup 2

This lowers the file size threshold for delta copying to 64K. It can further down if needs be.

eggshen :

May 17, 2018

Excellent! Thank you for the help and for your wonderful software!

eggshen :

May 22, 2018

Perhaps I spoke too soon.  I made the change you recommended but delta copying is still not working.  

Any help would be greatly appreciated

Alex Pankratov :

May 22, 2018



1.  If you haven't updated to the latest version, do it first.
      https://bvckup2.com/update

2. Go into "Running the backup" > Processing section of the log, pick the file that you think is not being delta-copied (it'll be one of "Updating file ..." entries), right-click on it and select "Copy block to clipboard". Then paste it in your reply and post it here. Feel free to mask the file name, but leave the rest unedited.

eggshen :

May 22, 2018

Here you go:

2018.05.21 18:28:48.932 (UTC-6) 2 2         1. Updating file Baroness\Purple\01 - Morningstar.flac
2018.05.21 18:28:48.932 (UTC-6) 3 3             Details
2018.05.21 18:28:48.932 (UTC-6) 3 4                 Source: 32.88 MB, created 2016.08.28 19:54:01.236, modified 2018.05.21 18:27:52.803, archive / not-indexed
2018.05.21 18:28:48.932 (UTC-6) 3 5                     Raw: 34480974 / 131169056412369557 / 131714188728038952 / 00002020
2018.05.21 18:28:48.932 (UTC-6) 3 4                 Backup: 32.88 MB, created 2016.08.28 19:54:01.236, modified 2016.11.27 13:38:16.755, (no attributes) / normal
2018.05.21 18:28:48.932 (UTC-6) 3 5                     Raw: 34480974 / 131169056412369557 / 131247454967558067 / 00000080
2018.05.21 18:28:48.932 (UTC-6) 3 3             Initializing delta copying data...
2018.05.21 18:28:49.553 (UTC-6) 3 3             Completed in 621 ms, copied in full

Thnx and LMK if you need anything else

Alex Pankratov :

May 22, 2018



Delta updating kicks in on a _second_ run after delta copying is enabled. This is because on the first run the program doesn't have any of the block hashes (from the previous run), so it cannot tell which blocks are modified and which aren't. Instead it merely computes these hashes and stores them to be used on the next (2nd) run.

See the "Initializing delta copying data..." log entry? It means that there was no prior delta state for this file, so it's its first delta copy run.

Try playing a track, making a backup, then playing it and backing it up again. *Then* you will see it getting delta-copied.

See "Quick Recap" section here https://bvckup2.com/wip/25042018 for a bit more detailed description.

eggshen :

May 23, 2018

Alright, so I did as you said--Listened to the same file making sure the tags updated and then re-ran B2.  Here's the log:
2018.05.22 18:21:13.614 (UTC-6) 2 2         2. Updating file Baroness\Purple\01 - Morningstar.flac
2018.05.22 18:21:13.614 (UTC-6) 3 3             Details
2018.05.22 18:21:13.614 (UTC-6) 3 4                 Source: 32.88 MB, created 2016.08.28 19:54:01.236, modified 2018.05.22 17:50:13.643, archive / not-indexed
2018.05.22 18:21:13.614 (UTC-6) 3 5                     Raw: 34480974 / 131169056412369557 / 131715030136435929 / 00002020
2018.05.22 18:21:13.614 (UTC-6) 3 4                 Backup: 32.88 MB, created 2016.08.28 19:54:01.236, modified 2018.05.21 18:27:52.803, archive
2018.05.22 18:21:13.614 (UTC-6) 3 5                     Raw: 34480974 / 131169056412369557 / 131714188728038952 / 00000020
2018.05.22 18:21:13.614 (UTC-6) 3 3             Opening delta state file...
2018.05.22 18:21:13.614 (UTC-6) 2 4                 LoadCrcFileHeader() failed with 56130
2018.05.22 18:21:14.146 (UTC-6) 3 3             Completed in 532 ms, copied in full

For the sake of full disclosure, The entire destination folder structure and all files were created via B2 so there should be no "first run" issues.  Also I went back and checked and as near as I can tell delta copy has never been invoked at any time (yes I double-checked and the delta option is on for this job). The LoadCrcFileHeader error has only appeared since I added the override.ini to the config directory as you instructed.

Just for grits and shins I played a single file then ran the backup again--no crc error.  I then replayed the file, re-ran B2 and BAM!  LoadCrcFileHeader appears.  I have verified this with 3 different files.

Here are the logs for those:
2018.05.22 18:49:36.773 (UTC-6) 2 2         1. Updating file Minutemen\The Politics of Time\Minutemen - The Politics of Time - Below the Belt.mp3
2018.05.22 18:49:36.773 (UTC-6) 3 3             Details
2018.05.22 18:49:36.773 (UTC-6) 3 4                 Source: 2.22 MB, created 2013.10.23 19:48:02.015, modified 2018.05.22 18:47:35.173, archive / not-indexed
2018.05.22 18:49:36.773 (UTC-6) 3 5                     Raw: 2324170 / 130270492820156250 / 131715064551737208 / 00002020
2018.05.22 18:49:36.773 (UTC-6) 3 4                 Backup: 2.22 MB, created 2013.10.23 19:48:02.015, modified 2017.04.27 22:46:49.253, (no attributes) / normal
2018.05.22 18:49:36.773 (UTC-6) 3 5                     Raw: 2324104 / 130270492820156250 / 131378248092536892 / 00000080
2018.05.22 18:49:36.773 (UTC-6) 3 3             Initializing delta copying data...
2018.05.22 18:49:36.887 (UTC-6) 3 3             Completed in 114 ms, copied in full

--------------------------------------

2018.05.22 18:52:58.109 (UTC-6) 2 2         2. Updating file Minutemen\The Politics of Time\Minutemen - The Politics of Time - Below the Belt.mp3
2018.05.22 18:52:58.109 (UTC-6) 3 3             Details
2018.05.22 18:52:58.109 (UTC-6) 3 4                 Source: 2.22 MB, created 2013.10.23 19:48:02.015, modified 2018.05.22 18:50:32.730, archive / not-indexed
2018.05.22 18:52:58.109 (UTC-6) 3 5                     Raw: 2324170 / 130270492820156250 / 131715066327306734 / 00002020
2018.05.22 18:52:58.109 (UTC-6) 3 4                 Backup: 2.22 MB, created 2013.10.23 19:48:02.015, modified 2018.05.22 18:47:35.173, archive
2018.05.22 18:52:58.109 (UTC-6) 3 5                     Raw: 2324170 / 130270492820156250 / 131715064551737208 / 00000020
2018.05.22 18:52:58.109 (UTC-6) 3 3             Opening delta state file...
2018.05.22 18:52:58.110 (UTC-6) 2 4                 LoadCrcFileHeader() failed with 56130
2018.05.22 18:52:58.231 (UTC-6) 3 3             Completed in 122 ms, copied in full

Any ideas?

Alex Pankratov :

May 23, 2018

Can you do something for me please?

1. Right-click on this backup job in main window, select Open Folder > Configuration and Logging. There you will see a folder called 'deltas'.

2. Rename 'deltas' to something else, e.g. 'deltas.bak'

3. Play a track, run the backup.

4. At this point you should see 'delta' folder reappear. Inside there will be a single subfolder with a 2-letter name, go in. There again will be another 2-letter subfolder, go in. There will be a file with a random looking name and .uc1 extension. Make a copy of it somewhere.

5. Play the same track again and re-run the backup. Check if you see the error.

Repeat #4 and then forward both copies of the .uc1 file to support@pipemetrics.com. I'll have a look what's going on.

PS. Delta copying _is_ enabled in your case and it is being used. It's just that it produces delta state that fails a self-consistency check on the next run (which is what the error is about). This is strange, so we need to have a look at the actual state files.

Alex Pankratov :

May 23, 2018

Scratch that. I think we can reproduce this locally.

Alex Pankratov :

May 23, 2018

OK, found it. There is an issue with a routine that validates file's delta state after reading it from the disk. In particular, it deems the state invalid if the last file block was modified on a previous run *and* the file size is not a multiple of 64K... which is normally the case.

We'll have this patched up before the end of today and the update will go out shortly after.

eggshen :

May 23, 2018

Interesting.  I'll keep an eye out for the update.
Nice work . . . and thanks so much for the quick responses

Alex Pankratov :

May 25, 2018



OK, should be resolved now in 79.3.

https://bvckup2.com/support/forum/topic/1060/5991

You may or may not see the update in the in-app updates, but it's already at https://bvckup2.com/update.

Please let me know how it goes and if we can close this issue.

eggshen :

May 25, 2018

That seems to have done the trick!  Thanks again for the help.

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Blog / RSS
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy