The support forum

Force one-time integrity check?

gmagana :

Jan 07, 2016

Is there a way to manually do a one-time integrity check/compare of the source and destination?

I have a backup configured to run every 6 hours. Every X days I would like to manually force an integrity check/comparison to make sure my backup is accurate. My backup destination is a network drive which sometimes gets disconnected mid-backup (because of network problems, phase of the moon, or who knows why). I can continue this backup by making sure the remote drive is accessible again and restarting the backup job. This _seems_ to work ok, but I want to make sure that the backup I think I have is indeed there and is accurate.

On "Detecting changes" I have it set to "Use destination snapshot" and then on "Copying" I have it set to "Use Delta Copying". I'm not sure if these settings are allowing me to have a potentially faulty backup if teh destination goes away mid-backup, which is why I want to check integrity fully every once in a while.

Thanks!

Alex Pankratov :

Jan 09, 2016

Good question.

Integrity checks (a.k.a. "backup verification") is one of two big outstanding features. The next release (R75) will get the other feature - SMART monitoring - and the one after (R76) will get the verification support.

The way it will work is that it won't just compare source against destination, but it will also have an option of hashing source files and storing their hashes between the runs. This will allow the app to understand where exactly a change occurred when two copies of a file no longer match. This in turn helps catching and recovering from the bitrot [1] both at destination *and* source.

[1] https://en.wikipedia.org/wiki/Data_degradation

---

With regards to "detecting changes" / "copying" settings - using delta copying may in fact leave file in a partially updated state if the update is interrupted mid-way through. This will however leave file tagged for an update, so it will be re-synced on the next run.

The "Detecting changes" setting is irrelevant here as it controls how the app scans the destination, rather than how it copies things.

gmagana :

Jan 09, 2016

Thank you! Looking forward to the updates!

Niek :

Mar 25, 2016

Hi Alex,

First: great program! Fast and easy to use :)

I'm using the beta for now but am thinking about getting the paid version. Any idea on when the "backup verification" will be available? I had some very recent trouble with corrupted back-up files. So a good verification method is very welcome!

Also had the same issue as described here: https://www.bvckup2.com/support/forum/topic/471 :an 0000 Timestamp on a QNAP NAS. Stopped the back-up and did a successful re-run with "re-scan destination" enabled.

And had a small problem with excluding folders: Lightroom creates 'Preview' folders that I wanted to exclude, folders ending with Previews.lrdata
First i tried */*.lrdata but did not work. Then I tried *.lrdata but this also did not work. But now with *Previews.lrdata* it seems to work correct. So ending with a * did the trick.

Regards,
Niek.

Alex Pankratov :

Mar 28, 2016

Any idea on when the "backup verification" will be available?


It is scheduled for Release 76. We are at Release 74 now, so it's in one major release. Time-wise, it's likely to be in the early summer.

And had a small problem with excluding folders...


This is odd. Just

    *.lrdata

should've worked. Perhaps the filter type was accidentally set to "Files" instead of "Files and folders" or "Folders"? If not, then the only explanation is that these folders DO have something else in their names after the lrdata part that doesn't show up in the Windows Explorer.

It won't be the first time when Adobe did some really weird stuff with the file names - https://bvckup2.com/wip/18062015

Niek :

Mar 28, 2016

Thanks!

Can you tell me how 'save' backing up with bvckup is? Or is safety not really a bvckup thing and more a hardware thing? Corrupted files are mostly caused by a hardware failure?

Niek :

Mar 29, 2016

edit: 'save' is 'safe' :)

Alex Pankratov :

Mar 30, 2016



Corruption is nearly always a hardware issue.

If there's a faulty RAM stick, then a file may get corrupted when it is copied (as its being read into memory first and then written out).

Files may also get corrupted at rest as well, which is a phenomenon known as "bit rot" [1]. You will basically see bits getting flipping from 0 to 1 and vice versa for no apparent reason.

With Bvckup 2 you may end up with a partially updated file if you are using delta copying AND if you are to abort the update mid-way through. This may result is some changes propagated from the source file to the backup and some - not. This is however detected on the next backup run and the file is updated accordingly.

[1] https://en.wikipedia.org/wiki/Data_rot

Niek :

Mar 30, 2016

@Alex: Thanks for your explanation!

Really close to purchasing :-) Really like your program, the speed, design, simplicity, features, support.

Off course did some browsing and reading about 'similar' tools, and did some speedtests with Robocopy, RichCopy and Bvckup (beta). My speedtest (on my machine, my network, back-up to NAS, 2 folders, 4.06 GB, 17 files ) was won by Bvckup.

I decided to update to Bvckup 2 for the 14-day trial.....

After updating from the beta to 74.22 I did the same speedtest again,  but the back-up was way slower: with the beta I got 1 min 29 sec / 1 min 31 sec. With 74.22 I got 1:55 sec..... uhhhh, that cannot be true. So I tried again and got 1:37, 1:38 and 1:33 (the last one after restart computer). The last runs got closer, but still slower than the beta. [1]

Any thoughts?

[1] http://s22.postimg.org/mq8q4l1ox/Capture.png

Niek :

Mar 31, 2016

And one other question: I use Bvckup 2 to make a backup from my Windows machine to a NAS. So a NTFS to EXT4 transfer. Is there a chance for 'timestamps problems'?

Reading https://bvckup2.com/wip/06042013 the program does a file system check. But when I look in the log file the destination drive shows up as a NTFS (throug Samba) and not as EXT4... can this cause problems?

Alex Pankratov :

Mar 31, 2016

Any thoughts?


That's within the statistical noise level. If you want to make an accurate benchmark comparison, grab the RamMap tool by SysInternals and use it to "Empty Standby List" before every run. This will clear Windows disk caches and level the playing field a bit more.

Is there a chance for 'timestamps problems'?


There shouldn't be. The app includes comprehensive timestamp resolution testing module, and there are also manual overrides just in case.

can this cause problems?


Samba shares will always show up as NTFS, this is absolutely normal.

Niek :

Apr 03, 2016

Did another ' speed test'... nothing 'scientific', just the way I'm using it. This time with more, but smaller files. 105 files, 2.34 GB. Used RamMap between each run, to clear cache (thanks).

http://s9.postimg.org/ve23ny8fz/Bvckup2.png

The first three runs with the beta: all three in 57 sec. Then upgraded to latest version 74.22: the first run was also 57 sec, but then it slowed down. 58 sec, 59 sec and 1 min 9 sec. Strange or not?

Stability and reliability are of course the most important factors in a program like this, that's why backup verification is a very welcome addition.

Even over a slow Wifi connection the beta was 1 minute faster than v74. 13 min 36 sec vs 14 min 37 sec.

I assume you do your own speed tests. Your tests showed different results?

Alex Pankratov :

Apr 04, 2016

Strange or not?


Not really. There's still lots of moving parts. It could be a File Indexing service kicking in during the latter runs, it could be an antivitus downloading its daily definitions update, etc. When you are measuring something you need to compare apples to apples or allow for a larger margin of statistical variation.

Your tests showed different results?


Of course. Any changes that may affect copying speeds are subject to benchmarking before being signed off to production.

Niek :

Apr 07, 2016

Thanks!

mastershakes :

Jul 22, 2017

Bvckup is great.  I buy it for all my customers.  One of my bigger customers I setup for VM backup using Delta copies.  I would like to do schedule a verify once a week or so to catch any gremlins.  I don't see an option for that.  Am I missing something?  If not do you have a timeline for implementation.  Thanks and fantastic piece of software you have created!!

Alex Pankratov :

Jul 24, 2017

It's not yet done, but it *is* one of three main outstanding features that separate Bvckup 2 from being feature-complete. Current plan is to have it done in the early-to-mid fall.

Thanks for the "fantastic" compliment, much appreciated!

number_one :

Aug 28, 2017

I'm unclear on the timing of the "backup verification" feature release.  The initial post about it back in Jan 2016 mentions it being a v76 feature.  We are now at v77 and I don't see an option for it.  I'm not trying to come off as being critical; just trying to understand what's going on.

justbackmeup :

Aug 28, 2017

My interpretation, just as a fellow reader of the forum who is interested in the same functionality, is that it had to be pushed back due to other more pressing issues and/or that it is a significant enough bit of functionality that it needed to wait for space in his programming schedule in order to tackle it. Based on what I've read above I'm hoping we'll see it implemented by the end of 2017. Again, just speaking as an observer, not as one who has any inside knowledge.

Alex Pankratov :

Aug 28, 2017



It's a fair question. Here's where we are at in terms of current development plan:

1. Release 78 is up next, slated to go out tomorrow. It will include reworked exclusion/inclusion filters, scheduler changes (support for "quiet times"), internal rework of certain parts and several smaller features and changes.

2. Release 79 will introduce support for the revised licensing [1] that has been pending for almost a year now. Tentatively this should be done before the September is out.

After that we'll be working on adding

    (a) backup verification
    (b) parallel copying
    (c) bandwidth control
    (d) SMART monitoring
    (e) backup grouping (UI)

as well as a handful of smaller features, tentatively in the order listed.

[1] https://bvckup2.com/support/forum/topic/840

Alex Pankratov :

Aug 28, 2017



On more general note - plans and priorities change.

We made a "detour" earlier this year and looked at making a hosted management console. Got it prototyped, looked at this, looked at that, but ended up shelving the idea for the time being, because become a service provider would've stretched us thinner than it was comfortable.

Similarly, you've probably heard about the CrashPlan announcement. What they did doesn't affect us *directly*, but the outcry that followed showed that there's a non-trivial demand for over-the-Internet P2P backups. And it just so happens that the p2p part aligns nicely with my past experience.

So I've been sitting here, twiddling a pencil and considering if we should prototype something of this kind. Most likely nothing will come out of this, not now, but the thing is that plans change, often, and this tends to lead to the delays like the one with the backup verification support.

justbackmeup :

Aug 28, 2017

Similarly, you've probably heard about the CrashPlan announcement. What they did doesn't affect us *directly*, but the outcry that followed showed that there's a non-trivial demand for over-the-Internet P2P backups. And it just so happens that the p2p part aligns nicely with my past experience.


I'd be interested. :) That would fit well with the below, which I'm sure you've already been pondering:
https://bvckup2.com/support/forum/topic/965/5330

I have a VPN connection between my two sites, so p2p is less critical for me, but still nice for when the VPN is down (eg, I had a router failure earlier this year and so had to fall back to an older model I had on hand that didn't have VPN functionality, but CrashPlan could then just route my backups via the p2p functionality, instead of going direct over the VPN).

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy