Speed seems Slow
niemeyer :
Mar 15, 2020
I am testing a couple of different Applications to copy only changed blocks "Delta" to a server in a different datacenter and I am comparing to the total amount copied and times against RoboCopy.
The version I am testing Bvckup 2 Pro for Windows Server Release 80.11.
I am running Windows Server 2012 Standard
Robocopy says that it would copy 32.8gig which would take about 11 hours.
Bvckup test was; Read 90.32, wrote 13.59 GB, throughput 1.13MBps / 1188549bps. Completed in 23 h 23 min with 84 errors.
So even though it is copying less it is taking twice as long.
The data being copy is a lot of folders and small files with some files up to 2 gig in size.
Because of the abundance of small files I can make RoboCopy more efficient by running multiple threads. I don't see the ethernet speed drop off when using multiple threads.
I have seen an option on your website that seems to allow this in Bvckup2, but the option is not in the version I am running.
Is there anyway to speed Bvckup 2 up or am I doing something wrong?
1
Alex Pankratov :
Mar 15, 2020
When copying lots of small(er) files over the network, an option of running several copies in parallel with each other is a must. Current production release (R80) doesn't do that. However it is *the* main new feature of the next release (R81).
R81 is few weeks away from a formal release, but there are stable preview builds that you can use to evaluate the speed. See here for details -
https://bvckup2.com/support/preview/topic/1264
For how to get a preview build see here -
https://bvckup2.com/support/preview/topic/1263
Sample benchmark of R80 vs R81 -
https://bvckup2.com/wip/03102019
The UI config of parallel processing -
https://bvckup2.com/wip/020320202
niemeyer :
Mar 18, 2020
Unfortunately I am not able to get the preview through the application.
3
Alex Pankratov :
Mar 19, 2020
Can you explain which way it fails exactly?
Here's a screencap of how to do it. Just don't forget to hold Ctrl down when selecting Help menu for the first time -
https://bvckup2.com/support/data/switching-to-preview-builds.gif4
niemeyer :
Mar 19, 2020
It says checking and then it says ;
Regrettably, the check has failed.
It's possible it could be a proxy / firewall issue. You shouldn't expect software to be able to get out to the internet. Some networks are secured or sites not allowed for security reasons.
5
Alex Pankratov :
Mar 19, 2020
There are no expectations of this sort for the mainstream releases, but for the "preview" builds being able to check for updates is generally a good thing to have. It's not required however. Here's a direct download link for the latest one -
https://bvckup2.com/files/bvckup2-setup-1.80.99.13.exe6
niemeyer :
Mar 20, 2020
Thank you Sir! Testing this version now.
7
loyukfai :
Apr 19, 2020
R81 is few weeks away from a formal release...
Is this schedule still on track?
Cheers.
8
Alex Pankratov :
Apr 19, 2020
Yes, it is. We are now at preview 14 and it's the last of the preview builds. The next up is the actual release 81.
9
niemeyer :
Apr 25, 2020
Still testing this version and I don't understand how the delta copying thing works in Bvckup. I have a 600m file. I make a small change near the beginning of the file and it copies almost the entire file; Read 598.48MB wrote 561.92MB. If I make the same change at the end of the file; Read 598.46MB wrote 26.84KB.
10
Alex Pankratov :
Apr 25, 2020
This is how delta copying works, in detail -
https://bvckup2.com/kb/delta-copying.
If you are seeing the file getting recopied nearly in full, it means that there _were_ in fact that many block-wise changes.
Keep in mind if you are to _insert_ something at the beginning of the file, it will shift all the data in the file and all subsequent blocks _will_ change. While it is possible to detect this sort of change and handle it incrementally, this is a change pattern that is rarely used in practice due to its sheer inefficiency.
11
niemeyer :
Apr 28, 2020
Thanks for the link, but I am not seeing what is described in the link. I have tested other competing products and don't see the same results based on your comment of inserting something near the beginning of the file. I performed another test where I didn't insert anything I changed one character from a 4 to a 5 and the whole file copied from the point of the changed. The demo that is shown in the link shows parts of a file being updated which is what I thought would have happened.
I did a test using DeltaCopy which uses Rsync where I inserted the same text as I did in my Bvcup2 tests and it only copied the change not the entire file from the change. So I am not sure why the difference but there is a client and server piece. I know you are asking then why don't I just use deltacopy, well because it screws up permissions on any file that gets updated so that won't work for me.
12
Alex Pankratov :
Apr 28, 2020
I can't comment on specifics of what you are seeing without knowing _exact_ details of your tests and seeing relevant log extracts. But if you are to touch just a single byte in a source file and nothing else, then only the block hosting that byte will be copied.
13
Alex Pankratov :
Apr 28, 2020
I should add that rsync DOES detect blocks of data being moved, so if you are seeing rsync-based solution picking up a discrete block change and bvckup2 not recognizing it, chances are that your test change IS shifting the data.