The support forum

Swap file failure on job with 3.077.383 files

Michael :

Jan 22, 2016

Hi,

I have 30 or some jobs on the same server backing up many TB from fileservers to offsite server.
All jobs runs correct, except one which had run successful but the other day it started to fail with this error:

Retrieving IDs of 3077383 new files...
- completed in 46 min 51 sec
Swap file failure
- Error: SwapReadPlannerStep() faield with 10006
-- Context: 5770380
File: C:\Users\%removed%\AppData\Local\Bvckup2\engine\backup-0018\~~planner.swp
Error: failed to complete the backup plan.

Its running on Srv2008R2 with 8GB RAM.
All sources are mapped network drives, all destinations are local disks.
Other jobs share network drive and destination drives, so its not a source or destination error.

Thanks.

Alex Pankratov :

Jan 22, 2016

Michael,

This happens when there's a folder with more than 1,000,000 files in it. It was put in place as a "sanity check", but apparently it was too conservative for the real world :)

It also happens one of very few hardcoded thresholds in the app, so it cannot be changed through an INI file edit. However this has been resolved in a development build. If you'd like to get a copy, drop me an email at support@pipemetrics.com (it's basically the latest production release plus a small patch to rework this sanity check).

Michael :

Jan 25, 2016

Hi Alex,

thanks for your quick reply.
I have sent an email, so I can try with a new build.

Michael :

Jan 26, 2016

I received the new build and so far it hasnt failed, moving 717649 out of 3.078.910 files today.
So I believe its OK now.

Thanks a lot for the instantaneous support and fix.

genl :

Nov 19, 2017

Release 78.11. Getting this error today:

2017.11.19 18:10:53.324 (*) 0 4                 Swap file failure
2017.11.19 18:10:53.324 (*) 3 5                     Error: SwapReadFileBlob() failed with 10002
2017.11.19 18:10:53.324 (*) 3 6                         Context: 480089022
2017.11.19 18:10:53.324 (*) 3 5                     File: C:\Users\*****\AppData\Local\Bvckup2\engine\backup-0001\~scanner-dst.swp

And the destination snapshot got wiped second time already, from what I see. Source and destination are both on local disks. The amount of files is more than 3 400 000.

Alex Pankratov :

Nov 19, 2017

Error 10002 means that the file name was empty (zero-sized) when a file record was read from the swap file. Technically, file names cannot be empty, but a program may end up seeing one if the name merely _starts_ with a NUL character. This in turn may happen if a file is created via so-called "native" (low-level) API and this is usually done to conceal the file from the normal programs. DRM software and rootkits used to do this routinely before, but when it would happen now - I'm not sure.

Now, if that's in fact the case (that is, you have a zero-sized file name), then you should also see a complain about that during the scanning phase. If you don't, then this probably means a bug in bvckup2 itself.

In this case, you should see ~scanner-dst.swp file being left intact in the job's configuration folder after the job completes. It's going to be very big, but it will also be very sparse, so it will compress really well. Please do just that and forward it to me at support@pipemetrics.com *together with the exact error* (just as you have above).

Keep in mind that .swp will contain names of files and folders in your backup, so if you'd rather not share these, let me know and we'll need to find some other way to tackle this.

genl :

Nov 20, 2017

There is no scanner-dst.swp file left. Also I can see that previous attempt ended with this:

2017.11.19 14:40:15.447 (*) 3 3             Preparing backup plan ...
2017.11.19 14:40:16.677 (*) 0 4                 Swap file failure
2017.11.19 14:40:16.677 (*) 3 5                     Error: SwapReadFileBlob() failed with 10010
2017.11.19 14:40:16.677 (*) 3 6                         Context: 19866641
2017.11.19 14:40:16.677 (*) 3 5                     File: C:\Users\*****\AppData\Local\Bvckup2\engine\backup-0001\~scanner-dst.swp
2017.11.19 14:40:16.713 (UTC+3) 0 4                 Error: failed to compile the backup plan

genl :

Nov 20, 2017

The same job was finished today with usual amount of changes and no errors...

Alex Pankratov :

Nov 20, 2017

Hmm. Either a code review or an exhaustive memtest is in order. Probably the former.

genl :

Nov 21, 2017

Actually, I think it's possible that my memory got corrupted as I'm noticing applications crash with no obvious reason sometimes, and it happens more often than few weeks ago. I only ran Windows Memory Diagnostic once and it returned no errors, but I suspect it's not a very good tool. I'll going to test it some more soon.

Alex Pankratov :

Nov 21, 2017

Interesting. Perhaps run http://www.memtest.org/ overnight?

Alex Pankratov :

Nov 27, 2017

Ok, I think I tracked this one down.

This is an incorrectly reported error reading a swap file. But it's an IO error nonetheless. More specifically, swap files are used as a temporary off-RAM storage for source/destination file indexes. When operating with a large number of files, the program will periodically write out and read back big chunks of memory to/from a swap file. If an attempt to read from a swap file would fail for some reason, the error wasn't handled right away, but passed on in a form of zero-filled memory chunk to the requesting part of the program. This part would then discard this chunk as corrupted and emit 2 or 3 different kinds of errors depending on the context.

We'll have this reworked for clarity, should be in the next maintenance release.

genl :

Dec 02, 2017

Thanks. I expected something like that since I know that my RAM is at 80%-90% most of times.

I did a full run with memtest and it only reported errors in hammer test. I'll try again after some adjustments later.

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy