The support forum

Scanning source progress..

Imprezadude :

Nov 21, 2017

I am migrating a 50TB+ NAS to Windows File Services. Right now i'm trying to copy a 140GB share with approx 6 mill. files in it. I am aware that it takes time, but it has scanned the source directory for like 5 hours now. I could use something that indicates some progress because I can't see if it's doing anything except high CPU usage.

Regards

Henrik Pedersen

Alex Pankratov :

Nov 21, 2017



What do you see at the bottom right corner of the backup tile? During the scanning phase it should be showing current file/folder counts there.

With 6 mil files you would want to bump up the size of the program's internal cache, so that it would swap less. This is done by setting these keys in %LocalAppData%\Bvckup2\engine\bvckup2-engine.ini as follows:

    swap.max_pages    64
    swap.page_size       8388608

This will set the cache to use 64 x 8 MB pages, so you'll be looking at 512MB of run-time RAM usage. In comparison, the default is 8 x 1MB setup.

If you have multiple gigs of RAM, then you can switch to using fully in-memory file indexes by changing the following key:

    conf.scanning.cap_memory_use    0

To edit the ini -
1. Shut down the program
2. Make the ini changes (any text editor will do, e.g. Notepad)
3. Restart the program

Any INI changes made while the program is running will be ignored and discarded.

For background on caching layer see https://bvckup2.com/wip/27052015

Imprezadude :

Nov 22, 2017

Alex,

It just stops counting at 144.61MB -  8086 Files . 35 folders. Status is Scanning Source. I have another job that right now have counted more than 400k files on another share. I will try to increase the memory.

/Henrik

Imprezadude :

Nov 22, 2017

The only folder in %LocalAppData%\Bvckup2\ is a "ui" folder and then .ini, .ini.copy and .ini.prev and log files. ??

/Henrik

Alex Pankratov :

Nov 22, 2017

144.61MB -  8086 Files . 35 folders


This is peanuts. The memory/swapping tweaks apply from 500K files and up. This means that the cause of the stoppage is external to bvckup2. Perhaps your NAS has problems coping with rapid-pace parallel scanning that bvckup2 uses by default. It's also possible that this scan triggers a bug in an antivirus or a deep-packet inspection filter between the machines or something even more exotic.

You can try and tell it to use sequential scanning, see if this helps. This is done by setting the following two keys in settings.ini for the job:

    conf.scan_threads.local           1
    conf.scan_threads.remote      1

NB - That's a different INI from the one I mentioned above.

The only folder in %LocalAppData%\Bvckup2\ is a "ui" folder and then .ini, .ini.copy and .ini.prev and log files. ??


You are running in a service mode, right?
The folder will be %ProgramData%\Bvckup2\engine then.

Imprezadude :

Nov 22, 2017

Okay, set it back to run as admin then the folders came and i could edit. Still looks like the same issue but i'll give it time. I have to run it as a service again but i assume that the settings in the .ini will apply also when it runs as a service?

/Henrik

Imprezadude :

Nov 22, 2017

Okay, discovered that the files under %programdata% still was default. Maybe a bug that it doesn't apply the settings from administrator mode to service mode?  Anyway, corrected it and job is running. Still the same, will try to let it run until tomorrow to see what happens.  If it works i'm buying it :)

/Henrik

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy