The support forum

How to avoid scanning/backing up/deleting more than selected?

ScottD-KMHA :

Mar 11, 2017

We recently ran into a bit of a nightmare scenario in which we set the "Backup from" value as "C:\" and "Backup to" value as "Y:\".  We set the "What to backup" to "Selected files and folders" / "Start with an empty list" and then selected to "include" only 15 specific folders from the source (OS C: drive).  Well, thinking that Bvckup2 will only be monitoring the 15 specific folders selected to be included, we went ahead and set the "Deleting" option to "Delete backup copies" as we don't wish to keep folders/files on the destination that have been deleted from any one of these 15 selected folders at the source.

To our dismay, on the first run, we found that Bvckup actually deleted as many files as it could OUTSIDE of the 15 folders we selected to include and that did not exist on the source...thereby destroying some of the OS related folders/files on the destination machine before it was noticed and the backup was killed prior to complete OS meltdown.

Here is how we were hoping Bvckup 2 can operate to meet our goals...

1) Backup from = C:\
2) Backup to = Y:\
3) What to backup = ONLY the selected/included folders/files.
4) Deleting = Delete backup copies

Problems when configuring the set...

In the "Start with an empty list" and "Then, include the following items" section, it currently does not allow one to check/select a folder and then uncheck individual sub-folders.  Instead your only choice, if you only want to backup certain sub-folders, is to leave the top level folder unchecked and then manually check/select each sub-folder.  If a folder contains 100 folders/files and you only wish to NOT backup 3 of them (temp, logs, cache)...you are left having to manually select 97 separate folders/files (one at a time).

On first run we would want it to...

a) ONLY scan the selected/included folders/files and NOT to scan the entire "backup from" root of source drive.  Otherwise, Bvckup may wind up having to scan an entire 5TB drive each time just to check on a few GB worth of folders/files.  The only alternative left not to scan the entire 5TB is to create 15 separate backup sets/modified clones (one for each folder we wish to backup from the root of C:).

b) ONLY folders/files that were specifically selected for inclusion, in the "What to backup" -> "Start with an empty list" -> "Then, include the following items" list should be considered for DELETION, Archiving, or comparison.

c) And when we do set the "Deleting" option to "Keep backup copies"...it would be less terrifying if the Bvckup logs did not show that it is actually deleting thousands of folder/files and then we have to right click one of those lines, "Copy line to clipboard", and then scroll all the way over to the end of the path (in Notepad) to see the words "not executed"- at which time we're finally able to exhale.

I don't know if there are already ways around these issues that we are just not aware of or if these are feature requests...but we would love to find out if there is something that can be done to improve the situation in these areas.

Alex Pankratov :

Mar 12, 2017



The short answer is that you will need to set up matching set of exclusion/inclusion rules for destination location -
https://bvckup2.com/support/forum/topic/502/2959

---

The moderately-sized answer - the app works by obtaining a file tree for the source and a file tree for destination, comparing them and compiling a list of basic steps (e.g. "create folder", "delete file", "update file", etc.) that, when executed in order, will make destination tree look like the source tree.

The exclusion/inclusion rules are applied during the scanning step and they control which items from the disk actually get into the trees that the backup planning module sees. Each tree has its own set of rules.

The rules you see in the UI (under Backup What section) are for the source. The destination rule set is simply "include *" by default. It too can be viewed/edited through the UI, but this requires jumping through a couple of hoops as per above link.

So if you are after backing up a set of folders into a populated folder, then you need to modify the destination rule set to match the one for the source. This will make sure that the app ignores all other items at destination and leave them alone.

---

Generally speaking there are two modes of operation for a backup program. It can treat its backup directory as an exclusive resource or as a shared one. Here's a 4 year old topic from about half-way through the beta - https://bvckup2.com/support/forum/topic/120

Each approach has its pluses and minuses. The principal difference is that the shared mode is far more error prone, especially with simpler users. So back then the decision was made to go with an exclusive mode.

---

There *is* a warning that is displayed when trying to use a non-empty location for backup - http://bvckup2.com/wip/dst-warning.png

You must've clicked through it without reading or, perhaps, you didn't have your destination location accessible at the time when you created the backup (so the warning wasn't shown).

---

ONLY scan the selected/included folders/files and NOT to scan the entire "backup from" root of source drive.


That's how it works right now. If you are to exclude, say, C:\Windows, then the scanning module will not descent into Windows, but ignore it altogether.

But it still does start from the top of the source location, so if your source is C:\ and you only included C:\Foo\Bar\file.ext, then the app will end up requesting a listing of C:\, C:\Foo and C:\Foo\Bar

---

And when we do set the "Deleting" option to "Keep backup copies"...it would be less terrifying if the Bvckup logs did not show that it is actually deleting thousands of folder/file


Ah, excellent point re: long lines! Will rework this part.

Though "not executed" line are already shown in gray color.


ScottD-KMHA :

Mar 13, 2017

Great.  I'll test out the Destination level filters to see if that does the trick.  

Regarding my comment "ONLY scan the selected/included folders/files and NOT to scan the entire backup from root of source drive."

What I mean is that if I have an F: drive that has 5TB in used space and I create two sets, one simply to "Backup from" the "F:\" and the other to "Backup from" a 25MB "F:\images\" folder...the first set (F:\) will end up with a scan size of 5TB and the 2nd set (F:\images\) will end up with a scan size of only 25MB.  This is expected.

Now, for the sake of example, if I create a third set to "Backup from" the "F:\" and then also configure the "What to backup" to "Selected files and folders" ("Start with am empty list") with only the "F:\images\" folder in the "Then, include the following items" list...I would expect to end up with a scan size of 25MB and not the entire drive's 5TB, but that's not the case.  Instead of just scanning the files I have specifically included, it scans the entire drive each time it has to scan.

Will the destination filtering address this as well?

Alex Pankratov :

Mar 13, 2017

I would expect to end up with a scan size of 25MB and not the entire drive's 5TB, but that's not the case.


It should be the case. It will list all root level items for F:\, ignore all by \images and then descend and scan just this folder.

How did you arrive at the "not the case" conclusion?

It may scan folders that aren't explicitly included, but this only happens if you have custom *wildcard* filters specified (in the bottom pane of the same configuration window).

ScottD-KMHA :

Mar 13, 2017

The phenomenon I describe seems to only occur when you make the "Backup to" value the root of drive (such as F:\).  If I point the "Backup to" at a sub-folder of the root (such as F:\Bvckup_TEST1\) , the behavior is as one would expect and only the sub-folder (C:\Bvckup_TEST1\ to F:\Bvckup_TEST1) contents are scanned.  But, if one does as follows...

Backup from = C:\
Backup to = F:\
What to backup = C:\Bvckup_TEST1\
Deleting = Don't delete anything at destination

It scans the entire F:\ drive, instead of just the "C:\Bvckup_TEST1\" to "F:\Bvckup_TEST1" folder/contents.  If you happen to forget to set the "Deleting" option to "Don't delete"...any folder that does not exist on the root of C: drive will be deleted from the root of F: drive.  If you're going to test this in-house there, please make sure you don't have anything you care about on the root of the "Backup to" drive (outside of the "include" list), because it will be deleted.

Again, if I do it as follows it works fine...

Backup from = C:\Bvckup_TEST1\
Backup to = F:\Bvckup_TEST1\
What to backup = C:\Bvckup_TEST1\

It only scans and deletes items that are within the "\Bvckup_TEST1\" folder.

I also noticed, if I do it as follows it works fine...

Backup from = C:\
Backup to = F:\Bvckup_TEST1\
What to backup = C:\Bvckup_TEST1\

It only scans the "\Bvckup_TEST1\" contents...though it now creates a "F:\Bvckup_TEST1\Bvckup_TEST1\", which at least makes sense.

Yet, as soon as I set the "Backup to" one more level up to the root of "F:\"...it scans the entire F:\ drive rather than just the"What to backup" "\Bvckup_TEST1\" folder.

Lastly, I did test the new destination filter feature and got the same behavior still...

Backup from = C:\
Backup to = F:\
What to backup (source) = C:\Bvckup_TEST1\
What to backup (destination) = F:\Bvckup_TEST1\
Deleting = Don't delete anything at destination

The entire F drive is scanned.  I was too afraid to test turning delete back on, as I need the data on the root of F and didn't want to risk deleting it...but I assume because it is scanning the entire drive, it will also be deleting based on that scan as well.

ScottD-KMHA :

Mar 13, 2017

Correction on that last statement, I tested again from scratch and found your new feature for destination filtering does address the issue of scanning the entire drive...

Backup from = C:\
Backup to = F:\
What to backup (source) = C:\Bvckup_TEST1\
What to backup (destination) = F:\Bvckup_TEST1\
Deleting = Don't delete anything at destination

Only the \Bvckup_TEST1\ is scanned.  Works great.  I'll have to find another drive to test the delete part, but will let you know.  Looks promising.

ScottD-KMHA :

Mar 13, 2017

I just confirmed the Destination filtering feature also addresses the delete all files on root issue as well.  Thank you.

The only bummer about it is that one has to either first manually create each folder/file on the destination (to match the source) in order to even use this destination filtering feature (or else the folders/files are not there to select for inclusion) or one has to allow the application to perform the full scan of the entire root drive (as formerly described) the first time around (just to get all desired files on the destination for inclusion selection) and then setup the destination filters as well as enable delete only after that has been done.  From that point on, it should work.

What would be also great is if you could release a version that has the "edit_dst_filters 1" line already built-in to the "bvckup2-ui.ini", so that the Ctrl + What to backup -> Details button feature is native to the application- for reaching the hidden Destination filter box.

I'm putting together instructions, so that other technicians can install this program into our wide ranging customer sites and there are going to be so many caveats I'll have to include into the instructions based on the OS involved and whether we are running Bvckup 2 as an app or service...I prefer to keep the instructions more simple and not having to manually find/update this file would help.

Currently, on a Windows 7 box, I find it in...

C:\Users\Administrator\AppData\Local\Bvckup2\ui\bvckup2-ui.ini

But I'm sure that will be different in the various OS environments we work with (from XP to 2012 Server), as well as if we convert the app to a service in some.

Alex Pankratov :

Mar 13, 2017

Scott, thanks for the follow-up and the comments.

In reverse order.

Re: the UI ini. You can always find it at -

        %LocalAppData%\Bvckup2\ui\bvckup2-ui.ini

unless it's an XP, which I suspect you no longer have in your rotation.

Re: usability issues -

1. What you can do is to put "edit_dst_filters 1" into bvckup2-ui-override.ini file and drop it into %LocalAppData%\Bvckup2\ui folder. Moreover, you can automate this process using scripted installation mode as per - https://bvckup2.com/support/forum/topic/801

2. I think it makes sense to add an option of initializing destination-side ruleset from the source ruleset. Or, even simpler, have an option of the former to always be the same as the latter. This should cut down configuration time by exactly half and it would also eliminate the need to tinker with destination filters altogether. I will try and get this into the next minor update (77.2)


ScottD-KMHA :

Mar 13, 2017

Sounds great.  Very helpful.  Thank you.

ScottD-KMHA :

Mar 13, 2017

Just a side note %LocalAppData%\Bvckup2\ui does not work for Windows Server 2003 either.  It is located under...

%HomePath%\Local Settings\Application Data\Bvckup2\ui\bvckup2-ui.ini

Yes, Microsoft no longer supports this OS...but we do have many customers who still have it on their production and backup server.  Unfortunately.

In the end, this just means I'll just have to add a total of two possible paths into my instructions, so not a big deal.

Alex Pankratov :

Mar 14, 2017



Ah, true. W2003 is the same as XP, it doesn't have %LocalAppData% envvar.

an option of initializing destination-side ruleset from the source ruleset


This is done now, shipped as a part of R77.2 just now -
https://bvckup2.com/support/forum/topic/900/4929

ScottD-KMHA :

Mar 14, 2017

Thank you.  Just to clarify...does this mean I no longer need to hold down the Ctrl + What to backup -> Details button  in order to configure the destination rules if the source and destination are intended to be the same?  If so, should I remove the "edit_dst_filters 1" line from the "bvckup2-ui.ini"...after adding the new "conf.dst.filters.as_source   1" line?  

I tested with R77.2 just adding this new line to the ini, but it still scans the entire destination drive when I don't perform the Ctrl + Details button destination configuration changes.  I also confirmed (when delete enabled) it deletes all data on the destination drive, outside of the folder I selected to include using the source detail settings alone.

Was this meant to address that issue or did I misunderstand the purpose of the new feature?

Alex Pankratov :

Mar 14, 2017



Make sure to exit the app (or stop the bvckup2 service when running in service mode) before making any changes to any of the INI files.

INI files are read on app's launch and recreated from scratch on app's exit. Any changes you make to them while the app's running will be ignored and lost.

does this mean I no longer need to hold down...


Correct.

You configure what needs to be included for the source location, then pop open the INI, exit the app, change "conf.dst.filters.as_source" to 1, save INI changes, start bvckup2 and that's it.

PS. You can use "Simulate" option from the right-click menu for the job's entry in main list to do a dry-run. The job will just go through scanning and planning phases, dump the plan into the log and stop.

ScottD-KMHA :

Mar 14, 2017

Thank you for the Simulate advice.  I did test again.  I closed Bvckup 2 from the Sys Tray, confirmed it is not a service, and ensured it was not still running as a process (via Task Manager).

I then opened "C:\Users\Administrator\AppData\Local\Bvckup2\ui\bvckup2-ui.ini", added the "conf.dst.filters.as_source   1" line, and closed/saved the file.

I then started the Bvckup 2 application, waited 10 seconds, and then closed it from the Sys Tray again.  I then checked the contents of the "C:\Users\Administrator\AppData\Local\Bvckup2\ui\bvckup2-ui.ini" file and found the "conf.dst.filters.as_source   1" line is nowhere in any line of the file.

It appears that Bvckup 2 completely removes it upon closing the application and doesn't seem to make use of it while the application is running.

I also tried to do it in the exact manner you suggested, just in case...

1) configure what needs to be included for the source location.
2) pop open the INI.
3) exit the app.
4) change "conf.dst.filters.as_source" to 1
5) save INI changes
6) start bvckup2.

I had trouble between step 1 and 2, as I was still in the "New backup" window and needed to click the "Create" button before my configurations would even save.  But, if I do so, it will start scanning immediately and defeat the purpose of the feature in the first place.  I guess I could kill the scan, open the INI (while bvckup2 is still running) and then exit the app?  So, I tried that approach (disabling the set), added the conf.dst.filters.as_source   1, saved the file (leaving the file open), and launched Bvckup2.  I then clicked Go button and found it still scans the entire root drive still, instead of just the folder select for inclusion.

I know there were some obvious things I should have read between the lines and assumed in the steps you gave me but I wanted my last test, before I replied, to be as literal as possible- just in case.

ScottD-KMHA :

Mar 14, 2017

Just to be clear, I did try the following as well...

I closed Bvckup 2 from the Sys Tray, confirmed it is not a service, and ensured it was not still running as a process (via Task Manager).

I then opened "C:\Users\Administrator\AppData\Local\Bvckup2\ui\bvckup2-ui.ini", added the "conf.dst.filters.as_source   1" line, and closed/saved the file.

I then started the Bvckup 2 application, created a new set as follows...

Backup from = C:\
Backup to = H:\
What to backup = Start with empty list -> include -> C:\Bvckup_TEST1\

It still scanned the entire H:\ drive, rather than just the \Bvckup_TEST1\ folder.  And the conf.dst.filters.as_source   1 line disappears from the ini as soon as I close Bvckup2 program.

ScottD-KMHA :

Mar 14, 2017

Also, could you possibly add some kind of visual indicator (in the UI) when this mode is successfully enabled in the app?  We would like to turn the delete option on, but that can be very dangerous if we are not SURE that this "conf.dst.filters.as_source" mode is successfully enabled in the app.

I could see someone forgetting to kill the app first and their ini changes getting blown away or not properly applied.  If I can just be able to say to the techs, in my instructions, "if you don't see such and such visual indicator in the UI...you've done something wrong and DO NOT proceed"...that would save us from many potential meltdowns.

Alex Pankratov :

Mar 15, 2017

I then opened "C:\Users\Administrator\AppData\Local\Bvckup2\ui\bvckup2-ui.ini"


It's the wrong .ini, that's why it didn't work for you.

bvckup2-ui.ini is for app-wide UI settings. There's bvckup2-engine.ini for the engine settings and there are also settings.ini for each backup job, which is where the actual job config is stored.

The src/dst filters are per-job settings so the "conf.dst.filters.as_source" line needs to go into the job's settings.ini. This will be in \Bvckup2\engine\backup-00xx folder, but it's easier to find it by right-clicking on the job and selecting Open Folder > Config & Logging.

Here's a general description of all the INI files in app's configuration -
https://bvckup2.com/support/forum/topic/480

Also, could you possibly add some kind of visual indicator (in the UI) when this mode is successfully enabled in the app?


It's a fair suggestion, but since it's a job-specific override and because there's a LOT of job overrides in general, these can't be shown in the UI directly.

What's done instead is that these are reflected in the backup log. For example, this particular override shows up as follows:

2017.03.14 21:57:19.978 (UTC+1) 2 1     Preparing ...
...
2017.03.14 21:57:19.978 (UTC+1) 2 2         Verifying configuration ...
...
2017.03.14 21:57:19.978 (UTC+1) 2 3             Note: using source scan selectors for destination scan

(sorry about the wrap)

ScottD-KMHA :

Mar 15, 2017

Ah, I see.  That worked for me.  Thank you for those additional resources as well.

So given that I have to create the set before I can make the conf.dst.filters.as_source adjustment, and upon creating the set it automatically starts the scanning of the entire 5TB (root directory level, rather than the single 25MB sub-folder I'm using in my current example)...I have no choice but to either let it scan the entire 5TB the first time around or to just quickly Stop/Disable the set (clicking Stop button 2 or 3 times) before it can really get going with the scanning of the source.  Then I can...

1) right click -> Open folder -> Configuration and logging and then close the Bvckup2 application from the Sys Tray.

2) change the already existing conf.dst.filters.as_source value from "0" to "1" in the "C:\Users\Administrator\AppData\Local\Bvckup2\engine\backup-0001\settings.ini" file, save, and close the file.

3) open the Bvckup2 application and click the Go button on the set in question.

That workflow has done the trick.  The root folders/files are no longer being scanned or deleted.  Again, I'm trying to make these steps as dublicatable and least likely for mishaps as possible...so, are there any steps above that I'm doing extra when there is a shorter way of doing the same?

I also wonder, is it hypothetically possible to create a batch file or utilize the "scripted installation mode" you mentioned earlier to...

1) Custom install the application so that it already has two backup sets in place (backup-0001 and backup-0002) with all configurations pre-populated?

2) conf.dst.filters.as_source value will be auto set to "1" in the "\Bvckup2\engine\backup-0001\settings.ini" file?

3) Both backup sets will be Disabled from the get-go, so that the user will have to click Go button to start them for the first time?

If this is possible, it could make these installs a breeze for us.

ScottD-KMHA :

Mar 15, 2017

While waiting, I did some testing to see if I can simply copy the "\Bvckup2\engine\backup-0001" and "\Bvckup2\engine\backup-0002" folders into the proper place and the backup sets show up into Bvckup2 UI as they were...and it seems to work.

So, is it safe to assume I can create the sets just as we need them and then copy these "\Bvckup2\engine\backup-0001" and "\Bvckup2\engine\backup-0002" folders into any new machine we install Bvckup2 on...as long as it is the same version?

If so, can we get away with just copying the entire "\Bvckup2\" folder?  If not, can we get away with copying the entire "\Bvckup2\engine" and/or ""\Bvckup2\ui" folders?  Or, would it only be safe to copy the "\Bvckup2\engine\backup-0001" and "\Bvckup2\engine\backup-0002" folder level?

ScottD-KMHA :

Mar 15, 2017

Also, I noticed that the "scripted installation mode" allows you to install the configuration files to a specific location (rather than the default).  Does this include the "engine" and "ui" folders?  In other words, instead of having these folders located in "C:\Users\Administrator\AppData\Local\Bvckup2", can I instead tell Bvckup2 I want these files to be stored and referenced from "F:\Bvckup2" instead?

The reason I ask is I noticed some of the files in the sub-folders of the "engine" folder can get pretty big (currently looking at one that is 24GB), so I can see the potential of some of our smaller C drives filling up with these files.  If I can have these files store on our larger F:\ drives, it won't present us this problem going forward.

ScottD-KMHA :

Mar 15, 2017

Alright, so I've confirmed that I'm able to copy the entire "C:\Users\Administrator\AppData\Local\Bvckup2\" folder from one computer to another computer, both using the same Bvckup2 v1.77.2.0, and this has allowed me to have all of the backup set pre-created just as we want them and with the new "conf.dst.filters.as_source" feature enabled as well as all sets disabled by default.

I know we'll need to always make sure the same version is on the new machine that were on the original machine the files/configurations were created on, as well as we will have to place the "\Bvckup2\" folder into the correct location depending on the OS involved...but do you see any other drawbacks to this approach that could come back to bite us?

Alex Pankratov :

Mar 16, 2017

Again, I'm trying to make these steps as dublicatable and least likely for mishaps as possible...so, are there any steps above that I'm doing extra when there is a shorter way of doing the same?


What you can do is to use an app-wide job template to override "conf.dst.filters.as_source" for all new jobs. This is done by creating a file called

        backup-override.ini

placing it into

        %LocalAppData%\Bvckup2\ui\

and adding a single line to it

        conf.dst.filters.as_source  1

Then, when you create a new job, it will be initialized with the defaults first and then adjusted using whatever settings you have in -override.ini

---

Furthermore, you can use the same mechanism to *augment* the source filter list for all new jobs. If you'd rather *replace* the source filters, you'd need to use backup-template.ini file instead.

See the last section in this post for details - https://bvckup2.com/support/forum/topic/480/2839

Alex Pankratov :

Mar 16, 2017

I also wonder, is it hypothetically possible to create a batch file or utilize the "scripted installation mode" you mentioned earlier to...


Yep, perfectly doable.

I wasn't sure if your source incl/excl rules are machine-specific, so I didn't suggest it, but if they are, then this will indeed be the simplest option.

So, is it safe to assume I can create the sets just as we need them and then copy these "\Bvckup2\engine\backup-0001" and "\Bvckup2\engine\backup-0002" folders into any new machine we install Bvckup2 on...as long as it is the same version?


Yes, it is.

Also, I noticed that the "scripted installation mode" allows you to install the configuration files to a specific location (rather than the default).  Does this include the "engine" and "ui" folders?


No, it actually doesn't. It allows installing *program* files where you want, but the configuration template is always copied into default config folder.

In other words, instead of having these folders located in "C:\Users\Administrator\AppData\Local\Bvckup2", can I instead tell Bvckup2 I want these files to be stored and referenced from "F:\Bvckup2" instead?


You can still do that if you are scripting the installation, but it just will need to be done from the .bat file after the installer completes (using something banal like "xcopy /s").

If you are to place config in a non-default location, then you will need to make bvckup2 aware of that. There are several ways to do that - from specifying the config folder path in command line, to dropping a redirect.ini file into default configuration folder.

See this post for details - https://bvckup2.com/support/forum/topic/480/2837

The reason I ask is I noticed some of the files in the sub-folders of the "engine" folder can get pretty big (currently looking at one that is 24GB)


To comment on this - the disk usage comes from delta copying. Basically, for every file that is delta-copied, the app saves its block hashes locally. The usage comes to about 0.06% of the original file sizes, but for larger backups these "peanuts" can really add up.

It might make sense to evaluate if you will in fact benefit from delta copying and if you could perhaps exclude some files from being processed using this method. There is a way to do that, see this post for details - https://bvckup2.com/support/forum/topic/739

Alex Pankratov :

Mar 16, 2017

Do you see any other drawbacks to this approach that could come back to bite us?


Looks OK to me. The configuration is portable by design.

The only bit of configuration that is machine-specific is the device tracking data. That is, when a backup is bound to a specific removable device and won't run unless that device is present - https://bvckup2.com/wip/09102014

I don't think you have that in your setup, so you should be OK.

ScottD-KMHA :

Apr 04, 2017

This has worked great for us so far, except one issue. It's not a big deal, but if you can let us know how to fix it...that would be less confusing to some of our installers.  When I copy the entire preset %LocalAppData%\ Bvckup2\, on some computers the position of the Bvckup2 window initially is almost all the way off the screen and we then have to pull it to the center.  

I assume this has to do with the coordinates/resolution in use on the machine this preset was first created with.  Is there a config I can modify somewhere that would make the preset more likely to present the window front and center when we initially open Bvckup2...or is this just something we will have to deal with given the method we are using?

Alex Pankratov :

Apr 04, 2017



Yep. Good guess. This is due to "window_pos" entry in Bvckup2\ui\bvckup2-ui.ini. Just remove it and the program should default to showing window in the center of primary desktop. May also want to remove "window_state" just in case.

ScottD-KMHA :

Apr 05, 2017

That worked.  Thank you.

ScottD-KMHA :

Apr 14, 2017

Hi.  Does "edit_dst_filters 1" get removed from the "%LocalAppData%\Bvckup2\ui\bvckup2-ui.ini" when Bvckup2 is auto upgraded from v77.2 to v77.3?  I just came across a machine that was updated and I could not find "edit_dst_filters 1" in "%LocalAppData%\Bvckup2\ui\bvckup2-ui.ini" anymore.  Another factor may be that I switched Bvckup mode from application to service and then back to application before I checked for this value.

So, could the update to v77.3 or the change from app to service to app wipe that value?

Alex Pankratov :

Apr 15, 2017

No, neither should cause this. But editing an INI while the engine is running (as a part of a desktop process or as a service) *will* cause this.

ScottD-KMHA :

Apr 27, 2017

Follow-up on this one.  Just to understand...if I am using v77.2:

1) My placing "conf.dst.filters.as_source  1" into "%LocalAppData%\Bvckup2\ui\backup-override.ini" will make it so that every new set created will automatically have "conf.dst.filters.as_source   1" in its "%LocalAppData%\Bvckup2\engine\backup-00xx\settings.ini"?

2) And for existing sets, does my already having "conf.dst.filters.as_source   1" in its "%LocalAppData%\Bvckup2\engine\backup-00xx\settings.ini" make it so that there is no need to place "edit_dst_filters 1" into "%LocalAppData%\Bvckup2\ui\bvckup2-ui-override.ini" or "%LocalAppData%\Bvckup2\ui\bvckup2-ui.ini"?

I'm trying to figure out why I'm seeing all of my folders/files deleted or archived at the root of the F:\ drive when Y:\ is my source location and F:\ is the destination.  I confirmed that "conf.dst.filters.as_source   1" is still in its "%LocalAppData%\Bvckup2\engine\backup-00xx\settings.ini" for the set in question...yet, it's still deleting everything in the destination that does not exist in the source.  Mind you, it is not doing this in most of our installations...so I'm thinking I must be doing something wrong in these few to trigger this.

Here are some differences with these that may be causing it.  Please let me know if you think something might be missed here...

Might right clicking and choosing to "Clone" a set make the cloned set somehow malfunction in this way?

So far I've definitely been able to verify that the following does cause the issue to occur...

Ctrl+GO button on existing set.

Once I clicked Ctrl+GO button for the set, it scanned and then started preparing files for archiving that were not selected to be in the scope of this set, but that were on the same root level of destination F:\ that did not exist in source Y:\.

Is there a way I can fix this or is this perhaps hole that needs to be patched in the application?

As it stands now, I've had to keep the set disabled because as soon as I enable it (in this condition) it will archive TB's of folders at the root of F:\ that I don't want it to and that are outside of the set's selection anyway.

Alex Pankratov :

Apr 28, 2017



Re 1) - Yes, correct.
Re 2) - correct as well.

Just keep in mind that "as_source" is a backup job setting, whereby "edit_dst_filters" is a job-agnostic UI setting. Moreover, editing destination filters for a job that has "as_source" enabled is allowed, but it will have no effect on the job, because destination filters will be re-initialized to match the source ones at run-time.

I confirmed that "conf.dst.filters.as_source   1" is still in its "%LocalAppData%\Bvckup2\engine\backup-00xx\settings.ini" for the set in question


Check the backup log. You should see a note there that the as_source is being used:

    2017.04.28 10:20:40.462      Preparing ...
    2017.04.28 10:20:40.462          Verifying configuration ...
    2017.04.28 10:20:40.463              Note: using source scan selectors for     destination scan
    2017.04.28 10:20:40.463                  As per "conf.dst.filters.as_source" override in settings.ini

If you don't see the note, then double-check the program version and check that it was restarted after the change was made.

If you do see the note, the check the exclusion/inclusion rules. There's probably a wildcard "include" involved, so it ends up pulling in all of F:\

It might be an explicit "include *" rule or it can be an implicit one if the rule set is of the "Include everything by default" kind.

Might right clicking and choosing to "Clone" a set make the cloned set somehow malfunction in this way?


No, this is certainly not the cause.

Is there a way I can fix this or is this perhaps hole that needs to be patched in the application?


I am guessing that it's either due to the app being at an older version OR some sort of mix-up with the settings.ini for the job - if you have the INI open in an editor, close all instances, then shut down Bvckup 2 (including stopping the bvckup2 service if it's in service mode), *then* open the INI and check "as_source" settings. It should be at 0 or it should be missing.

Internally this is an exceedingly simple feature - when a backup is being prepared for a run, the app first initializes source-side filters and then it looks at whether 'as_source' is set. If it is, then destination filters are initialized by simply copying them from source filters. If it's not set, then they are initialized from the specified rules.

ScottD-KMHA :

Apr 28, 2017

I think you've led me to water.  So, it turned out to be an unforeseen negative consequence of me trying to use what I call my "preset" by copying a pre-configured Bvckup2 folder into the %LocalAppData%\ just after installation.  Well, my pre-configured Bvckup2 folder contains "Included directories" for the source/destination for all possible scenarios we cover.  Problem is, I was trying to utilize this server to backup two different production server types into one backup server and so though one set shows only the folders that actually exist on the source server, in the background configuration of that set are the other possible folders included that only exist on the other source server.  

It's hard to explain, but I think we've got it figured out.  It's a one-off that I don't think well run into again anyway.

I was able to remove the unwanted "conf.filters.1.folders" from the settings.ini of the set.  I think that should do it.

ScottD-KMHA :

May 08, 2017

Ok, I think most of it got cleared up...but I'm still struggling with one directory that is mysteriously being moved to the "C:\$Archive (Bvckup 2)\" folder.  

I confirmed conf.dst.filters.as_source enabled...

"2017.05.08 10:53:05.801 (UTC-5) 3 4                 As per "conf.dst.filters.as_source" override in settings.ini"

I confirmed this C:\KMMI Data Management" folder is not in the "Include:" lists for source or destination...

"2017.05.08 10:53:05.802 (UTC-5) 2 2         Filtering rules
2017.05.08 10:53:05.802 (UTC-5) 3 3             Source
2017.05.08 10:53:05.803 (UTC-5) 3 4                 Exclude everything by default
2017.05.08 10:53:05.803 (UTC-5) 3 4                 Included directories
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\cert
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\inetpub\wwwroot\EXA_Cardio
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\cfg
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\dictation
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\document-scan
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\PACS\trans\bin
2017.05.08 10:53:05.803 (UTC-5) 3 3             Destination
2017.05.08 10:53:05.803 (UTC-5) 3 4                 Exclude everything by default
2017.05.08 10:53:05.803 (UTC-5) 3 4                 Included directories
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\cert
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\inetpub\wwwroot\EXA_Cardio
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\cfg
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\dictation
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\exa\document-scan
2017.05.08 10:53:05.803 (UTC-5) 3 5                     .\viztek\PACS\trans\bin"

Yet, it then goes into "Running archive maintenance" mode...
"2017.05.08 10:53:05.825 (UTC-5) 3 3             Location: C:\$Archive (Bvckup 2)\
Processing...
1. Archiving folder KMMI Data Management"

And then there it is.  KMMI Data Management folder is being moved to the "C:\$Archive (Bvckup 2)\" folder.  The only thing I can think of that I did differently with this one is that I did originally have this "C:\KMMI Data Management" folder contained in the Includes of this set in Bvckup2, but I then later removed it via Adjust Backup Settings -> What to Backup -> Details button -> unchecked KMMI Data Management folder from the "Then, include the following items" section.  Ever since I did that, it will eventually actually remove this KMMI Data Management folder (during its daily runs) from the destinations hard drive (C:\KMMI Data Management) and will move it to "C:\$Archive (Bvckup 2)\" folder...even though this folder actually exists both on the source and destination.  

It is as if Bvckup2 got confused when I removed this folder from the include area of the set and now always thinks it needs to remove it from the C:\ (even though it exists on the source/destination's C:\).  Given that it does exist on the source, I know its not a scenario where it is keeping it in the includes.  There is something else afoot here that Bvckup2 is seeing as a reason to move this folder to the "C:\$Archive (Bvckup 2)\" folder.  Did my removing it, after the fact, from the set cause it to be stuck in some kind of state where it thinks it needs to remove it from the drive?

Alex Pankratov :

May 09, 2017



Hmm. Check in Preparing section, see if it says

        Scanning destination...

or

        Loading destination snapshot...

Destination filters are applied only during the scanning phase. If your job is set to cache destination file tree (aka the "use destination snapshots" option in Backup Settings > Detecting Changes), then the app may in fact end up seeing some items that would've been otherwise excluded by the filters. If these filters were added after the snapshot was created.

That said, you should not see this folder getting archived _repeatedly_. If my guess is right, then the app will remove this folder from the snapshot once it is archived.

ScottD-KMHA :

May 09, 2017

Currently the logs show, just after "Scanning Source", it goes to "Loading destination snapshot"...but these are the logs for after the event occurred.  The logs from when the event occurred last are no longer in the logs as the logs don't go back that far now.

It seems to only  happen when I put the folder back in "F:\KMMI Data Management" location and then utilize Ctrl+GO to have it re-scan from scratch.  The funny thing seems to be, it doesn't happen immediately on the next scan.  It seems to happen later that it finally decides to archive this folder.  Is there a certain time of day or number of hours or some other interval Bvckup2 uses to determine when it will decide to check if folders/files need to be archived?  I can't seem to get this phenomenon to occur in front of me live time, but if I check back the next day, I find the directory archived again.

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy