The support forum

Cloud Backups

BvcDer :

Jun 12, 2014

Are there any plans to allow direct back-ups to cloud storage?  I'm using Amazon Cloud Drive for a remote back-up, but I have to make the Cloud Drive folder on my PC the back-up destination and then the Cloud Drive app takes care of the uploading.  This is a little inconvenient since the "real" location of the files to be backed-up is not the Cloud Drive folder and the back-up amounts to about 20GB of data, so I have had to make a duplicate source folder inside the Cloud Drive folder and copy the 20GB of stuff into there.

Lurch :

Jun 13, 2014

I think that's more for Amazon to allow access to an API as a solution rather than for Alex to try and find a way to use all these random services that don't actually allow you to do this from anything.

Most of these solutions to problems that don't exist will just extra junk to this lightweight program so I wouldn't want to see them supported.

On a specific note to Amazon Cloud Drive I don't think the business model will allow for releasing an API to access it directly as it wasn't intended for everyone to use as a remote drive, it was designed as a way for Amazon to share music it sells in a convenient manner. If you want a decent cloud drive with remote storage access then you are looking at S3 and then adding one of the programs to your machine that can map a drive to it.

Alex Pankratov :

Jun 16, 2014

Adding an option for backing up to a remote/cloud location is on ToDo list, right next to the file versioning. However there are some high-level nuances that aren't yet clear how to handle.

Specifically - (1) data protection and (2) delta updates.

(1) simply means that there needs to be some form of data encryption happening before uploading and while this is relatively easy to implement (even in a sense of crypto framework being "kosher" and what not), the issue is how to keep it bvckup-agnostic. That is if Bvckup dies, breaks, goes away, etc. - there should be a way to recover all backed up data, manually.

(2) - I haven't looked at Amazon Cloud Drive, but S3 doesn't allow in-place updates of already stored files, and given highly distributed nature of all cloud services I suspect there are very few that do. This means that if I touch file A here, then A will either need to be re-uploaded in full or the diff should be uploaded as a separate file. And then we are back to the same question - how to restore from this multi-piece backup should bvckup become unavailable.

tl;dr - yes, there are plans, but there's some research work that needs to be done first. In any case, this is going to be in the V3 the earliest and this is few months away. Optimistically :)

BvcDer :

Jun 16, 2014

Thanks for the replies - I'll keep my fingers crossed!

davidm_uk :

Jul 10, 2015

It's great to know that you're considering backup to cloud services in a future release. Amazon Cloud Drive in particular, which offering very generous storage (e.g. unlimited photo storage, including raw files, for a relatively small fee and 'free' for Prime subscribers) has an abysmal 'upload only' Windows application. The ability to synchronize files to Amazon Cloud Drive using Bvckup - even if requiring whole-file upload on changes and without encryption - would be incredibly useful.

EJL :

Jul 14, 2015

I would love to see backup to Amazon Cloud Drive as well. There is now an API available https://developer.amazon.com/public/apis/experience/cloud-drive/content/restful-api
Even without encryption the ability to keep synced backups of large media collections on the cloud using Bvckup 2 would be most welcome.  Amazon Cloud Drive seems to have very robust upload speeds.
At least 2 desktop programs allow backup to Amazon Cloud Drive: Allway Sync and SyncBack Pro. I'd much prefer if I could use Bvckup 2 for this.

fabiogferreira :

Jul 28, 2015

[ trimmed ]

Alex Pankratov :

Jul 28, 2015

@fabiogferreira - I've removed your message. Please don't use Bvckup 2 forum to promote your own products. I've sent you an email to the same effect and I would appreciate if we keep this thread on topic from now on. Ty.

Andre :

Sep 28, 2015

Alex, have you considered doing like Arq (for Mac) to be bvckup-agnostic? http://www.haystacksoftware.com/arq/

The author of Arq have open sourced the backup format itself and made a simple command line utility for restoring available on Github and in that way makes sure that the end customers data is retrievable no matter what happens to Arq itself.
http://sreitshamer.github.io/arq_restore/

Maybe could you even use the same format? Arq supports encrypted backups to Amazon S3 with delta uploads.

Regards,
André

Alex Pankratov :

Sep 30, 2015

André,

Yup, I've seen Stefan's specs and it's the certainly one of better ways to handle encrypted storage. It's definitely an option. Piggy-backing on his format though... I don't know, seems a bit too cheeky for comfort :)

harrisc :

Jul 22, 2016

Amazon Drive is now unlimited.

Can I backup to Amazon Drive?

Chris

sunk818 :

Jul 26, 2016

Maybe I misunderstand, but couldn't you set the destination to a folder inside Amazon Cloud Drive then there is only one 20GB?

Another alternative may be WebDrive. I'm not associated with SouthRiverTech, but I've used their WebDrive product in the past to mount a FTP site as a drive. The Unix equivalent would be FUSE with a Amazon Cloud Drive plug-in. If you point Bvckup to the drive letter, you can in theory back up directly to the Amazon Cloud Drive. I used WebDrive in the past for mapping a FTP site and that worked fine for several hundred files within each folder. It didn't work well when file counts went past 1,000 files  (too much time spent caching & comparing). I've never tried it with Amazon Cloud Drive, but in theory the final destination would be Amazon Cloud Drive without any backups on the local drive.

aweber :

Aug 02, 2016

I would like to use cloud storage too. As referenced, Amazon has "unlimited" storage for $60USD per year.  They DO have an API, but not sure if it will suffice for bvckup.  I am considering trying a 3rd party product to interface like "ExpanDrive" (I do not work for them or own the sw yet).  I doubt their virtual device has rsync/delta capability.  I will ask them.  If I do want to use them in any case...maybe as a secondary backup...I can toggle the delta copying OFF in bvckup for that job, right?

aweber :

Aug 02, 2016

Update: As generally expected, the rsync/delta mechanism wouldn't really work.  I am still considering using it as a secondary location/offsite backup.

Can you please just verify/clarify the following:
1) Use destination snapshot: This will keep a listing of files, folders and mod dates on my local PC so bvckup does not have to manually scan the remote/destination each time?  I think this will be crucial.

2) Copy files in full: I guess this does what it says...it will overwrite older versions of files with a full copy function (bad for big files).

3) Keep backup copies: If the cloud drive is "unlimited", I don't see why not turn this on, right?  Do locally deleted files still get moved to the Archive folder?

Thanks,
AJ

Alex Pankratov :

Aug 02, 2016



I read through the ACD API specs a couple of months ago and back then they had no support for selective file updates. This doesn't mean that incremental updates aren't possible to implement, but only if we store these updates as separate binary deltas. Meaning in turn that there will be "some assembly required" when trying to retrieve these files. That's not exactly elegant, nor does it fit into how Bvckup 2 works at the moment. So there's a bit more research required.

1) Correct.

2) Correct (correct).

3) With unlimited backup storage you would still probably want to set Deleting to Archive, but then disable age-based archive trimming (it's behind "Edit details" link next to the Archive option).

"Keep" option effectively suppresses all delete operations on the backup side. "Archive" replaces them with "move to $archive", and "Delete" just executes them. That's all there's to this option.

aweber :

Aug 02, 2016

Thanks for the feedback.

Have you thought of adding a "Priority" value to the backup jobs?  In addition, an "interruptable" option.  If a higher-priority job needs to run, it can interrupt a lower priority job.

Of course, you'd need a way to "resume" that lower priority job later, but if you kept the local snapshot continually updated during the run, I guess it wouldn't be too hard to implement.  (And you'd have to figure out what to do with Shadow Copies when you interrupt...maybe just release it.)

That job would only be consistent to the last full run time, which is not unlike any other job.

Alex Pankratov :

Aug 02, 2016



To be honest, this looks too complicated to be comfortable.

There is already support for running jobs in parallel if needed and we will be adding support for multiple job queues shortly (so that jobs in each queue are executed one by one, but queues themselves execute in parallel).

Also, there's also pre/post-backup commands and command-line control, so you can put them together and have a high-priority job suspend other jobs with its pre-script and resume them from its post-script.

So between these two options I don't think it makes sense to add any more smarts to the scheduler.

EJL :

Oct 26, 2016

Is there any further progress on backup capability to Amazon Cloud Drive? I would love to see this functionality, even if at the moment their API doesn't support the delta copying that helps make Bvckup so powerful.

STBOUK :

Jun 16, 2017

It was a joy to find this thread when searching to see if it existed before starting my own.  

Firstly; yes, I use Bvckup 2 to enable a multi-tiered backup strategy that as one element includes physically taking a NAS off-site once a month and replacing it with its sister before repeating the next month and so on.

I too an am Amazon Prime customer and want to take advantage of the unlimited photo storage in Amazon Drive that I am entitled to to at least get my huge photo collection (raw and jpg) duplicated off site in near real time.  I currently have to do this by using Bvckup 2 to selectively copy the photo files from the source folders (leaving behind all the *.mpg and *.mp4 files since not being photos they fall outside my free allocation) in to the Amazon Drive monitored folder on my PC - which takes up 1.2TB of storage that I prefer not to consume.

As a huge fan of Bvckup 2 I hope a Cloud backup feature that includes Amazon Cloud is now imminent. :)

New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy