company location

Get In Touch

Why not join us on Facebook? We only post about useful things like new releases, and unlike some others we promise we won't spam you with messages!

           MLtek Limited on Facebook       

 

 

ArchiverFS - How To Archive Old Files To The Cloud.

Product Home | Pricing | Download | Documentation | Release Notes | Customers

How to setup ArchiverFS to archive old files to the cloud

Before starting::

ArchiverFS is capable of archiving old files to most cloud storage offerings.of integrating with most cloud storage offerings. It can migrate old files to any cloud storage that can be accessed via a UNC path (e.g. \\Host\Share\ ). This includes offerings from Oracle, Amazon and others.

Before deciding to setup cloud archiving there are a couple of items that have to be considered:

- How much data do you want to migrate?

- How fast is your internet connectivity??

- How large are the largest files that users may want to access via seamless links?

All three of these items are crucial to the performance of the cloud archiving system. First files will need to be migrated to the second tier archive storage across the local internet connection, and then when users want to access those files they will need to be recalled over it.

If you are able to migrate everything that hasn't been used in 3 years to second line storage, then as a guide we recommend you allow for 75% of your total file system to be moved. Obviously the exact percentage will vary, and you can always turn up or down the age settings as required..

First we recommend that you calculate how long it will take to migrate 75% of your file system to cloud storage over the local internet connection. This is a simple calculation, but remember not to mix up MB's (MegaBytes) and Mb's (MegaBits). File system size will normally be calculated in MB's, and line speed will normally be calculated in Mb's. Divide Mb's by 8 to convert to MB's.

Next you should establish what the largest size file is that your users will typically need to access, and work out how long it will take a user to open that size file of file over the internet connection. You can set size limits on files to be migrated, which may help if there are a small number of files that are significantly larger than the average.

 If your files are particularly large, if the internet connection isn't fast enough or if it will take a really long time to migrate all your files to cloud storage then you may be better off using stand alone storage on site like a NAS device.

Archiving over a VPN:

Once you have your cloud VM up and running with a Windows OS you can setup your site-to-site VPN. Almost all cloud providers will let you configure a site-to-site VPN between your cloud provision and your on-site infrastructure. This VPN is crucial, it will allow connectivity between your cloud service and your internal infrastructure, it is across this VPN that files will be transferred and retrieved.

When configuring outbound connections for your VPN you need to ensure that the server ArchiverFS will run on has access to it, and also ensure that all of your workstations can access it. When a workstation click on a link left behind by ArchiverFS all of the IO goes directly from the workstation to the second line storage, it doesn't go via the ArchiverFS server.

Then when configuring the inbound connections you need to ensure the cloud archive server can connect to at least one domain controller on your network. The cloud server will need to be a member of your domain if you want to maintain permissions (we are assuming you do).

Once you have setup your VPN and configured any firewall rules you should be able to join your cloud server to your domain as a member server. Next, create a test share on your archive server and ensure both the ArchiverFS server and your users can access the new test share over the VPN..

Once you have configured and tested your cloud server and VPN you should work through the User Guide or the Quick Setup Guide to complete the setup.

Archiving to Amazon Glacier

Amazon Glacier offers almost unparalled per GB storage costs, they are roughly on par with a locally deployed NAS device (depends heavily on the specific device, service life, etc). Amazon Glacier is a fantastic product, but there is one problem.

It is difficult to integrate Amazon Glacier with on-premise services. At the time of writing, you can only interact with it via the Amazon Glacier via the Amazon console or via the HTTP API.

There is no way to directly share Amazon Glacier storage to the network, or mount Glacier storage to a VM as a volume. Luckily there are ways round this.

Amazon Storage Gateway ((https://aws.amazon.com/storagegateway/) allows you to present Amazon storage volumes to your local network as if they were hosted on your local on-site servers. In addition, you can setup policies to move old files from your Amazon storage volumes to Glacier storage automatically.

In principal this would allow you to deploy an Amazon Storage Gateway on site and use it to present an Amazon storage volume to the local network. This local volume can then be used as storage for one or more archive jobs, and as long as shortcuts aren't used in place of migrated files the old files can be automatically migrated to Amazon Glacier.

Getting further help:

If you have any further questions then pleae don't hesitate to contact us via [email protected]