MLtek on Facebook        MLtek on Twitter

We have recently signed up to CrowdReviews.com and we would really appreciate it if you could take a few minutes to leave us some feedback. All reviews are visible to everyone, and your review will be given particular weight if you can verify it with your LinkedIn profile.

 

 

ArchiverFS - Archiving a File Server to Amazon Glacier

Product Home | Pricing | Download | Documentation | Release Notes | Customers

How to use ArchiverFS to archive to Amazon Glacier

What is Amazon Glacier?

Amazon are one of the biggest players in the online cloud computing sector, and amongst the many services they offer is a storage tier called 'Glacier'. Amazon Glacier is designed to provide cheap storage for cloud users, and it is excellent value for money.

Storage costs start from $0.004 per GB per month and it's easy to see why this would be attractive to businesses looking to place data into cheap long term storage.

There is one major issue though, and that is you can't mount an Amazon Glacier volume to a machine and share it to the network like you can with a normal volume. Normally you can only move data to a Glacier volume via either an API, the Amazon Console tools or via a storage and migration policy on an Amazon Storage Gateway. This makes it really hard to use Glacier to provide archive storage for an on site file server.

By working in tandem with an Amazon Storage Gateway, ArchiverFS is now capable of moving your old and outdated content seamlessly from your live file servers to an Amazon volume hosted on a Storage Gateway. From there it can be migrated to Amazon Glacier automatically for long term storage..

How does it work?

By following the steps below you will be able to archive the old and outdated files on your file server to Amazon Glacier, via an Amazon Storage Gateway.

Step 1 - Setup your storage gateway: First of all, you should have a look at the Amazon Storage Gateway getting started guide. This will walk you through the setup of a Storage Gateway.

Step 2 - Create your archive share: Next you'll need to create a share on the Storage Gateway that will be presented to the network. It is this share that you will configure ArchiverFS to move old files too, and it must be accessible across the network as a UNC path (e.g. \\server\share). The share will most likely need to be AD integrated so you can maintain your NTFS permissions when files are initially archived, Amazon have created this guide which explains how to setup the share.

Step 3 - Setup your Amazon lifecycle migration policies: These are the policies that will move files from your archive share (the share that ArchiverFS archives too) off to Amazon Glacier. Like the previous steps, Amazon have a good guide to setting up a lifecycle policy. We sugest moving files to Amazon Glacier once they have been sat on the archive volume for a few months, but it is totally up to you how you configure this.

Step 4 - Setup ArchiverFS: Assuming you already have a configured and working copy of ArchiverFS, the last thing you'll need to do will be to setup a new archive job and configure the correct maintenance job settings. When setting up your new archive job choose either 'Move - No Links' or 'Move - File Shortcuts' as the desired action and set the target to be the new share you created on the Storage Gateway. If you selected 'Move - File Shortcuts' then open up the maintenance job options and tick the 'Delete shortcuts (.lnk) in the live file system that point to non-existent archived files'. If you haven't already got a copy of ArchiverFS setup then we'd recommend having a good read of the QuickStart Guide and\or the full User Guide before installing ArchiverFS.

Future Enhancements:

The above configuration works well, and it should be compatible with almost all storage hardware (AD integrated and formatted with NTFS). However, we would very much like to have true seamless (hard) links working with the storage gateway.

Initial work on this looks encouraging, but a lot more testing and some code changes are needed before seamless (hard) links are fully supported with the Amazon Storage Gateway.