Migrating to a New Server — Install First Migration
This guide will walk through how to migrate a LiquidFiles system from an existing system to a newly installed system using the Installation First Migration option. We call the existing system the source system and the new system the destination system.
You only really need to use this option if you want to use NFS for storage or want to use a separate data disk using Amazon AWS or Microsoft Azure. If you want to use a separate data disk on a local deployment, please install LiquidFiles using the ISO image using the Custom Disk Installation Guide.
For documentation how the migration works on a high level, please see the Migration Overview including how to prepare your environment for Production Change-Over.
Installing the new system
The first step is to install a new LiquidFiles system that will become the destination system for this migration.
Please follow any of the Installation Guides to install a new LiquidFiles system in your environment. The figure above has this new destination system installed in the same on the same network as the source system, but you can use this guide to migrate your data to another LiquidFiles system installed anywhere. You can install the destination system in the same data centre, a different data centre, migrate to or from your Amazon AWS or Microsoft Azure private cloud space — from anywhere to anywhere.
The only limitation is that during migration, the new destination system will connect to the source system using Rsync that uses TCP/873 so this connection needs to be permitted in any firewall or similar that resides between the destination and source system.
Once you have installed the new LiquidFiles system that will become the destination system for this migration, please come back to this guide.
Difference from a installation of a complete new system
When you follow the Installation Guides and you'll get to the Getting Started page, you'll need to select a couple of different options.
For this example, we assume that your source system has the following configuration:
- Public Hostname: files.company.com
- License Code: 5ed230b1604e77a3fc6e4a59720400655285239c
When configuring this new destination system, we are going to use the same details. First on the Public Hostname page.
And then on the license page.
The rest of the configuration will strictly not matter as they will be overwritten during the migration anyway.
Preparing the Source System
Create locally authenticated Sysadmin Account
On the existing LiquidFiles source system, we recommend that you first ensure that you have a locally authenticated sysadmin account with no 2-factor authentication enabled. Please navigate to Admin → Users on your existing source system, and either change an existing account or create a new Sysadmin account that doesn't use LDAP/SAML or any 2-factor authentication.
The reason we recommend this is because if you rely on LDAP/SAML or require 2-factor authentication to authenticate users, there's a risk that the new system can't reach the LDAP server, that the SAML server isn't configured to match the new system, that time is not correct for TOTP, you cannot reach the SMS OTP delivery system, DUO and similar issues. Once the new destination system is up and running and you have tested all authentication options you have configured, you can delete this Sysadmin account, or re-enable the required options on the new destination system.
Starting the Migration on the source system
To start the migration, login as root on on the source system, either on the console or using SSH, and run the ft migrate command. Please see the Console Access Guide for more details how to access the console. It will look similar to this:
A couple of things to note about the source system:
- Please note the password chokochea in this screenshot that you will need to enter on the destination system when you start this migration.
- The function to migration from this source system is only available while the ft migrate command is running. If you interrupt it with control c, or the SSH connection is terminated, ft migrate will stop running and you cannot start or complete the migration and you will need to restart it.
Continue and disable option
What the Continue and disable option does is that it blocks connections to the LiquidFiles application on the source system. This is a temporary block. It does not delete, remove or permanently changes anything on the source system. When you've completed the production change-over and for whatever reason decides that you need to rollback. When you stop ft migrate by pressing control c, it will remove this block and your users can once again connect to the LiquidFiles application and send files, exactly how it was before.
The reason the Continue and disable option exist is that if there's any changes on the source system after you've started ft migrate, those changes will not be migrated to the destination system. ft migrate makes a copy of the database that is copied over to the destination system and anything that happens on the source system after you've run ft migrate will not be included in the database backup and will not be migrated to the destination system.
Should you select the Continue and disable option?
- Are you performing any form of test migration or multistage migration where another final migration will come after this migration — select Continue and don't disable.
- Final migration in a multistep migration or single-step migration — select Continue and disable.
Start the Migration on the Destination system
Your source system is now configured and ready for you to migrate the data across to your new destination system.
When you've finished setting up your system as you need, please login to the System Console and run ft migrate and select the options as below.
The configuration options include:
Hostname or IP Address
The safest option here is to use an IP address. The hostname might not resolve and there's been issues in some cases where if you're migrating from and to the same hostname, that the destination system resolves the name to itself.
Migration Password
You'll get this password from the console on the source system after you've run ft migrate. This is a random password that's going to be unique each time you start ft migrate.
Migration Certificates
You can choose to copy the certificates from the source system or not. If you choose not to migrate the certificates and later want to migrate the certificate over, please go to Admin → System → Certificate and click on Upload Certificate on both the source and destination system and you can manually copy the private key, certificate and any certificate chain certificates.
Not Migrated
The following settings are not migrated as part of this migration:
- IP Address configuration including default gateway.
- DNS servers.
- NTP servers.
- Static routes.
- root password.
- System email notification address.
- Syslog Forwarder
- System Hostname (the system hostname will be set to the Public Hostname).
Begin Migration
When all details are configured, you can click on Begin Migration. At this point the migration will start with the destination system connecting to the source system using Rsync (TCP/873). You will see a lot of output in the interface as all data and configuration are copied over.
When completed you will be notified that you can continue.
Video Walkthrough — Install First migration
Here's a video walkthrough of the migration procedure using the Installation First migration methodology.
Testing & Troubleshooting
When the system come back online after the migration, you should be able to login with the sysadmin account you configured or created.
Keep the Public Hostname the same
When you're testing the migration, before the Production Change-Over, please make sure you keep the Public Hostname the same on the source and destination system. The license is tied to the Public Hostname. If you change this on the destination system, the source system will get an invalid license error and stop working properly.
If you want to change the Public Hostname, please do that after you've decommissioned the old source system.
Common Issues
If there's any issues, it's usually one of two different issues we've seen.
Cannot authenticate with sysadmin account?
If you didn't create a locally authenticated sysadmin account, and the new destination system can't authenticate (because you cannot reach the LDAP server or similar). Please login to the System Console and run the command ft add_admin to create a new locally authenticated sysadmin account, or ft reset_admin to reset the first sysadmin account on the system.
You get redirected to the source system?
It's possible to configure LiquidFiles with strict hostname validation. If your Public Hostname is https://files.company.com, with strict hostname validation you will get redirected to this hostname. If you haven't done the Production Change-Over, this will redirect you to the source system.
You can disable this strict hostname validation by logging in to the System Console and run the command ft hostconfig. You will be asked a few configuration question. The important one in this instance is the strict hostname validation that needs to be disabled.
Production (Phase 2) Migration
If you're planning to use a multiphase migration where you first do a test migration and then want to do an additional production migration, this section is for you. You can repeat this migration procedure as many times as you wish. Each time you do, the destination system will become a replica of the source system.
If you're satisfied and want to skip this step, please go do the Production Change-Over section.
For subsequent migrations, and you can run this as many times as you need, you will need to login to the System Console and run the ft migrate command both on the source and destination system.
When you run a multistage migration, only the difference is copied after the first migration. So if your source system has 1TB of data and only 2GB has been uploaded since the last migration, only 2GB of data will be copied instead of 1TB. This is therefore also a good way of minimizing downtime where the final Production Change-Over migration will be much faster with often a fraction of the data to copy.
Source System
On the source system, the procedure is identical to the previous time in the Preparing the Source system section. Please note though that you will need to restart ft migrate each time you want to run the migration. If you've haven't interrupted this since the last migration, the same data will be migrated over.
Destination System
On the destination system, please login to the Console and run the command ft migrate and follow the options below. You will notice that the migration password is different each time ft migrate is run on the source system. Please take note of the current migration password and enter the correct one here.
Production Change-Over
When you have completed the migration, the destination system will be identical to the source system when it comes to any LiquidFiles related data such as users, groups, configuration, messages, data, ...
When you're ready to change over the source and destination system, making the destination system the new production system, there are three basic options. Each of these options are outlined in the Migration Overview guide.