Migrating to a New Server — Console Based Migration

This guide will walk through how to migrate a LiquidFiles system from an existing system to a newly installed system using the Console Based Migration option. We call the existing system the source system and the new system the destination system.

For documentation how the migration works on a high level, please see the Migration Overview including how to prepare your environment for Production Change-Over.

Installing the new system

The first step is to install a new LiquidFiles system that will become the destination system for this migration.

Please follow any of the Installation Guides to install a new LiquidFiles system in your environment. The figure above has this new destination system installed in the same on the same network as the source system, but you can use this guide to migrate your data to another LiquidFiles system installed anywhere. You can install the destination system in the same data centre, a different data centre, migrate to or from your Amazon AWS or Microsoft Azure private cloud space — from anywhere to anywhere.

The only limitation is that during migration, the new destination system will connect to the source system using Rsync that uses TCP/873 so this connection needs to be permitted in any firewall or similar that resides between the destination and source system.

Once you have installed the new LiquidFiles system that will become the destination system for this migration, please come back to this guide.

Preparing the Source System

Create locally authenticated Sysadmin Account

On the existing LiquidFiles source system, we recommend that you first ensure that you have a locally authenticated sysadmin account with no 2-factor authentication enabled. Please navigate to Admin → Users on your existing source system, and either change an existing account or create a new Sysadmin account that doesn't use LDAP/SAML or any 2-factor authentication.

The reason we recommend this is because if you rely on LDAP/SAML or require 2-factor authentication to authenticate users, there's a risk that the new system can't reach the LDAP server, that the SAML server isn't configured to match the new system, that time is not correct for TOTP, you cannot reach the SMS OTP delivery system, DUO and similar issues. Once the new destination system is up and running and you have tested all authentication options you have configured, you can delete this Sysadmin account, or re-enable the required options on the new destination system.

Starting the Migration on the source system

To start the migration, login as root on on the source system, either on the console or using SSH, and run the ft migrate command. Please see the Console Access Guide for more details how to access the console. It will look similar to this:

A couple of things to note about the source system:

• Please note the password chokochea in this screenshot that you will need to enter on the destination system when you start this migration.
• The function to migration from this source system is only available while the ft migrate command is running. If you interrupt it with control c, or the SSH connection is terminated, ft migrate will stop running and you cannot start or complete the migration and you will need to restart it.

Continue and disable option

What the Continue and disable option does is that it blocks connections to the LiquidFiles application on the source system. This is a temporary block. It does not delete, remove or permanently changes anything on the source system. When you've completed the production change-over and for whatever reason decides that you need to rollback. When you stop ft migrate by pressing control c, it will remove this block and your users can once again connect to the LiquidFiles application and send files, exactly how it was before.

The reason the Continue and disable option exist is that if there's any changes on the source system after you've started ft migrate, those changes will not be migrated to the destination system. ft migrate makes a copy of the database that is copied over to the destination system and anything that happens on the source system after you've run ft migrate will not be included in the database backup and will not be migrated to the destination system.

Should you select the Continue and disable option?

• Are you performing any form of test migration or multistage migration where another final migration will come after this migration — select Continue and don't disable.
• Final migration in a multistep migration or single-step migration — select Continue and disable.

Start the Migration on the Destination system

Your source system is now configured and ready for you to migrate the data across to your new destination system.

For this Console based migration, the console it will look like this. Please please F2 to reach the setup menu.

The configuration options include:

Hostname or IP Address

The safest option here is to use an IP address. The hostname might not resolve and there's been issues in some cases where if you're migrating from and to the same hostname, that the destination system resolves the name to itself.

Migration Password

You'll get this password from the console on the source system after you've run ft migrate. This is a random password that's going to be unique each time you start ft migrate.

Migration Certificates

You can choose to copy the certificates from the source system or not. If you choose not to migrate the certificates and later want to migrate the certificate over, please go to Admin → System → Certificate and click on Upload Certificate on both the source and destination system and you can manually copy the private key, certificate and any certificate chain certificates.

Root Password

The root password is not migrated from the source system. If you set a root password here, it will be set on this destination system once the migration has completed successfully.

It is recommended setting the root password. In case something goes wrong or you can't access the destination system after migration. Logging in to the System Console as root, it will almost always be able to update the system to gain access.

Not Migrated

The following settings are not migrated as part of this migration:

• IP Address configuration including default gateway.
• DNS servers.
• NTP servers.
• Static routes.
• root password.
• System email notification address.
• Syslog Forwarder
• System Hostname (the system hostname will be set to the Public Hostname).

Begin Migration

After the configuration options have been filled out, the system will begin migration. At this point the migration will start with the destination system connecting to the source system using Rsync (TCP/873). You will see a lot of output in the interface as all data and configuration are copied over.

When completed you will be notified that you can continue.

Video Walkthrough — Console migration

Here's a video walkthrough of the migration procedure, including installing a new LiquidFiles instance using Amazon AWS.

Testing & Troubleshooting

When the system come back online after the migration, you should be able to login with the sysadmin account you configured or created.

Common Issues

If the migration has completed successfully but you cannot access the system properly, it's usually one of the following two common issues.

Cannot authenticate with sysadmin account?

If you didn't create a locally authenticated sysadmin account, and the new destination system can't authenticate (because you cannot reach the LDAP server or similar). Please login to the System Console and run the command ft add_admin to create a new locally authenticated sysadmin account, or ft reset_admin to reset the first sysadmin account on the system.

You get redirected to the source system?

It's possible to configure LiquidFiles with strict hostname validation. If your Public Hostname is https://files.company.com, with strict hostname validation you will get redirected to this hostname. If you haven't done the Production Change-Over, this will redirect you to the source system.

You can disable this strict hostname validation by logging in to the System Console and run the command ft hostconfig. You will be asked a few configuration question. The important one in this instance is the strict hostname validation that needs to be disabled.

Production (Phase 2) Migration

If you're planning to use a multiphase migration where you first do a test migration and then want to do an additional production migration, this section is for you. You can repeat this migration procedure as many times as you wish. Each time you do, the destination system will become a replica of the source system.

If you're satisfied and want to skip this step, please go do the Production Change-Over section.

For subsequent migrations, and you can run this as many times as you need, you will need to login to the System Console and run the ft migrate command both on the source and destination system.

When you run a multistage migration, only the difference is copied after the first migration. So if your source system has 1TB of data and only 2GB has been uploaded since the last migration, only 2GB of data will be copied instead of 1TB. This is therefore also a good way of minimizing downtime where the final Production Change-Over migration will be much faster with often a fraction of the data to copy.

Source System

On the source system, the procedure is identical to the previous time in the Preparing the Source system section. Please note though that you will need to restart ft migrate each time you want to run the migration. If you've haven't interrupted this since the last migration, the same data will be migrated over.

Destination System

On the destination system, please login to the Console and run the command ft migrate and follow the options below. You will notice that the migration password is different each time ft migrate is run on the source system. Please take note of the current migration password and enter the correct one here.

Production Change-Over

When you have completed the migration, the destination system will be identical to the source system when it comes to any LiquidFiles related data such as users, groups, configuration, messages, data, ...

When you're ready to change over the source and destination system, making the destination system the new production system, there are three basic options. Each of these options are outlined in the Migration Overview guide.