2.7.1 to 2.8.0

This document describes the upgrade instructions from 2.7.1 to 2.8.0.

Aerie v2.8.0 contains significant changes to our PostgreSQL database structure, along with changes to our required environment variables and configuration. It therefore requires a different upgrade process than usual, especially if you plan to migrate your data forward to v2.8.0. Make sure to read all instructions below carefully before continuing, and please open a Github issue or ask for help in our Slack room if you have any questions.

For Aerie End-Users

(ie. those running Aerie from the released Deployment.zip or the deployment folder in the repository)

Changes to docker-compose configuration

Several changes have been made to the variables used in our docker-compose.yml config file. If you are using the supplied config as-is, you don't need to do anything - but if you are using a modified version of this file for your own deployment, you'll need to make some updates. You can either:

• Take a fresh copy of our docker-compose.yml file and recreate any modifications you made
• Or see the full detailed list of changes to the config variables in the appendix of this document, and manually apply these changes to your own version of the file.

Changes to .env file

Several new environment variables are now required. Compare the new version of the .env file in the deployment directory to your existing .env file and copy over any new variables which are missing in your .env.

• existing values, especially AERIE_USERNAME, AERIE_PASSWORD, POSTGRES_USER and POSTGRES_PASSWORD, must remain unchanged from your previous version if you plan to migrate your data
• You must fill in (arbitrary) values for the password variables, which are left blank in the released .env file

Other notes

• Users should no longer run the createSchedulingSpecification mutation after creating a plan via the API
• If you plan to migrate your data, follow the migration instructions below before running Aerie v2.8.0 for the first time.

For Aerie Developers

(ie. anyone running Aerie from a cloned version of the aerie repository)

• Run ./gradlew clean before checking out develop or v2.8.0 in order to clean up old SQL files.
  • Or if you have already checked it out, run rm -rf deployment/postgres-init-db/sql && git restore deployment/postgres-init-db/sql
• Several new environment variables are now required. Compare the new version of env.template in the repository root to your existing .env file and copy over any new variables which are missing in your .env. Keep any of your existing values, and fill in (arbitrary) values for the password variables which are blank in the template.
• If you plan to migrate your data, follow the migration instructions below before running Aerie v2.8.0 for the first time
• When running v2.8.0 for the first time, make sure to rebuild the backend, and build/pull all containers, ie.:
  • docker compose down --remove-orphans && ./gradlew assemble && docker compose up --pull always -d --build

For Anyone Migrating Data to v2.8.0

These instructions apply if you wish to save your data and migrate it forward to work with v2.8.0, whether you're a user or a developer.

Migration Prerequisites

• Your database must already be up-to-date with the migrations from the prior release, 2.7.1. If you are upgrading from an earlier version, you should upgrade and migrate to v2.7.1 first before continuing.
• You should have already followed one set of instructions above to modify either your docker-compose file or your .env file.
• Make sure you have the psql command installed and you can run it from your shell. If not:
  • Mac users can run brew install libpq
  • For other platforms, download the Postgres installer, and uncheck "PostgreSQL Server" when installing
• From the deployment folder, run python -m pip install -r requirements.txt to make sure you have all necessary Python packages installed
• If you have ever changed the default names for the Aerie databases (aerie_merlin, aerie_scheduler, aerie_sequencing, aerie_ui) - this is no longer allowed, and you must rename them to their default names.

Migration Process

1. Starting from the deployment directory, run cd merge_aerie_db
2. Run ./merge_db.sh -h to list the available command line flags and see if any apply to your deployment - ie. if you are using non-default config file locations or connecting to a non-local database
3. Run ./merge_db.sh, adding any necessary command line flags, to start the migration
4. As a precaution, your old databases have been preserved at this point. If the last step failed for any reason, you can safely drop the aerie database, troubleshoot the problem and re-run the merge_db.sh script
5. After the migration has run successfully, run drop_old_dbs.sh to drop your old databases.

Your data should now be fully migrated and ready to run Aerie v2.8.0! Note that, due to breaking changes to the mission model framework in v2.7.0, you may need to recompile and re-upload your mission models if you are upgrading from a version before v2.7.0.

Appendix

Full list of docker-compose.yml variable changes

Below are all of the changes to the variables in our docker-compose.yml file introduced in v2.8.0, if you plan to modify your version of this file manually:

• aerie_gateway:
  • removed envvars:
    • POSTGRES_AERIE_MERLIN_DB
    • POSTGRES_USER
    • POSTGRES_PASSWORD
  • renamed envvars:
    • POSTGRES_HOST -> AERIE_DB_HOST
    • POSTGRES_PORT -> AERIE_DB_PORT
  • added envvars:
    • GATEWAY_DB_USER: "${GATEWAY_USERNAME}"
    • GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"
• aerie_merlin:
  • removed envvars: MERLIN_DB
  • renamed envvars:
    • MERLIN_DB_SERVER -> AERIE_DB_HOST
    • MERLIN_DB_PORT -> AERIE_DB_PORT
  • MERLIN_DB_USER should now be mapped at the envvar "${MERLIN_USERNAME}"
  • MERLIN_DB_PASSWORD should now be mapped at the envvar "${MERLIN_PASSWORD}"
• aerie_merlin_workers: (for each worker)
  • removed envvars:
    • MERLIN_WORKER_DB
    • MERLIN_WORKER_DB_USER
    • MERLIN_WORKER_DB_PASSWORD
  • renamed envvars:
    • MERLIN_WORKER_DB_SERVER -> AERIE_DB_HOST
    • MERLIN_WORKER_DB_PORT -> AERIE_DB_PORT
  • added envvars:
    • MERLIN_DB_USER: "${MERLIN_USERNAME}"
    • MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"
• aerie_scheduler:
  • removed envvars: SCHEDULER_DB
  • renamed envvars:
    • SCHEDULER_DB_SERVER -> AERIE_DB_SERVER
    • SCHEDULER_DB_PORT -> AERIE_DB_PORT
  • SCHEDULER_DB_USER should now be mapped at the envvar "${SCHEDULER_USERNAME}"
  • SCHEDULER_DB_PASSWORD should now be mapped at the envvar "${SCHEDULER_PASSWORD}"
• aerie_scheduler_workers: (for each worker)
  • removed envvars:
    • SCHEDULER_WORKER_DB
    • SCHEDULER_WORKER_DB_USER
    • SCHEDULER_WORKER_DB_PASSWORD
  • renamed envvars:
    • SCHEDULER_WORKER_DB_SERVER -> AERIE_DB_SERVER
    • SCHEDULER_WORKER_DB_PORT -> AERIE_DB_PORT
  • added envvars:
    • SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"
    • SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"
• aerie_sequencing:
  • removed envvars: SEQUENCING_DB
  • renamed envvars:
    • SEQUENCING_DB_SERVER -> AERIE_DB_HOST
    • SEQUENCING_DB_PORT -> AERIE_DB_PORT
  • SEQUENCING_DB_USER should now be mapped at the envvar "${SEQUENCING_USERNAME}"
  • SEQUENCING_DB_PASSWORD should now be mapped at the envvar "${SEQUENCING_PASSWORD}"
• hasura:
  • removed envvars:
    • AERIE_MERLIN_DATABASE_URL
    • AERIE_SCHEDULER_DATABASE_URL
    • AERIE_SEQUENCING_DATABASE_URL
    • AERIE_UI_DATABASE_URL
  • added envvars:
    • AERIE_DATABASE_URL: "postgres://${AERIE_USERNAME}:${AERIE_PASSWORD}@postgres:5432/aerie?options=-c%20search_path%3Dutil_functions%2Chasura%2Cpermissions%2Ctags%2Cmerlin%2Cscheduler%2Csequencing%2Cpublic"
• postgres:
  • added envvars:
    • GATEWAY_DB_USER: "${GATEWAY_USERNAME}"
    • GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"
    • MERLIN_DB_USER: "${MERLIN_USERNAME}"
    • MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"
    • SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"
    • SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"
    • SEQUENCING_DB_USER: "${SEQUENCING_USERNAME}"
    • SEQUENCING_DB_PASSWORD: "${SEQUENCING_PASSWORD}"