Skip to main content

CI for Databases on GitLab

GitLab is a popular, open-source alternative to GitHub. In addition to a self-hosted version, GitLab also offers a hosted version at gitlab.com. Similar to GitHub, GitLab offers users storage for Git repositories, issue tracking, and CI/CD pipelines.

In this guide we will demonstrate how to use GitLab CI and Atlas to setup CI pipelines for your database schema changes.

Prerequisites​

Setup​

Install Atlas​

To download and install the latest release of the Atlas CLI, simply run the following in your terminal:

curl -sSf https://atlasgo.sh | sh

Login to Atlas​

After installing Atlas locally, you will need to login to your Atlas account. If you don't have an account yet, you will be prompted to create one for free during the login process.

You can do this by running the following command:

atlas login

Push your migrations to Atlas​

Once you have logged in, you can push your migrations to Atlas by running the following command:

atlas migrate push app \
  --dev-url "docker://postgres/15/dev?search_path=public"

Atlas will print a URL leading to your migrations on Atlas Cloud. You can visit this URL to view your migrations.

Create a Bot Token​

To connect GitLab CI to Atlas, you will need to create a bot token. To learn how to create a bot token, please visit the Bot Tokens guide.

Setup GitLab CI​

With your migrations pushed to Atlas, you can now setup GitLab CI to run your migrations on every commit.

Create a variable for your bot token​

In order to connect GitLab CI to Atlas, you will need to create a variable for your bot token. To do this:

  1. Go to your projects Settings page.
  2. Click on the CI/CD tab.
  3. Expand the Variables tab.
  4. Click "Add Variable".
  5. Use ATLAS_CLOUD_TOKEN as the variable key and paste the bot token you created earlier as the variable value.
  6. Set the "Mask Variable" checkbox to true to prevent the token from being printed in the CI logs.
  7. Unset the "Protect Variable" checkbox so that this job can be run on all branches (necessary for running on merge requests).

Create a .gitlab-ci.yml file​

Next, you will need to create a .gitlab-ci.yml file in the root of your repository. This file will tell GitLab CI how to run your Atlas CI pipeline.

image: ubuntu:latest

stages:
  - lint
  - push

before_script:
  - apt-get update -q && apt-get install -y curl
  - 'which atlas || (curl -sSf https://atlasgo.sh | sh)'
  - atlas login --token $ATLAS_CLOUD_TOKEN

lint migrations:
  stage: lint
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      when: always
    - if: '$CI_MERGE_REQUEST_ID'
      changes:
        - migrations/*
      when: always
      allow_failure: false
  script:
    - |
      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
        URL="${CI_PROJECT_URL}/-/merge_requests/${CI_MERGE_REQUEST_IID}"
      else
        URL="${CI_PROJECT_URL}/-/commit/${CI_COMMIT_SHA}"
      fi

      ATLAS_CONTEXT=$(cat <<EOF
      {
        "repo": "$CI_PROJECT_PATH",
        "branch": "$CI_COMMIT_REF_NAME",
        "commit": "$CI_COMMIT_SHA",
        "url": "$URL"
      }
      EOF
      )
    - atlas migrate lint --base atlas://gitlab -w --dev-url "sqlite://demo?mode=memory" --context "$ATLAS_CONTEXT"

push migrations:
  stage: push
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
  script:
    - |
      ATLAS_CONTEXT=$(cat <<EOF
      {
        "repo": "$CI_PROJECT_PATH",
        "branch": "$CI_COMMIT_REF_NAME",
        "commit": "$CI_COMMIT_SHA",
        "url": "${CI_PROJECT_URL}"
      }
      EOF
      )
    - atlas migrate push gitlab --dev-url "sqlite://demo?mode=memory" --context "$ATLAS_CONTEXT"

Let's break down what this file is doing:

  1. The image directive tells GitLab CI to use the ubuntu:latest Docker image to run your CI jobs.
  2. The stages directive defines the stages that your CI pipeline will run. In this case, we have two stages: lint and push. In the lint stage, we will run the atlas migrate lint command to verify that our migrations are valid and safe. In the push stage, we will run the atlas migrate push command to push our migrations to Atlas.
  3. The before_script directive defines a set of commands that will be run before each job. In this case, we are installing the atlas CLI and logging into Atlas using the bot token we created earlier.
  4. The lint migrations job will run the atlas migrate lint command. This job will run on all commits to your default branch and on all merge requests. This job will fail if the migrations are invalid.
  5. The push migrations job will run the atlas migrate push command. This job will only run on commits to your default branch.

Commit and push your changes​

Now that you have created your .gitlab-ci.yml file, you can commit and push your changes to GitLab.

From this point on, whenever you make a change to your migrations and push them to GitLab, GitLab CI will run your Atlas CI pipeline to verify their safety. To view a detailed analysis of your schema changes, look at the logs for a direct link to your Atlas Cloud account:

$ which atlas || (curl -sSf https://atlasgo.sh | sh)
Downloading https://release.ariga.io/atlas/atlas-linux-amd64-latest
Installation successful!

$ atlas login --token $ATLAS_CLOUD_TOKEN
You are now connected to "rotemtam85" on Atlas Cloud.

$ atlas migrate lint --base atlas://gitlab -w --dev-url "sqlite://demo?mode=memory" --context "$ATLAS_CONTEXT"
https://rotemtam85.atlasgo.cloud/ci-runs/8589934616
Cleaning up project directory and file based variables

Once you merge your changes to your default branch, the pipeline will push your latest changes to Atlas, so they can be deployed. Similarly, to get a visual representation of your database schema, follow the link from the logs to Atlas Cloud:

$ which atlas || (curl -sSf https://atlasgo.sh | sh)
Downloading https://release.ariga.io/atlas/atlas-linux-amd64-latest
Installation successful!

$ atlas login --token $ATLAS_CLOUD_TOKEN
You are now connected to "rotemtam85" on Atlas Cloud.

$ atlas migrate push gitlab --dev-url "sqlite://demo?mode=memory" --context "$ATLAS_CONTEXT"
https://rotemtam85.atlasgo.cloud/dirs/4294967302

Next Steps: Deployments​

Congratulations! You have successfully setup GitLab CI to lint and push your migration directory to Atlas Cloud. The next step is to deploy your migrations to your production database directly from Atlas Cloud using the CLI.