Atlas vs Classic Migration Tools

Introduction

What are schema migrations?

Most server-side applications are backed by a database. This database usually has a schema that reflects the application's data model. Over time, the application's data model evolves and the database schema must follow suit.

Schema migrations are a common approach in our industry to automate schema changes. Following this approach, the user creates a versioned migration script, which is the set of commands that should be executed on the database to upgrade the schema from one version to the next, thereby migrating the database to the next version.

What are migration tools?

Over the years, hundreds of tools have been created to facilitate the process of schema migrations. Migration tools typically provide a structured way of defining migration scripts, versioning them, and a program to execute them on a target database.

Because most database schema changes are not idempotent (they cannot be re-run successfully once applied), migration tools commonly maintain a "Changelog Table" on the target database to keep track of which migrations have already been applied.

Which migration tool should you use?

Choosing the right migration tool for your project can be daunting due to the sheer number of options available. In this document, we compare Atlas, a modern, database schema-as-code tool to more "classic" options that have been developed over the years.

We will take a high-level look at multiple candidates:

• Liquibase and Flyway - established JVM-based migration tools
• ORM-based solutions - that ship with the most popular ORM and backend frameworks
• Atlas - a modern, database schema-as-code tool

The candidates

	Atlas	Liquibase	Flyway	ORMs
Tagline	Database schema as code	Database DevOps	Database migrations made easy	Varies
Open-sourced	2021	2006	2010	Varies
Written in	Go	Java	Java	Varies
Docker Image Size	22 MB	198 MB	242 MB	Varies
Migration Format	SQL	XML/JSON/SQL/Java	SQL	Native Code *
Commercial Support Available	✅	✅	✅	☒ *

Liquibase and Flyway - are well-established projects that have been around since 2006 and 2010 respectively. Both are written in Java and require a JVM to execute. Both tools have an open-source distribution and are backed by commercial companies.

Liquibase and Flyway alike operate by letting the user define migration scripts (Flyway in plain SQL, Liquibase in XML, SQL, or Java) and execute them on behalf of the user.

ORM-based Solutions - many backend developers use application development frameworks or ORM libraries to interact with their underlying database. Virtually all of them provide some support for schema management which is crucial for supporting developers in any realistic production environment.

ORM-based migration tools vary greatly in quality and sophistication which makes it a bit difficult to treat them as a single group. In relevant categories where certain tools especially stand out, we will try to mention them specifically. ORM tend to provide native programming language based DSLs for defining migrations (such as Sequelize Migrations, with a few exceptions which use plain SQL.

ORMs tend to be community-based projects without a commercial entity backing them (with the exception of Prisma. This means that the level of support and maintenance can vary greatly between different options.

Atlas - is a database schema-as-code tool that applies modern DevOps principles to the ancient art of database schema management. Many of its users call it a "Terraform for Databases" because of its schema-first, declarative approach.

Atlas is an open-core project developed by Ariga and is available under both a commercial and an open-source license.

Full Disclosure

In case you missed it, this document is written and maintained by the team behind Atlas 😊

The Comparison

Summary (tl;dr)

To assist you in deciding which migration tool is right for you, we will compare the different possibilities along these categories:

	Description	Atlas	Liquibase	Flyway	ORMs
Schema-as-code	Schema-first approach?	✅	Revisions	Revisions	Mix
Auto migration planning	Are changes planned by the tool?	✅	Manual	Manual	Manual*
Auto code review	Are automated quality checks available?	✅	Some	Some	Some
CLI Support	Is CLI support available?	✅	✅	✅	✅
Kubernetes Operator	Can users natively deploy from k8s?	✅	☒	☒	☒
Terraform Provider	Does the tool integrate with IaC?	✅	☒	☒	☒
Database Support	Database driver count	9	60	31	Varies

* Exceptions exist

Schema-as-code

What is the source-of-truth for your database schema?

	Atlas	Liquibase	Flyway	ORMs
ORM Models	✅	☒	☒	Mix
Migrations	✅	✅	✅	Mix
SQL Schema	✅	☒	☒	☒
HCL	✅	☒	☒	☒

One of the most important principles that came from the DevOps movement is the idea that to achieve effective automation, you need to be able to build everything, deterministically, from a single source of truth.

This is especially important when it comes to databases! The database schema is a critical part of our application and we better have a way to ensure it is compatible with the code we are deploying.

Let's compare the approach to defining the source of truth schema between different migration tools:

Flyway and Liquibase are "version-based", which means that to obtain the current source of truth schema, one needs to replay all migrations on an empty database and inspect the result.

ORMs and frameworks are more difficult to classify. On one hand, ORMs and frameworks revolve around a code-first definition of the different entities in the application data model.

On the other hand, migrations are defined as revisions, in a version-based approach. Aside from Prisma and Django, most frameworks do not supply a mechanism to ensure that the planned migrations are consistent with the application data model as it is defined in the ORM.

Atlas is a database schema-as-code tool that takes the "schema-based" approach. Developers provide Atlas with the source-of-truth schema in various ways (SQL, HCL, and many other programming languages), and Atlas uses that to automatically plan changes. This way, the source of truth schema is always readily available and legible to users.

Using Atlas with ORMs

Atlas provides a set of plugins called "schema loaders" for integrating with ORMs such as SQLAlchemy, GORM and Hibernate.

By using schema loaders, developers can keep defining the application data model using their favorite ORM and programming language while offloading schema management to Atlas.

Planning Migrations

Who is responsible for planning schema changes?

	Atlas	Liquibase	Flyway	ORMs
Auto migrations from code	✅	☒	☒	Some
Declarative Migrations	✅	☒	☒	Dev-mode
Advanced DB Objects (triggers, functions, etc)	✅	☒	☒	Some plugins

The next category in our comparison is migration planning. People deliberating which migration tool to use should ask themselves, who is going to plan the schema changes? If our database is in some version N, and we want to get to version N+1, we need to calculate the plan of going from one version to the next.

Flyway and Liquibase rely on the user to plan schema changes. If a developer wishes to evolve the database schema to a new state, it is on them to look into their target database's documentation and find out the correct way to do so. Sometimes these changes are trivial, but many caveats and unpleasant surprises exist.

ORMs and frameworks most ORMs and frameworks provide a basic "auto migration" functionality that can automatically plan changes for you. This is used primarily for local development on disposable databases.

However, when it comes to planning production-grade changes, most frameworks leave planning to the user.

As we mentioned above, some tools (such as Django) have stood out in this perspective, by providing automatic migration planning capabilities for many years. Even so, these tools tend to focus on a pretty narrow set of database capabilities often neglecting power tools such as functions, triggers, and stored procedures.

Atlas includes a powerful automatic migration planning engine that supports over 95% of all database features. Atlas migration flows come in two flavors:

1. Fully declarative - Similar to Terraform, Atlas takes the desired state of the database (defined "as-code"), compares it to a live database, and suggests a migration plan at runtime.

2. Versioned - Atlas compares the desired state of the database ("as-code") with the current state of your migration directory and produces a migration file to reconcile between the two.

Both flows support "diffing policies" that provide Atlas with additional context and considerations about how to plan changes (such as: "always create indexes concurrently" or "verify columns are empty before dropping").

Additionally, Atlas detects ambiguous change types such as resource renames (which can also be interpreted as drop-and-add), and prompts the user to disambiguate.

Automated Quality Checks

	Atlas	Liquibase	Flyway	ORMs
Enforce Naming Conventions	✅	Paid	Paid	☒
Detect Destructive Changes	✅	Paid	☒	☒
Detect Table Locks	✅	☒	☒	☒
Detect Breaking Changes	✅	☒	☒	☒
Detect Potential Constraint Violations	✅	☒	☒	☒
Pre-migration checks	✅	✅	☒	☒
Post-migration checks	🔜	☒	☒	☒
Migration conflict detection	✅	☒	☒	☒

Schema migrations are a risky business, if you are not careful you can accidentally introduce destructive changes, break a data contract with your server or a downstream consumer, lock a table for writes causing downtime, or cause a deployment to fail because of a constraint violation.

Traditionally, the responsibility for reviewing proposed migrations was placed on humans, specifically those with database expertise. With the growing complexity of modern tech stacks, people with deep database knowledge are becoming increasingly rare, exposing teams to potential risks.

The modern approach to de-risking such operations is of course by automating quality checks during the CI phase of the software development life cycle. Let's see how the different options compare in this respect:

Flyway and Liquibase - provide a limited set of automated quality checks as part of their commercial offering but not in the free/open source tier.

ORMs and frameworks - generally do not provide automated quality checks for migrations. Over the years some notable exceptions have emerged, such as the ankane/strong_migration Ruby Gem that provides automated migration quality checks for Rails, and the Shadow Database functionality from "Prisma Migrate".

Atlas - Atlas natively provides a feature called "Migration Linting" which is available both as a CLI tool and as a CI integration (for example with GitHub Actions). Migration Linting is similar to static code analysis in that it is based on a semantic analysis of changes and their examination through configurable policies.

Atlas ships with over 30 automated checks to detect critical migration risks such as:

• Destructive change detection
• Data-dependent changes that may fail due to constraint violations
• Table-locking operations that can cause database downtime
• Breaking change detection
• .. many more database-specific quirks and edge cases

In addition, Atlas supports pre-migration checks that can be used to enforce certain predicates before a migration is applied. For example, you can enforce that a column is empty before dropping it, or that a column is not null before adding a constraint.

To assist teams in preventing such negative outcomes, these checks can also be automatically generated by Atlas during the automatic migration planning phase. These checks are respected by Atlas in the migration safety verification phase (i.e a migration dropping a table can be automatically approved if it contains no data).

Migration conflict detection

When a project succeeds to the point that it has many developers working on it, it's common to see developers working on different features that require different changes to the database schema. This can lead to conflicts when developers try to merge their changes together.

Classic migration tools don't provide a good way to detect and handle this situation. Because each migration script is created in its own file, common conflict detection tools like git can't help you. Your source control system can't tell you if two developers are working on the same table, or if they are adding columns with the same name.

For this reason, it's common to see teams surprised in production when conflicting migrations are applied. Even worse, in some cases migrations may be applied out of order or skipped entirely, leading to an inconsistent and unknown state of the database.

Atlas supports this by enforcing a linear history and Migration Directory Integrity during local development and in CI.

Deploying Migrations

	Atlas	Liquibase	Flyway	ORMs
CLI	✅	✅	✅	✅
Kubernetes Operator	✅	☒	☒	☒
Terraform Provider	✅	☒	☒	☒
GitHub Action	✅	✅	☒	☒
GitLab CI Component	🔜	☒	☒	☒
CircleCI Orb	🔜	Community	☒	☒
Native Code	Go SDK	Java SDK	Java SDK	✅

After planning and verifying the safety of migrations are considered, engineers comparing different migration tools should evaluate the provided deployment strategies.

As you can see, virtually all tools provide a CLI tool that can be used to deploy migrations, making it possible to integrate schema management into CI/CD pipelines.

However, modern deployment solutions often provide a modular plugin system that allows tool creators to develop simple integrations to streamline the deployment process.

For instance, organizations that manage their source code on GitHub often use GitHub Actions to automate their workflows. Tools that provide a native GitHub Actions integration make it easier for these organizations to integrate schema management into their CI/CD pipelines. As of writing this document, Atlas and Liquibase are the only tools that provide a comprehensive GitHub Actions integration.

Similarly, organizations that deploy to Kubernetes benefit from using Operators to extend the Kubernetes API with custom resources. By providing a Kubernetes Operator, a schema management makes it straightforward to include schemas as native Kubernetes resources. Atlas is the only tool that provides a Kubernetes Operator for schema management.

Finally, organizations that manage their infrastructure using Terraform rely on Terraform Providers to manage all of their resources. By providing a Terraform Provider, a schema management tool enables organizations to include schema management in their infrastructure-as-code workflows. Atlas is the only tool under comparison here that provides a Terraform Provider.

Database Support

Naturally, when comparing migration tools, support for various database engines is a differentiating factor.

While all tools under comparison in this document support popular open-source RDBMSs such as PostgreSQL, MySQL and SQLite, support for other databases varies.

The following table summarizes the support matrix for different databases between the different tools:

	Atlas	Liquibase	Flyway	ORMs
PostgreSQL	✅	✅	✅	✅
MySQL	✅	✅	✅	✅
MariaDB	✅	✅	✅	✅
SQLite	✅	✅	✅	✅
LibSQL	✅	☒	☒	Some
SQL Server	✅	✅	✅	Some
ClickHouse	✅	✅	✅	☒
Redshift	✅	✅	✅	☒
Oracle	🔜	✅	✅	Some
TiDB	✅	☒	✅	Some
CockroachDB	✅	✅	✅	Some
MongoDB	☒	✅	✅	Some
Apache Derby	☒	✅	✅	☒
Full List	Here	47 more..	18 more..	-

As you can see from the table, Liquibase and Flyway, having been around for a longer time support the largest amount of databases. Atlas, being a newer tool, supports fewer databases but is rapidly expanding its support.

Conclusion

We created Atlas to provide a modern, schema-as-code approach to database schema management. Atlas by no means is the best solution in every case and for every team but we hope this document has provided you with some insight into how Atlas compares to other tools and where it may be a good fit for your project.

If you are interested in learning more about Atlas, we recommend you check out the Quickstart Guide and continue exploring the documentation.

If you have any questions about Atlas or want to engage with a community of Atlas users, you are welcome to join the discussion on the Atlas Discord server.