6 posts tagged with "clickhouse"

Hey everyone!

We're excited to announce Atlas v1.3. This release delivers on a promise we made in v1.2: the security graph is now live in Atlas Cloud, mapping your entire database attack surface. It also introduces Atlas Scripts for running data operations as code, safer and faster migrations, and a new command group for working with the Atlas Registry from the CLI.

Here is what you can find in this release:

• Security Graph for Databases - Map your entire database attack surface: permissions, misconfigurations, and CVEs across every object, with severity tiers and blast-radius tracing.
• Atlas Scripts: DataOps as Code - Backfills, GDPR purges, and reports declared as code: transactional mutations, masked queries, and batched loops, reviewed and tested in CI.
• atlas cloud: Manage Registry Resources from the CLI - Repositories, databases, and deployment history from the terminal, with JSON output for scripts, CI gates, and AI agents, plus security graph and lineage graph exports.
• Schema Annotations - Attach type-safe metadata to schema resources and render GraphQL, OpenAPI, or any other format from it with templates.
• Pre-Apply Drift Detection - Block a deployment when the database drifted from the expected state, with the diff printed before any statement runs.
• Database Driver Improvements - Lock-safe NOT NULL and 8x-10x faster schema tests on PostgreSQL, Oracle roles and storage, Snowflake Iceberg tables, ClickHouse row policies, and more.
• Simpler Licensing and No AI Training - One agreement replaces the separate EULA and SaaS terms, with a written commitment that we never train AI models on your data.

We're excited to announce Atlas v1.1.

This release delivers on a promise we made in v1.0: Database Security as Code is now available for Atlas Pro users.

We're also shipping declarative data management for lookup tables and seed data, expanding database coverage with Aurora DSQL, Azure Fabric, and CockroachDB Cloud, and further improving our drivers and Atlas Cloud.

Here is what you can find in this release:

• Database Security as Code - Manage roles, users, and permissions declaratively across various popular Atlas-supported databases.
• Declarative Data Management - Define lookup tables and seed data as code using various sync modes.
• Aurora DSQL Support - Manage schemas on AWS's new serverless distributed SQL database.
• Azure Fabric Support - Use Atlas to manage your Microsoft Azure Fabric databases.
• Spanner Enhancements - Added support for Spanner's PostgreSQL dialect and vector indexes.
• ClickHouse Improvements - Cluster mode and data retention on table recreation.
• CockroachDB Support - Full Atlas support for CockroachDB Cloud and self-hosted deployments.
• PostgreSQL Enhancements - CAST definitions and replica identity support.
• Slack Integration - Connect to our native Slack app for real-time notifications.
• Schema Exporters - Declarative export for schema inspection, drift and diff reporting, and more.

Hey everyone!

Some time has passed since our previous release, and we're very excited to bring you another large batch of exciting additions in Atlas v0.37.

• Databricks Driver Beta - Atlas now supports managing Databricks databases in beta.
• ClickHouse Support Additions - We've expanded the support for ClickHouse to include clusters, user-defined functions, table projections, table partitions, and experiment types.
• SQL Server Support Additions - Our support for SQL Server has been extended to include SQL Server 2008, 2012, 2014, and 2016.
• Broader Scope for Linting Analyzers - Atlas now supports configuring analyzers to follow object deprecation workflows, enforce checks, block nolint usage, and allow or block specific SQL statements in migrations.
• Custom Migration Rules - Similar to custom schema rules, Atlas Pro users can now write rules for schema changes in their migrations.
• Pre-Execution Checks for Versioned Migrations - Added support for policy rules that run before migration execution. Teams can now allow or deny migrations based on conditions such as the number of pending files or specific SQL statements (e.g., blocking CREATE INDEX during peak hours).
• Cloud Databases as a Data Source - Users can now dynamically retrieve the migration status of different environments using the cloud_databases data source.
• Support for Hashicorp Vault - Atlas Pro users can now retrieve database credentials stored in Hashicorp Vault.
• Discover Database Instances for Schema Monitoring - Use the Atlas Agent to discover all database instances in your environment automatically in order to monitor them in Atlas Cloud.
• Protected Flows by Default - Atlas Cloud users can configure their settings to enable protected flows on all new projects.

Automatic ER Diagrams and Docs for ClickHouse​

When working with a relational database like ClickHouse, understanding the database schema becomes essential for many functions in the organization. Who cares about the schema? Almost everyone who interacts with your data:

• Software engineers and architects use knowledge about the schema to make design decisions when building software.
• Data engineers need to have an accurate understanding of schemas to build correct and efficient data pipelines.
• Data analysts rely on familiarity with the schema to write accurate queries and derive meaningful insights.
• DevOps, SREs, and Production Engineers use schema information (especially recent changes to it) to triage database-related production issues.

Having clear, centralized documentation of your database's schema and its changes can be a valuable asset to foster efficient work and collaboration. Knowing this, many teams have developed some form of strategy to provide this kind of documentation:

• Diagramming tools. Teams use generic diagramming tools like Miro or Draw.io to maintain ER (Entity-Relation) Diagrams representing their database schema. While this is easy to set up, it requires manually updating the documents whenever something changes, often causing documents to go stale and become obsolete.
• Data modeling tools. Alternatively, teams use database modeling software like DataGrip or DBeaver. While these tools automatically inspect your database, understand its schema, and provide interactive diagrams, they have two main downsides: 1) Since they run locally, they require a direct connection and credentials introducing a potential security risk; 2) They do not enable any collaboration, discussion, or sharing of information.
• Enterprise Data Catalogs like Atlan or Alation, provide extensive schema documentation and monitoring; however, they can be quite pricey and difficult to set up.

Enter: Atlas Schema Monitoring​

Atlas offers an automated, secure, and cost-effective solution for monitoring and documenting your ClickHouse schema.

With Atlas, you can:

• Generate ER Diagrams: Visualize your database schema with up-to-date, easy-to-read diagrams.
• Create Searchable Code Docs: Enable your team to quickly find schema details and usage examples.
• Track Schema Changes: Keep a detailed changelog to understand what's changed and why.
• Receive Alerts: Get notified about unexpected or breaking changes to your schema.

All without granting everyone on your team direct access to your production database.

Hi everyone,

We are excited to share our latest release with you! Here's what's new:

• Pre-migration Checks: Before migrating your schema, you can now add SQL checks that will be verified to help avoid risky migrations.
• Schema Docs: Atlas lets you manage your database schema as code. One of the things we love most about code, is that because of its formal structure, it's possible to automatically generate documentation from it. With this release, we're introducing a new feature that lets you generate code-grade documentation for your database schema.
• SQL Server Trigger Support: Atlas now supports managing triggers in SQL Server.
• ClickHouse Materialized View Support: Atlas now supports managing materialized views in ClickHouse.

Let's dive in.

Hi everyone,

It's been a while since our last version announcement and today I'm happy to share with you v0.16, which includes some very exciting improvements for Atlas:

• ClickHouse Beta Support - ClickHouse is a high-performance, columnar database optimized for analytics and real-time query processing. Support for ClickHouse in Atlas has been one of the top requested features by our community in the past year. Today, we are happy to announce that ClickHouse is officially in Beta!
• Hibernate Provider - Atlas now supports loading the desired state of your database directly from your Hibernate code. Hibernate developers can now join developers from the GORM, Sequelize, TypeORM and more communities who can now use Atlas to manage their database schema.
• Baseline Schemas - In some cases, your migrations rely on certain database objects to exist apriori to your application schema, for example extensions or legacy tables. Atlas now supports defining a baseline schema which will be loaded before automatically planning and applying your migrations.
• Proactive conflict detection - Teams that have connected their project to Atlas Cloud will get a prompt in the CLI if their migration directory is out of sync with the latest version in Atlas Cloud. This ensures that new migration files are added in a sequential order, preventing unexpected behavior.
• Mermaid Support - Atlas now supports generating a Mermaid diagram of your database schema. This is a great way to visualize your database schema and share it with your team.
• Review Policies - Users working with declarative migrations can now define "review policies" which can define thresholds for which kinds of changes require human review and which can be auto-applied.
• Postgres Sequences - Another long awaited feature, Atlas now supports managing sequences in PostgreSQL.

I know that's quite a list, so let's dive right in!