3 posts tagged with "schema monitoring"

Hi everyone!

We're excited to share with you the release of Atlas v0.38, filled with many new features and enhancements for you to try.

• Oracle Triggers and Views - We've expanded the support for Oracle schemas to include triggers and views.
• Snowflake Additions - Our library of supported resources for Snowflake has also expanded with the additions of stages, external tables, hybrid tables, and dynamic tables.
• Google Spanner Additions - Spanner users can now manage geo-partitioning placements, locality groups, sequences, and change streams with Atlas.
• Expanded Analyzer Detection - Our linting analyzers now detect SQL injections in SQL schema and migration files, and incorrect usage of transactions in migration files.
• HTTP Data Source - Users can now use HTTP endpoints as data sources in the Atlas configuration file.
• PII Detection - Objects containing potentially sensitive or PII data can now be automatically or manually tagged in the Atlas Registry.
• Pre/Post-migration Hooks - Pre- and post-migration hooks enable teams to run custom logic before and after applying migrations.
• Atlas Monitoring - The Atlas Agent can now automatically discover and monitor RDS instances across multiple AWS accounts using IAM role assumption.
• Azure DevOps Repos CI/CD Integration - Atlas now provides native integration with Azure DevOps Pipelines and Azure Repos, including a dedicated Azure DevOps extension for seamless database schema CI/CD workflows.

Automatic ER Diagrams and Docs for ClickHouse​

When working with a relational database like ClickHouse, understanding the database schema becomes essential for many functions in the organization. Who cares about the schema? Almost everyone who interacts with your data:

• Software engineers and architects use knowledge about the schema to make design decisions when building software.
• Data engineers need to have an accurate understanding of schemas to build correct and efficient data pipelines.
• Data analysts rely on familiarity with the schema to write accurate queries and derive meaningful insights.
• DevOps, SREs, and Production Engineers use schema information (especially recent changes to it) to triage database-related production issues.

Having clear, centralized documentation of your database's schema and its changes can be a valuable asset to foster efficient work and collaboration. Knowing this, many teams have developed some form of strategy to provide this kind of documentation:

• Diagramming tools. Teams use generic diagramming tools like Miro or Draw.io to maintain ER (Entity-Relation) Diagrams representing their database schema. While this is easy to set up, it requires manually updating the documents whenever something changes, often causing documents to go stale and become obsolete.
• Data modeling tools. Alternatively, teams use database modeling software like DataGrip or DBeaver. While these tools automatically inspect your database, understand its schema, and provide interactive diagrams, they have two main downsides: 1) Since they run locally, they require a direct connection and credentials introducing a potential security risk; 2) They do not enable any collaboration, discussion, or sharing of information.
• Enterprise Data Catalogs like Atlan or Alation, provide extensive schema documentation and monitoring; however, they can be quite pricey and difficult to set up.

Enter: Atlas Schema Monitoring​

Atlas offers an automated, secure, and cost-effective solution for monitoring and documenting your ClickHouse schema.

With Atlas, you can:

• Generate ER Diagrams: Visualize your database schema with up-to-date, easy-to-read diagrams.
• Create Searchable Code Docs: Enable your team to quickly find schema details and usage examples.
• Track Schema Changes: Keep a detailed changelog to understand what's changed and why.
• Receive Alerts: Get notified about unexpected or breaking changes to your schema.

All without granting everyone on your team direct access to your production database.

Happy new year everyone, and welcome to our first release of 2025, Atlas v0.30! We have some exciting new features and improvements to share with you.

In this release you will find:

1. Simplified Schema Monitoring: Previously you needed to install a long-running agent on your database VPC to monitor your schema. Schema monitoring is now even simpler with the introduction of a new agentless monitoring mode.
2. Drizzle Support: We now support Drizzle, a popular ORM for Node.js. You can now use Atlas to automate schema migrations for your Drizzle projects.
3. Bitbucket Pipelines: We have added support for Bitbucket Pipelines, making it easier to integrate Atlas into your Bitbucket CI/CD workflows.
4. Custom Kubernetes Configurations: Atlas Pro users can now provide custom atlas.hcl configurations for their Kubernetes Custom Resource Definitions (CRDs) using the Atlas Operator.
5. txtar Editor Support: The Atlas JetBrains plugin now supports editing txtar files, used by the Atlas CLI to define pre-migration checks but also useful for other purposes.