Agent Quickstart

Overview

Atlas Agent supports configuring schema monitoring declaratively using HCL (HashiCorp Configuration Language) files. This "as code" approach allows you to version control your monitoring configuration, manage multiple database instances consistently, and automate agent deployments.

With monitoring configuration as code, you can:

• Version control your monitoring configuration alongside your application code and avoid "ClickOps" mistakes
• Automate agent deployments with consistent configurations
• Manage multiple database instances and monitoring scopes declaratively
• Override default monitoring settings for specific databases or schemas
• Discover RDS instances automatically using data sources

Prerequisites

• An Atlas Cloud account
• A bot token for the agent
• Atlas Agent installed and running
• Access to your database instances

Basic Configuration

A basic Atlas Agent configuration file (agent.hcl) consists of:

1. An agent block with authentication and logging settings
2. One or more instance blocks defining database connections
3. monitor or default_monitor blocks specifying what to monitor

Here's a minimal example:

agent.hcl

agent {
  token = env("ATLAS_TOKEN")
}

instance "production" {
  driver = "postgres"
  
  connection {
    host     = "prod-db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"
    
    auth {
      env_var "DATABASE_PASSWORD" {}
    }
  }
  
  default_monitor "all" {
    snapshot_interval = "1h"
    
    statistics {
      enabled = true
    }
  }
}

Configuration Structure

Agent Block

The agent block configures the agent's connection to Atlas Cloud and optional settings:

agent {
  token     = env("ATLAS_TOKEN")  # Required: Atlas Cloud token
  
  log {
    level  = "INFO"  # DEBUG, INFO, WARN, ERROR
    format = "TEXT"  # TEXT or JSON
  }
  
  metrics {
    enabled = true
    port    = 8081
  }
}

Instance Block

Each instance block defines a database instance to monitor:

instance "my-database" {
  name   = "Production Database"  # Human-friendly name
  slug   = "prod-db"              # Unique identifier
  driver = "postgres"             # Database driver
  
  tags = {
    environment = "production"
    team        = "backend"
  }
  
  connection {
    host     = "db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"
    
    auth {
      # Authentication method (see below)
    }
  }
  
  # Monitoring configuration (see below)
}

Connection Authentication

Atlas Agent supports multiple authentication methods for database connections:

Environment Variable

Read the database password from an environment variable:

connection {
  auth {
    env_var "DATABASE_PASSWORD" {}
  }
}

AWS IAM (RDS)

Use AWS IAM authentication for RDS databases:

connection {
  auth {
    aws_iam {
      endpoint    = "prod-db.abc123.us-east-1.rds.amazonaws.com"
      region      = "us-east-1"
      assume_role = "arn:aws:iam::123456789012:role/AtlasRDSRole"
    }
  }
}

AWS Secrets Manager

Retrieve the password from AWS Secrets Manager:

connection {
  auth {
    aws_secrets_manager {
      name   = "prod/database/password"
      region = "us-east-1"
      key    = "password"  # Optional: JSON key path (e.g., "password")
    }
  }
}

Monitoring Configuration

Each instance must have at least one monitoring configuration using either default_monitor or monitor blocks.

Default Monitor

A default_monitor block applies monitoring settings to all databases and schemas in the instance:

instance "production" {
  # ... connection configuration ...
  
  default_monitor "all" {
    snapshot_interval = "1h"  # How often to take snapshots (minimum: 1m)
    
    # Monitor specific schemas
    schemas = ["public", "app"]
    
    # Or use include/exclude patterns
    include = ["app_*"]
    exclude = ["temp_*", "test_*"]
    
    # Exclude entire databases
    exclude_databases = ["postgres", "rdsadmin"]
    
    # Initial database to connect to (for multi-database instances)
    initial_database = "postgres"
    
    statistics {
      enabled = true  # Collect table statistics
    }
  }
}

Specific Monitor Blocks

Use monitor blocks to override default settings for specific databases or schemas:

instance "production" {
  # ... connection configuration ...
  
  default_monitor "all" {
    snapshot_interval = "1h"
    statistics {
      enabled = true
    }
  }
  
  # Override for a specific database
  monitor "analytics" {
    database          = "analytics_db"
    snapshot_interval = "6h"  # Less frequent snapshots
    
    statistics {
      enabled = false  # Disable statistics for this database
    }
  }
  
  # Monitor specific schemas in another database
  monitor "app-schemas" {
    database = "app_db"
    schemas  = ["users", "orders", "payments"]
  }
}

Advanced Examples

Multi-Database Instance

Monitor multiple databases on the same instance:

instance "multi-tenant" {
  driver = "postgres"
  
  connection {
    host     = "db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"  # Initial connection database
    
    auth {
      env_var "DB_PASSWORD" {}
    }
  }
  
  default_monitor "all" {
    initial_database  = "postgres"
    exclude_databases = ["rdsadmin"]
    snapshot_interval = "2h"
    
    statistics {
      enabled = true
    }
  }
  
  # Custom monitoring for specific tenant databases
  monitor "tenant-1" {
    database          = "tenant_1"
    snapshot_interval = "1h"
  }
  
  monitor "tenant-2" {
    database          = "tenant_2"
    snapshot_interval = "1h"
  }
}

RDS Discovery

Automatically discover RDS instances using the rds_discovery data source and create instance blocks for each discovered instance.

How RDS Discovery Works

The rds_discovery data source supports two modes of operation:

1. Default credentials mode: If no roles are provided, the agent uses the default AWS credentials (from the environment, IAM role, or credentials file) to discover instances in the current AWS account.

2. Role-based discovery: If roles are provided, the agent only uses those roles to discover instances. It will not use default credentials. This allows you to discover instances across multiple AWS accounts by assuming different IAM roles.

Cross-Account Discovery

If you need to discover instances in both the current account and other accounts, you have two options:

1. Use multiple data blocks: Create separate rds_discovery blocks—one without roles (for current account) and one with roles (for other accounts).

2. Include a role for the current account: Add an IAM role ARN for the current account to the roles list along with roles for other accounts.

Filtering Discovered Instances

The rds_discovery data source supports filtering to narrow down which instances are discovered:

• tags: Filter instances by AWS resource tags. You can specify tag keys with specific values, or use null to match any instance that has the tag key (regardless of value).
• name_regex: Filter instances by name using a regular expression pattern.

Filters are applied together (AND logic), so an instance must match all specified filters to be included in the results.

data "rds_discovery" "production" {
  region = "us-east-1"
  
  filter {
    # Match instances with specific tag values
    tags = {
      Environment = "production"
      Team        = "backend"
    }
    
    # Match instances whose name ends with "-prod"
    name_regex = ".*-prod$"
  }
}

Example: Discovering Instances in Current Account

# Discover instances using default credentials (current account)
data "rds_discovery" "current_account" {
  region = "us-east-1"
  # No roles specified - uses default credentials
  
  filter {
    tags = {
      Environment = "production"
    }
  }
}

Example: Discovering Instances Across Multiple Accounts

# Discover instances in multiple accounts using IAM roles
data "rds_discovery" "production" {
  region = "us-east-1"
  roles  = [
    "arn:aws:iam::123456789012:role/AtlasRDSRole",  # Account 1
    "arn:aws:iam::987654321098:role/AtlasRDSRole",  # Account 2
  ]
  
  filter {
    tags = {
      Environment = "production"
    }
    name_regex = ".*-prod$"
  }
}

# Create an instance block for each discovered RDS instance
instance "rds" {
  for_each = toset(data.rds_discovery.production.instances)
  
  name   = each.value.identifier
  driver = each.value.driver
  
  tags = each.value.tags
  
  connection {
    host     = each.value.address
    port     = each.value.port
    user     = each.value.master_user
    database = each.value.initial_database
    
    auth {
      aws_iam {
        endpoint   = each.value.address
        region     = "us-east-1"
        # assume_role is optional - only include if the instance was discovered using a role
        assume_role = each.value.assume_role
      }
    }
  }
  
  default_monitor "all" {
    snapshot_interval = "1h"
    
    statistics {
      enabled = true
    }
  }
}

Example: Discovering in Current Account and Other Accounts

# Discover instances in current account using default credentials
data "rds_discovery" "current_account" {
  region = "us-east-1"
  # No roles - uses default credentials
  
  filter {
    tags = {
      Environment = "production"
    }
  }
}

# Discover instances in other accounts using roles
data "rds_discovery" "other_accounts" {
  region = "us-east-1"
  roles  = [
    "arn:aws:iam::987654321098:role/AtlasRDSRole",  # Other account
  ]
  
  filter {
    tags = {
      Environment = "production"
    }
  }
}

# Combine instances from both data sources
locals {
  all_instances = concat(
    data.rds_discovery.current_account.instances,
    data.rds_discovery.other_accounts.instances
  )
}

instance "rds" {
  for_each = toset(local.all_instances)
  
  name   = each.value.identifier
  driver = each.value.driver
  
  tags = each.value.tags
  
  connection {
    host     = each.value.address
    port     = each.value.port
    user     = each.value.master_user
    database = each.value.initial_database
    
    auth {
      aws_iam {
        endpoint   = each.value.address
        region     = "us-east-1"
        assume_role = each.value.assume_role
      }
    }
  }
  
  default_monitor "all" {
    snapshot_interval = "1h"
    
    statistics {
      enabled = true
    }
  }
}

RDS Discovery Data Source Fields

The rds_discovery data source returns instances with the following fields:

• identifier - Unique identifier for the instance (used as the map key)
• driver - Database driver (e.g., "postgres", "mysql")
• address - Database endpoint address
• port - Database port number
• initial_database - Initial database to connect to
• master_user - Master username for the database
• assume_role - IAM role ARN used when discovering this instance (empty string if default credentials were used)
• iam_database_authentication_enabled - Whether IAM authentication is enabled
• tags - Map of tags associated with the instance
• arn - AWS ARN of the RDS instance

Pattern-Based Schema Selection

Use include/exclude patterns to selectively monitor schemas:

instance "production" {
  # ... connection configuration ...
  
  default_monitor "all" {
    # Monitor all schemas matching the pattern
    include = ["app_*", "public"]
    
    # Exclude temporary or test schemas
    exclude = ["temp_*", "test_*", "dev_*"]
    
    snapshot_interval = "1h"
    
    statistics {
      enabled = true
    }
  }
}

Running the Agent using Docker

docker run -d \
  -e ATLAS_TOKEN="your-token" \
  -e DATABASE_PASSWORD="your-password" \
  -v $(pwd)/agent.hcl:/etc/atlas-agent/agent.hcl \
  arigaio/atlas-agent \
  --config /etc/atlas-agent/agent.hcl

Next Steps

• Deploying with Helm - Deploy the Atlas Agent on Kubernetes using Helm
• Deploying with Terraform - Deploy the Atlas Agent to AWS using Terraform
• Drift Detection - Set up automated drift detection
• Webhooks - Configure notifications for schema changes
• How it works - Learn about the architecture
• Security - Security considerations
• HCL Reference - Complete HCL configuration reference