Skip to main content

Agent Quickstart

Overview​

Atlas Agent supports configuring schema monitoring declaratively using HCL (HashiCorp Configuration Language) files. This "as code" approach allows you to version control your monitoring configuration, manage multiple database instances consistently, and automate agent deployments.

With monitoring configuration as code, you can:

  • Version control your monitoring configuration alongside your application code and avoid "ClickOps" mistakes
  • Automate agent deployments with consistent configurations
  • Manage multiple database instances and monitoring scopes declaratively
  • Override default monitoring settings for specific databases or schemas
  • Discover RDS and GCP instances automatically using data sources

Prerequisites​

  • An Atlas Cloud account
  • A bot token for the agent
  • Access to your database instances

Basic Configuration​

A basic Atlas Agent configuration file (agent.hcl) consists of:

  1. An agent block with authentication and logging settings
  2. One or more instance blocks defining database connections
  3. monitor or default_monitor blocks specifying what to monitor

Here's a minimal example:

agent.hcl
agent {
  token = getenv("ATLAS_TOKEN")
}

instance "production" {
  driver = "postgres"

  connection {
    host     = "prod-db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"

    auth {
      env_var "DATABASE_PASSWORD" {}
    }
  }

  default_monitor "all" {
    snapshot_interval = "1h"

    statistics {
      enabled = true
    }
  }
}

Configuration Structure​

Agent Block​

The agent block configures the agent's connection to Atlas Cloud and optional settings:

agent {
  token = getenv("ATLAS_TOKEN")  # Required: Atlas Cloud token

  log {
    level  = "INFO"  # DEBUG, INFO, WARN, ERROR
    format = "TEXT"  # TEXT or JSON
  }

  metrics {
    enabled = true
    port    = 8081
  }
}

Instance Block​

Each instance block defines a database instance to monitor. The optional metadata attribute accepts a map of key-value pairs (e.g. region, engine, cluster info). When using RDS Discovery or GCP Discovery, each discovered instance exposes a metadata field that you can pass through to this attribute.

instance "my-database" {
  name   = "Production Database"  # Human-friendly name
  slug   = "prod-db"              # Unique identifier
  driver = "postgres"             # Database driver

  tags = {
    environment = "production"
    team        = "backend"
  }

  # Optional: custom metadata (e.g. from discovery; see RDS and GCP Discovery sections)
  metadata = {
    region = "us-east-1"
    source = "rds"
  }

  connection {
    host     = "db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"

    auth {
      # Authentication method (see below)
    }
  }

  # Monitoring configuration (see below)
}

Connection Authentication​

Atlas Agent supports multiple authentication methods for database connections:

Read the database password from an environment variable:

connection {
  auth {
    env_var "DATABASE_PASSWORD" {}
  }
}

Custom Query Parameters​

Pass custom query parameters to the database connection.

For example, you can pass the sslmode parameter to the database connection to enable or disable SSL:

atlas-agent.hcl
instance "my-database" {
  name   = "Production Database"
  slug   = "prod-db"
  driver = "postgres"

  connection {
    host     = "db.example.com"
    port     = 5432
    user     = "atlas_monitor"
    database = "postgres"

    auth {
      env_var "DATABASE_PASSWORD" {}
    }
    query = {
      sslmode = "disable"
    }
  }
}

Monitoring Configuration​

Each instance must have at least one monitoring configuration using either default_monitor or monitor blocks.

Default Monitor​

A default_monitor block applies monitoring settings to all databases and schemas in the instance:

instance "production" {
  # ... connection configuration ...

  default_monitor "all" {
    snapshot_interval = "1h"  # How often to take snapshots (minimum: 1m)

    # Monitor specific schemas
    schemas = ["public", "app"]

    # Or use include/exclude patterns
    include = ["app_*"]
    exclude = ["temp_*", "test_*"]

    # Exclude entire databases
    exclude_databases = ["postgres", "rdsadmin"]

    # Initial database to connect to (for multi-database instances)
    initial_database = "postgres"

    statistics {
      enabled = true  # Collect table statistics
    }
  }
}

Specific Monitor Blocks​

Use monitor blocks to override default settings for specific databases or schemas:

instance "production" {
  # ... connection configuration ...

  default_monitor "all" {
    snapshot_interval = "1h"
    statistics {
      enabled = true
    }
  }

  # Override for a specific database
  monitor "analytics" {
    database          = "analytics_db"
    snapshot_interval = "6h"  # Less frequent snapshots

    statistics {
      enabled = false  # Disable statistics for this database
    }
  }

  # Monitor specific schemas in another database
  monitor "app-schemas" {
    database = "app_db"
    schemas  = ["users", "orders", "payments"]
  }
}

RDS Discovery​

Automatically discover RDS instances using the rds_discovery data source and create instance blocks for each discovered instance.

How RDS Discovery Works​

The rds_discovery data source supports two modes of operation:

  1. Default credentials mode: If no roles are provided, the agent uses the default AWS credentials (from the environment, IAM role, or credentials file) to discover instances in the current AWS account.

  2. Role-based discovery: If roles are provided, the agent only uses those roles to discover instances. It will not use default credentials. This allows you to discover instances across multiple AWS accounts by assuming different IAM roles.

Cross-Account Discovery

If you need to discover instances in both the current account and other accounts, you have two options:

  1. Use multiple data blocks: Create separate rds_discovery blocks—one without roles (for current account) and one with roles (for other accounts).

  2. Include a role for the current account: Add an IAM role ARN for the current account to the roles list along with roles for other accounts.

Filtering Discovered Instances​

The rds_discovery data source supports filtering to narrow down which instances are discovered:

  • tags: Filter instances by AWS resource tags. You can specify tag keys with specific values, or use null to match any instance that has the tag key (regardless of value).
  • name_regex: Filter instances by name using a regular expression pattern.
  • exclude_tags: Exclude instances that have specific tags. You can specify tag keys with specific values, or use null to exclude any instance that has the tag key (regardless of value).

Filters are applied together (AND logic), so an instance must match all specified filters to be included in the results.

data "rds_discovery" "production" {
  region = "us-east-1"

  filter {
    # Match instances with specific tag values
    tags = {
      Environment = "production"
      Team        = "backend"
    }

    # Exclude instances with specific tags (regardless of value)
    exclude_tags = {
      Deprecated = null  # Exclude instances that have the "Deprecated" tag
    }

    # Match instances whose name ends with "-prod"
    name_regex = ".*-prod$"
  }
}

Example: Discovering Instances in Current Account​

# Discover instances using default credentials (current account)
data "rds_discovery" "current_account" {
  region = "us-east-1"
  # No roles specified - uses default credentials

  filter {
    tags = {
      Environment = "production"
    }
  }
}

Example: Discovering Instances Across Multiple Accounts​

# Discover instances in multiple accounts using IAM roles
data "rds_discovery" "production" {
  region = "us-east-1"
  roles  = [
    "arn:aws:iam::123456789012:role/AtlasRDSRole",  # Account 1
    "arn:aws:iam::987654321098:role/AtlasRDSRole",  # Account 2
  ]

  filter {
    tags = {
      Environment = "production"
    }
    name_regex = ".*-prod$"
  }
}

# Create an instance block for each discovered RDS instance
instance {
  for_each = toset(data.rds_discovery.production.instances)

  name     = each.value.identifier
  driver   = each.value.driver
  tags     = each.value.tags
  metadata = each.value.metadata

  connection {
    host     = each.value.address
    port     = each.value.port
    user     = each.value.master_user
    database = each.value.initial_database

    auth {
      aws_iam {
        endpoint   = each.value.address
        region     = "us-east-1"
        # assume_role is optional - only include if the instance was discovered using a role
        assume_role = each.value.assume_role
      }
    }
  }

  default_monitor "all" {
    snapshot_interval = "1h"

    statistics {
      enabled = true
    }
  }
}

Example: Discovering in Current Account and Other Accounts​

# Discover instances in current account using default credentials
data "rds_discovery" "current_account" {
  region = "us-east-1"
  # No roles - uses default credentials

  filter {
    tags = {
      Environment = "production"
    }
  }
}

# Discover instances in other accounts using roles
data "rds_discovery" "other_accounts" {
  region = "us-east-1"
  roles  = [
    "arn:aws:iam::987654321098:role/AtlasRDSRole",  # Other account
  ]

  filter {
    tags = {
      Environment = "production"
    }
  }
}

# Combine instances from both data sources
locals {
  all_instances = concat(
    data.rds_discovery.current_account.instances,
    data.rds_discovery.other_accounts.instances
  )
}

instance {
  for_each = toset(local.all_instances)

  name     = each.value.identifier
  driver   = each.value.driver
  tags     = each.value.tags
  metadata = each.value.metadata

  connection {
    host     = each.value.address
    port     = each.value.port
    user     = each.value.master_user
    database = each.value.initial_database

    auth {
      aws_iam {
        endpoint   = each.value.address
        region     = "us-east-1"
        assume_role = each.value.assume_role
      }
    }
  }

  default_monitor "all" {
    snapshot_interval = "1h"

    statistics {
      enabled = true
    }
  }
}

RDS Discovery Data Source Fields​

The rds_discovery data source returns instances with the following fields:

  • identifier - Unique identifier for the instance (used as the map key)
  • driver - Database driver (e.g., "postgres", "mysql")
  • address - Database endpoint address
  • port - Database port number
  • initial_database - Initial database to connect to
  • master_user - Master username for the database
  • assume_role - IAM role ARN used when discovering this instance (empty string if default credentials were used)
  • iam_database_authentication_enabled - Whether IAM authentication is enabled
  • tags - Map of tags associated with the instance
  • arn - AWS ARN of the RDS instance
  • metadata - Map of instance metadata (e.g. region, engine, cluster info); pass through to the instance block's metadata attribute

GCP Discovery​

Automatically discover GCP database instances (AlloyDB, Cloud SQL, and Spanner) using the gcp_discovery data source and create instance blocks for each discovered instance.

How GCP Discovery Works​

The gcp_discovery data source uses your GCP credentials (from the environment, e.g. GOOGLE_APPLICATION_CREDENTIALS, or default application credentials) to discover instances in a single GCP project. You must enable at least one of alloydb, cloudsql, or spanner to discover that type of instance.

Filtering Discovered Instances​

The gcp_discovery data source supports filtering to narrow down which instances are discovered:

  • labels: Filter instances by GCP resource labels. You can specify label keys with specific values, or use null to match any instance that has the label key (regardless of value).
  • name_regex: Filter instances by name (or identifier) using a regular expression pattern.

Filters are applied together (AND logic), so an instance must match all specified filters to be included in the results.

data "gcp_discovery" "production" {
  project   = "my-gcp-project"
  alloydb   = true
  cloudsql  = true
  spanner   = true

  filter {
    labels = {
      Environment = "production"
      Team        = "backend"
    }
    name_regex = ".*-prod$"
  }
}

Example: Discovering AlloyDB and Cloud SQL Instances​

AlloyDB and Cloud SQL instances use host/port connections. Use the discovered address, port, initial_database, and configure authentication (e.g. Cloud SQL IAM or environment variable for password):

data "gcp_discovery" "relational" {
  project  = "my-gcp-project"
  alloydb  = true
  cloudsql = true

  filter {
    labels = {
      Environment = "production"
    }
  }
}

# Create an instance block for each discovered AlloyDB and Cloud SQL instance
locals {
  relational_instances = concat(
    data.gcp_discovery.relational.alloydb_instances,
    data.gcp_discovery.relational.cloudsql_instances
  )
}

instance {
  for_each = { for inst in local.relational_instances : inst.identifier => inst }

  name     = each.value.display_name
  driver   = each.value.driver
  tags     = each.value.labels
  metadata = each.value.metadata

  connection {
    host     = each.value.address
    port     = each.value.port
    user     = "atlas_monitor"
    database = each.value.initial_database != "" ? each.value.initial_database : "postgres"

    auth {
      env_var "DATABASE_PASSWORD" {}
    }
  }

  default_monitor "all" {
    snapshot_interval = "1h"
    statistics {
      enabled = true
    }
  }
}

Example: Discovering Spanner Instances​

Spanner instances use a connection path (project/instance/database). Each Spanner instance can have multiple databases; you can create one instance block per Spanner instance and use monitor blocks per database, or structure your config to match your discovery output:

data "gcp_discovery" "spanner" {
  project = "my-gcp-project"
  spanner = true

  filter {
    labels = {
      Environment = "production"
    }
  }
}

# One instance block per Spanner instance
instance {
  for_each = { for inst in data.gcp_discovery.spanner.spanner_instances : inst.identifier => inst }

  name     = each.value.display_name
  driver   = each.value.driver
  tags     = each.value.labels
  metadata = each.value.metadata

  connection {
    path = each.value.address
  }
}

# One monitor per database (Spanner instance + database)
monitor "all" {
  for_each = flatten([
    for inst in data.gcp_discovery.spanner.spanner_instances : [
      for db in inst.databases : merge(db, { instance_id : inst.identifier })
    ]
  ])
  instance = instance[each.value.instance_id]
  database = each.value.name
  statistics {
    enabled = true
  }
}

GCP Discovery Data Source Fields​

The gcp_discovery data source returns:

  • alloydb_instances – List of AlloyDB instances (one per cluster, preferring a reader instance). Each element has: name, identifier, display_name, driver (e.g. "postgresql"), address, port, initial_database, labels, metadata (map of instance metadata; pass through to the instance block's metadata attribute).
  • cloudsql_instances – List of Cloud SQL instances. Each element has: name, identifier, display_name, driver (e.g. "postgresql", "mysql", "sqlserver"), address, port, initial_database, labels, metadata (map of instance metadata; pass through to the instance block's metadata attribute).
  • spanner_instances – List of Spanner instances. Each element has: name, identifier, display_name, driver ("spanner"), address (e.g. projects/{project}/instances/{instance}), databases (list of { name, dialect, state }), labels, metadata (map of instance metadata; pass through to the instance block's metadata attribute).

Running the Agent using Docker​

docker run -d \
  --name atlas-agent \
  -e ATLAS_TOKEN="your-token" \
  -e DATABASE_PASSWORD="your-password" \
  -v $(pwd)/agent.hcl:/etc/atlas-agent/agent.hcl \
  arigaio/atlas-agent \
  --config /etc/atlas-agent/agent.hcl

Next Steps​