Guidance for Atlas Scalability

Features for Atlas Scalability

Auto-scaling enables clusters to automatically adjust their tier, storage capacity, or both, in response to real-time use. Atlas analyzes the CPU and memory utilization to determine when and whether to scale the cluster tier up and down. See Cluster tier scaling to learn more about the conditions under which Atlas scales up or down your cluster nodes. You can also specify a range of maximum and minimum cluster sizes that your cluster can automatically scale to in order to guarantee minimum performance or control costs. Atlas won't scale a cluster if the new tier falls outside of your specified size range or if your memory usage would exceed the capacity of the new tier. Auto-scaling is throttled with a delay to scale a cluster tier up or down to ensure that auto-scaling doesn't cause any application impact. Therefore, it is best suited for a steadily growing or declining application load, not sudden spikes in which the database is being inundated with usage. If your workload experiences frequent spikes or if you are expecting a large increase in traffic because of an event or a launch, MongoDB recommends that you pre-scale programmatically.

Atlas deployment templates, as referenced in the Recommended Deployment Topologies, provide you with horizontal and vertical scaling options. Specifically, sharding distributes data across numerous machines, which is useful when no single server can handle your workloads. Sharding follows a shared-nothing architecture, a distributed computing architecture where none of the nodes share any resources with each other. See Choose a Shard Key to learn more about the ideal choice of shard key that allows MongoDB to distribute documents evenly throughout your cluster while facilitating common query patterns. Furthermore, see Performance Best Practices: Sharding to learn about the key sharding strategies, such as ranged sharding, hashed sharding, and zoned sharding.

Upgrading an Atlas cluster to the next available Atlas tier is available through the Atlas control plane GUI, the Atlas Administration API, or through IaC tools, such as the Atlas Kubernetes Operator, the MongoDB & HashiCorp Terraform, or the Atlas CLI. See Guidance for Atlas Automated Infrastructure Provisioning to learn more. Changing an Atlas tier, either upscaling or downscaling, allows zero downtime. The tier changes in a rolling fashion, which involves electing a secondary member as a replacement, promoting this secondary member to become the new primary, then restoring or replacing the failing member to ensure that the cluster is returned to its target configuration as soon as possible. Horizontal scaling occurs post-deployment based on Administrator action, which can be triggered from a programmatic script. Some cluster templates require sharded clusters. Starting with MongoDB version 8.0, you may make use of embedded config servers to reduce costs associated with config servers on small sharded clusters.

The low CPU option in Atlas helps applications that require higher memory but not as much processing power. This option provides instances with half the vCPUs compared to the General tier of the same cluster size, reducing costs for workloads that are memory-intensive but not CPU-bound.

Data tiering and archival allows you to archive data at low-cost storage while still enabling queries alongside live cluster data, which is particularly useful for long-term record retention. To optimize this process, MongoDB recommends that you automate data archiving with simple, configurable rules. See Archive Data to learn more about the criteria that you can specify in an archiving rule. For scenarios where data retention is not a priority, Atlas offers the option to automatically delete unused data based on date criteria. For infrequently accessed data, TTL indexes are special single-field indexes that automatically delete documents from a collection after a specified period or at a set clock time. This is particularly useful for data like logs, session information, or event data that only needs to be retained for a limited time. To create a TTL index, you can define an index on a field that holds date values and specify a time-to-live duration in seconds.

Atlas also provides you with automated tools, such as the Performance Advisor, to identify and optimize inefficient queries by adding or removing an index or changing your client's query structure. You can reduce unnecessary compute time and resource consumption by following the actionable recommendations to enhance your query performance. Additionally, you can leverage intelligent index recommendations provided by Atlas to further improve data retrieval efficiency and minimize the resources needed for database operations.

Recommendations for Atlas Scalability

For development and testing environments, do not enable auto-scaling compute and auto-scaling storage. This saves costs in your non-production environments.

For staging and production environments, we recommend that you:

• Enable auto-scaling for compute and storage for instances where your application grows organically from small to medium.

  If you use IaC tools, leverage settings to ignore resource drift caused by auto-scaling. For example, in Terraform, if disk_gb_enabled is true, Atlas will automatically scale disk size up and down. This will cause the value of disk_size_gb returned to potentially be different than what is specified in the Terraform config and if one then applies a plan, not noting this, Terraform will scale the cluster disk size back to the original disk_size_gb value. To prevent this a lifecycle customization should be used, i.e.: lifecycle { ignore_changes = [disk_size_gb] }.

  Similarly, in Terraform, if compute_enabled is true, then Atlas will automatically scale up to the maximum provided and down to the minimum, if provided. This will cause the value of provider_instance_size_name returned to potentially be different than what is specified in the Terraform config, and if one then applies a plan, not noting this, Terraform will scale the cluster back to the original instanceSizeName value. To prevent this a lifecycle customization should be used, i.e.: lifecycle { ignore_changes = [provider_instance_size_name] }.

Automation Examples: Atlas Scalability

The following examples enable auto-scaling compute and storage using Atlas tools for automation.

These examples also apply other recommended configurations, including:

{
    "clusterType": "REPLICASET",
    "links": [],
    "name": "CustomerPortalProd",
    "mongoDBMajorVersion": "8.0",
    "replicationSpecs": [
      {
        "numShards": 1,
        "regionConfigs": [
          {
            "electableSpecs": {
              "instanceSize": "M30",
              "nodeCount": 3
            },
            "priority": 7,
            "providerName": "GCP",
            "regionName": "EASTERN_US",
            "analyticsSpecs": {
              "nodeCount": 0,
              "instanceSize": "M30"
            },
            "autoScaling": {
              "compute": {
                "enabled": true,
                "scaleDownEnabled": true
              },
              "diskGB": {
                "enabled": true
              }
            },
            "readOnlySpecs": {
              "nodeCount": 0,
              "instanceSize": "M30"
            }
          }
        ],
        "zoneName": "Zone 1"
      }
    ]
  }

atlas cluster create --projectId 5e2211c17a3e5a48f5497de3 --file cluster.json

# Create a Group to Assign to Project
resource "mongodbatlas_team" "project_group" {
  org_id = var.atlas_org_id
  name   = var.atlas_group_name
  usernames = [
    "user1@example.com",
    "user2@example.com"
  ]
}
# Create a Project
resource "mongodbatlas_project" "atlas-project" {
  org_id = var.atlas_org_id
  name = var.atlas_project_name
  # Assign the Project the Group with Specific Roles
  team_id    = mongodbatlas_team.project_group.team_id
  role_names = ["GROUP_READ_ONLY", "GROUP_CLUSTER_MANAGER"]
}
# Create an Atlas Advanced Cluster
resource "mongodbatlas_advanced_cluster" "atlas-cluster" {
  project_id = mongodbatlas_project.atlas-project.id
  name = "ClusterPortalProd"
  cluster_type = "REPLICASET"
  mongo_db_major_version = var.mongodb_version
  replication_specs {
    region_configs {
      electable_specs {
        instance_size = var.cluster_instance_size_name
        node_count    = 3
      }
      priority      = 7
      provider_name = var.cloud_provider
      region_name   = var.atlas_region
    }
  }
  tags {
    key   = "BU"
    value = "ConsumerProducts"
  }
  tags {
    key   = "TeamName"
    value = "TeamA"
  }
  tags {
    key   = "AppName"
    value = "ProductManagementApp"
  }
  tags {
    key   = "Env"
    value = "Production"
  }
  tags {
    key   = "Version"
    value = "8.0"
  }
  tags {
    key   = "Email"
    value = "marissa@acme.com"
  }
}
# Outputs to Display
output "atlas_cluster_connection_string" { value = mongodbatlas_advanced_cluster.atlas-cluster.connection_strings.0.standard_srv }
output "project_name"      { value = mongodbatlas_project.atlas-project.name }

# Atlas Organization ID
variable "atlas_org_id" {
  type        = string
  description = "Atlas Organization ID"
}
# Atlas Project Name
variable "atlas_project_name" {
  type        = string
  description = "Atlas Project Name"
}
# Atlas Project Environment
variable "environment" {
  type        = string
  description = "The environment to be built"
}
# Cluster Instance Size Name
variable "cluster_instance_size_name" {
  type        = string
  description = "Cluster instance size name"
}
# Cloud Provider to Host Atlas Cluster
variable "cloud_provider" {
  type        = string
  description = "AWS or GCP or Azure"
}
# Atlas Region
variable "atlas_region" {
  type        = string
  description = "Atlas region where resources will be created"
}
# MongoDB Version
variable "mongodb_version" {
  type        = string
  description = "MongoDB Version"
}
# Storage Auto-scaling Enablement Flag
variable "auto_scaling_disk_gb" {
  type        = boolean
  description = "Flag that specifies whether disk auto-scaling is enabled"
}
# Compute Auto-scaling Enablement Flag
variable "auto_scaling_compute" {
  type        = boolean
  description = "Flag that specifies whether cluster tier auto-scaling is enabled"
}
# Disk Size in GB
variable "disk_size_gb" {
  type        = int
  description = "Disk Size in GB"
}

atlas_org_id = "32b6e34b3d91647abb20e7b8"
atlas_project_name = "Customer Portal - Prod"
environment = "prod"
cluster_instance_size_name = "M30"
cloud_provider = "AWS"
atlas_region = "US_WEST_2"
mongodb_version = "8.0"
auto_scaling_disk_gb_enabled = "true"
auto_scaling_compute_enabled = "true"
disk_size_gb = "40000"

# Define the MongoDB Atlas Provider
terraform {
  required_providers {
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
    }
  }
  required_version = ">= 0.13"
}

terraform init

terraform plan

terraform apply