Object storage

You can configure object storage in GDK.

Prerequisites:

You can enable the object store by adding the following to your gdk.yml:

object_store:
  enabled: true
  port: 9000

The object store has the following default settings:

SettingDefaultDescription
enabledfalseEnable or disable MinIO.
port9000Port to bind MinIO.
console_port9002Port to bind MinIO Console.
access keyminioAccess key needed by MinIO to log in via its web UI. Cannot be changed.
secret keygdk-minioSecret key needed by MinIO to log in via its web UI. Cannot be changed.

Changing settings requires gdk reconfigure to be run.

Object storage configuration

Available configuration keys for object_store are shown in the gdk.example.yml file.

For each type of object storage (for example, lfs, packages, or uploads) you can also set three additional keys that are available to all object storage types:

  • bucket, to change the bucket name if required.
  • enabled and proxy_download, to override the common setting for a specific object storage type.

You should set consolidated_form to true. For the full list of object storage settings, see the relevant documentation.

The following sections provide a quick guide for configuring object storage for:

  • External S3 providers.
  • Google Cloud Storage.
  • Microsoft Azure.

External S3 providers

object_store:
  enabled: true
  consolidated_form: true
  connection:
    provider: 'AWS'
    aws_access_key_id: '<YOUR AWS ACCESS KEY ID>'
    aws_secret_access_key: '<YOUR AWS SECRET ACCESS KEY>'
  objects:
    artifacts:
      bucket: artifacts
    backups:
      bucket: backups
    external_diffs:
      bucket: external-diffs
    lfs:
      bucket: lfs-objects
    uploads:
      bucket: uploads
    packages:
      bucket: packages
    dependency_proxy:
      bucket: dependency_proxy
    terraform_state:
      bucket: terraform
    pages:
      bucket: pages
    ci_secure_files:
      bucket: ci-secure-files

Google Cloud Storage

object_store:
  enabled: true
  consolidated_form: true
  connection:
    provider: 'Google'
    google_project: '<YOUR GOOGLE PROJECT ID>'
    google_json_key_location: '<YOUR PATH TO GCS CREDENTIALS>'
  objects:
    artifacts:
      bucket: artifacts
    backups:
      bucket: backups
    external_diffs:
      bucket: external-diffs
    lfs:
      bucket: lfs-objects
    uploads:
      bucket: uploads
    packages:
      bucket: packages
    dependency_proxy:
      bucket: dependency_proxy
    terraform_state:
      bucket: terraform
    pages:
      bucket: pages
    ci_secure_files:
      bucket: ci-secure-files

Microsoft Azure Blob storage

To make Microsoft Azure Blob storage work, consolidated_form must be set to true:

object_store:
  enabled: true
  consolidated_form: true
  connection:
    provider: 'AzureRM'
    azure_storage_account_name: '<YOUR AZURE STORAGE ACCOUNT>'
    azure_storage_access_key: '<YOUR AZURE STORAGE ACCESS KEY>'
  objects:
    artifacts:
      bucket: artifacts
    backups:
      bucket: backups
    external_diffs:
      bucket: external-diffs
    lfs:
      bucket: lfs-objects
    uploads:
      bucket: uploads
    packages:
      bucket: packages
    dependency_proxy:
      bucket: dependency_proxy
    terraform_state:
      bucket: terraform
    pages:
      bucket: pages
    ci_secure_files:
      bucket: ci-secure-files

Backups

To set the object storage config for backups, configure the bucket in object_store.backup_remote_directory, for example:

object_store:
  enabled: false
  backup_remote_directory: 'backups'

MinIO errors

If you cannot start MinIO, you may have an old version not supporting the --compat parameter.

gdk tail minio shows a crash loop with the following error:

Incorrect Usage: flag provided but not defined: -compat

Upgrading MinIO to the latest version fixes it.

Creating a new bucket

In order to start using MinIO from your GitLab instance you have to create buckets first. You can create a new bucket by accessing http://127.0.0.1:9000/ (default configuration).

Last updated on