Skip to content

Mirror GitLab component projects on self-managed

Problem

With GitLab-maintained component projects available on the CI/CD Catalog on SaaS, as well as other popular 3rd-party resources, self-managed instances cannot reference them because we don't allow cross-instance use of template components today.

Goal

We want to allow self-managed instances to use component projects available in the SaaS CI/CD Catalog.

  1. Our primary focus is to allow GitLab official components to be used on self-managed.
  2. Secondary, we can allow any arbitrary components from SaaS catalog to be used on self-managed.
  3. Finally, the end goal is to be able to deprecate CI templates in favour of a SSoT CI/CD Catalog on SaaS and the ability for admins to import them.

Challenges

We need to look for ways that can allow fast, efficient and secure use of CI components, while also retaining the ability for self-managed admins to control what is being imported from SaaS.

Proposal - MVC

As per #415638 (comment 1658581000) we are primarily exploring the idea of mirroring projects from SaaS into self-managed and provide admins with tools to do this easily:

  1. Import GitLab-maintained component projects (e.g. the whole group gitlab.com/components) into the self-managed instance.
  2. Provide CLI for admins to use to import entire groups, or selected projects.
  3. The CLI will also publish in the local CI/CD catalog all, the most recent or selected releases of each imported project.
  4. The CLI can be used in a scheduled CI pipeline to automatically update the projects and publish new releases.
  5. The CLI is distributed with GitLab product.

As discussed also in https://gitlab.com/gitlab-org/gitlab/-/issues/433441#note_1679589231 we should look into packaging important GitLab component projects with the GitLab installation, so that at the end of the GitLab self-managed installation these components are available in gitlab.mycompany.com/components group without requiring internet access.

  • The CLI can be used to import extra component projects and to upgrade component projects ad-hoc with latest versions from SaaS.
  • For self-managed in air-gapped environments they can update their component projects when they upgrade the GitLab version.

Related tasks

  • Create a PoC that shows the workflow of importing projects in a self-managed instance (example: GDK) and have them available in the local catalog.
    • Imported projects could be internal or public based on the GitLab instance settings. During the installation/upgrade a user/admin must be used to setup those component projects. No other permissions are needed because users should not really contribute to those projects and only consume them. Maybe those projects can be automatically archived (read only).
  • Update the architecture design doc with the decision.
Edited by Dov Hershkovitch