██FR█████ █INTELL███████████

Use minio as an external object storage with Gitlab


This content originally appeared on DEV Community and was authored by Aboubacar Doucoure

Context and reasoning

Gitlab can be installed on a Kubernetes cluster with the help of the official Gitlab Helm chart. I have installed a full version of Gitlab on a Rancher managed Kubernetes cluster with the usual griefs that come with the installation of such a behemoth: optimizing resources, picking the right subchart to install and how to install it. Everything was working fine until we had several machines crashes and I had to reinstall Gitlab from the custom chart I created. The installation only kept the gitaly, postgresql and redis PVCs, and to my devopsy sorrow, the minio storage was gone. There is no way around it, minio has to be installed seprately for a production ready self managed Gitlab. Here’s how.

Pre-requisites

  • A working Kubernetes installation and a distributes block storage for Kubernetes (I am using Longhorn on Rancher)
  • A working helm tooling

Install a minio chart

We’ll use a Bitnami chart to install a standalone Minio instance:

minio-external:
  mode: standalone
  enabled: true
  # https://github.com/bitnami/charts/blob/main/bitnami/minio/values.yaml
  persistence:
    enabled: true
    size: 10Gi
    annotations:
      helm.sh/resource-policy: keep
  auth:
    rootUser: <REDACTED>
    rootPassword: <REDACTED>
  provisioning:
    enabled: true
    users:
      - username: <REDACTED>
        password: <REDACTED>
        disabled: false
        policies:
          - readwrite
          - consoleAdmin
          - diagnostics  
        setPolicies: false
  accessKey:
    password: <REDACTED>
  secretKey:
    password: <REDACTED>
  # volumePermissions:
  #   enabled: true
  defaultBuckets: default,gitlab-registry-storage,gitlab-lfs,gitlab-artifacts,gitlab-uploads,gitlab-packages,gitlab-mr-diffs,gitlab-terraform-state,gitlab-ci-secure-files,gitlab-dependency-proxy,gitlab-pages

Use this configuration to kickstart a Minio instance with the buckets needed by Gitlab (defaultBuckets). These buckets are needed by different services of Gitlab and they will be mentionned in appConfig later.

Disable managed minio

Since w’ere using an external Minio instance we do not need the Gitlab managed one anymore. In your Gitlab values file disable minio:

global:
  ...
  minio:
    enabled: false

WARNING: This will delete your Gitlab Minio instance and all its data.

Create a secret

We’ll use a Bitnami chart to install a standalone Minio instance:

apiVersion: v1  
kind: Secret
metadata:  
    name: gitlab-object-storage
stringData:
  connection: |
    provider: AWS
    region: us-east-1
    aws_access_key_id: <REDACTED>
    aws_secret_access_key: <REDACTED>
    endpoint: "gitlab-minio-external:9000"

Update Gitlab with the new Minio 

  global:
    registry:
      bucket: gitlab-registry-storage
    appConfig: 
      ...  
      object_store:
        enabled: true
        proxy_download: true
        storage_options:
          {}
          # server_side_encryption:
          # server_side_encryption_kms_key_id
        connection:
          secret: gitlab-object-storage
          key: connection
      lfs:
        enabled: true
        proxy_download: true
        bucket: gitlab-lfs   
      artifacts:
        enabled: true
        proxy_download: true
        bucket: gitlab-artifacts
      uploads:
        enabled: true
        proxy_download: true
        bucket: gitlab-uploads
      packages:
        enabled: true
        proxy_download: true
        bucket: gitlab-packages
      externalDiffs:
        enabled: true
        when:
        proxy_download: true
        bucket: gitlab-mr-diffs
      terraformState:
        enabled: true
        bucket: gitlab-terraform-state
      ciSecureFiles:
        enabled: true   
bucket: gitlab-ci-secure-files
        # connection:
        #   secret: gitlab-object-storage
      dependencyProxy:
        enabled: true
        proxy_download: true
        bucket: gitlab-dependency-proxy
      pages:
        enabled: true
        proxy_download: true
        bucket: gitlab-pages

Voilà!

Inspirations and references


This content originally appeared on DEV Community and was authored by Aboubacar Doucoure