Containerise dependencies required to run tasks in dev workflow (#53)

banjoh · scottrigby · web-flow · commit c36cb6994787 · 2025-05-01T06:22:27.000+01:00
* feat: Dockerfile containing all dependencies for tasks Signed-off-by: Evans Mungai <evans@replicated.com> * download helmfile Signed-off-by: Evans Mungai <evans@replicated.com> * feat: Dockerfile containing all dependencies for tasks Signed-off-by: Evans Mungai <evans@replicated.com> * download helmfile Signed-off-by: Evans Mungai <evans@replicated.com> * Add initial docker env taskfile Signed-off-by: Evans Mungai <evans@replicated.com> * Move docker tasks into wg-easy application Signed-off-by: Evans Mungai <evans@replicated.com> * Update dev documentation Signed-off-by: Evans Mungai <evans@replicated.com> * Copy replicated auth token to container Signed-off-by: Evans Mungai <evans@replicated.com> * Update dev documentation Signed-off-by: Evans Mungai <evans@replicated.com> * Fix grammer Signed-off-by: Evans Mungai <evans@replicated.com> * Add SHELL env required by replicated shell subcommand Signed-off-by: Evans Mungai <evans@replicated.com> * Default to Podman, allow Docker optionally (#54) Signed-off-by: Scott Rigby <scott@r6by.com> * Prepend DEV to dev container variables Signed-off-by: Evans Mungai <evans@replicated.com> * Changes as per review comments Signed-off-by: Evans Mungai <evans@replicated.com> * Add shell completions and some colour to terminal Signed-off-by: Evans Mungai <evans@replicated.com> * Remove duplicate completion Signed-off-by: Evans Mungai <evans@replicated.com> * Update applications/wg-easy/container/Containerfile Co-authored-by: Scott Rigby <scott@r6by.com> * Update applications/wg-easy/docs/development-workflow.md Co-authored-by: Scott Rigby <scott@r6by.com> * Remove unnecessary shell-implementation task Signed-off-by: Evans Mungai <evans@replicated.com> * Improvements in Containerfile Signed-off-by: Evans Mungai <evans@replicated.com> * Remove unnecessary comment Signed-off-by: Evans Mungai <evans@replicated.com> * Build and publish wg-easy tools image to ghcr Signed-off-by: Evans Mungai <evans@replicated.com> * Update paths to check in workflow file Signed-off-by: Evans Mungai <evans@replicated.com> * Fix repo name Signed-off-by: Evans Mungai <evans@replicated.com> * Push in PR Signed-off-by: Evans Mungai <evans@replicated.com> * Publish multiarch images Signed-off-by: Evans Mungai <evans@replicated.com> * Download architecture specific binaries Signed-off-by: Evans Mungai <evans@replicated.com> * Only push when in main branch Signed-off-by: Evans Mungai <evans@replicated.com> * Check if image exists locally before pulling or building When starting the development container, check if the image exists locally before pulling or building it. Signed-off-by: Evans Mungai <evans@replicated.com> * Remove step to build image in docs dev:shell automatically builds the image if it doesn't exist locally Signed-off-by: Evans Mungai <evans@replicated.com> --------- Signed-off-by: Evans Mungai <evans@replicated.com> Signed-off-by: Scott Rigby <scott@r6by.com> Co-authored-by: Scott Rigby <scott@r6by.com>
diff --git a/.github/workflows/wg-easy-image.yml b/.github/workflows/wg-easy-image.yml
@@ -0,0 +1,64 @@
+name: WG-Easy Image CI
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - 'applications/wg-easy/**'
+      - '.github/workflows/wg-easy-image.yml'
+  pull_request:
+    paths:
+      - 'applications/wg-easy/**'
+      - '.github/workflows/wg-easy-image.yml'
+  workflow_dispatch:
+
+env:
+  DEV_CONTAINER_REGISTRY: ghcr.io
+  DEV_CONTAINER_IMAGE: replicatedhq/platform-examples/wg-easy-tools
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.DEV_CONTAINER_REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.DEV_CONTAINER_REGISTRY }}/${{ env.DEV_CONTAINER_IMAGE }}
+          tags: |
+            type=raw,value=latest,enable={{is_default_branch}}
+            type=sha,format=short
+            type=ref,event=branch
+            type=ref,event=pr
+
+      - name: Build and push image
+        uses: docker/build-push-action@v6
+        with:
+          context: applications/wg-easy
+          file: applications/wg-easy/container/Containerfile
+          platforms: linux/amd64,linux/arm64
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
diff --git a/applications/wg-easy/Taskfile.yaml b/applications/wg-easy/Taskfile.yaml
@@ -2,31 +2,39 @@ version: "3"
 
 includes:
   utils: ./taskfiles/utils.yml
+  dev: ./taskfiles/container.yml
 
 vars:
   # Application configuration
   APP_NAME: '{{.REPLICATED_APP | default "wg-easy"}}'
-  
+
   # Cluster configuration
   CLUSTER_NAME: '{{.CLUSTER_NAME | default "test-cluster"}}'
   K8S_VERSION: '{{.K8S_VERSION | default "1.32.2"}}'
   DISK_SIZE: '{{.DISK_SIZE | default "100"}}'
   INSTANCE_TYPE: '{{.INSTANCE_TYPE | default "r1.small"}}'
   DISTRIBUTION: '{{.DISTRIBUTION | default "k3s"}}'
   KUBECONFIG_FILE: './{{.CLUSTER_NAME}}.kubeconfig'
-  
+
   # Ports configuration
   EXPOSE_PORTS:
     - port: 30443
       protocol: https
     - port: 30080
       protocol: http
-    
+
   # GCP default configuration
   GCP_PROJECT: '{{.GCP_PROJECT | default "replicated-qa"}}'
   GCP_ZONE: '{{.GCP_ZONE | default "us-central1-a"}}'
   VM_NAME: '{{.VM_NAME | default (printf "%s-dev" (or (env "GUSER") "user"))}}'
 
+  # Container workflow configuration
+  DEV_CONTAINER_REGISTRY: '{{.DEV_CONTAINER_REGISTRY | default "ghcr.io"}}'
+  DEV_CONTAINER_IMAGE: '{{.DEV_CONTAINER_IMAGE | default "replicatedhq/platform-examples/wg-easy-tools"}}'
+  DEV_CONTAINER_TAG: '{{.DEV_CONTAINER_TAG | default "latest"}}'
+  DEV_CONTAINER_NAME: '{{.DEV_CONTAINER_NAME | default "wg-easy-tools"}}'
+  CONTAINER_RUNTIME: '{{.CONTAINER_RUNTIME | default "podman"}}'
+
 tasks:
   default:
     desc: Show available tasks
@@ -91,7 +99,7 @@ tasks:
             echo "Removing old kubeconfig file"
             rm -f {{.KUBECONFIG_FILE}}
           fi
-        fi 
+        fi
 
   setup-kubeconfig:
     desc: Get kubeconfig and prepare cluster for application deployment
@@ -112,7 +120,7 @@ tasks:
           true
         fi
     deps:
-      - create-cluster
+      - cluster-create
       - verify-kubeconfig
 
   dependencies-update:
@@ -138,7 +146,7 @@ tasks:
         if [ -z "$CLUSTER_ID" ]; then
           exit 1
         fi
-        
+
         # Check if all ports are already exposed
         expected_count={{len .EXPOSE_PORTS}}
         port_checks=""
@@ -147,7 +155,7 @@ tasks:
         {{end}}
         # Remove trailing "or "
         port_checks="${port_checks% or }"
-        
+
         PORT_COUNT=$(replicated cluster port ls $CLUSTER_ID --output json | jq -r ".[] | select($port_checks) | .upstream_port" | wc -l | tr -d ' ')
         [ "$PORT_COUNT" -eq "$expected_count" ]
     cmds:
@@ -169,10 +177,10 @@ tasks:
           echo "Error: Could not find cluster with name {{.CLUSTER_NAME}}"
           exit 1
         fi
-        
+
         # Get exposed URLs
         ENV_VARS=$(task utils:port-operations OPERATION=getenv CLUSTER_NAME={{.CLUSTER_NAME}})
-        
+
         # Deploy with helmfile
         echo "Using $ENV_VARS"
         eval "KUBECONFIG={{.KUBECONFIG_FILE}} $ENV_VARS helmfile sync --wait"
@@ -193,7 +201,7 @@ tasks:
           echo "No clusters found with name {{.CLUSTER_NAME}}"
           exit 0
         fi
-        
+
         for id in $CLUSTER_IDS; do
           echo "Deleting cluster ID: $id"
           replicated cluster rm "$id"
@@ -213,7 +221,7 @@ tasks:
       - echo "Preparing release files..."
       - rm -rf ./release
       - mkdir -p ./release
-      
+
       # Copy all non-config.yaml files
       - echo "Copying non-config YAML files to release folder..."
       - find . -path '*/replicated/*.yaml' -not -name 'config.yaml' -exec cp {} ./release/ \;
@@ -237,27 +245,27 @@ tasks:
           yq '.spec.chart.chartVersion = strenv(version) | .spec.chart.chartVersion style="single"' $directory/$helmChartName | tee release/$helmChartName
 
         done < <(find . -maxdepth 2 -mindepth 2 -type d -name replicated)
-      
+
       # Merge config.yaml files
       - echo "Merging config.yaml files..."
       - |
         # Start with an empty config file
         echo "{}" > ./release/config.yaml
-        
+
         # Merge all app config.yaml files first (excluding root replicated)
         for config_file in $(find . -path '*/replicated/config.yaml' | grep -v "^./replicated/"); do
           echo "Merging $config_file..."
           yq eval-all '. as $item ireduce ({}; . * $item)' ./release/config.yaml "$config_file" > ./release/config.yaml.new
           mv ./release/config.yaml.new ./release/config.yaml
         done
-        
+
         # Merge root config.yaml last
         if [ -f "./replicated/config.yaml" ]; then
           echo "Merging root config.yaml last..."
           yq eval-all '. as $item ireduce ({}; . * $item)' ./release/config.yaml "./replicated/config.yaml" > ./release/config.yaml.new
           mv ./release/config.yaml.new ./release/config.yaml
         fi
-      
+
       # Package Helm charts
       - echo "Packaging Helm charts..."
       - |
@@ -267,7 +275,7 @@ tasks:
           # Navigate to chart directory, package it, and move the resulting .tgz to release folder
           (cd "$chart_dir" && helm package . && mv *.tgz ../release/)
         done
-      
+
       - echo "Release files prepared in ./release/ directory"
     deps:
       - update-version
diff --git a/applications/wg-easy/container/Containerfile b/applications/wg-easy/container/Containerfile
@@ -0,0 +1,78 @@
+# Base image for all shared Containerfiles for taskfiles
+# Use this image as base image for app specific container files
+FROM --platform=$BUILDPLATFORM ubuntu:24.04
+
+ARG TARGETOS
+ARG TARGETARCH
+
+WORKDIR /tools
+
+# Set environment variables
+ENV DEBIAN_FRONTEND=noninteractive \
+    HOME=/home/devuser \
+    SHELL=/bin/bash
+
+# Install debian packages
+RUN apt-get update && apt-get install -y \
+    curl \
+    jq \
+    less \
+    yq \
+    gnupg \
+    bash-completion \
+
+    # Install Google Cloud CLI
+    && echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list \
+    && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg \
+    && apt-get update \
+    && apt-get install google-cloud-cli -y \
+
+    # Clean up
+    && apt-get purge -y gnupg \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install helm
+RUN curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash \
+
+    # Install kubectl
+    && curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${TARGETARCH}/kubectl" \
+    && chmod +x kubectl \
+    && mv kubectl /usr/local/bin/ \
+
+    # Install Task
+    && sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin \
+
+    # Install Helmfile
+    && curl -Ls $(curl -s https://api.github.com/repos/helmfile/helmfile/releases/latest \
+    | grep "browser_download_url.*linux_${TARGETARCH}.tar.gz" \
+    | cut -d : -f 2,3 \
+    | tr -d \") -o helmfile.tar.gz \
+    && tar xf helmfile.tar.gz helmfile && rm helmfile.tar.gz \
+    && mv helmfile /usr/local/bin/helmfile \
+
+    # Install Replicated CLI
+    && curl -Ls $(curl -s https://api.github.com/repos/replicatedhq/replicated/releases/latest \
+    | grep "browser_download_url.*linux_amd64.tar.gz" \
+    | cut -d : -f 2,3 \
+    | tr -d \") -o replicated.tar.gz \
+    && tar xf replicated.tar.gz replicated && rm replicated.tar.gz \
+    && mv replicated /usr/local/bin/replicated
+
+# Create a non-root user for better security
+RUN groupadd -r devuser && useradd -r -g devuser -m -s /bin/bash devuser
+
+# Copy shell completion scripts
+COPY container/tool-completions.sh tool-completions.sh
+
+# Copy entrypoint script
+COPY container/entrypoint.sh entrypoint.sh
+RUN chmod +x entrypoint.sh
+
+# Set working directory
+WORKDIR /workspace
+
+# Switch to non-root user
+USER devuser
+
+# Set entrypoint
+ENTRYPOINT ["/tools/entrypoint.sh", "-l"]
diff --git a/applications/wg-easy/container/entrypoint.sh b/applications/wg-easy/container/entrypoint.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Uncomment force_color_prompt in bashrc
+sed -i 's/#force_color_prompt=yes/force_color_prompt=yes/' $HOME/.bashrc
+
+# Source the tool completions
+echo "source /tools/tool-completions.sh" >> $HOME/.bashrc
+
+# Execute the passed command or default to bash
+exec "$@"
diff --git a/applications/wg-easy/container/tool-completions.sh b/applications/wg-easy/container/tool-completions.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# kubectl completion
+source <(kubectl completion bash)
+alias k=kubectl
+complete -o default -F __start_kubectl k
+
+# helm completion
+source <(helm completion bash)
+
+# task completion
+source <(task --completion bash)
+
+# helmfile completion
+source <(helmfile completion bash)
+
+# replicated completion
+source <(replicated completion bash)
+
+# gcloud completion
+source /usr/share/google-cloud-sdk/completion.bash.inc
diff --git a/applications/wg-easy/docs/development-workflow.md b/applications/wg-easy/docs/development-workflow.md
@@ -18,14 +18,9 @@ The core philosophy of this workflow is to start simple and add complexity incre
 Before starting the development workflow, ensure you have the following tools installed:
 
 - **Task:** The task runner used in this project. ([Installation Guide](https://taskfile.dev/installation/))
-- **Replicated CLI:** For managing test clusters and application releases. ([Installation Guide](https://docs.replicated.com/reference/replicated-cli-installing))
-- **Helm:** The Kubernetes package manager. ([Installation Guide](https://helm.sh/docs/intro/install/))
-- **Helmfile:** For orchestrating Helm chart deployments. ([Installation Guide](https://github.com/helmfile/helmfile#installation))
-- **kubectl:** The Kubernetes command-line tool. ([Installation Guide](https://kubernetes.io/docs/tasks/tools/install-kubectl/))
-- **jq:** A command-line JSON processor. ([Download Page](https://stedolan.github.io/jq/download/))
-- **yq:** A command-line YAML processor. ([Installation Guide](https://github.com/mikefarah/yq#install))
-- **gcloud CLI:** Google Cloud command-line interface (optional, only required for GCP-specific tasks). ([Installation Guide](https://cloud.google.com/sdk/docs/install))
-- **Standard Unix Utilities:** `find`, `xargs`, `grep`, `awk`, `wc`, `tr`, `cp`, `mv`, `rm`, `mkdir`, `echo`, `sleep`, `test`, `eval` (typically available by default on Linux and macOS).
+- **Container runtime tool** Either [Podman](https://podman.io/docs/installation) (default) or [Docker](https://docs.docker.com/get-docker/) for local development
+
+All other tools will be automatically provided through task commands and containers.
 
 ## Workflow Stages
 
@@ -97,6 +92,17 @@ Configure chart values and create or modify templates.
 
 ### Stage 3: Local Validation with helm template
 
+> [!IMPORTANT]
+> Tools required by tasks in this project will be made available in a container. Run the commands below to start the dev environment
+
+```
+# Open shell to execute tasks
+task dev:shell
+
+# Start/restart tools container. Idempotent.
+task dev:restart
+```
+
 Validate chart templates locally without deploying to a cluster.
 
 1. Run helm template to render the chart and inspect manifests:
@@ -177,7 +183,7 @@ Test multiple charts working together using Helmfile orchestration.
    # Check if issuers are correctly using cert-manager
    kubectl get clusterissuers
    kubectl get issuers -A
-   
+
    # Verify Traefik routes
    kubectl get ingressroutes -A
    ```
diff --git a/applications/wg-easy/taskfiles/container.yml b/applications/wg-easy/taskfiles/container.yml
