R510 -> R730XD/Proxmox migration helpers (consistent DB dumps, B2/Kopia snapshot refresh, cutover final backup) plus the previously-untracked gitea-portainer-registry cutover plan under docs/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1701 lines
48 KiB
Markdown
1701 lines
48 KiB
Markdown
# Gitea Portainer Registry Cutover Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Move homelab deployment source control from GitHub to local Gitea, keep GitHub as a temporary rollback mirror, move Gitea and the Docker registry into the management plane, publish famapp images to `registry.ginnoir.com`, and recreate Portainer stacks with `stack.env` as the only app-stack environment source.
|
|
|
|
**Architecture:** The management plane becomes the dependency root for Portainer, Vault, Gitea, Gitea runner, Docker registry, registry UI, and Watchtower. Portainer app stacks are recreated to poll `https://gitea.ginnoir.com/ginnoir/homelabstack.git`, with GitHub retained only as a temporary push mirror. Famapp release automation moves to Gitea Actions and publishes images to the self-hosted Docker registry.
|
|
|
|
**Tech Stack:** Docker Compose, Portainer CE Git stacks, Gitea, Gitea Actions `act_runner`, Docker registry v2, Caddy, PowerShell, SSH to `ginnoir@valhalla`, famapp Next.js Docker image.
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
Homelab repo files:
|
|
|
|
- Modify `portainer-compose.yml`
|
|
- Add Gitea, `postgres_gitea`, `gitea_runner`, `registry`, and `registry_ui` to the raw management compose file.
|
|
- Add registry auth bind mounts and environment.
|
|
- Keep `portainer`, `vault`, and `watchtower` in place.
|
|
- Remove `github-runner` only after Gitea Actions and Caddy reloads are proven.
|
|
- Modify `.env`
|
|
- Add management-plane registry credential variables and Gitea runner token variables.
|
|
- Retain GitHub token variables during the staged mirror period.
|
|
- Modify `stacks/dev/docker-compose.yml`
|
|
- Remove Gitea, `postgres_gitea`, `gitea_runner`, `registry`, and `registry_ui`.
|
|
- Convert remaining services away from `${VAR}` interpolation by storing container-native variables directly in `stacks/dev/stack.env`.
|
|
- Modify `stacks/dev/stack.env`
|
|
- Preserve existing secret values while adding the names that containers actually read: `APP_KEY`, `DB_PASS`, `MYSQL_ROOT_PASSWORD`, `MYSQL_PASSWORD`, `SECRET_KEY`, `DATABASE_URL`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `MINIO_USER`, `MINIO_PASSWORD`, `POSTGRES_PASSWORD`, `MINIO_ROOT_USER`, and `MINIO_ROOT_PASSWORD`.
|
|
- Remove Gitea-only variables after Gitea is moved to `.env`.
|
|
- Modify `stacks/resume/docker-compose.yml`
|
|
- Remove MinIO OIDC `${VAR}` interpolation by making `MINIO_IDENTITY_OPENID_CLIENT_ID` and `MINIO_IDENTITY_OPENID_CLIENT_SECRET` direct `stack.env` keys.
|
|
- Modify `stacks/resume/stack.env`
|
|
- Add `MINIO_IDENTITY_OPENID_CLIENT_ID` with the current `MINIO_OIDC_CLIENT_ID` value.
|
|
- Add `MINIO_IDENTITY_OPENID_CLIENT_SECRET` with the current `MINIO_OIDC_CLIENT_SECRET` value.
|
|
- Modify `stacks/famapp/docker-compose.yml`
|
|
- Change `famapp` image from `ghcr.io/ginnoir/famapp:latest` to `registry.ginnoir.com/ginnoir/famapp:latest`.
|
|
- Keep `pull_policy: always` during the staged cutover.
|
|
- Modify `stacks/proxy/docker-compose.yml`
|
|
- Update comments from GitHub runner reloads to Gitea Actions reloads after the workflow is proven.
|
|
- Modify `Caddyfile`
|
|
- Keep `registry.ginnoir.com` internal-only.
|
|
- Keep `gitea.ginnoir.com` Git/API bypasses.
|
|
- Adjust upstreams only if service names or networks change.
|
|
- Modify `CLAUDE.md` and `AGENTS.md`
|
|
- Replace GitHub-primary deploy language with Gitea-primary, GitHub-temporary-mirror language.
|
|
- Record that Portainer app stacks must have no Portainer UI env vars.
|
|
- Create `scripts/check-portainer-stack-env.ps1`
|
|
- Fail if any non-comment line in `stacks/*/docker-compose.yml` contains `${`.
|
|
- Fail if a service that requires `env_file: stack.env` lacks it.
|
|
- Allow explicit, reviewed exemptions in `scripts/portainer-stack-env-exemptions.json`.
|
|
- Create `scripts/portainer-stack-env-exemptions.json`
|
|
- List services that intentionally need no `stack.env`.
|
|
- Create `scripts/new-registry-auth.ps1`
|
|
- Generate registry push/pull credentials and a bcrypt `htpasswd` file without printing passwords to the terminal after creation.
|
|
|
|
Famapp repo files:
|
|
|
|
- Modify `.github/workflows/ci.yml` only if GitHub CI remains as temporary mirror validation.
|
|
- Create `.gitea/workflows/ci.yml`
|
|
- Mirror current CI checks in Gitea Actions.
|
|
- Create `.gitea/workflows/release.yml`
|
|
- Build and push `registry.ginnoir.com/ginnoir/famapp`.
|
|
- Modify `.release-it.json`
|
|
- Disable GitHub release creation or move release publication to Gitea-compatible tagging only.
|
|
- Modify `README.md`, `docs/RUNBOOK.md`, `docs/ENV.md`, `deploy/README.md`, and `deploy/compose.example.yaml`
|
|
- Replace GHCR image references with `registry.ginnoir.com/ginnoir/famapp`.
|
|
- Replace GitHub-primary clone/release text with Gitea-primary text.
|
|
- Verify `.git/config`
|
|
- Confirm there are no `manta-cyber` or `RTSOK` remotes.
|
|
- Leave GitHub `origin` only as a temporary mirror until final cleanup.
|
|
|
|
Live system targets:
|
|
|
|
- `ginnoir@valhalla`
|
|
- `~/valhalla-lab/portainer-compose.yml`
|
|
- `~/valhalla-lab/.env`
|
|
- `/config/registry/auth/htpasswd`
|
|
- `/config/portainer`
|
|
- `/storage1/labdata/registry`
|
|
- `/config/gitea`
|
|
- `/storage1/labdata/gitea/repositories`
|
|
- `/storage1/labdata/gitea/lfs`
|
|
|
|
## Task 1: Baseline Live State and Protect Against Cross-Repo Drift
|
|
|
|
**Files:**
|
|
- Read: `portainer-compose.yml`
|
|
- Read: `stacks/dev/docker-compose.yml`
|
|
- Read: `stacks/famapp/docker-compose.yml`
|
|
- Read: `C:\Users\MattC\Documents\famapp\.git\config`
|
|
|
|
- [ ] **Step 1: Confirm homelab repo state**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git -C C:\Users\MattC\Documents\homelabstack status --short --branch
|
|
git -C C:\Users\MattC\Documents\homelabstack remote -v
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
## main...origin/main
|
|
gitea https://gitea.ginnoir.com/ginnoir/homelabstack.git (fetch)
|
|
gitea https://gitea.ginnoir.com/ginnoir/homelabstack.git (push)
|
|
origin https://github.com/ginnoir/homelabstack.git (fetch)
|
|
origin https://github.com/ginnoir/homelabstack.git (push)
|
|
origin https://gitea.ginnoir.com/ginnoir/homelabstack.git (push)
|
|
```
|
|
|
|
- [ ] **Step 2: Confirm famapp has no work-repo remote**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git -C C:\Users\MattC\Documents\famapp remote -v
|
|
Select-String -Path C:\Users\MattC\Documents\famapp\.git\config -Pattern 'manta-cyber|RTSOK'
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
gitea https://gitea.ginnoir.com/ginnoir/famapp.git (fetch)
|
|
gitea https://gitea.ginnoir.com/ginnoir/famapp.git (push)
|
|
origin https://github.com/ginnoir/famapp.git (fetch)
|
|
origin https://github.com/ginnoir/famapp.git (push)
|
|
origin https://gitea.ginnoir.com/ginnoir/famapp.git (push)
|
|
```
|
|
|
|
`Select-String` should return no matches.
|
|
|
|
- [ ] **Step 3: Confirm live dependency containers**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes -o ConnectTimeout=8 ginnoir@valhalla "docker ps --format '{{.Names}} {{.Image}} {{.Status}}' | egrep '^(portainer|gitea|postgres_gitea|gitea_runner|registry|registry_ui|github_runner|caddy|famapp) '"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
portainer ... Up ...
|
|
gitea ... Up ...
|
|
postgres_gitea ... Up ...
|
|
gitea_runner ... Up ...
|
|
registry ... Up ...
|
|
registry_ui ... Up ...
|
|
github_runner ... Up ...
|
|
caddy ... Up ...
|
|
famapp ... Up ...
|
|
```
|
|
|
|
- [ ] **Step 4: Confirm registry and Gitea endpoints**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes -o ConnectTimeout=8 ginnoir@valhalla "curl -skI https://registry.ginnoir.com/v2/ | head -5; curl -skI https://gitea.ginnoir.com/ | head -5"
|
|
```
|
|
|
|
Expected before registry auth:
|
|
|
|
```text
|
|
HTTP/2 200
|
|
...
|
|
HTTP/2 200
|
|
```
|
|
|
|
Expected after registry auth in later tasks:
|
|
|
|
```text
|
|
HTTP/2 401
|
|
www-authenticate: Basic realm="Valhalla Registry"
|
|
...
|
|
HTTP/2 200
|
|
```
|
|
|
|
- [ ] **Step 5: Commit baseline-free state**
|
|
|
|
No commit is needed in this task. If any command shows unexpected dirty files, stop and identify whether they are user changes before continuing.
|
|
|
|
## Task 2: Add Stack Env Guardrails
|
|
|
|
**Files:**
|
|
- Create: `scripts/check-portainer-stack-env.ps1`
|
|
- Create: `scripts/portainer-stack-env-exemptions.json`
|
|
|
|
- [ ] **Step 1: Create the exemption file**
|
|
|
|
Create `scripts/portainer-stack-env-exemptions.json`:
|
|
|
|
```json
|
|
{
|
|
"authentik": ["authentik-redis"],
|
|
"dev": ["dbx", "redis_plane"],
|
|
"foundry": ["5etools"],
|
|
"monitoring": ["uptime-kuma"],
|
|
"owncloud": ["redis"]
|
|
}
|
|
```
|
|
|
|
Do not include `registry` or `registry_ui` here because they will move to the management plane.
|
|
|
|
- [ ] **Step 2: Create the guard script**
|
|
|
|
Create `scripts/check-portainer-stack-env.ps1`:
|
|
|
|
```powershell
|
|
[CmdletBinding()]
|
|
param(
|
|
[string]$StacksRoot = (Join-Path $PSScriptRoot "..\stacks"),
|
|
[string]$ExemptionsPath = (Join-Path $PSScriptRoot "portainer-stack-env-exemptions.json")
|
|
)
|
|
|
|
$ErrorActionPreference = "Stop"
|
|
$failures = New-Object System.Collections.Generic.List[string]
|
|
$exemptions = @{}
|
|
|
|
if (Test-Path $ExemptionsPath) {
|
|
$raw = Get-Content -Raw -LiteralPath $ExemptionsPath | ConvertFrom-Json
|
|
foreach ($property in $raw.PSObject.Properties) {
|
|
$exemptions[$property.Name] = @($property.Value)
|
|
}
|
|
}
|
|
|
|
function Get-ServiceBlocks {
|
|
param([string[]]$Lines)
|
|
|
|
$inServices = $false
|
|
$currentName = $null
|
|
$currentLines = New-Object System.Collections.Generic.List[string]
|
|
$blocks = New-Object System.Collections.Generic.List[object]
|
|
|
|
foreach ($line in $Lines) {
|
|
if ($line -match '^services:\s*$') {
|
|
$inServices = $true
|
|
continue
|
|
}
|
|
|
|
if (-not $inServices) {
|
|
continue
|
|
}
|
|
|
|
if ($line -match '^[A-Za-z0-9_-]+:\s*$') {
|
|
break
|
|
}
|
|
|
|
if ($line -match '^ ([A-Za-z0-9_-]+):\s*$') {
|
|
if ($null -ne $currentName) {
|
|
$blocks.Add([pscustomobject]@{
|
|
Name = $currentName
|
|
Lines = @($currentLines)
|
|
})
|
|
}
|
|
$currentName = $Matches[1]
|
|
$currentLines = New-Object System.Collections.Generic.List[string]
|
|
continue
|
|
}
|
|
|
|
if ($null -ne $currentName) {
|
|
$currentLines.Add($line)
|
|
}
|
|
}
|
|
|
|
if ($null -ne $currentName) {
|
|
$blocks.Add([pscustomobject]@{
|
|
Name = $currentName
|
|
Lines = @($currentLines)
|
|
})
|
|
}
|
|
|
|
return $blocks
|
|
}
|
|
|
|
Get-ChildItem -LiteralPath $StacksRoot -Directory | Sort-Object Name | ForEach-Object {
|
|
$stackName = $_.Name
|
|
$composePath = Join-Path $_.FullName "docker-compose.yml"
|
|
$stackEnvPath = Join-Path $_.FullName "stack.env"
|
|
|
|
if (-not (Test-Path $composePath)) {
|
|
return
|
|
}
|
|
|
|
if (-not (Test-Path $stackEnvPath)) {
|
|
$failures.Add("${stackName}: missing stack.env")
|
|
}
|
|
|
|
$lines = Get-Content -LiteralPath $composePath
|
|
$interpolations = Select-String -LiteralPath $composePath -Pattern '\$\{' |
|
|
Where-Object { $_.Line -notmatch '^\s*#' }
|
|
|
|
foreach ($match in $interpolations) {
|
|
$failures.Add("${stackName}: compose interpolation at $($match.Path):$($match.LineNumber): $($match.Line.Trim())")
|
|
}
|
|
|
|
$allowedMissing = @()
|
|
if ($exemptions.ContainsKey($stackName)) {
|
|
$allowedMissing = @($exemptions[$stackName])
|
|
}
|
|
|
|
foreach ($service in Get-ServiceBlocks -Lines $lines) {
|
|
$hasEnvFile = $service.Lines | Where-Object { $_ -match '^\s+env_file:\s*$' -or $_ -match '^\s+- stack\.env\s*$' }
|
|
if (-not $hasEnvFile -and $allowedMissing -notcontains $service.Name) {
|
|
$failures.Add("${stackName}/${service.Name}: missing env_file: stack.env")
|
|
}
|
|
}
|
|
}
|
|
|
|
if ($failures.Count -gt 0) {
|
|
$failures | ForEach-Object { Write-Error $_ }
|
|
exit 1
|
|
}
|
|
|
|
Write-Host "Portainer stack env checks passed."
|
|
```
|
|
|
|
- [ ] **Step 3: Run the guard and prove current failure**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\check-portainer-stack-env.ps1
|
|
```
|
|
|
|
Expected before fixes:
|
|
|
|
```text
|
|
Write-Error: dev: compose interpolation at ...
|
|
Write-Error: resume: compose interpolation at ...
|
|
```
|
|
|
|
- [ ] **Step 4: Commit guardrails**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add scripts/check-portainer-stack-env.ps1 scripts/portainer-stack-env-exemptions.json
|
|
git commit -m "test: add portainer stack env guard"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `test: add portainer stack env guard`.
|
|
|
|
## Task 3: Prepare Registry Auth Material
|
|
|
|
**Files:**
|
|
- Create: `scripts/new-registry-auth.ps1`
|
|
- Modify: `.env`
|
|
|
|
- [ ] **Step 1: Create registry credential generator**
|
|
|
|
Create `scripts/new-registry-auth.ps1`:
|
|
|
|
```powershell
|
|
[CmdletBinding()]
|
|
param(
|
|
[string]$OutputDirectory = (Join-Path $env:TEMP "valhalla-registry-auth"),
|
|
[string]$PushUser = "registry_push",
|
|
[string]$PullUser = "registry_pull"
|
|
)
|
|
|
|
$ErrorActionPreference = "Stop"
|
|
|
|
function New-Secret {
|
|
$bytes = New-Object byte[] 32
|
|
[System.Security.Cryptography.RandomNumberGenerator]::Fill($bytes)
|
|
return [Convert]::ToBase64String($bytes).TrimEnd("=")
|
|
}
|
|
|
|
New-Item -ItemType Directory -Force -Path $OutputDirectory | Out-Null
|
|
|
|
$pushPassword = New-Secret
|
|
$pullPassword = New-Secret
|
|
$htpasswdPath = Join-Path $OutputDirectory "htpasswd"
|
|
$envPath = Join-Path $OutputDirectory "registry.env"
|
|
|
|
$pushLine = docker run --rm --entrypoint htpasswd httpd:2 -Bbn $PushUser $pushPassword
|
|
$pullLine = docker run --rm --entrypoint htpasswd httpd:2 -Bbn $PullUser $pullPassword
|
|
|
|
Set-Content -LiteralPath $htpasswdPath -NoNewline -Value ($pushLine + "`n" + $pullLine + "`n")
|
|
Set-Content -LiteralPath $envPath -NoNewline -Value @"
|
|
REGISTRY_PUSH_USERNAME=$PushUser
|
|
REGISTRY_PUSH_PASSWORD=$pushPassword
|
|
REGISTRY_PULL_USERNAME=$PullUser
|
|
REGISTRY_PULL_PASSWORD=$pullPassword
|
|
"@
|
|
|
|
Write-Host "Created:"
|
|
Write-Host " $htpasswdPath"
|
|
Write-Host " $envPath"
|
|
Write-Host "Append registry.env values to .env, then copy htpasswd to valhalla:/config/registry/auth/htpasswd."
|
|
```
|
|
|
|
- [ ] **Step 2: Generate credentials**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\new-registry-auth.ps1
|
|
```
|
|
|
|
Expected: the script prints concrete paths under `$env:TEMP\valhalla-registry-auth` for `htpasswd` and `registry.env`.
|
|
|
|
- [ ] **Step 3: Add generated registry credentials to `.env`**
|
|
|
|
Open the generated `registry.env`, copy these four generated key/value lines into `.env` under a new section:
|
|
|
|
Add this section to `.env`, using the exact generated values from `$env:TEMP\valhalla-registry-auth\registry.env`:
|
|
|
|
```dotenv
|
|
# =============================================================
|
|
# DOCKER REGISTRY
|
|
# =============================================================
|
|
REGISTRY_PUSH_USERNAME=registry_push
|
|
REGISTRY_PUSH_PASSWORD=value generated by scripts/new-registry-auth.ps1
|
|
REGISTRY_PULL_USERNAME=registry_pull
|
|
REGISTRY_PULL_PASSWORD=value generated by scripts/new-registry-auth.ps1
|
|
```
|
|
|
|
Replace the two `value generated by...` strings with the actual generated password values before saving `.env`.
|
|
|
|
- [ ] **Step 4: Commit generator and `.env` credential additions**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add scripts/new-registry-auth.ps1 .env
|
|
git commit -m "chore: add registry auth credentials"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `chore: add registry auth credentials`.
|
|
|
|
## Task 4: Move Gitea and Registry to the Management Plane
|
|
|
|
**Files:**
|
|
- Modify: `portainer-compose.yml`
|
|
- Modify: `.env`
|
|
- Modify: `stacks/dev/docker-compose.yml`
|
|
- Modify: `stacks/dev/stack.env`
|
|
|
|
- [ ] **Step 1: Add management services to `portainer-compose.yml`**
|
|
|
|
In `portainer-compose.yml`, add these services after `vault` and before `github-runner`:
|
|
|
|
```yaml
|
|
postgres_gitea:
|
|
container_name: postgres_gitea
|
|
image: postgres:16-alpine
|
|
restart: unless-stopped
|
|
labels:
|
|
- "com.centurylabs.watchtower.enable=false"
|
|
networks:
|
|
- gitea_internal
|
|
environment:
|
|
POSTGRES_USER: gitea
|
|
POSTGRES_PASSWORD: ${GITEA_DB_PASSWORD}
|
|
POSTGRES_DB: gitea
|
|
volumes:
|
|
- /config/gitea/postgres:/var/lib/postgresql/data
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U gitea -d gitea"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
|
|
gitea:
|
|
container_name: gitea
|
|
image: gitea/gitea:latest
|
|
restart: unless-stopped
|
|
networks:
|
|
- gitea_internal
|
|
- edge
|
|
environment:
|
|
- USER_UID=1000
|
|
- USER_GID=1000
|
|
- GITEA__database__DB_TYPE=postgres
|
|
- GITEA__database__HOST=postgres_gitea:5432
|
|
- GITEA__database__NAME=gitea
|
|
- GITEA__database__USER=gitea
|
|
- GITEA__database__PASSWD=${GITEA_DB_PASSWORD}
|
|
- GITEA__server__DOMAIN=gitea.ginnoir.com
|
|
- GITEA__server__ROOT_URL=https://gitea.ginnoir.com
|
|
- GITEA__server__SSH_DOMAIN=gitea.ginnoir.com
|
|
- GITEA__server__SSH_PORT=2222
|
|
- GITEA__repository__ROOT=/repos
|
|
- GITEA__lfs__PATH=/repos-lfs
|
|
volumes:
|
|
- /config/gitea:/data
|
|
- /storage1/labdata/gitea/repositories:/repos
|
|
- /storage1/labdata/gitea/lfs:/repos-lfs
|
|
- /etc/localtime:/etc/localtime:ro
|
|
ports:
|
|
- "3030:3000"
|
|
- "2222:22"
|
|
depends_on:
|
|
postgres_gitea:
|
|
condition: service_healthy
|
|
labels:
|
|
- "com.centurylabs.watchtower.enable=false"
|
|
|
|
gitea-runner:
|
|
container_name: gitea_runner
|
|
image: gitea/act_runner:latest
|
|
restart: unless-stopped
|
|
networks:
|
|
- gitea_internal
|
|
environment:
|
|
- GITEA_INSTANCE_URL=https://gitea.ginnoir.com
|
|
- GITEA_RUNNER_REGISTRATION_TOKEN=${GITEA_RUNNER_REGISTRATION_TOKEN}
|
|
- GITEA_RUNNER_NAME=valhalla-management-runner
|
|
- GITEA_RUNNER_LABELS=ubuntu-latest:docker://node:24-bookworm,node-24:docker://node:24-bookworm
|
|
volumes:
|
|
- /config/gitea/act_runner:/data
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
- /home/ginnoir/valhalla-lab:/valhalla-lab
|
|
- /config/caddy:/config/caddy
|
|
depends_on:
|
|
gitea:
|
|
condition: service_started
|
|
labels:
|
|
- "com.centurylabs.watchtower.enable=false"
|
|
|
|
registry:
|
|
container_name: registry
|
|
image: registry:2
|
|
restart: unless-stopped
|
|
networks:
|
|
- registry_internal
|
|
- edge
|
|
environment:
|
|
- REGISTRY_AUTH=htpasswd
|
|
- REGISTRY_AUTH_HTPASSWD_REALM=Valhalla Registry
|
|
- REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd
|
|
- REGISTRY_STORAGE_DELETE_ENABLED=true
|
|
volumes:
|
|
- /storage1/labdata/registry:/var/lib/registry
|
|
- /config/registry/auth:/auth:ro
|
|
ports:
|
|
- "5000:5000"
|
|
labels:
|
|
- "com.centurylabs.watchtower.enable=false"
|
|
|
|
registry-ui:
|
|
container_name: registry_ui
|
|
image: joxit/docker-registry-ui:latest
|
|
restart: unless-stopped
|
|
networks:
|
|
- registry_internal
|
|
- edge
|
|
environment:
|
|
- SINGLE_REGISTRY=true
|
|
- REGISTRY_TITLE=Valhalla Registry
|
|
- NGINX_PROXY_PASS_URL=http://registry:5000
|
|
- SHOW_CONTENT_DIGEST=true
|
|
- DELETE_IMAGES=true
|
|
depends_on:
|
|
- registry
|
|
labels:
|
|
- "com.centurylabs.watchtower.enable=false"
|
|
```
|
|
|
|
- [ ] **Step 2: Add management networks to `portainer-compose.yml`**
|
|
|
|
At the bottom of `portainer-compose.yml`, add:
|
|
|
|
```yaml
|
|
gitea_internal:
|
|
name: gitea_internal
|
|
driver: bridge
|
|
registry_internal:
|
|
name: registry_internal
|
|
driver: bridge
|
|
```
|
|
|
|
- [ ] **Step 3: Remove moved services from `stacks/dev/docker-compose.yml`**
|
|
|
|
Remove these service blocks from `stacks/dev/docker-compose.yml`:
|
|
|
|
```text
|
|
gitea
|
|
gitea-runner
|
|
postgres_gitea
|
|
registry
|
|
registry-ui
|
|
```
|
|
|
|
Keep `code-server`, `bookstack`, `mariadb_bookstack`, `dbx`, and all Plane services in `stacks/dev/docker-compose.yml`.
|
|
|
|
- [ ] **Step 4: Move Gitea variables out of `stacks/dev/stack.env`**
|
|
|
|
Remove this key from `stacks/dev/stack.env` after confirming the same value exists in root `.env`:
|
|
|
|
```dotenv
|
|
GITEA_DB_PASSWORD=...
|
|
```
|
|
|
|
Keep `GITEA_DB_PASSWORD` in root `.env` for management-plane compose interpolation.
|
|
|
|
- [ ] **Step 5: Validate raw compose locally**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
docker compose -f .\portainer-compose.yml --env-file .\.env config | Out-Null
|
|
```
|
|
|
|
Expected: the command exits `0` with no error output.
|
|
|
|
- [ ] **Step 6: Commit management-plane extraction**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add portainer-compose.yml stacks/dev/docker-compose.yml stacks/dev/stack.env .env
|
|
git commit -m "refactor: move gitea and registry to management plane"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `refactor: move gitea and registry to management plane`.
|
|
|
|
## Task 5: Convert Remaining App Stack Interpolation to `stack.env`
|
|
|
|
**Files:**
|
|
- Modify: `stacks/dev/docker-compose.yml`
|
|
- Modify: `stacks/dev/stack.env`
|
|
- Modify: `stacks/resume/docker-compose.yml`
|
|
- Modify: `stacks/resume/stack.env`
|
|
|
|
- [ ] **Step 1: Convert BookStack variable names**
|
|
|
|
In `stacks/dev/stack.env`, add direct container keys preserving existing values:
|
|
|
|
```dotenv
|
|
APP_KEY=copy the current BOOKSTACK_APP_KEY value
|
|
DB_PASS=copy the current BOOKSTACK_DB_PASSWORD value
|
|
MYSQL_ROOT_PASSWORD=copy the current BOOKSTACK_DB_ROOT_PASSWORD value
|
|
MYSQL_PASSWORD=copy the current BOOKSTACK_DB_PASSWORD value
|
|
```
|
|
|
|
Replace each `copy the current ... value` string with the exact current secret value from `stacks/dev/stack.env`.
|
|
|
|
In `stacks/dev/docker-compose.yml`, replace BookStack environment entries that use `${...}` with direct names:
|
|
|
|
```yaml
|
|
- PUID=1000
|
|
- PGID=1000
|
|
- TZ
|
|
- APP_URL=https://docs.ginnoir.com
|
|
- APP_KEY
|
|
- DB_HOST=mariadb_bookstack
|
|
- DB_PORT=3306
|
|
- DB_USER=bookstack
|
|
- DB_PASS
|
|
- DB_DATABASE=bookstack
|
|
- AUTH_METHOD=oidc
|
|
- AUTH_AUTO_INITIATE=false
|
|
- OIDC_NAME=Authentik
|
|
- OIDC_ISSUER=https://auth.ginnoir.com/application/o/bookstack/
|
|
- OIDC_ISSUER_DISCOVER=true
|
|
- OIDC_END_SESSION_ENDPOINT=true
|
|
```
|
|
|
|
For `mariadb_bookstack`, replace environment with:
|
|
|
|
```yaml
|
|
- PUID=1000
|
|
- PGID=1000
|
|
- TZ
|
|
- MYSQL_ROOT_PASSWORD
|
|
- MYSQL_DATABASE=bookstack
|
|
- MYSQL_USER=bookstack
|
|
- MYSQL_PASSWORD
|
|
```
|
|
|
|
- [ ] **Step 2: Convert code-server `TZ` mapping**
|
|
|
|
In `stacks/dev/docker-compose.yml`, replace:
|
|
|
|
```yaml
|
|
- TZ=${TZ}
|
|
```
|
|
|
|
with:
|
|
|
|
```yaml
|
|
- TZ
|
|
```
|
|
|
|
- [ ] **Step 3: Convert Plane variables**
|
|
|
|
In `stacks/dev/stack.env`, add direct container keys preserving existing values:
|
|
|
|
```dotenv
|
|
SECRET_KEY=copy the current PLANE_SECRET_KEY value
|
|
DATABASE_URL=postgresql://plane:copy-the-current-PLANE-DB-PASSWORD-value@postgres_plane:5432/plane
|
|
AWS_ACCESS_KEY_ID=copy the current PLANE_MINIO_USER value
|
|
AWS_SECRET_ACCESS_KEY=copy the current PLANE_MINIO_PASSWORD value
|
|
MINIO_USER=copy the current PLANE_MINIO_USER value
|
|
MINIO_PASSWORD=copy the current PLANE_MINIO_PASSWORD value
|
|
POSTGRES_PASSWORD=copy the current PLANE_DB_PASSWORD value
|
|
MINIO_ROOT_USER=copy the current PLANE_MINIO_USER value
|
|
MINIO_ROOT_PASSWORD=copy the current PLANE_MINIO_PASSWORD value
|
|
```
|
|
|
|
Replace each `copy the current ... value` string with the exact current secret value from `stacks/dev/stack.env`. In `DATABASE_URL`, replace only `copy-the-current-PLANE-DB-PASSWORD-value` with the actual `PLANE_DB_PASSWORD` value.
|
|
|
|
In all Plane service environment blocks, replace `${...}` entries with direct variables. For example, `plane-api` should include:
|
|
|
|
```yaml
|
|
- SECRET_KEY
|
|
- DEBUG=0
|
|
- DATABASE_URL
|
|
- REDIS_URL=redis://redis_plane:6379/
|
|
- CELERY_BROKER_URL=redis://redis_plane:6379/0
|
|
- USE_MINIO=1
|
|
- AWS_REGION=us-east-1
|
|
- AWS_ACCESS_KEY_ID
|
|
- AWS_SECRET_ACCESS_KEY
|
|
- AWS_S3_ENDPOINT_URL=http://plane_minio:9000
|
|
- AWS_S3_BUCKET_NAME=uploads
|
|
- WEB_URL=https://plane.ginnoir.com
|
|
- CORS_ALLOWED_ORIGINS=https://plane.ginnoir.com
|
|
- GUNICORN_WORKERS=2
|
|
```
|
|
|
|
`postgres_plane` should use:
|
|
|
|
```yaml
|
|
environment:
|
|
POSTGRES_USER: plane
|
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
|
POSTGRES_DB: plane
|
|
```
|
|
|
|
Then change it again to list form to avoid compose interpolation:
|
|
|
|
```yaml
|
|
environment:
|
|
- POSTGRES_USER=plane
|
|
- POSTGRES_PASSWORD
|
|
- POSTGRES_DB=plane
|
|
```
|
|
|
|
`plane-create-bucket` should use:
|
|
|
|
```yaml
|
|
environment:
|
|
- MINIO_USER
|
|
- MINIO_PASSWORD
|
|
```
|
|
|
|
`plane-minio` should use:
|
|
|
|
```yaml
|
|
environment:
|
|
- MINIO_ROOT_USER
|
|
- MINIO_ROOT_PASSWORD
|
|
```
|
|
|
|
- [ ] **Step 4: Convert resume MinIO OIDC variables**
|
|
|
|
In `stacks/resume/stack.env`, add direct container keys preserving existing values:
|
|
|
|
```dotenv
|
|
MINIO_IDENTITY_OPENID_CLIENT_ID=copy the current MINIO_OIDC_CLIENT_ID value
|
|
MINIO_IDENTITY_OPENID_CLIENT_SECRET=copy the current MINIO_OIDC_CLIENT_SECRET value
|
|
```
|
|
|
|
Replace each `copy the current ... value` string with the exact current secret value from `stacks/resume/stack.env`.
|
|
|
|
In `stacks/resume/docker-compose.yml`, replace:
|
|
|
|
```yaml
|
|
- MINIO_IDENTITY_OPENID_CLIENT_ID=${MINIO_OIDC_CLIENT_ID}
|
|
- MINIO_IDENTITY_OPENID_CLIENT_SECRET=${MINIO_OIDC_CLIENT_SECRET}
|
|
```
|
|
|
|
with:
|
|
|
|
```yaml
|
|
- MINIO_IDENTITY_OPENID_CLIENT_ID
|
|
- MINIO_IDENTITY_OPENID_CLIENT_SECRET
|
|
```
|
|
|
|
- [ ] **Step 5: Run the env guard**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\check-portainer-stack-env.ps1
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Portainer stack env checks passed.
|
|
```
|
|
|
|
- [ ] **Step 6: Validate all stack compose files**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
Get-ChildItem .\stacks -Directory | ForEach-Object {
|
|
$compose = Join-Path $_.FullName "docker-compose.yml"
|
|
$env = Join-Path $_.FullName "stack.env"
|
|
if ((Test-Path $compose) -and (Test-Path $env)) {
|
|
Push-Location $_.FullName
|
|
docker compose --env-file stack.env config | Out-Null
|
|
Pop-Location
|
|
}
|
|
}
|
|
```
|
|
|
|
Expected: the command exits `0` with no error output.
|
|
|
|
- [ ] **Step 7: Commit env model cleanup**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add stacks/dev/docker-compose.yml stacks/dev/stack.env stacks/resume/docker-compose.yml stacks/resume/stack.env scripts/check-portainer-stack-env.ps1 scripts/portainer-stack-env-exemptions.json
|
|
git commit -m "fix: make portainer stacks use stack env only"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `fix: make portainer stacks use stack env only`.
|
|
|
|
## Task 6: Apply Management Plane and Verify Gitea/Registry Stability
|
|
|
|
**Files:**
|
|
- Use: `apply-compose.ps1`
|
|
- Use: `portainer-compose.yml`
|
|
- Use: `.env`
|
|
|
|
- [ ] **Step 1: Copy registry `htpasswd` to valhalla**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "mkdir -p /config/registry/auth"
|
|
scp "$env:TEMP\valhalla-registry-auth\htpasswd" "ginnoir@valhalla:/tmp/registry-htpasswd"
|
|
ssh -o BatchMode=yes ginnoir@valhalla "sudo mv /tmp/registry-htpasswd /config/registry/auth/htpasswd && sudo chmod 640 /config/registry/auth/htpasswd"
|
|
```
|
|
|
|
Expected: each command exits `0` with no error output.
|
|
|
|
- [ ] **Step 2: Push management `.env` to valhalla**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\apply-compose.ps1 -EnvFile
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Pushing .env ...
|
|
```
|
|
|
|
- [ ] **Step 3: Apply management plane**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\apply-compose.ps1 -Portainer
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Applying management plane (portainer, vault, github-runner, watchtower) ...
|
|
```
|
|
|
|
The status text can still mention `github-runner` until docs/scripts are updated later.
|
|
|
|
- [ ] **Step 4: Verify moved containers**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes -o ConnectTimeout=8 ginnoir@valhalla "docker ps --format '{{.Names}} {{.Status}} {{.Label \"com.docker.compose.project\"}}' | egrep '^(gitea|postgres_gitea|gitea_runner|registry|registry_ui|portainer) '"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
gitea Up ... valhalla-lab
|
|
postgres_gitea Up ... valhalla-lab
|
|
gitea_runner Up ... valhalla-lab
|
|
registry Up ... valhalla-lab
|
|
registry_ui Up ... valhalla-lab
|
|
portainer Up ... valhalla-lab
|
|
```
|
|
|
|
- [ ] **Step 5: Verify registry auth challenge**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "curl -skI https://registry.ginnoir.com/v2/ | head -8"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
HTTP/2 401
|
|
www-authenticate: Basic realm="Valhalla Registry"
|
|
docker-distribution-api-version: registry/2.0
|
|
```
|
|
|
|
- [ ] **Step 6: Verify Gitea endpoint and SSH**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "curl -skI https://gitea.ginnoir.com/ | head -5; ssh -o BatchMode=yes -o StrictHostKeyChecking=no -p 2222 git@localhost 2>&1 | head -5"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
HTTP/2 200
|
|
...
|
|
Hi there, ginnoir! You've successfully authenticated...
|
|
```
|
|
|
|
The SSH greeting username can differ if the key maps to a different Gitea account.
|
|
|
|
## Task 7: Add Gitea Actions for Homelab Caddy Reload
|
|
|
|
**Files:**
|
|
- Create: `.gitea/workflows/deploy-caddy.yml`
|
|
- Keep temporarily: `.github/workflows/deploy.yml`
|
|
- Modify later: `stacks/proxy/docker-compose.yml`
|
|
- Modify later: `CLAUDE.md`
|
|
|
|
- [ ] **Step 1: Create Gitea deploy workflow**
|
|
|
|
Create `.gitea/workflows/deploy-caddy.yml`:
|
|
|
|
```yaml
|
|
name: Deploy Caddyfile to valhalla
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- Caddyfile
|
|
- .gitea/workflows/deploy-caddy.yml
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
deploy:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Push Caddyfile and reload Caddy
|
|
run: |
|
|
cp Caddyfile /config/caddy/Caddyfile
|
|
docker exec caddy caddy reload --config /etc/caddy/Caddyfile
|
|
```
|
|
|
|
- [ ] **Step 2: Commit workflow**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add .gitea/workflows/deploy-caddy.yml
|
|
git commit -m "ci: add gitea caddy deploy workflow"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `ci: add gitea caddy deploy workflow`.
|
|
|
|
- [ ] **Step 3: Push to Gitea and GitHub mirror**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git push origin main
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
To https://github.com/ginnoir/homelabstack.git
|
|
To https://gitea.ginnoir.com/ginnoir/homelabstack.git
|
|
```
|
|
|
|
- [ ] **Step 4: Verify Gitea workflow run**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker logs --tail 200 gitea_runner"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
... Deploy Caddyfile to valhalla ...
|
|
... completed ...
|
|
```
|
|
|
|
- [ ] **Step 5: Verify Caddy is still serving**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "curl -skI https://gitea.ginnoir.com/ | head -5; curl -skI https://registry.ginnoir.com/v2/ | head -8"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
HTTP/2 200
|
|
...
|
|
HTTP/2 401
|
|
```
|
|
|
|
## Task 8: Recreate Portainer Stacks Against Gitea With No UI Env Vars
|
|
|
|
**Files:**
|
|
- Use: `stacks/*/docker-compose.yml`
|
|
- Use: `stacks/*/stack.env`
|
|
- Use: `scripts/check-portainer-stack-env.ps1`
|
|
|
|
- [ ] **Step 1: Run env guard before touching Portainer**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\check-portainer-stack-env.ps1
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Portainer stack env checks passed.
|
|
```
|
|
|
|
- [ ] **Step 2: Create a Gitea read-only PAT for Portainer**
|
|
|
|
In Gitea, create a token for a deployment user or `ginnoir` with repository read access for `ginnoir/homelabstack`.
|
|
|
|
Record these values for Portainer stack creation:
|
|
|
|
```text
|
|
Repository URL: https://gitea.ginnoir.com/ginnoir/homelabstack.git
|
|
Repository reference: refs/heads/main
|
|
Authentication type: Basic
|
|
Username: ginnoir
|
|
Password/token: the Gitea read-only PAT created for Portainer
|
|
```
|
|
|
|
- [ ] **Step 3: Recreate one low-risk stack first**
|
|
|
|
Use Portainer UI for the first stack to avoid API shape surprises:
|
|
|
|
```text
|
|
Stack name: remote
|
|
Repository URL: https://gitea.ginnoir.com/ginnoir/homelabstack.git
|
|
Repository reference: refs/heads/main
|
|
Compose path: stacks/remote/docker-compose.yml
|
|
Authentication: Basic, ginnoir, Gitea read-only PAT
|
|
GitOps updates: enabled
|
|
Mechanism: polling
|
|
Fetch interval: 5 minutes
|
|
Environment variables in Portainer UI: empty
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
remote stack deploys successfully
|
|
Portainer UI shows no manually declared environment variables
|
|
```
|
|
|
|
- [ ] **Step 4: Verify recreated low-risk stack**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker ps --format '{{.Names}} {{.Status}} {{.Label \"com.docker.compose.project\"}}' | egrep '^(hbbr|hbbs) '"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
hbbr Up ... remote
|
|
hbbs Up ... remote
|
|
```
|
|
|
|
- [ ] **Step 5: Recreate remaining app stacks**
|
|
|
|
Repeat the same Portainer Git stack settings for each stack:
|
|
|
|
```text
|
|
proxy stacks/proxy/docker-compose.yml
|
|
media stacks/media/docker-compose.yml
|
|
foundry stacks/foundry/docker-compose.yml
|
|
owncloud stacks/owncloud/docker-compose.yml
|
|
resume stacks/resume/docker-compose.yml
|
|
famapp stacks/famapp/docker-compose.yml
|
|
authentik stacks/authentik/docker-compose.yml
|
|
notify stacks/notify/docker-compose.yml
|
|
monitoring stacks/monitoring/docker-compose.yml
|
|
roms stacks/roms/docker-compose.yml
|
|
romhacks stacks/romhacks/docker-compose.yml
|
|
share stacks/share/docker-compose.yml
|
|
backup stacks/backup/docker-compose.yml
|
|
notes stacks/notes/docker-compose.yml
|
|
dev stacks/dev/docker-compose.yml
|
|
```
|
|
|
|
For each stack:
|
|
|
|
```text
|
|
Environment variables in Portainer UI: empty
|
|
GitOps updates: enabled
|
|
Mechanism: polling
|
|
Fetch interval: 5 minutes
|
|
Force redeployment: disabled
|
|
Re-pull image: enabled for stacks using mutable image tags
|
|
```
|
|
|
|
- [ ] **Step 6: Verify stack env values landed from `stack.env`**
|
|
|
|
For each recreated stack, inspect one representative container:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker inspect famapp --format '{{json .Config.Env}}' | jq -r '.[]' | sort | egrep '^(DATABASE_URL|AUTH_URL|MINIO_ROOT_USER)='"
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker inspect bookstack --format '{{json .Config.Env}}' | jq -r '.[]' | sort | egrep '^(APP_KEY|DB_PASS|OIDC_CLIENT_ID)='"
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker inspect plane_api --format '{{json .Config.Env}}' | jq -r '.[]' | sort | egrep '^(SECRET_KEY|DATABASE_URL|AWS_ACCESS_KEY_ID)='"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
The expected keys are present.
|
|
No value contains a trailing carriage return.
|
|
No value is empty unless intentionally empty in stack.env.
|
|
```
|
|
|
|
- [ ] **Step 7: Verify Portainer is polling Gitea**
|
|
|
|
Make a harmless comment-only commit in `stacks/remote/docker-compose.yml`, push it, and watch Portainer redeploy `remote`.
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git commit --allow-empty -m "test: verify portainer gitea polling"
|
|
git push origin main
|
|
```
|
|
|
|
Expected within 5 minutes:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker ps --format '{{.Names}} {{.Status}} {{.Label \"com.docker.compose.project\"}}' | egrep '^(hbbr|hbbs) '"
|
|
```
|
|
|
|
The `remote` container creation time should refresh or Portainer activity should show a GitOps update from Gitea.
|
|
|
|
## Task 9: Move Famapp Image Build and Release to Gitea Actions
|
|
|
|
**Files in `C:\Users\MattC\Documents\famapp`:**
|
|
- Create: `.gitea/workflows/ci.yml`
|
|
- Create: `.gitea/workflows/release.yml`
|
|
- Modify: `.release-it.json`
|
|
- Modify: `README.md`
|
|
- Modify: `docs/RUNBOOK.md`
|
|
- Modify: `docs/ENV.md`
|
|
- Modify: `deploy/README.md`
|
|
- Modify: `deploy/compose.example.yaml`
|
|
|
|
- [ ] **Step 1: Create Gitea CI workflow**
|
|
|
|
Create `C:\Users\MattC\Documents\famapp\.gitea\workflows\ci.yml`:
|
|
|
|
```yaml
|
|
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
|
|
jobs:
|
|
checks:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Enable pnpm
|
|
run: |
|
|
corepack enable
|
|
corepack prepare pnpm@10.33.3 --activate
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Typecheck
|
|
run: pnpm typecheck
|
|
|
|
- name: Lint
|
|
run: pnpm lint
|
|
|
|
- name: Format check
|
|
run: pnpm format:check
|
|
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Enable pnpm
|
|
run: |
|
|
corepack enable
|
|
corepack prepare pnpm@10.33.3 --activate
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Build
|
|
run: pnpm build
|
|
env:
|
|
DATABASE_URL: postgres://ci_user:ci_password@localhost:5432/ci_database
|
|
AUTH_SECRET: ci-auth-secret-for-build
|
|
NEXT_PUBLIC_APP_URL: http://localhost:3000
|
|
```
|
|
|
|
- [ ] **Step 2: Create Gitea release workflow**
|
|
|
|
Create `C:\Users\MattC\Documents\famapp\.gitea\workflows\release.yml`:
|
|
|
|
```yaml
|
|
name: Release Image
|
|
|
|
on:
|
|
push:
|
|
tags:
|
|
- "v*"
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
build-and-push:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Derive image tags
|
|
id: meta
|
|
shell: bash
|
|
run: |
|
|
VERSION="${GITEA_REF_NAME#v}"
|
|
MAJOR_MINOR="$(printf '%s' "$VERSION" | awk -F. '{print $1"."$2}')"
|
|
{
|
|
echo "version=$VERSION"
|
|
echo "major_minor=$MAJOR_MINOR"
|
|
echo "image=registry.ginnoir.com/ginnoir/famapp"
|
|
} >> "$GITEA_OUTPUT"
|
|
|
|
- name: Login to registry
|
|
run: |
|
|
echo "${{ secrets.REGISTRY_PUSH_PASSWORD }}" | docker login registry.ginnoir.com \
|
|
--username "${{ secrets.REGISTRY_PUSH_USERNAME }}" \
|
|
--password-stdin
|
|
|
|
- name: Build image
|
|
run: |
|
|
docker build \
|
|
-t "${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.version }}" \
|
|
-t "${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.major_minor }}" \
|
|
-t "${{ steps.meta.outputs.image }}:latest" \
|
|
.
|
|
|
|
- name: Push image
|
|
run: |
|
|
docker push "${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.version }}"
|
|
docker push "${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.major_minor }}"
|
|
docker push "${{ steps.meta.outputs.image }}:latest"
|
|
```
|
|
|
|
- [ ] **Step 3: Add Gitea repository secrets**
|
|
|
|
In Gitea repository `ginnoir/famapp`, add repository-level Actions secrets:
|
|
|
|
```text
|
|
REGISTRY_PUSH_USERNAME=copy the current homelabstack .env REGISTRY_PUSH_USERNAME value
|
|
REGISTRY_PUSH_PASSWORD=copy the current homelabstack .env REGISTRY_PUSH_PASSWORD value
|
|
```
|
|
|
|
Use the exact values from `C:\Users\MattC\Documents\homelabstack\.env` when saving the Gitea secrets.
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Secrets are visible by name in Gitea repository settings.
|
|
Secret values are not visible after saving.
|
|
```
|
|
|
|
- [ ] **Step 4: Disable GitHub release creation in `.release-it.json`**
|
|
|
|
In `C:\Users\MattC\Documents\famapp\.release-it.json`, replace:
|
|
|
|
```json
|
|
"github": {
|
|
"release": true,
|
|
"releaseName": "v${version}"
|
|
},
|
|
```
|
|
|
|
with:
|
|
|
|
```json
|
|
"github": {
|
|
"release": false
|
|
},
|
|
```
|
|
|
|
- [ ] **Step 5: Update famapp docs and compose example**
|
|
|
|
Replace GHCR image references with:
|
|
|
|
```text
|
|
registry.ginnoir.com/ginnoir/famapp
|
|
```
|
|
|
|
Required replacements:
|
|
|
|
```text
|
|
ghcr.io/ginnoir/famapp:latest -> registry.ginnoir.com/ginnoir/famapp:latest
|
|
ghcr.io/ginnoir/famapp:v0.x.y -> registry.ginnoir.com/ginnoir/famapp:v0.x.y
|
|
FAMAPP_IMAGE=ghcr.io/ginnoir/famapp:v0.4.7 -> FAMAPP_IMAGE=registry.ginnoir.com/ginnoir/famapp:v0.4.7
|
|
```
|
|
|
|
- [ ] **Step 6: Run local famapp checks**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm --dir C:\Users\MattC\Documents\famapp typecheck
|
|
pnpm --dir C:\Users\MattC\Documents\famapp lint
|
|
pnpm --dir C:\Users\MattC\Documents\famapp format:check
|
|
docker build -t registry.ginnoir.com/ginnoir/famapp:local-test C:\Users\MattC\Documents\famapp
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
typecheck passes
|
|
lint exits 0
|
|
format:check exits 0
|
|
docker build completes
|
|
```
|
|
|
|
- [ ] **Step 7: Commit famapp workflow migration**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git -C C:\Users\MattC\Documents\famapp add .gitea/workflows/ci.yml .gitea/workflows/release.yml .release-it.json README.md docs/RUNBOOK.md docs/ENV.md deploy/README.md deploy/compose.example.yaml
|
|
git -C C:\Users\MattC\Documents\famapp commit -m "ci: publish famapp images to valhalla registry"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `ci: publish famapp images to valhalla registry`.
|
|
|
|
- [ ] **Step 8: Push famapp to Gitea and GitHub mirror**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git -C C:\Users\MattC\Documents\famapp push origin main
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
To https://github.com/ginnoir/famapp.git
|
|
To https://gitea.ginnoir.com/ginnoir/famapp.git
|
|
```
|
|
|
|
## Task 10: Publish and Deploy Famapp From the Self-Hosted Registry
|
|
|
|
**Files:**
|
|
- Modify: `stacks/famapp/docker-compose.yml`
|
|
|
|
- [ ] **Step 1: Push a famapp release tag**
|
|
|
|
Run from famapp:
|
|
|
|
```powershell
|
|
git -C C:\Users\MattC\Documents\famapp tag v0.5.4
|
|
git -C C:\Users\MattC\Documents\famapp push origin v0.5.4
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
To https://github.com/ginnoir/famapp.git
|
|
To https://gitea.ginnoir.com/ginnoir/famapp.git
|
|
```
|
|
|
|
- [ ] **Step 2: Verify image exists in registry**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
$envContent = Get-Content C:\Users\MattC\Documents\homelabstack\.env
|
|
$pullUser = ($envContent | Where-Object { $_ -match '^REGISTRY_PULL_USERNAME=' }) -replace '^REGISTRY_PULL_USERNAME=', ''
|
|
$pullPass = ($envContent | Where-Object { $_ -match '^REGISTRY_PULL_PASSWORD=' }) -replace '^REGISTRY_PULL_PASSWORD=', ''
|
|
ssh -o BatchMode=yes ginnoir@valhalla "echo '$pullPass' | docker login registry.ginnoir.com --username '$pullUser' --password-stdin && docker pull registry.ginnoir.com/ginnoir/famapp:latest"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Login Succeeded
|
|
latest: Pulling from ginnoir/famapp
|
|
Status: Downloaded newer image for registry.ginnoir.com/ginnoir/famapp:latest
|
|
```
|
|
|
|
- [ ] **Step 3: Point homelab famapp compose at local registry**
|
|
|
|
In `stacks/famapp/docker-compose.yml`, replace:
|
|
|
|
```yaml
|
|
image: ghcr.io/ginnoir/famapp:latest
|
|
```
|
|
|
|
with:
|
|
|
|
```yaml
|
|
image: registry.ginnoir.com/ginnoir/famapp:latest
|
|
```
|
|
|
|
- [ ] **Step 4: Commit famapp compose switch**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add stacks/famapp/docker-compose.yml
|
|
git commit -m "fix(famapp): pull image from valhalla registry"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `fix(famapp): pull image from valhalla registry`.
|
|
|
|
- [ ] **Step 5: Push and wait for Portainer redeploy**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git push origin main
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
To https://github.com/ginnoir/homelabstack.git
|
|
To https://gitea.ginnoir.com/ginnoir/homelabstack.git
|
|
```
|
|
|
|
- [ ] **Step 6: Verify famapp is using registry image**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker inspect famapp --format '{{.Config.Image}}'; curl -skI https://fam.ginnoir.com/ | head -5"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
registry.ginnoir.com/ginnoir/famapp:latest
|
|
HTTP/2 200
|
|
```
|
|
|
|
## Task 11: Update Docs and Retire GitHub Runner After Staged Verification
|
|
|
|
**Files:**
|
|
- Modify: `CLAUDE.md`
|
|
- Modify: `AGENTS.md`
|
|
- Modify: `apply-compose.ps1`
|
|
- Modify: `portainer-compose.yml`
|
|
- Modify: `stacks/proxy/docker-compose.yml`
|
|
- Modify: `.github/workflows/deploy.yml`
|
|
|
|
- [ ] **Step 1: Update homelab docs**
|
|
|
|
In `CLAUDE.md` and `AGENTS.md`, update deployment channel language:
|
|
|
|
```text
|
|
stacks/domain-name/* -> git push to Gitea main -> Portainer polls every 5 minutes
|
|
Caddyfile -> git push to Gitea main -> Gitea Actions copies Caddyfile and reloads Caddy
|
|
GitHub -> temporary mirror only during staged migration
|
|
Portainer app stacks -> no Portainer UI env vars; use stack.env via env_file
|
|
```
|
|
|
|
- [ ] **Step 2: Update proxy comment**
|
|
|
|
In `stacks/proxy/docker-compose.yml`, replace:
|
|
|
|
```text
|
|
hot-reloaded by the GitHub Actions runner on push
|
|
```
|
|
|
|
with:
|
|
|
|
```text
|
|
hot-reloaded by the Gitea Actions runner on push
|
|
```
|
|
|
|
- [ ] **Step 3: Update apply script management text**
|
|
|
|
In `apply-compose.ps1`, replace:
|
|
|
|
```powershell
|
|
Write-Host "Applying management plane (portainer, vault, github-runner, watchtower) ..."
|
|
```
|
|
|
|
with:
|
|
|
|
```powershell
|
|
Write-Host "Applying management plane (portainer, vault, gitea, registry, watchtower) ..."
|
|
```
|
|
|
|
- [ ] **Step 4: Remove GitHub runner after two successful Gitea deploy cycles**
|
|
|
|
Remove the `github-runner` service block from `portainer-compose.yml` only after:
|
|
|
|
```text
|
|
Gitea Actions has reloaded Caddy successfully at least twice.
|
|
Portainer has redeployed at least one stack from Gitea polling.
|
|
Famapp has been built by Gitea Actions and pulled from registry.ginnoir.com.
|
|
```
|
|
|
|
- [ ] **Step 5: Keep `.github/workflows/deploy.yml` until final GitHub mirror retirement**
|
|
|
|
During the temporary mirror period, disable rather than delete the GitHub workflow by changing triggers to manual only:
|
|
|
|
```yaml
|
|
name: Deploy to valhalla
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
deploy:
|
|
runs-on: self-hosted
|
|
steps:
|
|
- name: Disabled
|
|
run: echo "Deployment moved to Gitea Actions."
|
|
```
|
|
|
|
- [ ] **Step 6: Commit documentation and runner retirement**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add CLAUDE.md AGENTS.md apply-compose.ps1 portainer-compose.yml stacks/proxy/docker-compose.yml .github/workflows/deploy.yml
|
|
git commit -m "docs: document gitea-first homelab deploys"
|
|
```
|
|
|
|
Expected: commit exits `0` and prints a one-line summary for `docs: document gitea-first homelab deploys`.
|
|
|
|
## Task 12: Final Verification and Vault Write-Back
|
|
|
|
**Files:**
|
|
- Read: `CLAUDE.md`
|
|
- Read: `AGENTS.md`
|
|
- Use: Obsidian note `Homelab/_Claude.md` when vault MCP is available
|
|
|
|
- [ ] **Step 1: Run repo verification**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\check-portainer-stack-env.ps1
|
|
docker compose -f .\portainer-compose.yml --env-file .\.env config | Out-Null
|
|
git status --short --branch
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
Portainer stack env checks passed.
|
|
docker compose config exits 0
|
|
## main...origin/main
|
|
```
|
|
|
|
- [ ] **Step 2: Verify live management plane**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "docker compose -f ~/valhalla-lab/portainer-compose.yml --env-file ~/valhalla-lab/.env ps"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
portainer Up
|
|
vault Up
|
|
gitea Up
|
|
postgres_gitea Up
|
|
gitea_runner Up
|
|
registry Up
|
|
registry_ui Up
|
|
watchtower Up
|
|
```
|
|
|
|
- [ ] **Step 3: Verify public/internal endpoints**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
ssh -o BatchMode=yes ginnoir@valhalla "curl -skI https://portainer.ginnoir.com/ | head -5; curl -skI https://gitea.ginnoir.com/ | head -5; curl -skI https://registry.ginnoir.com/v2/ | head -8; curl -skI https://fam.ginnoir.com/ | head -5"
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
HTTP/2 200
|
|
HTTP/2 200
|
|
HTTP/2 401
|
|
HTTP/2 200
|
|
```
|
|
|
|
- [ ] **Step 4: Verify no app stack relies on Portainer UI env vars**
|
|
|
|
For each recreated Portainer stack, open Portainer UI and confirm:
|
|
|
|
```text
|
|
Stack > Environment variables: empty
|
|
Stack > GitOps updates: enabled
|
|
Repository URL: https://gitea.ginnoir.com/ginnoir/homelabstack.git
|
|
```
|
|
|
|
Record any stack with non-empty UI env vars as a blocker and recreate it before continuing.
|
|
|
|
- [ ] **Step 5: Write durable findings to Obsidian**
|
|
|
|
When `mcp__obsidian__vault_patch` or the Obsidian MCP is available, append to `Homelab/_Claude.md`:
|
|
|
|
```markdown
|
|
## Gitea-first deploy architecture
|
|
|
|
- Homelab app stacks now poll `https://gitea.ginnoir.com/ginnoir/homelabstack.git` from Portainer.
|
|
- GitHub is only a temporary mirror during the cutover period.
|
|
- Gitea, `postgres_gitea`, `gitea_runner`, `registry`, and `registry_ui` are management-plane services, not Portainer app-stack services.
|
|
- Portainer app stacks must not define UI environment variables; stack config comes from repo-local `stack.env` via `env_file: stack.env`.
|
|
- Famapp images publish to `registry.ginnoir.com/ginnoir/famapp` from Gitea Actions.
|
|
- Registry auth uses separate push and pull credentials; Portainer/Docker pulls use the pull credential.
|
|
```
|
|
|
|
- [ ] **Step 6: Final commit if vault/docs changed**
|
|
|
|
If only the vault was patched through MCP, no repo commit is needed. If repo docs changed during final verification, run:
|
|
|
|
```powershell
|
|
git add CLAUDE.md AGENTS.md
|
|
git commit -m "docs: record gitea-first deploy verification"
|
|
```
|
|
|
|
Expected only if docs changed: commit exits `0` and prints a one-line summary for `docs: record gitea-first deploy verification`.
|
|
|
|
## Rollback Plan
|
|
|
|
- If Gitea management-plane apply fails:
|
|
- Re-apply the previous `portainer-compose.yml` commit with `apply-compose.ps1 -Portainer`.
|
|
- Gitea data is bind-mounted and remains under `/config/gitea` and `/storage1/labdata/gitea`.
|
|
- If registry auth blocks famapp pulls:
|
|
- Temporarily set `stacks/famapp/docker-compose.yml` back to `ghcr.io/ginnoir/famapp:latest`.
|
|
- Push to Gitea/GitHub mirror.
|
|
- Recreate Portainer registry credentials before switching back.
|
|
- If Portainer cannot poll Gitea:
|
|
- Keep GitHub mirror stack definitions active.
|
|
- Recreate only the affected stack against GitHub until Gitea auth/TLS is fixed.
|
|
- If `dev` stack recreation fails after removing Gitea/registry:
|
|
- Management-plane Gitea and registry remain independent.
|
|
- Restore the previous `stacks/dev/docker-compose.yml` only for non-moved services if needed.
|
|
|
|
## Self-Review
|
|
|
|
- Spec coverage:
|
|
- Gitea-first migration is covered by Tasks 4, 7, 8, 11, and 12.
|
|
- Temporary GitHub mirror is covered by Tasks 7, 8, 9, 10, and 11.
|
|
- Docker registry setup and famapp image publishing are covered by Tasks 3, 4, 9, and 10.
|
|
- Portainer `stack.env` enforcement is covered by Tasks 2, 5, 8, and 12.
|
|
- Work-repo remote protection is covered by Task 1.
|
|
- Red-flag scan:
|
|
- Secret values are intentionally generated at execution time by `scripts/new-registry-auth.ps1`; plan text names exact keys and commands without embedding secrets.
|
|
- No implementation step depends on an unnamed file or undefined command.
|
|
- Type and name consistency:
|
|
- Registry image path is consistently `registry.ginnoir.com/ginnoir/famapp`.
|
|
- Gitea repository path is consistently `https://gitea.ginnoir.com/ginnoir/homelabstack.git`.
|
|
- Guard script and exemption file paths are consistent across tasks.
|