diff --git a/AGENTS.md b/AGENTS.md index efcfeec4..1b65311e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,6 +27,16 @@ just instance show # Print active instance number just instance use 5 # Set active instance for this workspace ``` +**Instance env files**: + +```bash +just instance-init 5 # Initialize .instances/instance-5/*.env +just instance-env init 5 # Create from app/.env or app/.env.example +just instance-env update 5 # Re-apply instance-scoped vars +just instance-env copy 5 6 # Copy env setup from instance 5 -> 6 +just instance-env show 6 # Show file status and computed values +``` + **URLs**: - Frontend: `http://localhost:${5173 + instance*100}` @@ -43,6 +53,7 @@ Local development runs as **multiple app instances** (PM2) on top of **one share - Per-instance apps: `shipsec-{frontend,backend,worker}-N`. - Isolation is via per-instance DB + Temporal namespace/task queue + Kafka topic suffixing + instance-scoped Kafka consumer groups/client IDs (not per-instance infra containers). - The workspace can have an **active instance** (stored in `.shipsec-instance`, gitignored). +- Instance env files are stored at `.instances/instance-N/{backend,worker,frontend}.env` and can be managed with `just instance-env ...`. **Agent rule:** before running any dev commands, ensure you’re targeting the intended instance. @@ -50,7 +61,7 @@ Local development runs as **multiple app instances** (PM2) on top of **one share - If the task is ambiguous (logs, curl, E2E, “run locally”, etc.), ask the user which instance to use. - If the user says “use instance N”, prefer either: - `just instance use N` then run `just dev` / `bun run test:e2e`, or - - explicit instance invocation (`just dev N ...`) for one-off commands. + - explicit env override (`SHIPSEC_INSTANCE=N just dev ...`) for one-off commands. **Ports / URLs** @@ -61,7 +72,7 @@ Local development runs as **multiple app instances** (PM2) on top of **one share **E2E tests** - E2E targets the backend for `SHIPSEC_INSTANCE` (or the active instance). -- When asked to run E2E, confirm the instance and ensure that instance is running: `just dev N start`. +- When asked to run E2E, confirm the instance and ensure that instance is running: `SHIPSEC_INSTANCE=N just dev start` (or `just instance use N` then `just dev start`). **Keep docs in sync** diff --git a/README.md b/README.md index a99b3f7a..9c5f6aa4 100644 --- a/README.md +++ b/README.md @@ -137,8 +137,12 @@ Run multiple isolated dev instances on one machine for parallel feature work: # Instance 0 (default) just dev -# Instance 1 — offset ports (frontend :5273, backend :3311) -SHIPSEC_INSTANCE=1 just dev +# Switch active workspace instance +just instance use 1 +just dev + +# Manage per-instance env files +just instance-env init 1 ``` Each instance gets its own frontend port, backend port, database, and Temporal namespace while sharing a single Docker infra stack. See [Multi-Instance Development Guide](docs/MULTI-INSTANCE-DEV.md) for full details. diff --git a/docs/MULTI-INSTANCE-DEV.md b/docs/MULTI-INSTANCE-DEV.md index acd7ed7f..8b94fc47 100644 --- a/docs/MULTI-INSTANCE-DEV.md +++ b/docs/MULTI-INSTANCE-DEV.md @@ -8,15 +8,15 @@ ShipSec Studio supports running multiple isolated dev instances (0-9) on one mac # Instance 0 (default) — works exactly as before just dev -# Instance 1 — runs on offset ports (frontend :5273, backend :3311) -SHIPSEC_INSTANCE=1 just dev +# Check active instance for this workspace +just instance show -# Or persist the choice for this workspace -echo 1 > .shipsec-instance -just dev # now uses instance 1 +# Persist instance 1 for this workspace +just instance use 1 +just dev # now starts instance 1 # Stop your instance -SHIPSEC_INSTANCE=1 just dev stop +just dev stop ``` ## How It Works @@ -41,9 +41,36 @@ The instance is resolved in this order: SHIPSEC_INSTANCE=2 just dev # Per-workspace (persistent) -echo 2 > .shipsec-instance +just instance use 2 +just dev +``` + +## Instance Env Files + +PM2 can read per-instance env files from: + +- `.instances/instance-N/backend.env` +- `.instances/instance-N/worker.env` +- `.instances/instance-N/frontend.env` + +Use `just instance-env` to manage them: + +```bash +# Create env files for an instance (uses app/.env, or falls back to app/.env.example) +just instance-env init 2 + +# Re-apply only instance-scoped values (ports, DB, Temporal namespace) +just instance-env update 2 + +# Copy env files between instances and rescope values +just instance-env copy 2 5 + +# Show file presence + computed values +just instance-env show 5 ``` +`just dev` auto-initializes env files for the active instance when missing (without overwriting existing files). + ## Port Map Ports are offset by `N * 100`: @@ -82,23 +109,29 @@ The Vite dev server proxies `/api` calls to the correct backend port automatical ## Commands -All commands respect `SHIPSEC_INSTANCE`: +All commands respect `SHIPSEC_INSTANCE` (or the active instance selected by `just instance use N`): ```bash +# Select instance once per workspace +just instance use 1 + # Start -SHIPSEC_INSTANCE=1 just dev +just dev # Stop (only stops PM2 apps; infra stays running for other instances) -SHIPSEC_INSTANCE=1 just dev stop +just dev stop # Logs (filtered to your instance's PM2 apps) -SHIPSEC_INSTANCE=1 just dev logs +just dev logs # Status -SHIPSEC_INSTANCE=1 just dev status +just dev status # Clean (stops PM2 apps; only tears down infra if instance 0) -SHIPSEC_INSTANCE=1 just dev clean +just dev clean + +# One-off override without changing active workspace instance +SHIPSEC_INSTANCE=3 just dev ``` When stopping/cleaning instance 0, Docker infra is also torn down. For non-zero instances, only the PM2 apps are stopped (since other instances may still need the shared infra). @@ -128,9 +161,10 @@ lsof -i :3311 # backend instance 1 ### Instance is unhealthy but infra is fine ```bash -SHIPSEC_INSTANCE=1 just dev logs -SHIPSEC_INSTANCE=1 just dev stop -SHIPSEC_INSTANCE=1 just dev +just instance use 1 +just dev logs +just dev stop +just dev ``` ### Infra conflicts / stuck containers diff --git a/justfile b/justfile index 8d21b0e8..ba937f69 100644 --- a/justfile +++ b/justfile @@ -85,6 +85,11 @@ dev action="start": exit 1 fi + # Auto-init instance env files if missing (never overwrites) + if [ "$INST" != "0" ] || [ ! -d ".instances/instance-0" ]; then + ./scripts/instance-env.sh init "$INST" + fi + if [ "$SECURE_MODE" = "true" ]; then echo "🔐 Starting development environment (Clerk auth, instance ${INST})..." @@ -541,6 +546,46 @@ build: docker compose -f docker/docker-compose.full.yml build echo "✅ Images built" +# Manage active multi-instance selection +instance action="show" value="": + #!/usr/bin/env bash + set -euo pipefail + + case "{{action}}" in + show) + ./scripts/active-instance.sh get + ;; + use) + if [ -z "{{value}}" ]; then + echo "Usage: just instance use <0-9>" + exit 1 + fi + ./scripts/active-instance.sh set "{{value}}" + ;; + *) + echo "Usage: just instance [show|use <0-9>]" + exit 1 + ;; + esac + +# === Instance Environment === + +# Initialize instance env files (creates from .env or .env.example, never overwrites) +instance-init instance="": + #!/usr/bin/env bash + set -euo pipefail + INST="{{instance}}" + if [ -z "$INST" ]; then + INST="$(./scripts/active-instance.sh get)" + fi + ./scripts/instance-env.sh init "$INST" + +# Manage instance env files (init, update, copy, show) +instance-env +args: + #!/usr/bin/env bash + set -euo pipefail + ./scripts/instance-env.sh {{args}} + # === Help === help: @@ -576,6 +621,15 @@ help: @echo " just infra logs View infrastructure logs" @echo " just infra clean Remove infrastructure data" @echo "" + @echo "Multi-Instance:" + @echo " just instance show Show active instance" + @echo " just instance use N Persist active instance in .shipsec-instance" + @echo " just instance-init [N] Init env files for instance N" + @echo " just instance-env init [N] [--force] Generate env files" + @echo " just instance-env update [N] Patch instance-specific vars" + @echo " just instance-env copy SRC DEST Copy env between instances" + @echo " just instance-env show [N] Show instance config" + @echo "" @echo "Utilities:" @echo " just status Show status of all services" @echo " just db-reset Reset database" diff --git a/scripts/instance-env.sh b/scripts/instance-env.sh new file mode 100755 index 00000000..41a194f2 --- /dev/null +++ b/scripts/instance-env.sh @@ -0,0 +1,340 @@ +#!/usr/bin/env bash +# Manage per-instance env files for local multi-instance development. +# +# Usage: +# ./scripts/instance-env.sh init [N] [--force] +# ./scripts/instance-env.sh update [N] +# ./scripts/instance-env.sh copy SOURCE DEST [--force] +# ./scripts/instance-env.sh show [N] + +set -euo pipefail + +APPS=(backend worker frontend) +BASE_BACKEND_PORT=3211 +BASE_FRONTEND_PORT=5173 +BASE_DB_NAME="shipsec" +BASE_TEMPORAL_NS="shipsec-dev" + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +INSTANCES_DIR="$ROOT_DIR/.instances" + +log_info() { echo "[info] $*"; } +log_ok() { echo "[ok] $*"; } +log_warn() { echo "[warn] $*"; } +log_err() { echo "[err] $*" >&2; } + +fail() { + log_err "$*" + exit 1 +} + +validate_instance() { + local instance="$1" + [[ "$instance" =~ ^[0-9]+$ ]] || fail "Instance must be a number 0-9. Got: $instance" + [ "$instance" -ge 0 ] && [ "$instance" -le 9 ] || fail "Instance must be 0-9. Got: $instance" +} + +instance_dir() { + echo "$INSTANCES_DIR/instance-$1" +} + +get_backend_port() { + echo $((BASE_BACKEND_PORT + $1 * 100)) +} + +get_frontend_port() { + echo $((BASE_FRONTEND_PORT + $1 * 100)) +} + +get_db_name() { + if [ "$1" -eq 0 ]; then + echo "$BASE_DB_NAME" + else + echo "${BASE_DB_NAME}_instance_$1" + fi +} + +get_temporal_ns() { + if [ "$1" -eq 0 ]; then + echo "$BASE_TEMPORAL_NS" + else + echo "${BASE_TEMPORAL_NS}-$1" + fi +} + +get_db_url() { + echo "postgresql://shipsec:shipsec@localhost:5433/$(get_db_name "$1")" +} + +get_studio_api_url() { + echo "http://localhost:$(get_backend_port "$1")/api/v1" +} + +get_vite_api_url() { + echo "http://localhost:$(get_backend_port "$1")" +} + +resolve_source_env() { + local app="$1" + local env_path="$ROOT_DIR/$app/.env" + local example_path="$ROOT_DIR/$app/.env.example" + + if [ -f "$env_path" ]; then + echo "$env_path" + elif [ -f "$example_path" ]; then + echo "$example_path" + else + echo "" + fi +} + +set_var() { + local file="$1" + local key="$2" + local value="$3" + + if grep -qE "^${key}=" "$file" 2>/dev/null; then + sed -i.bak "s|^${key}=.*|${key}=${value}|" "$file" + rm -f "${file}.bak" + else + printf '\n%s=%s\n' "$key" "$value" >>"$file" + fi +} + +apply_instance_vars() { + local file="$1" + local instance="$2" + local app="$3" + + set_var "$file" "DATABASE_URL" "$(get_db_url "$instance")" + set_var "$file" "TEMPORAL_NAMESPACE" "$(get_temporal_ns "$instance")" + set_var "$file" "TEMPORAL_TASK_QUEUE" "$(get_temporal_ns "$instance")" + + case "$app" in + backend) + set_var "$file" "PORT" "$(get_backend_port "$instance")" + ;; + worker) + set_var "$file" "STUDIO_API_BASE_URL" "$(get_studio_api_url "$instance")" + ;; + frontend) + set_var "$file" "VITE_API_URL" "$(get_vite_api_url "$instance")" + ;; + *) + fail "Unknown app: $app" + ;; + esac +} + +show_summary() { + local instance="$1" + + echo "Backend port: $(get_backend_port "$instance")" + echo "Frontend port: $(get_frontend_port "$instance")" + echo "Database: $(get_db_name "$instance")" + echo "Temporal NS: $(get_temporal_ns "$instance")" + echo "API URL: $(get_vite_api_url "$instance")" + echo "Studio API: $(get_studio_api_url "$instance")" +} + +parse_init_args() { + local instance="" + local force="false" + + while [ "$#" -gt 0 ]; do + case "$1" in + --force) + force="true" + ;; + *) + if [ -z "$instance" ]; then + instance="$1" + else + fail "Unexpected argument: $1" + fi + ;; + esac + shift + done + + if [ -z "$instance" ]; then + instance="0" + fi + + printf '%s %s\n' "$instance" "$force" +} + +cmd_init() { + local parsed + parsed="$(parse_init_args "$@")" + local instance + local force + instance="$(echo "$parsed" | awk '{print $1}')" + force="$(echo "$parsed" | awk '{print $2}')" + + validate_instance "$instance" + + local dir + dir="$(instance_dir "$instance")" + mkdir -p "$dir" + + log_info "Initializing env files for instance $instance" + + for app in "${APPS[@]}"; do + local dest="$dir/$app.env" + + if [ -f "$dest" ] && [ "$force" = "false" ]; then + log_warn "$app.env already exists (use --force to overwrite)" + continue + fi + + local src + src="$(resolve_source_env "$app")" + if [ -z "$src" ]; then + log_warn "No source found for $app (.env or .env.example)" + continue + fi + + cp "$src" "$dest" + apply_instance_vars "$dest" "$instance" "$app" + if [ "$force" = "true" ]; then + log_ok "$app.env overwritten" + else + log_ok "$app.env created" + fi + done + + log_info "Env files path: $dir" + show_summary "$instance" +} + +cmd_update() { + local instance="${1:-0}" + [ "$#" -le 1 ] || fail "Usage: ./scripts/instance-env.sh update [N]" + validate_instance "$instance" + + local dir + dir="$(instance_dir "$instance")" + + for app in "${APPS[@]}"; do + local file="$dir/$app.env" + [ -f "$file" ] || fail "Missing $file. Run: ./scripts/instance-env.sh init $instance" + done + + for app in "${APPS[@]}"; do + local file="$dir/$app.env" + apply_instance_vars "$file" "$instance" "$app" + log_ok "$app.env updated" + done + + show_summary "$instance" +} + +cmd_copy() { + local src="${1:-}" + local dest="${2:-}" + local force="false" + + [ -n "$src" ] && [ -n "$dest" ] || fail "Usage: ./scripts/instance-env.sh copy SOURCE DEST [--force]" + + if [ "${3:-}" = "--force" ]; then + force="true" + elif [ "${3:-}" != "" ]; then + fail "Unexpected argument: ${3}" + fi + + validate_instance "$src" + validate_instance "$dest" + [ "$src" != "$dest" ] || fail "Source and destination must be different" + + local src_dir + local dest_dir + src_dir="$(instance_dir "$src")" + dest_dir="$(instance_dir "$dest")" + + for app in "${APPS[@]}"; do + [ -f "$src_dir/$app.env" ] || fail "Missing source file: $src_dir/$app.env" + done + + mkdir -p "$dest_dir" + + for app in "${APPS[@]}"; do + local target="$dest_dir/$app.env" + + if [ -f "$target" ] && [ "$force" = "false" ]; then + log_warn "$app.env already exists in destination (use --force to overwrite)" + continue + fi + + cp "$src_dir/$app.env" "$target" + apply_instance_vars "$target" "$dest" "$app" + log_ok "$app.env copied" + done + + log_info "Copied from instance $src to $dest" + show_summary "$dest" +} + +cmd_show() { + local instance="${1:-0}" + [ "$#" -le 1 ] || fail "Usage: ./scripts/instance-env.sh show [N]" + validate_instance "$instance" + + local dir + dir="$(instance_dir "$instance")" + + for app in "${APPS[@]}"; do + if [ -f "$dir/$app.env" ]; then + log_ok "$app.env exists" + else + log_warn "$app.env missing" + fi + done + + show_summary "$instance" +} + +usage() { + cat <<'USAGE' +Instance Env Manager + +Commands: + init [N] [--force] Create .instances/instance-N/{backend,worker,frontend}.env + update [N] Patch only instance-specific variables + copy SOURCE DEST [--force] Copy env files between instances, then re-scope values + show [N] Show file presence and effective instance values + +Notes: + - Instance defaults to 0. + - Source templates are app/.env (preferred) or app/.env.example. +USAGE +} + +main() { + local cmd="${1:-}" + shift || true + + case "$cmd" in + init) + cmd_init "$@" + ;; + update) + cmd_update "$@" + ;; + copy) + cmd_copy "$@" + ;; + show) + cmd_show "$@" + ;; + -h|--help|help|"") + usage + ;; + *) + fail "Unknown command: $cmd" + ;; + esac +} + +main "$@"