diff --git a/README.md b/README.md index 93ed0ee..d38ab32 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,11 @@ A Lambda starter kit for Node.js TypeScript serverless functions. -This project provides a solid foundation for building AWS Lambda functions using Node.js and TypeScript, with AWS CDK for infrastructure as code, Jest for testing, and modern development tooling. +## Overview + +This project provides a solid foundation for implementing Serverless Microservice Patterns with AWS Lambda functions using Node.js and TypeScript. The project uses the AWS CDK for infrastructure as code, Jest for testing, and modern development tooling. + +There are many Serverless Microservice Patterns which may be implemented with AWS Lambda functions. This project illustrates the "Simple Web Service" pattern, which is one of the most frequently used. ## Getting started @@ -11,25 +15,24 @@ This project provides a solid foundation for building AWS Lambda functions using Before you begin, ensure you have the following installed: - **[Node Version Manager (NVM)](https://github.com/nvm-sh/nvm)** - Manages Node.js versions -- **Node.js 24+** - JavaScript runtime (install via NVM) +- **Node.js** - JavaScript runtime (install via NVM) - **npm** - Package manager (comes with Node.js) - **AWS CLI** - For AWS credentials and configuration (recommended) #### Setting up Node.js with NVM -This project uses Node.js version `24.11.1` as specified in `.nvmrc`. +This project uses the Node.js version specified in `.nvmrc`. See the [official nvm guide](https://github.com/nvm-sh/nvm) for additional information. ```bash # Install NVM (if not already installed) -curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash # Install and use the correct Node.js version nvm install nvm use # Verify installation -node --version # Should output v24.11.1 -npm --version +node --version # Should output same version as in .nvmrc ``` #### Installing Dependencies @@ -41,19 +44,27 @@ npm install ## Project structure +This is a high-level overview of the project structure. This structure separates the infrastructure as code from the Lambda application code. Within the Lambda microservice component, directories provide structure to implement DRY (Don't Repeat Yourself) code which follows the SRP (Single Responsibility Principle). + ``` /docs # Project documentation + /infrastructure # AWS CDK infrastructure code /stacks # CDK stack definitions /utils # CDK utilities and helpers app.ts # CDK app entry point cdk.json # CDK configuration + jest.config.ts # Infrastructure Jest configuration + package.json # Infrastructure dependencies and scripts + tsconfig.json # Infrastructure TypeScript configuration + .env.example # Infrastructure example .env + /src # Application source code /handlers # Lambda function handlers /models # Data models and types /services # Business logic services /utils # Utility functions and helpers -/coverage # Test coverage reports (generated) + eslint.config.mjs # ESLint configuration jest.config.ts # Jest testing configuration package.json # Project dependencies and scripts @@ -110,13 +121,13 @@ npm run test:watch - **Language:** TypeScript - **Platform:** AWS Lambda -- **Runtime:** Node.js 24+ -- **AWS SDK:** v3 (modular packages) +- **Runtime:** Node.js 24+ (see .nvmrc) +- **Package Manager:** npm +- **AWS SDK:** v3 - **Testing:** Jest - **Linting/Formatting:** ESLint + Prettier - **Validation:** Zod - **Logging:** Pino + Pino Lambda -- **Package Manager:** npm - **Infrastructure:** AWS CDK - **DevOps:** GitHub Actions @@ -127,6 +138,7 @@ npm run test:watch - **[@aws-sdk/client-dynamodb](https://www.npmjs.com/package/@aws-sdk/client-dynamodb)** - AWS SDK v3 DynamoDB client - **[@aws-sdk/lib-dynamodb](https://www.npmjs.com/package/@aws-sdk/lib-dynamodb)** - DynamoDB document client utilities - **[zod](https://www.npmjs.com/package/zod)** - TypeScript-first schema validation +- **[pino](https://getpino.io/)** - Low overhead, fast logger for JavaScript ### Development Dependencies @@ -148,3 +160,7 @@ Each environment has its own AWS account and configuration. ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. + +## Further Reading + +- [Project Documentation](./docs/README.md) diff --git a/docs/DevOpsGuide.md b/docs/DevOpsGuide.md new file mode 100644 index 0000000..b8c30a8 --- /dev/null +++ b/docs/DevOpsGuide.md @@ -0,0 +1,97 @@ +# DevOps Guide + +## Overview + +This guide documents the DevOps automation for the project, focusing on the use of GitHub Actions for CI/CD and operational workflows. It is intended for software engineers and DevOps engineers responsible for maintaining and deploying the project to AWS. + +--- + +## GitHub Actions Workflows + +The project uses GitHub Actions to automate the following tasks: + +- **Continuous Integration (CI):** + - Linting, building, and testing the application and infrastructure code on every push and pull request. +- **Continuous Deployment (CD):** + - Automated deployment to AWS for specific branches or tags (e.g., `main`, `prd`). +- **Code Quality Gates:** + - Enforces code formatting, linting, and test coverage thresholds before merging. +- **Release Automation:** + - Optionally, creates releases and tags for production deployments. + +--- + +## Workflow Summary + +The project utilizes the following workflows. + +| Workflow Name | Purpose | Triggers | +| ---------------------- | ----------------------------- | -------------------------------------- | +| Continuous Integration | Lint, build, test | pull_request, manual | +| Deploy to DEV | Deploy to DEV environment | manual | +| Code Quality | Generate code quality reports | push to main branch, scheduled, manual | + +--- + +## Workflow Configuration + +Workflows are defined in `.github/workflows/` as YAML files. Each workflow is triggered by specific events (push, pull_request, release, etc.). + +### Example Workflow Structure + +```yaml +name: CI +on: + push: + branches: [main, dev, qat, prd] + pull_request: + branches: [main, dev, qat, prd] +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: '24' + - run: npm ci + - run: npm run lint + - run: npm run build + - run: npm test +``` + +--- + +## Environment Variables and Secrets + +Workflows are configured using GitHub Actions variables and secrets: + +- **Variables:** + - Used for non-sensitive configuration (e.g., environment names, deployment flags). + - Set in the repository or organization settings under "Variables". +- **Secrets:** + - Used for sensitive data (e.g., AWS credentials, tokens). + - Set in the repository or organization settings under "Secrets". + +### Common Variables and Secrets + +See the [Configuration Guide](./ConfigurationGuide.md) for a comprehensive list of variables and secrets. + +--- + +## Adding or Modifying Workflows + +- Add new workflow files to `.github/workflows/`. +- Reference official GitHub Actions and community actions for best practices. +- Use secrets for all sensitive data. +- Review workflow logs in the GitHub Actions tab for troubleshooting. + +--- + +## Further Reading + +- [GitHub Actions Documentation](https://docs.github.com/en/actions) +- [AWS CDK Documentation](https://docs.aws.amazon.com/cdk/) +- [Project Infrastructure Guide](./InfrastructureGuide.md) +- [Project Configuration Guide](./ConfigurationGuide.md) diff --git a/docs/InfrastructureGuide.md b/docs/InfrastructureGuide.md index 204a2d8..0a9c808 100644 --- a/docs/InfrastructureGuide.md +++ b/docs/InfrastructureGuide.md @@ -1,1177 +1,177 @@ # Infrastructure Guide -A comprehensive guide to AWS Cloud Development Kit (CDK) and the infrastructure implementation for the lambda-starter project. - -## Table of Contents - -- [Introduction to AWS CDK](#introduction-to-aws-cdk) -- [Why CDK?](#why-cdk) -- [CDK Concepts](#cdk-concepts) -- [Project Infrastructure](#project-infrastructure) -- [Getting Started](#getting-started) -- [Configuration](#configuration) -- [Infrastructure Resources](#infrastructure-resources) -- [Stack Management](#stack-management) -- [Commands Reference](#commands-reference) -- [Deployment Guide](#deployment-guide) -- [Testing Infrastructure](#testing-infrastructure) -- [Troubleshooting](#troubleshooting) -- [Best Practices](#best-practices) -- [Additional Resources](#additional-resources) - ---- - -## Introduction to AWS CDK - -The AWS Cloud Development Kit (CDK) is an open-source software development framework for defining cloud infrastructure using familiar programming languages. Rather than writing JSON or YAML templates, you write code in TypeScript, Python, Java, or other supported languages. - -### What is Infrastructure as Code? - -Infrastructure as Code (IaC) is the practice of managing and provisioning infrastructure through code rather than manual processes. This approach: - -- **Increases consistency**: Same code produces identical infrastructure -- **Enables version control**: Track changes over time -- **Facilitates collaboration**: Multiple developers can work together -- **Speeds up deployment**: Automated provisioning reduces errors -- **Improves documentation**: Code serves as living documentation - -### CDK vs CloudFormation - -While CDK ultimately generates CloudFormation templates, it offers several advantages: - -| Feature | CloudFormation (YAML/JSON) | AWS CDK | -| -------------- | --------------------------- | ----------------------------------- | -| Language | YAML or JSON | TypeScript, Python, Java, C#, Go | -| Type Safety | None | Compile-time checking | -| Code Reuse | Limited (via nested stacks) | Full programming language features | -| Testing | Manual validation | Unit tests with Jest, PyTest, etc. | -| Abstractions | Low-level resources | High-level constructs with defaults | -| Learning Curve | CloudFormation-specific | Use existing programming skills | - -### How CDK Works - -``` -┌─────────────────┐ -│ CDK Code (TS) │ -│ app.ts │ -└────────┬────────┘ - │ - ▼ - ┌─────────────┐ - │ cdk synth │ - └──────┬──────┘ - │ - ▼ -┌──────────────────┐ -│ CloudFormation │ -│ Template │ -└────────┬─────────┘ - │ - ▼ - ┌─────────────┐ - │ cdk deploy │ - └──────┬──────┘ - │ - ▼ -┌──────────────────┐ -│ AWS Resources │ -│ (DynamoDB, etc.) │ -└──────────────────┘ -``` - -1. **Write** infrastructure code in TypeScript -2. **Synthesize** CloudFormation templates using `cdk synth` -3. **Deploy** to AWS using `cdk deploy` -4. **Update** by modifying code and redeploying +This guide provides a concise overview of the AWS CDK infrastructure for the project. It is intended for software and DevOps engineers deploying and maintaining the project on AWS. --- -## Why CDK? - -### Benefits for This Project - -1. **Type Safety**: TypeScript catches errors at compile time -2. **Testability**: Unit tests ensure infrastructure correctness -3. **Reusability**: Share constructs across multiple stacks -4. **Maintainability**: Clear, readable code vs. verbose YAML -5. **Productivity**: Less boilerplate, more functionality -6. **Integration**: Seamless integration with application code - -### When to Use CDK +## Stacks Overview -CDK is ideal when: +The infrastructure is organized into two main AWS CDK stacks: -- You want to use programming languages for infrastructure -- Your team has development experience -- You need complex, conditional infrastructure -- You want to write unit tests for infrastructure -- You need to create reusable infrastructure components - -### When Not to Use CDK - -Consider alternatives when: - -- Your team prefers declarative YAML/JSON -- You're using simple, static infrastructure -- You need multi-cloud support (consider Terraform) -- You have significant investment in existing CloudFormation +| Stack Name Pattern | Purpose | +| ------------------------- | ------------------------------------------ | +| `{app-name}-data-{env}` | Manages DynamoDB tables and data resources | +| `{app-name}-lambda-{env}` | Manages Lambda functions and API Gateway | --- -## CDK Concepts - -### Constructs - -Constructs are the basic building blocks of CDK applications. There are three levels: - -**L1 Constructs (CloudFormation Resources)** - -- Direct mapping to CloudFormation resources -- Named with `Cfn` prefix (e.g., `CfnBucket`) -- Require all properties to be specified - -**L2 Constructs (AWS Constructs)** - -- Higher-level, opinionated abstractions -- Provide sensible defaults -- Example: `Table` for DynamoDB - -**L3 Constructs (Patterns)** +## Data Stack -- Multi-resource patterns -- Represent common architectures -- Example: `ApplicationLoadBalancedFargateService` +**Purpose:** Manages DynamoDB tables and data-related resources. -### Stacks +**Key Resources:** -A **Stack** is a unit of deployment. It maps to a CloudFormation stack and contains a group of related resources. +| Resource | Name Pattern | Key Properties | +| -------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| DynamoDB Table | `{app-name}-task-{env}` | Partition Key: `pk` (String), On-demand billing, SSE encryption, PITR (prd only), Removal Policy: `RETAIN` (prd), `DESTROY` (dev/qat) | -```typescript -export class DataStack extends cdk.Stack { - constructor(scope: Construct, id: string, props?: cdk.StackProps) { - super(scope, id, props); - // Resources defined here - } -} -``` - -### Apps - -An **App** is the root construct that contains one or more stacks. - -```typescript -const app = new cdk.App(); -new DataStack(app, 'DataStack-dev'); -``` +**Outputs:** -### Environments - -Environments specify the AWS account and region where stacks are deployed: - -```typescript -{ - env: { - account: '123456789012', - region: 'us-east-1' - } -} -``` +| Output Name | Export Name Pattern | Description | +| --------------- | ---------------------------------- | ------------------------------------- | +| `TaskTableName` | `{app-name}-task-table-name-{env}` | Table name (exported as stack output) | +| `TaskTableArn` | `{app-name}-task-table-arn-{env}` | Table ARN (exported as stack output) | --- -## Project Infrastructure - -### Architecture Overview - -The lambda-starter infrastructure uses a modular, environment-aware design: +## Lambda Stack -``` -┌─────────────────────────────────────────┐ -│ CDK Application │ -│ │ -│ ┌────────────────────────────────────┐ │ -│ │ Configuration Layer │ │ -│ │ • Environment Variables (.env) │ │ -│ │ • Zod Validation │ │ -│ │ • AWS Account/Region Resolution │ │ -│ │ • Logging Configuration │ │ -│ └────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────┐ │ -│ │ Data Stack │ │ -│ │ • DynamoDB Tables │ │ -│ │ • Environment-specific configs │ │ -│ │ • Resource tagging │ │ -│ └────────────────────────────────────┘ │ -│ │ -│ ┌────────────────────────────────────┐ │ -│ │ Lambda Stack │ │ -│ │ • Lambda Functions │ │ -│ │ • API Gateway REST API │ │ -│ │ • CloudWatch Logs (JSON format) │ │ -│ │ • Environment-specific retention │ │ -│ └────────────────────────────────────┘ │ -└─────────────────────────────────────────┘ -``` +**Purpose:** Manages Lambda functions, API Gateway, and application runtime resources. -### Design Principles +**Key Resources:** -1. **Environment Isolation**: Separate AWS accounts per environment -2. **Configuration Validation**: Zod ensures correct configuration -3. **Resource Tagging**: All resources tagged for cost allocation -4. **Type Safety**: TypeScript prevents configuration errors -5. **Testability**: Unit tests validate infrastructure logic -6. **Modularity**: Separate stacks for different resource types +| Resource | Name Pattern | Purpose/Notes | +| --------------- | ------------------------------ | ------------------------------------ | +| Lambda Function | `{app-name}-list-tasks-{env}` | List all tasks (DynamoDB Scan) | +| Lambda Function | `{app-name}-get-task-{env}` | Get a task by ID (DynamoDB GetItem) | +| Lambda Function | `{app-name}-create-task-{env}` | Create a new task (DynamoDB PutItem) | +| Lambda Function | `{app-name}-update-task-{env}` | Update a task (DynamoDB UpdateItem) | +| Lambda Function | `{app-name}-delete-task-{env}` | Delete a task (DynamoDB DeleteItem) | +| API Gateway | `{app-name}-api-{env}` | REST API for Lambda functions | -### Technology Stack +**Outputs:** -- **AWS CDK**: https://docs.aws.amazon.com/cdk/api/v2/ -- **TypeScript**: https://www.typescriptlang.org/ -- **Node.js**: https://nodejs.org/ -- **Testing**: https://jestjs.io/ -- **Validation**: https://zod.dev/ +| Output Name | Export Name Pattern | Description | +| ----------------------- | ---------------------------------- | ------------------------------- | +| `ApiUrl` | `{app-name}-api-url-{env}` | API Gateway endpoint URL | +| `ApiId` | `{app-name}-api-id-{env}` | API Gateway ID | +| `ListTasksFunctionArn` | `{app-name}-list-tasks-arn-{env}` | List Tasks Lambda function ARN | +| `GetTaskFunctionArn` | `{app-name}-get-task-arn-{env}` | Get Task Lambda function ARN | +| `CreateTaskFunctionArn` | `{app-name}-create-task-arn-{env}` | Create Task Lambda function ARN | +| `UpdateTaskFunctionArn` | `{app-name}-update-task-arn-{env}` | Update Task Lambda function ARN | +| `DeleteTaskFunctionArn` | `{app-name}-delete-task-arn-{env}` | Delete Task Lambda function ARN | --- -## Getting Started - -### Prerequisites - -1. **Node.js v24+** installed - - ```bash - node --version - ``` - -2. **AWS CLI** configured - - ```bash - aws configure - aws sts get-caller-identity - ``` - -3. **AWS CDK** installed globally (optional) - - ```bash - npm install -g aws-cdk - ``` - -4. **Bootstrap CDK** (once per account/region) - ```bash - cdk bootstrap aws://ACCOUNT-ID/REGION - ``` - -### Initial Setup - -1. **Navigate to infrastructure directory** - - ```bash - cd infrastructure - ``` - -2. **Install dependencies** - - ```bash - npm install - ``` - -3. **Configure environment** - - ```bash - cp .env.example .env - # Edit .env with your settings - ``` - -4. **Build the project** - - ```bash - npm run build - ``` - -5. **Run tests** +## Resource Tagging - ```bash - npm test - ``` +All resources are tagged for cost allocation and management: -6. **Synthesize CloudFormation** - ```bash - npm run synth - ``` +| Tag | Source | Example Value | +| ------- | -------------- | ------------------- | +| `App` | `CDK_APP_NAME` | `lambda-starter` | +| `Env` | `CDK_ENV` | `dev`, `qat`, `prd` | +| `OU` | `CDK_OU` | `leanstacks` | +| `Owner` | `CDK_OWNER` | `platform-team` | --- -## Configuration +## Configuration & DevOps -### Environment Variables - -All configuration is managed through environment variables prefixed with `CDK_`: - -| Variable | Required | Description | Default | -| ------------------------- | -------- | -------------------------- | ---------------- | -| `CDK_APP_NAME` | No | Application name | `lambda-starter` | -| `CDK_ENV` | Yes | Environment | - | -| `CDK_ACCOUNT` | No | AWS account ID override | From AWS CLI | -| `CDK_REGION` | No | AWS region override | From AWS CLI | -| `CDK_OU` | No | Organizational unit | `leanstacks` | -| `CDK_OWNER` | No | Resource owner | `unknown` | -| `CDK_APP_LOGGING_ENABLED` | No | Enable application logging | `true` | -| `CDK_APP_LOGGING_LEVEL` | No | Application logging level | `info` | -| `CDK_APP_LOGGING_FORMAT` | No | Application logging format | `json` | - -### Configuration Validation - -Configuration is validated using Zod schemas. Invalid configurations fail at synthesis time with descriptive errors. - -### AWS Account Resolution - -The CDK automatically detects AWS credentials: - -- **CDK_DEFAULT_ACCOUNT**: From your AWS profile -- **CDK_DEFAULT_REGION**: From your AWS profile - -Override by setting `CDK_ACCOUNT` and `CDK_REGION` in `.env`. - -### Environment-Specific Configuration - -Each environment can have different settings: - -**Development (dev)** - -- Pay-per-request billing -- No point-in-time recovery -- `DESTROY` removal policy - -**Production (prd)** - -- Pay-per-request billing -- Point-in-time recovery enabled -- `RETAIN` removal policy +- For environment variables, configuration, and validation, see the [Configuration Guide](./ConfigurationGuide.md). +- For CI/CD, GitHub Actions, and DevOps automation, see the [DevOps Guide](./DevOpsGuide.md). --- -## Infrastructure Resources - -### Data Stack - -**Purpose**: Manages DynamoDB tables and data-related resources. - -**Stack Naming**: `{app-name}-data-{env}` (e.g., `lambda-starter-data-dev`) - -#### Task Table - -**Resource Type**: DynamoDB Table - -**Configuration**: - -- **Table Name**: `{app-name}-task-{env}` -- **Partition Key**: `pk` (String) -- **Billing Mode**: Pay-per-request (on-demand) -- **Encryption**: AWS managed (SSE) -- **Point-in-time Recovery**: Enabled for `prd` only -- **Removal Policy**: - - `RETAIN` for `prd` (preserved on deletion) - - `DESTROY` for `dev`/`qat` (deleted on deletion) - -**Outputs**: - -- `TaskTableName`: Table name (exported as `{env}-task-table-name`) -- `TaskTableArn`: Table ARN (exported as `{env}-task-table-arn`) - -**Usage Example**: - -```typescript -// Reference in Lambda code -const tableName = process.env.TASK_TABLE_NAME; -``` - -### Lambda Stack - -**Purpose**: Manages Lambda functions, API Gateway, and application runtime resources. - -**Stack Naming**: `{app-name}-lambda-{env}` (e.g., `lambda-starter-lambda-dev`) - -#### List Tasks Function - -**Resource Type**: AWS Lambda Function - -**Configuration**: - -- **Function Name**: `{app-name}-list-tasks-{env}` -- **Runtime**: Node.js 24.x -- **Handler**: `handler` (bundled with esbuild) -- **Memory**: 256 MB -- **Timeout**: 10 seconds -- **Log Format**: JSON (structured logging) -- **Bundling**: Automatic TypeScript compilation with esbuild -- **Environment Variables**: - - `TASKS_TABLE`: DynamoDB table name - - `LOGGING_ENABLED`: Logging enabled flag (from `CDK_APP_LOGGING_ENABLED`) - - `LOGGING_LEVEL`: Minimum log level (from `CDK_APP_LOGGING_LEVEL`) - - `LOGGING_FORMAT`: Log output format (from `CDK_APP_LOGGING_FORMAT`) - -**CloudWatch Logs**: - -- **Log Group**: `/aws/lambda/{app-name}-list-tasks-{env}` -- **Log Retention**: - - `prd`: 30 days - - Other environments: 7 days -- **Removal Policy**: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -**IAM Permissions**: - -- **DynamoDB**: Read access (Scan) to the Task table -- **CloudWatch Logs**: Write access to its log group - -#### Get Task Function - -**Resource Type**: AWS Lambda Function - -**Configuration**: - -- **Function Name**: `{app-name}-get-task-{env}` -- **Runtime**: Node.js 24.x -- **Handler**: `handler` (bundled with esbuild) -- **Memory**: 256 MB -- **Timeout**: 10 seconds -- **Log Format**: JSON (structured logging) -- **Bundling**: Automatic TypeScript compilation with esbuild -- **Environment Variables**: - - `TASKS_TABLE`: DynamoDB table name - - `LOGGING_ENABLED`: Logging enabled flag (from `CDK_APP_LOGGING_ENABLED`) - - `LOGGING_LEVEL`: Minimum log level (from `CDK_APP_LOGGING_LEVEL`) - - `LOGGING_FORMAT`: Log output format (from `CDK_APP_LOGGING_FORMAT`) - -**CloudWatch Logs**: - -- **Log Group**: `/aws/lambda/{app-name}-get-task-{env}` -- **Log Retention**: - - `prd`: 30 days - - Other environments: 7 days -- **Removal Policy**: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -**IAM Permissions**: - -- **DynamoDB**: Read access (GetItem) to the Task table -- **CloudWatch Logs**: Write access to its log group - -#### Create Task Function - -**Resource Type**: AWS Lambda Function - -**Configuration**: - -- **Function Name**: `{app-name}-create-task-{env}` -- **Runtime**: Node.js 24.x -- **Handler**: `handler` (bundled with esbuild) -- **Memory**: 256 MB -- **Timeout**: 10 seconds -- **Log Format**: JSON (structured logging) -- **Bundling**: Automatic TypeScript compilation with esbuild -- **Environment Variables**: - - `TASKS_TABLE`: DynamoDB table name - - `LOGGING_ENABLED`: Logging enabled flag (from `CDK_APP_LOGGING_ENABLED`) - - `LOGGING_LEVEL`: Minimum log level (from `CDK_APP_LOGGING_LEVEL`) - - `LOGGING_FORMAT`: Log output format (from `CDK_APP_LOGGING_FORMAT`) - -**CloudWatch Logs**: - -- **Log Group**: `/aws/lambda/{app-name}-create-task-{env}` -- **Log Retention**: - - `prd`: 30 days - - Other environments: 7 days -- **Removal Policy**: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -**IAM Permissions**: - -- **DynamoDB**: Write access (PutItem) to the Task table -- **CloudWatch Logs**: Write access to its log group - -#### Update Task Function - -**Resource Type**: AWS Lambda Function - -**Configuration**: - -- **Function Name**: `{app-name}-update-task-{env}` -- **Runtime**: Node.js 24.x -- **Handler**: `handler` (bundled with esbuild) -- **Memory**: 256 MB -- **Timeout**: 10 seconds -- **Log Format**: JSON (structured logging) -- **Bundling**: Automatic TypeScript compilation with esbuild -- **Environment Variables**: - - `TASKS_TABLE`: DynamoDB table name - - `LOGGING_ENABLED`: Logging enabled flag (from `CDK_APP_LOGGING_ENABLED`) - - `LOGGING_LEVEL`: Minimum log level (from `CDK_APP_LOGGING_LEVEL`) - - `LOGGING_FORMAT`: Log output format (from `CDK_APP_LOGGING_FORMAT`) - -**CloudWatch Logs**: - -- **Log Group**: `/aws/lambda/{app-name}-update-task-{env}` -- **Log Retention**: - - `prd`: 30 days - - Other environments: 7 days -- **Removal Policy**: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -**IAM Permissions**: - -- **DynamoDB**: Read-write access (GetItem, UpdateItem) to the Task table -- **CloudWatch Logs**: Write access to its log group - -#### Delete Task Function - -**Resource Type**: AWS Lambda Function - -**Configuration**: - -- **Function Name**: `{app-name}-delete-task-{env}` -- **Runtime**: Node.js 24.x -- **Handler**: `handler` (bundled with esbuild) -- **Memory**: 256 MB -- **Timeout**: 10 seconds -- **Log Format**: JSON (structured logging) -- **Bundling**: Automatic TypeScript compilation with esbuild -- **Environment Variables**: - - `TASKS_TABLE`: DynamoDB table name - - `LOGGING_ENABLED`: Logging enabled flag (from `CDK_APP_LOGGING_ENABLED`) - - `LOGGING_LEVEL`: Minimum log level (from `CDK_APP_LOGGING_LEVEL`) - - `LOGGING_FORMAT`: Log output format (from `CDK_APP_LOGGING_FORMAT`) - -**CloudWatch Logs**: - -- **Log Group**: `/aws/lambda/{app-name}-delete-task-{env}` -- **Log Retention**: - - `prd`: 30 days - - Other environments: 7 days -- **Removal Policy**: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -**IAM Permissions**: - -- **DynamoDB**: Read-write access (DeleteItem) to the Task table -- **CloudWatch Logs**: Write access to its log group - -#### Lambda Starter API - -**Resource Type**: API Gateway REST API - -**Configuration**: - -- **API Name**: `{app-name}-api-{env}` -- **Description**: "Lambda Starter API for {env} environment" -- **Stage**: `{env}` (e.g., `dev`, `prd`) -- **CORS**: Enabled with preflight OPTIONS support - - Allowed Headers: `Content-Type`, `Authorization` - - Allowed Methods: Configured per resource - - Allowed Origins: Configured per environment -- **Throttling**: - - Rate limits configured per stage - - Burst limits configured per stage - -**API Resources**: - -- **GET /tasks**: List all tasks - - Integration: Lambda proxy integration with List Tasks Function - - Response: JSON array of tasks - -- **GET /tasks/{taskId}**: Get a specific task by ID - - Integration: Lambda proxy integration with Get Task Function - - Path Parameter: `taskId` - The unique identifier of the task - - Response: JSON object with the requested task - - Error Responses: - - 404 Not Found: Task ID does not exist or path parameter is missing - - 500 Internal Server Error: Failed to retrieve task - -- **POST /tasks**: Create a new task - - Integration: Lambda proxy integration with Create Task Function - - Request Body: JSON object with task properties - - `title` (required): string, max 100 characters - - `detail` (optional): string, max 1000 characters - - `dueAt` (optional): ISO8601 timestamp - - `isComplete` (optional): boolean, defaults to false - - Response: JSON object with created task including ID and timestamps - - Success Status: 201 Created - - Error Responses: - - 400 Bad Request: Invalid request body or validation error - - 500 Internal Server Error: Failed to create task - -- **PUT /tasks/{taskId}**: Update an existing task - - Integration: Lambda proxy integration with Update Task Function - - Path Parameter: `taskId` - The unique identifier of the task - - Request Body: JSON object with task properties - - `title` (required): string, max 100 characters - - `detail` (optional): string, max 1000 characters - omit to remove from task - - `dueAt` (optional): ISO8601 timestamp - omit to remove from task - - `isComplete` (required): boolean - - Response: JSON object with updated task - - Success Status: 200 OK - - Error Responses: - - 400 Bad Request: Invalid request body or validation error - - 404 Not Found: Task ID does not exist - - 500 Internal Server Error: Failed to update task - -- **DELETE /tasks/{taskId}**: Delete an existing task - - Integration: Lambda proxy integration with Delete Task Function - - Path Parameter: `taskId` - The unique identifier of the task - - Response: No content on successful deletion - - Success Status: 204 No Content - - Error Responses: - - 404 Not Found: Task ID does not exist or path parameter is missing - - 500 Internal Server Error: Failed to delete task - -**Outputs**: - -- `ApiUrl`: The API Gateway endpoint URL (e.g., `https://abc123.execute-api.us-east-1.amazonaws.com/dev/`) -- `ApiId`: The API Gateway ID -- `ListTasksFunctionArn`: The List Tasks Lambda function ARN -- `GetTaskFunctionArn`: The Get Task Lambda function ARN -- `CreateTaskFunctionArn`: The Create Task Lambda function ARN -- `UpdateTaskFunctionArn`: The Update Task Lambda function ARN -- `DeleteTaskFunctionArn`: The Delete Task Lambda function ARN - -**Logging Configuration**: - -The Lambda stack uses environment variables to configure application behavior: - -- **CDK_APP_LOGGING_ENABLED**: Controls whether logging is enabled in Lambda functions - - Set to `true` (default) or `false` - - Passed to Lambda as `LOGGING_ENABLED` environment variable - - When disabled, Lambda functions produce minimal log output - -- **CDK_APP_LOGGING_LEVEL**: Sets the minimum log level for application logs - - Valid values: `debug`, `info` (default), `warn`, `error` - - Passed to Lambda as `LOGGING_LEVEL` environment variable - - Controls verbosity of application logging - -- **CDK_APP_LOGGING_FORMAT**: Sets the log output format - - Valid values: `text`, `json` (default) - - Passed to Lambda as `LOGGING_FORMAT` environment variable - - `json`: Structured JSON logs ideal for CloudWatch Logs Insights and log aggregation - - `text`: Human-readable text format with stringified context - -**Example Usage**: - -```bash -# Development with debug logging in text format -CDK_APP_LOGGING_ENABLED=true -CDK_APP_LOGGING_LEVEL=debug -CDK_APP_LOGGING_FORMAT=text - -# Production with info logging in JSON format -CDK_APP_LOGGING_ENABLED=true -CDK_APP_LOGGING_LEVEL=info -CDK_APP_LOGGING_FORMAT=json - -# Disable logging (not recommended) -CDK_APP_LOGGING_ENABLED=false -``` - -### Resource Tagging - -All resources are automatically tagged: - -| Tag | Source | Purpose | -| ------- | -------------- | -------------------------- | -| `App` | `CDK_APP_NAME` | Application identification | -| `Env` | `CDK_ENV` | Environment tracking | -| `OU` | `CDK_OU` | Organizational unit | -| `Owner` | `CDK_OWNER` | Resource ownership | - -Tags enable: - -- Cost allocation reports -- Resource filtering -- Compliance tracking -- Access control policies - ---- - -## Stack Management - -### Viewing Stacks - -List all stacks: - -```bash -npm run cdk list -``` - -View stack details: - -```bash -npm run cdk list --long -``` - -### Synthesizing Templates - -Generate CloudFormation templates: - -```bash -npm run synth -``` - -Output location: `cdk.out/` - -View specific stack template: - -```bash -npm run cdk synth lambda-starter-data-stack-dev -``` - -### Comparing Changes - -See differences before deploying: - -```bash -npm run diff -``` - -Compare specific stack: - -```bash -npm run cdk diff lambda-starter-data-stack-dev -``` - -### Stack Outputs - -View stack outputs: - -```bash -aws cloudformation describe-stacks \ - --stack-name lambda-starter-data-dev \ - --query 'Stacks[0].Outputs' -``` - ---- - -## Commands Reference - -### NPM Scripts - -| Command | Description | -| ----------------------- | ------------------------- | -| `npm run build` | Compile TypeScript | -| `npm run clean` | Remove build artifacts | -| `npm test` | Run unit tests | -| `npm run test:coverage` | Run tests with coverage | -| `npm run test:watch` | Watch mode for tests | -| `npm run synth` | Synthesize CloudFormation | -| `npm run deploy` | Deploy all stacks | -| `npm run diff` | Show differences | -| `npm run destroy` | Destroy all stacks | - -### CDK CLI Commands - -| Command | Description | -| --------------- | ------------------------------- | -| `cdk init` | Initialize new CDK project | -| `cdk bootstrap` | Bootstrap CDK in account/region | -| `cdk synth` | Synthesize CloudFormation | -| `cdk deploy` | Deploy stacks | -| `cdk diff` | Compare deployed vs local | -| `cdk destroy` | Remove stacks | -| `cdk list` | List stacks | -| `cdk doctor` | Check CDK setup | - -### Advanced Commands - -**Deploy with approval bypass** (CI/CD): - -```bash -npm run cdk deploy -- --require-approval never -``` - -**Deploy specific stack**: - -```bash -npm run cdk deploy lambda-starter-data-stack-dev -``` - -**Destroy with force**: - -```bash -npm run cdk destroy --all --force -``` - -**Watch mode** (auto-deploy on changes): - -```bash -npm run cdk watch -``` - ---- - -## Deployment Guide - -### Development Deployment - -```bash -# 1. Configure for dev -cat > .env << EOF -CDK_ENV=dev -CDK_APP_LOGGING_ENABLED=true -CDK_APP_LOGGING_LEVEL=debug -EOF - -# 2. Review changes -npm run diff - -# 3. Deploy -npm run deploy -``` - -### Production Deployment - -```bash -# 1. Configure for production -cat > .env << EOF -CDK_APP_NAME=lambda-starter -CDK_ENV=prd -CDK_ACCOUNT=123456789012 -CDK_REGION=us-east-1 -CDK_OU=leanstacks -CDK_OWNER=platform-team -CDK_APP_LOGGING_ENABLED=true -CDK_APP_LOGGING_LEVEL=info -EOF - -# 2. Review changes carefully -npm run diff - -# 3. Deploy with confirmation -npm run deploy -``` - -### CI/CD Deployment - -```bash -# Set environment variables -export CDK_ENV=prd -export CDK_ACCOUNT=123456789012 -export CDK_REGION=us-east-1 -export CDK_APP_LOGGING_ENABLED=true -export CDK_APP_LOGGING_LEVEL=info - -# Deploy without prompts -npm run deploy -- --require-approval never -``` - -### Rollback Procedure - -If deployment fails or issues arise: - -```bash -# 1. Identify the stack -aws cloudformation describe-stacks - -# 2. Rollback to previous version -aws cloudformation rollback-stack \ - --stack-name lambda-starter-data-prd - -# 3. Or destroy and redeploy -npm run cdk destroy lambda-starter-data-stack-prd -# Fix issue in code -npm run deploy -``` - ---- - -## Testing Infrastructure - -### Unit Tests - -Run all tests: - -```bash -npm test -``` - -Run with coverage: - -```bash -npm run test:coverage -``` - -Watch mode for development: - -```bash -npm run test:watch -``` - -### What's Tested - -1. **Configuration Validation** - - Environment variable parsing - - Zod schema validation - - Default value handling - -2. **Stack Resources** - - Correct resource creation - - Property values - - Dependencies - -3. **Environment Logic** - - Different behaviors per environment - - Conditional resource creation - -4. **Tagging** - - All resources properly tagged - - Tag inheritance +## Best Practices -### Test Structure +### Security -```typescript -describe('DataStack', () => { - describe('dev environment', () => { - it('should create a Task table', () => { - // Arrange - const app = new cdk.App(); - const stack = new DataStack(app, 'TestStack', { - envName: 'dev', - }); +1. **Never commit secrets**: Use `.env` for local configuration only +2. **Use AWS Secrets Manager**: Store sensitive values in AWS Secrets Manager or SSM Parameter Store +3. **Least privilege**: Grant only necessary IAM permissions +4. **Enable encryption**: All data at rest should be encrypted +5. **Separate accounts**: Use different AWS accounts for each environment - // Act - const template = Template.fromStack(stack); +### Development - // Assert - template.hasResourceProperties('AWS::DynamoDB::Table', { - TableName: 'task-dev', - }); - }); - }); -}); -``` +1. **Test before deploying**: Always run `npm test` before deployment +2. **Review diffs**: Use `npm run diff` to review changes before applying +3. **Use descriptive names**: Follow naming conventions for resources +4. **Document changes**: Update README when adding new stacks or resources +5. **Type safety**: Leverage TypeScript for compile-time error detection -### Best Practices +### Operations -1. Test each stack in isolation -2. Test environment-specific behavior -3. Verify resource properties -4. Check outputs and exports -5. Test tag application -6. Use CDK assertions library +1. **Tag everything**: Ensure all resources have proper tags +2. **Monitor costs**: Use cost allocation tags to track spending +3. **Backup production**: Enable point-in-time recovery for critical databases +4. **Retain production resources**: Use `RETAIN` removal policy for production +5. **Version control**: Commit infrastructure changes to source control --- ## Troubleshooting -### Common Issues - -#### Configuration Validation Failed +### Configuration Validation Errors -**Problem**: `CDK configuration validation failed: CDK_ENV: Required` +**Problem:** `CDK configuration validation failed` -**Solutions**: +**Solutions:** -1. Ensure `.env` file exists in `infrastructure/` directory -2. Verify `CDK_ENV` is set to valid value (`dev`, `qat`, `prd`) -3. Check for typos in variable names +1. Verify `.env` file exists in the infrastructure directory +2. Check that `CDK_ENV` is set to a valid value (`dev`, `qat`, `prd`) +3. Ensure all required variables are set -#### TypeScript Compilation Errors +### TypeScript Compilation Errors -**Problem**: Build fails with TypeScript errors +**Problem:** Build fails with TypeScript errors -**Solutions**: +**Solutions:** -1. Install dependencies: `npm install` -2. Clean and rebuild: `npm run clean && npm run build` -3. Check Node.js version: `node --version` (should be v24+) -4. Verify TypeScript version: `npx tsc --version` +1. Ensure dependencies are installed: `npm install` +2. Verify Node.js version: `node --version` (should be v24+) +3. Check for syntax errors in TypeScript files +4. Clean and rebuild: `npm run clean && npm run build` -#### Deployment Failures +### Deployment Failures -**Problem**: Stack deployment fails +**Problem:** Stack deployment fails -**Solutions**: +**Solutions:** 1. Verify AWS credentials: `aws sts get-caller-identity` -2. Check IAM permissions -3. Review CloudFormation events in AWS Console -4. Check for resource naming conflicts -5. Ensure CDK is bootstrapped: `cdk bootstrap` +2. Check account and region: Ensure `CDK_ACCOUNT` and `CDK_REGION` match your AWS profile +3. Confirm IAM permissions: Verify you have necessary permissions +4. Review CloudFormation events in AWS Console for detailed error messages +5. Check for resource naming conflicts -#### Bootstrap Version Mismatch +### CDK Bootstrap Issues -**Problem**: `This stack requires bootstrap stack version X` +**Problem:** `This stack requires bootstrap stack version X` -**Solution**: +**Solution:** ```bash cdk bootstrap aws://ACCOUNT-ID/REGION --force ``` -#### Resource Already Exists - -**Problem**: `Resource already exists` error during deployment - -**Solutions**: +### Node Version Warnings -1. Check if resource exists in AWS Console -2. Import existing resource: - ```bash - npm run cdk import lambda-starter-data-stack-dev - ``` -3. Or destroy and recreate: - ```bash - npm run cdk destroy lambda-starter-data-stack-dev - npm run deploy - ``` +**Problem:** Warning about untested Node.js version -#### Node Version Warnings - -**Problem**: Warning about untested Node.js version - -**Solution**: +**Solution:** ```bash export JSII_SILENCE_WARNING_UNTESTED_NODE_VERSION=1 ``` -Or use supported Node.js version (22.x or 20.x). - -### Debugging Tips - -1. **Enable verbose output**: - - ```bash - npm run cdk deploy -- --verbose - ``` - -2. **Check synthesized template**: - - ```bash - npm run synth - cat cdk.out/lambda-starter-data-stack-dev.template.json - ``` - -3. **View CloudFormation events**: - - ```bash - aws cloudformation describe-stack-events \ - --stack-name lambda-starter-data-dev - ``` - -4. **Check CDK context**: - - ```bash - npm run cdk context - ``` - -5. **Validate IAM permissions**: - ```bash - aws iam simulate-principal-policy \ - --policy-source-arn arn:aws:iam::ACCOUNT:user/USERNAME \ - --action-names cloudformation:* - ``` +Or use a supported Node.js version (22.x or 20.x). --- -## Best Practices - -### Security - -1. **Never commit secrets**: Use `.env` for local only -2. **Use AWS Secrets Manager**: Store sensitive values securely -3. **Implement least privilege**: Grant minimal IAM permissions -4. **Enable encryption**: Use AWS managed keys by default -5. **Separate accounts**: Use different AWS accounts per environment -6. **Enable MFA**: Require MFA for production deployments -7. **Audit regularly**: Review CloudTrail logs - -### Development - -1. **Test before deploying**: Always run `npm test` -2. **Review diffs**: Use `npm run diff` before deployment -3. **Use descriptive names**: Follow naming conventions -4. **Document changes**: Update README and comments -5. **Type everything**: Leverage TypeScript's type system -6. **Version dependencies**: Lock dependency versions -7. **Write unit tests**: Test infrastructure code - -### Operations - -1. **Tag everything**: Enable cost tracking and filtering -2. **Monitor costs**: Set up budget alerts -3. **Backup production**: Enable point-in-time recovery -4. **Retain critical resources**: Use `RETAIN` policy for production -5. **Version control**: Commit all infrastructure changes -6. **Use CI/CD**: Automate deployments -7. **Document runbooks**: Create operational procedures - -### Code Organization - -1. **One stack per concern**: Separate data, compute, networking -2. **Reusable constructs**: Create custom constructs for patterns -3. **Environment configuration**: Externalize environment-specific values -4. **Consistent naming**: Use clear, consistent resource names -5. **Modular structure**: Keep files small and focused - -### Deployment - -1. **Deploy to dev first**: Test changes in development -2. **Use change sets**: Review changes before applying -3. **Implement gradual rollout**: Deploy to environments sequentially -4. **Have rollback plan**: Document rollback procedures -5. **Monitor deployments**: Watch CloudFormation events -6. **Communicate changes**: Notify team of deployments - ---- - -## Additional Resources - -### Official Documentation +## Further Reading - [AWS CDK Documentation](https://docs.aws.amazon.com/cdk/latest/guide/) -- [AWS CDK API Reference](https://docs.aws.amazon.com/cdk/api/v2/) -- [CDK Workshop](https://cdkworkshop.com/) -- [CDK Patterns](https://cdkpatterns.com/) -- [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) - -### Community Resources - -- [AWS CDK GitHub](https://github.com/aws/aws-cdk) -- [CDK Slack Channel](https://cdk.dev) -- [AWS Developer Forums](https://forums.aws.amazon.com/forum.jspa?forumID=351) -- [CDK Examples Repository](https://github.com/aws-samples/aws-cdk-examples) - -### Learning Resources - -- [AWS CDK Workshop](https://cdkworkshop.com/) -- [AWS CDK Best Practices](https://docs.aws.amazon.com/cdk/latest/guide/best-practices.html) -- [Infrastructure as Code Best Practices](https://docs.aws.amazon.com/prescriptive-guidance/latest/infrastructure-as-code/welcome.html) -- [TypeScript Handbook](https://www.typescriptlang.org/docs/handbook/intro.html) - -### Tools - -- [CDK Construct Hub](https://constructs.dev/) -- [AWS CLI Documentation](https://docs.aws.amazon.com/cli/) -- [CloudFormation Documentation](https://docs.aws.amazon.com/cloudformation/) -- [AWS Console](https://console.aws.amazon.com/) - ---- - -## Support - -For questions or issues: - -1. Check this guide and the [troubleshooting section](#troubleshooting) -2. Review the [infrastructure README](../infrastructure/README.md) -3. Search [GitHub Issues](https://github.com/leanstacks/lambda-starter/issues) -4. Contact the platform team -5. Create a new issue with detailed information - ---- - -**Last Updated**: November 30, 2025 -**Version**: 1.0.0 -**Maintained By**: LeanStacks Platform Team +- [Project Configuration Guide](./ConfigurationGuide.md) +- [Project DevOps Guide](./DevOpsGuide.md) diff --git a/docs/README.md b/docs/README.md index 6f0ed78..0c27025 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,85 +1,11 @@ # Documentation -Welcome to the lambda-starter project documentation. This directory contains comprehensive guides and references for working with the project. - -## Table of Contents - -### Configuration - -- **[Configuration Guide](ConfigurationGuide.md)** - Complete guide to application and infrastructure configuration - - Application environment variables and validation - - Infrastructure CDK configuration variables - - Resource tagging standards - - Environment-specific configuration recommendations - - Configuration flow from infrastructure to application - - Troubleshooting and best practices - -### Infrastructure - -- **[Infrastructure Guide](InfrastructureGuide.md)** - Complete guide to AWS CDK infrastructure implementation - - Introduction to AWS CDK and Infrastructure as Code - - Project architecture and design principles - - Configuration and environment management - - Resource documentation (DynamoDB tables) - - Deployment procedures and best practices - - Testing infrastructure code - - Troubleshooting common issues - - Additional resources and learning materials - ---- - -## Quick Start - -New to the project? Start here: - -1. Review the [main README](../README.md) for project overview -2. Read the [Configuration Guide](ConfigurationGuide.md) to understand environment variables -3. Read the [Infrastructure Guide](InfrastructureGuide.md) to understand the AWS CDK setup -4. Follow the [Getting Started](InfrastructureGuide.md#getting-started) section to deploy infrastructure +This directory contains technical documentation for the project. For an overview of the project, see the main [README](../README.md). --- -## Contributing to Documentation - -When adding new documentation: - -1. Create the document in this directory -2. Use Markdown format (`.md`) -3. Add an entry to this README's Table of Contents -4. Include a clear title and table of contents in the document -5. Follow the existing documentation style and structure -6. Update the "Last Updated" date in the document - -### Documentation Standards - -- Use clear, descriptive headings -- Include code examples where appropriate -- Add links to related documentation -- Keep content up-to-date with code changes -- Use tables for structured information -- Include troubleshooting sections for complex topics - ---- - -## Document Index - -| Document | Description | Last Updated | -| ---------------------------------------------- | -------------------------------------------- | ------------ | -| [Configuration Guide](ConfigurationGuide.md) | Application and infrastructure configuration | 2025-12-02 | -| [Infrastructure Guide](InfrastructureGuide.md) | AWS CDK infrastructure documentation | 2025-11-30 | - ---- - -## Support - -For questions or issues: - -1. Check the relevant guide in this documentation -2. Review the [troubleshooting sections](InfrastructureGuide.md#troubleshooting) -3. Search [GitHub Issues](https://github.com/leanstacks/lambda-starter/issues) -4. Contact the platform team -5. Create a new issue with detailed information - ---- +## Table of Contents -**Maintained By**: LeanStacks Platform Team +- [Configuration Guide](ConfigurationGuide.md): Application and infrastructure configuration, environment variables, and validation. +- [Infrastructure Guide](InfrastructureGuide.md): AWS CDK stacks, resource summary, and tagging. +- [DevOps Guide](DevOpsGuide.md): GitHub Actions workflows, CI/CD, and automation. diff --git a/infrastructure/README.md b/infrastructure/README.md index cdce4da..e86f5d2 100644 --- a/infrastructure/README.md +++ b/infrastructure/README.md @@ -1,89 +1,56 @@ # Infrastructure -AWS CDK infrastructure for the lambda-starter project. - -## Table of Contents - -- [Overview](#overview) -- [What is AWS CDK?](#what-is-aws-cdk) -- [Technology Stack](#technology-stack) -- [Project Structure](#project-structure) -- [Prerequisites](#prerequisites) -- [Getting Started](#getting-started) -- [Configuration](#configuration) -- [Available Commands](#available-commands) -- [Stacks](#stacks) -- [Resource Tagging](#resource-tagging) -- [Deployment](#deployment) -- [Testing](#testing) -- [Best Practices](#best-practices) -- [Troubleshooting](#troubleshooting) -- [Additional Resources](#additional-resources) - -## Overview - -This directory contains the AWS CDK infrastructure code for deploying the lambda-starter application. The infrastructure is written in TypeScript and uses AWS CDK v2 for infrastructure as code. - -The CDK application is designed to be environment-aware, supporting deployment to multiple environments (dev, qat, prd) with environment-specific configurations and safeguards. - -## What is AWS CDK? - -The AWS Cloud Development Kit (CDK) is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. - -**Key Benefits:** - -- **Familiar Languages**: Write infrastructure in TypeScript, Python, Java, or C# -- **Higher-Level Abstractions**: Use constructs that provide sensible defaults -- **Type Safety**: Catch errors at compile time -- **Reusability**: Share and reuse infrastructure components -- **Testing**: Write unit tests for your infrastructure - -**How it Works:** - -1. Write infrastructure code in TypeScript -2. CDK synthesizes CloudFormation templates -3. CloudFormation provisions AWS resources -4. Changes are deployed as stack updates +AWS CDK infrastructure for this project. ## Technology Stack -- **AWS CDK:** v2.178.0+ +- **AWS CDK:** v2 - **Language:** TypeScript 5.9+ - **Node.js:** v24+ - **Package Manager:** npm - **Testing:** Jest - **Validation:** Zod -- **AWS SDK:** v3 ## Project Structure ``` /infrastructure - /stacks - data-stack.ts # DynamoDB tables and data resources - data-stack.test.ts # Unit tests for data stack - /utils - config.ts # Configuration management with Zod validation - config.test.ts # Unit tests for config - app.ts # CDK application entry point - cdk.json # CDK configuration and feature flags - jest.config.ts # Jest configuration - jest.setup.ts # Jest setup - package.json # Dependencies and scripts - tsconfig.json # TypeScript configuration - .env # Local environment variables (not committed) - .env.example # Example environment configuration - README.md # This file + /stacks + data-stack.ts # DynamoDB tables and data resources + data-stack.test.ts # Unit tests for data stack + lambda-stack.ts # Lambda/API Gateway resources + lambda-stack.test.ts # Unit tests for lambda stack + /utils + config.ts # Configuration management with Zod validation + config.test.ts # Unit tests for config + app.ts # CDK application entry point + cdk.context.json # CDK context values + cdk.json # CDK configuration and feature flags + jest.config.ts # Jest configuration + jest.setup.ts # Jest setup + package.json # Dependencies and scripts + tsconfig.json # TypeScript configuration + .env # Local environment variables (not committed) + .env.example # Example environment configuration + README.md # This file ``` ## Prerequisites Before you begin, ensure you have the following installed and configured: -1. **Node.js v24 or later** +1. **Node.js v24 or later** (use NVM to manage Node.js versions) ```bash - node --version # Should be v24.x.x or higher + # Install NVM (if not already installed) + curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash + + # Install and use the Node.js version specified in .nvmrc + nvm install + nvm use + + # Verify installation + node --version # Should match the version in .nvmrc ``` 2. **AWS CLI** configured with appropriate credentials @@ -161,26 +128,14 @@ This generates CloudFormation templates in the `cdk.out` directory. Review them ### 6. Deploy (Optional) ```bash -npm run deploy +npm run deploy:all ``` ## Configuration ### Environment Variables -All CDK configuration is managed through environment variables prefixed with `CDK_` to avoid conflicts with application code. - -| Variable | Required | Description | Valid Values | Default | -| ------------------------- | -------- | ------------------------------------------ | -------------------------------- | ---------------------------------- | -| `CDK_APP_NAME` | No | Application name for stack naming and tags | Any string | `lambda-starter` | -| `CDK_ENV` | Yes | Infrastructure environment | `dev`, `qat`, `prd` | - | -| `CDK_ACCOUNT` | No | Override AWS account ID | 12-digit account ID | `CDK_DEFAULT_ACCOUNT` from AWS CLI | -| `CDK_REGION` | No | Override AWS region | Valid AWS region | `CDK_DEFAULT_REGION` from AWS CLI | -| `CDK_OU` | No | Organizational unit | Any string | `leanstacks` | -| `CDK_OWNER` | No | Team or owner name | Any string | `unknown` | -| `CDK_APP_LOGGING_ENABLED` | No | Enable application logging | `true`, `false` | `true` | -| `CDK_APP_LOGGING_LEVEL` | No | Application logging level | `debug`, `info`, `warn`, `error` | `info` | -| `CDK_APP_LOGGING_FORMAT` | No | Application logging format | `text`, `json` | `json` | +See the [Configuration Guide](../docs/ConfigurationGuide.md) for a comprehensive list of variables and secrets. ### AWS Account and Region Resolution @@ -225,10 +180,10 @@ Error: CDK configuration validation failed: CDK_ENV: CDK_ENV must be one of: dev npm run cdk list # Show differences for a specific stack -npm run cdk diff LambdaStarterDataStack-dev +npm run cdk diff lambda-starter-data-dev # Deploy a specific stack -npm run cdk deploy LambdaStarterDataStack-dev +npm run cdk deploy lambda-starter-data-dev # View stack outputs npm run cdk list --long @@ -239,320 +194,11 @@ npm run cdk destroy --all ## Stacks -### Data Stack - -**Purpose:** Manages DynamoDB tables and data-related resources. - -**Stack Name:** `lambda-starter-data-{env}` (e.g., `lambda-starter-data-dev`) - -**Resources:** - -- **Task Table** (`task-{env}`) - - Partition Key: `id` (String) - - Billing Mode: Pay-per-request (on-demand) - - Encryption: AWS managed (SSE) - - Point-in-time Recovery: Enabled for `prd` only - - Removal Policy: - - `RETAIN` for `prd` (table preserved on stack deletion) - - `DESTROY` for other environments (table deleted on stack deletion) - -**Outputs:** - -- `TaskTableName`: The DynamoDB table name (exported as `{env}-task-table-name`) -- `TaskTableArn`: The DynamoDB table ARN (exported as `{env}-task-table-arn`) - -### Lambda Stack - -**Purpose:** Manages Lambda functions, API Gateway, and application runtime resources. - -**Stack Name:** `lambda-starter-lambda-{env}` (e.g., `lambda-starter-lambda-dev`) - -**Resources:** - -- **List Tasks Function** (`list-tasks-{env}`) - - Runtime: Node.js 24.x - - Memory: 256 MB - - Timeout: 10 seconds - - Handler: Retrieves all tasks from DynamoDB - - IAM Permissions: Read access to Task table (Scan) - -- **Get Task Function** (`get-task-{env}`) - - Runtime: Node.js 24.x - - Memory: 256 MB - - Timeout: 10 seconds - - Handler: Retrieves a single task by ID - - IAM Permissions: Read access to Task table (GetItem) - -- **Create Task Function** (`create-task-{env}`) - - Runtime: Node.js 24.x - - Memory: 256 MB - - Timeout: 10 seconds - - Handler: Creates a new task in DynamoDB - - IAM Permissions: Write access to Task table (PutItem) - -- **Update Task Function** (`update-task-{env}`) - - Runtime: Node.js 24.x - - Memory: 256 MB - - Timeout: 10 seconds - - Handler: Updates an existing task in DynamoDB - - IAM Permissions: Read-write access to Task table (GetItem, UpdateItem) - -- **Delete Task Function** (`delete-task-{env}`) - - Runtime: Node.js 24.x - - Memory: 256 MB - - Timeout: 10 seconds - - Handler: Deletes an existing task from DynamoDB - - IAM Permissions: Read-write access to Task table (DeleteItem) - -**Common Lambda Configuration:** - -- Log Format: JSON (structured logging) -- Log Retention: - - `prd`: 30 days - - Other environments: 7 days -- Log Removal Policy: - - `prd`: `RETAIN` (logs preserved on stack deletion) - - Other environments: `DESTROY` (logs deleted on stack deletion) - -- **Lambda Starter API** (`lambda-starter-api-{env}`) - - Type: REST API - - Endpoints: - - `GET /tasks` - List all tasks - - `GET /tasks/{taskId}` - Get a specific task - - `POST /tasks` - Create a new task - - `PUT /tasks/{taskId}` - Update an existing task - - `DELETE /tasks/{taskId}` - Delete an existing task - - CORS: Enabled with preflight OPTIONS support - - Throttling: Rate and burst limits configured - - Stage: `{env}` (e.g., `dev`, `prd`) - -**Outputs:** - -- `ApiUrl`: The API Gateway endpoint URL -- `ApiId`: The API Gateway ID -- `ListTasksFunctionArn`: The List Tasks Lambda function ARN -- `GetTaskFunctionArn`: The Get Task Lambda function ARN -- `CreateTaskFunctionArn`: The Create Task Lambda function ARN -- `UpdateTaskFunctionArn`: The Update Task Lambda function ARN -- `DeleteTaskFunctionArn`: The Delete Task Lambda function ARN - -**Logging Configuration:** - -The Lambda stack uses the `CDK_APP_LOGGING_ENABLED`, `CDK_APP_LOGGING_LEVEL`, and `CDK_APP_LOGGING_FORMAT` environment variables to configure application logging: - -- **CDK_APP_LOGGING_ENABLED**: Controls whether logging is enabled in the Lambda functions -- **CDK_APP_LOGGING_LEVEL**: Sets the minimum log level (`debug`, `info`, `warn`, `error`) -- **CDK_APP_LOGGING_FORMAT**: Sets the log output format (`text` or `json`) - -These values are passed to the Lambda functions as environment variables (`LOGGING_ENABLED`, `LOGGING_LEVEL`, and `LOGGING_FORMAT`) and control both CloudWatch log output and application-level logging behavior. - -**Log Format Options:** - -- **json** (default): Structured JSON logs ideal for CloudWatch Logs Insights and log aggregation tools. Each log entry is a JSON object with fields like `timestamp`, `level`, `message`, and any additional context fields. -- **text**: Human-readable text format with timestamp, level, and message. Context is stringified and appended to the message. - -## Resource Tagging - -All resources are automatically tagged for cost allocation, compliance, and management: - -| Tag | Description | Example Value | Source | -| ------- | ------------------- | ------------------- | ----------------------------------- | -| `App` | Application name | `lambda-starter` | `CDK_APP_NAME` environment variable | -| `Env` | Environment | `dev`, `qat`, `prd` | `CDK_ENV` environment variable | -| `OU` | Organizational unit | `leanstacks` | `CDK_OU` environment variable | -| `Owner` | Team or owner | `platform-team` | `CDK_OWNER` environment variable | - -Tags are applied at the app level and automatically inherited by all stacks and resources. - -## Deployment - -### Deploy to Development - -```bash -# Ensure .env is configured for dev -echo "CDK_ENV=dev" > .env - -# Review what will be deployed -npm run synth - -# Deploy -npm run deploy -``` - -### Deploy to Production - -```bash -# Configure for production -cat > .env << EOF -CDK_ENV=prd -CDK_ACCOUNT=123456789012 -CDK_REGION=us-east-1 -CDK_OU=leanstacks -CDK_OWNER=platform-team -EOF - -# Review changes -npm run diff - -# Deploy with manual approval -npm run deploy -``` - -### CI/CD Deployment - -For automated deployments, set environment variables directly: - -```bash -export CDK_ENV=prd -export CDK_ACCOUNT=123456789012 -export CDK_REGION=us-east-1 - -npm run deploy -- --require-approval never -``` - -## Testing - -### Unit Tests - -Run all unit tests: - -```bash -npm test -``` - -### Test Coverage - -Generate a coverage report: - -```bash -npm run test:coverage -``` - -Coverage reports are generated in the `coverage` directory and can be viewed in your browser: - -```bash -open coverage/lcov-report/index.html -``` - -### Watch Mode - -Run tests in watch mode for development: - -```bash -npm run test:watch -``` - -### What is Tested? - -- **Configuration Validation**: Ensures environment variables are properly validated -- **Stack Resources**: Verifies correct resource creation and properties -- **Environment-Specific Logic**: Tests different behaviors for dev vs prd -- **Tagging**: Confirms all resources are properly tagged - -## Best Practices - -### Security - -1. **Never commit secrets**: Use `.env` for local configuration only -2. **Use AWS Secrets Manager**: Store sensitive values in AWS Secrets Manager or SSM Parameter Store -3. **Least privilege**: Grant only necessary IAM permissions -4. **Enable encryption**: All data at rest should be encrypted -5. **Separate accounts**: Use different AWS accounts for each environment - -### Development - -1. **Test before deploying**: Always run `npm test` before deployment -2. **Review diffs**: Use `npm run diff` to review changes before applying -3. **Use descriptive names**: Follow naming conventions for resources -4. **Document changes**: Update README when adding new stacks or resources -5. **Type safety**: Leverage TypeScript for compile-time error detection - -### Operations - -1. **Tag everything**: Ensure all resources have proper tags -2. **Monitor costs**: Use cost allocation tags to track spending -3. **Backup production**: Enable point-in-time recovery for critical databases -4. **Retain production resources**: Use `RETAIN` removal policy for production -5. **Version control**: Commit infrastructure changes to source control - -## Troubleshooting - -### Configuration Validation Errors - -**Problem:** `CDK configuration validation failed` - -**Solutions:** +See the [Infrastructure Guide](../docs/InfrastructureGuide.md) for a comprehensive list of stacks, resources, and outputs. -1. Verify `.env` file exists in the infrastructure directory -2. Check that `CDK_ENV` is set to a valid value (`dev`, `qat`, `prd`) -3. Ensure all required variables are set - -### TypeScript Compilation Errors - -**Problem:** Build fails with TypeScript errors - -**Solutions:** - -1. Ensure dependencies are installed: `npm install` -2. Verify Node.js version: `node --version` (should be v24+) -3. Check for syntax errors in TypeScript files -4. Clean and rebuild: `npm run clean && npm run build` - -### Deployment Failures - -**Problem:** Stack deployment fails - -**Solutions:** - -1. Verify AWS credentials: `aws sts get-caller-identity` -2. Check account and region: Ensure `CDK_ACCOUNT` and `CDK_REGION` match your AWS profile -3. Confirm IAM permissions: Verify you have necessary permissions -4. Review CloudFormation events in AWS Console for detailed error messages -5. Check for resource naming conflicts - -### CDK Bootstrap Issues - -**Problem:** `This stack requires bootstrap stack version X` - -**Solution:** - -```bash -cdk bootstrap aws://ACCOUNT-ID/REGION --force -``` - -### Node Version Warnings - -**Problem:** Warning about untested Node.js version - -**Solution:** - -```bash -export JSII_SILENCE_WARNING_UNTESTED_NODE_VERSION=1 -``` - -Or use a supported Node.js version (22.x or 20.x). - -## Additional Resources - -### AWS CDK Documentation +## Further Reading +- [Project Documentation](../docs/README.md) - [AWS CDK Developer Guide](https://docs.aws.amazon.com/cdk/latest/guide/) - [AWS CDK API Reference](https://docs.aws.amazon.com/cdk/api/v2/) -- [CDK Workshop](https://cdkworkshop.com/) -- [CDK Patterns](https://cdkpatterns.com/) - -### Best Practices - -- [CDK Best Practices](https://docs.aws.amazon.com/cdk/latest/guide/best-practices.html) -- [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) -- [Infrastructure as Code Best Practices](https://docs.aws.amazon.com/prescriptive-guidance/latest/infrastructure-as-code/welcome.html) - -### Community - -- [AWS CDK GitHub](https://github.com/aws/aws-cdk) -- [CDK Slack Channel](https://cdk.dev) -- [AWS Developer Forums](https://forums.aws.amazon.com/forum.jspa?forumID=351) - ---- - -For questions or issues, please contact the platform team or open an issue in the repository. +- [AWS CDK Best Practices](https://docs.aws.amazon.com/cdk/latest/guide/best-practices.html) diff --git a/infrastructure/stacks/data-stack.test.ts b/infrastructure/stacks/data-stack.test.ts index 938fb7d..c3d1663 100644 --- a/infrastructure/stacks/data-stack.test.ts +++ b/infrastructure/stacks/data-stack.test.ts @@ -59,7 +59,7 @@ describe('DataStack', () => { it('should export table name', () => { template.hasOutput('TaskTableName', { Export: { - Name: 'dev-task-table-name', + Name: 'lambda-starter-task-table-name-dev', }, }); }); @@ -67,7 +67,7 @@ describe('DataStack', () => { it('should export table ARN', () => { template.hasOutput('TaskTableArn', { Export: { - Name: 'dev-task-table-arn', + Name: 'lambda-starter-task-table-arn-dev', }, }); }); diff --git a/infrastructure/stacks/data-stack.ts b/infrastructure/stacks/data-stack.ts index d36a9ca..b905249 100644 --- a/infrastructure/stacks/data-stack.ts +++ b/infrastructure/stacks/data-stack.ts @@ -48,14 +48,14 @@ export class DataStack extends cdk.Stack { new cdk.CfnOutput(this, 'TaskTableName', { value: this.taskTable.tableName, description: 'The name of the Task DynamoDB table', - exportName: `${props.envName}-task-table-name`, + exportName: `${props.appName}-task-table-name-${props.envName}`, }); // Output the table ARN new cdk.CfnOutput(this, 'TaskTableArn', { value: this.taskTable.tableArn, description: 'The ARN of the Task DynamoDB table', - exportName: `${props.envName}-task-table-arn`, + exportName: `${props.appName}-task-table-arn-${props.envName}`, }); } }