Testing

This guide covers testing strategies, tools, and practices for TMI development including unit tests, integration tests, API tests, and end-to-end tests.

Testing Philosophy

TMI follows a comprehensive testing approach:

Unit Tests - Fast tests with no external dependencies
Integration Tests - Tests with real database and services
API Tests - Complete API workflow testing with Postman/Newman
E2E Tests - Full user journey testing with Cypress

Test Pyramid

        /\
       /E2E\          Few, slow, expensive
      /------\
     /  API  \        Some, medium speed
    /----------\
   /Integration\     More, medium speed
  /--------------\
 /   Unit Tests  \   Many, fast, cheap
/------------------\

Testing Principles

Test business logic thoroughly - Unit test all business rules
Test integration points - Verify components work together
Test user workflows - Ensure complete features work end-to-end
Automate everything - All tests should be automated
Fast feedback - Unit tests run in seconds
Realistic testing - Integration tests use real databases

Unit Testing

Server Unit Tests (Go)

TMI server uses Go's built-in testing framework.

Running Unit Tests

# Run all unit tests
make test-unit

# Run specific test
go test -v ./api -run TestCreateThreatModel

# Run with coverage
make test-coverage-unit

Writing Unit Tests

Test File Naming: *_test.go

Example Test:

// api/threat_model_test.go
package api

import (
    "testing"
    "github.com/stretchr/testify/assert"
)

func TestCreateThreatModel(t *testing.T) {
    // Arrange
    tm := ThreatModel{
        Name:        "Test Threat Model",
        Description: stringPtr("Test description"),
    }

    // Act
    result, err := createThreatModelLogic(tm)

    // Assert
    assert.NoError(t, err)
    assert.NotEmpty(t, result.ID)
    assert.Equal(t, tm.Name, result.Name)
}

Test Patterns

Table-Driven Tests:

func TestAuthorizationRoles(t *testing.T) {
    tests := []struct {
        name     string
        role     string
        canRead  bool
        canWrite bool
        canDelete bool
    }{
        {"owner", "owner", true, true, true},
        {"writer", "writer", true, true, false},
        {"reader", "reader", true, false, false},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            assert.Equal(t, tt.canRead, canRead(tt.role))
            assert.Equal(t, tt.canWrite, canWrite(tt.role))
            assert.Equal(t, tt.canDelete, canDelete(tt.role))
        })
    }
}

Mocking External Dependencies:

type MockDatabase struct {
    mock.Mock
}

func (m *MockDatabase) GetThreatModel(id string) (*ThreatModel, error) {
    args := m.Called(id)
    return args.Get(0).(*ThreatModel), args.Error(1)
}

func TestWithMock(t *testing.T) {
    // Create mock
    mockDB := new(MockDatabase)
    mockDB.On("GetThreatModel", "123").Return(&ThreatModel{
        ID: "123",
        Name: "Test",
    }, nil)

    // Use mock in test
    tm, err := mockDB.GetThreatModel("123")

    assert.NoError(t, err)
    assert.Equal(t, "123", tm.ID)
    mockDB.AssertExpectations(t)
}

Web App Unit Tests (Angular/TypeScript)

TMI-UX uses Vitest for unit testing.

Running Unit Tests

# Run all tests
pnpm run test

# Run in watch mode
pnpm run test:watch

# Run with UI
pnpm run test:ui

# Run specific test
pnpm run test -- src/app/pages/tm/tm.component.spec.ts

# Coverage report
pnpm run test:coverage

Writing Unit Tests

Test File Naming: *.spec.ts

Example Component Test:

// src/app/pages/tm/tm.component.spec.ts
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TmComponent } from './tm.component';
import { ApiService } from '../../core/services/api.service';
import { of } from 'rxjs';

describe('TmComponent', () => {
  let component: TmComponent;
  let fixture: ComponentFixture<TmComponent>;
  let mockApiService: jasmine.SpyObj<ApiService>;

  beforeEach(async () => {
    // Create mock
    mockApiService = jasmine.createSpyObj('ApiService', ['getThreatModels']);

    await TestBed.configureTestingModule({
      imports: [TmComponent],
      providers: [
        { provide: ApiService, useValue: mockApiService }
      ]
    }).compileComponents();

    fixture = TestBed.createComponent(TmComponent);
    component = fixture.componentInstance;
  });

  it('should create', () => {
    expect(component).toBeTruthy();
  });

  it('should load threat models on init', () => {
    // Arrange
    const mockThreatModels = [
      { id: '1', name: 'TM 1' },
      { id: '2', name: 'TM 2' }
    ];
    mockApiService.getThreatModels.and.returnValue(of(mockThreatModels));

    // Act
    component.ngOnInit();

    // Assert
    expect(mockApiService.getThreatModels).toHaveBeenCalled();
    expect(component.threatModels).toEqual(mockThreatModels);
  });
});

Service Test:

// src/app/core/services/api.service.spec.ts
import { TestBed } from '@angular/core/testing';
import { HttpClientTestingModule, HttpTestingController } from '@angular/common/http/testing';
import { ApiService } from './api.service';

describe('ApiService', () => {
  let service: ApiService;
  let httpMock: HttpTestingController;

  beforeEach(() => {
    TestBed.configureTestingModule({
      imports: [HttpClientTestingModule],
      providers: [ApiService]
    });

    service = TestBed.inject(ApiService);
    httpMock = TestBed.inject(HttpTestingController);
  });

  afterEach(() => {
    httpMock.verify();
  });

  it('should fetch threat models', () => {
    const mockThreatModels = [{ id: '1', name: 'TM 1' }];

    service.getThreatModels().subscribe(tms => {
      expect(tms).toEqual(mockThreatModels);
    });

    const req = httpMock.expectOne('/api/threat_models');
    expect(req.request.method).toBe('GET');
    req.flush(mockThreatModels);
  });
});

Integration Testing

Integration tests verify that components work correctly with real databases and services.

Server Integration Tests (Go)

Running Integration Tests

# Run all integration tests (automatic setup and cleanup)
make test-integration

# This automatically:
# 1. Starts PostgreSQL container
# 2. Starts Redis container
# 3. Runs migrations
# 4. Starts server
# 5. Runs tests
# 6. Cleans up everything

Test Configuration

Integration tests use dedicated ports to avoid conflicts:

PostgreSQL: Port 5434 (vs 5432 for development)
Redis: Port 6381 (vs 6379 for development)
Server: Port 8080

Writing Integration Tests

Test File Naming: *_integration_test.go

Example:

// api/threat_model_integration_test.go
package api

import (
    "testing"
    "net/http"
    "net/http/httptest"
    "github.com/stretchr/testify/assert"
)

func TestDatabaseThreatModelIntegration(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    // Create threat model
    threatModelData := map[string]interface{}{
        "name": "Integration Test TM",
        "description": "Test with real database",
    }

    req := suite.makeAuthenticatedRequest("POST", "/threat_models", threatModelData)
    w := suite.executeRequest(req)

    assert.Equal(t, http.StatusCreated, w.Code)

    // Verify in database
    var tm ThreatModel
    err := suite.db.First(&tm).Error
    assert.NoError(t, err)
    assert.Equal(t, "Integration Test TM", tm.Name)
}

Test Data Management

Predictable Test Users (using login hints):

func createTestUser(hint string) (*User, string) {
    // Create specific test user 'alice@test.tmi' instead of random
    resp, _ := http.Get(
        "http://localhost:8080/oauth2/authorize?idp=test&login_hint=" + hint
    )

    // Parse token from response
    token := parseTokenFromResponse(resp)
    return &User{Email: hint + "@test.tmi"}, token
}

func TestMultiUserScenario(t *testing.T) {
    alice, aliceToken := createTestUser("alice")
    bob, bobToken := createTestUser("bob")

    // Test with both users
}

Test Patterns

Complete Entity Lifecycle:

func TestThreatModelLifecycle(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    // 1. Create
    createReq := suite.makeAuthenticatedRequest("POST", "/threat_models", data)
    createW := suite.executeRequest(createReq)
    assert.Equal(t, http.StatusCreated, createW.Code)
    tmID := parseID(createW.Body)

    // 2. Read
    getReq := suite.makeAuthenticatedRequest("GET", "/threat_models/" + tmID, nil)
    getW := suite.executeRequest(getReq)
    assert.Equal(t, http.StatusOK, getW.Code)

    // 3. Update
    updateReq := suite.makeAuthenticatedRequest("PUT", "/threat_models/" + tmID, updatedData)
    updateW := suite.executeRequest(updateReq)
    assert.Equal(t, http.StatusOK, updateW.Code)

    // 4. Delete
    deleteReq := suite.makeAuthenticatedRequest("DELETE", "/threat_models/" + tmID, nil)
    deleteW := suite.executeRequest(deleteReq)
    assert.Equal(t, http.StatusNoContent, deleteW.Code)

    // 5. Verify deletion
    verifyReq := suite.makeAuthenticatedRequest("GET", "/threat_models/" + tmID, nil)
    verifyW := suite.executeRequest(verifyReq)
    assert.Equal(t, http.StatusNotFound, verifyW.Code)
}

Authorization Testing:

func TestAuthorizationMatrix(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    alice, aliceToken := createTestUser("alice")
    bob, bobToken := createTestUser("bob")

    // Alice creates threat model
    tm := createThreatModel(aliceToken)

    // Test reader permissions
    addAuthorization(tm.ID, bob.Email, "reader", aliceToken)

    // Bob can read
    getReq := makeRequestWithToken("GET", "/threat_models/" + tm.ID, nil, bobToken)
    assert.Equal(t, http.StatusOK, suite.executeRequest(getReq).Code)

    // Bob cannot write
    updateReq := makeRequestWithToken("PUT", "/threat_models/" + tm.ID, data, bobToken)
    assert.Equal(t, http.StatusForbidden, suite.executeRequest(updateReq).Code)

    // Bob cannot delete
    deleteReq := makeRequestWithToken("DELETE", "/threat_models/" + tm.ID, nil, bobToken)
    assert.Equal(t, http.StatusForbidden, suite.executeRequest(deleteReq).Code)
}

API Testing

TMI uses Postman collections and Newman for comprehensive API testing.

Running API Tests

# Run all API tests
make test-api

# Or run manually
cd postman
./run-tests.sh

Test Collections

Located in /postman directory:

comprehensive-test-collection.json - Main test suite
unauthorized-tests-collection.json - 401 error testing
threat-crud-tests-collection.json - Threat CRUD operations
metadata-tests-collection.json - Metadata operations
permission-matrix-tests-collection.json - Authorization testing
bulk-operations-tests-collection.json - Batch operations

Test Coverage

API tests cover:

✅ 70+ endpoints
✅ 91 workflow methods
✅ All HTTP status codes (200, 201, 204, 400, 401, 403, 404, 409, 422, 500)
✅ Authentication and authorization
✅ CRUD operations for all entities
✅ Metadata operations
✅ Batch operations
✅ Error scenarios

Writing Postman Tests

Basic Test:

pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

pm.test("Response has threat models", function () {
    const response = pm.response.json();
    pm.expect(response).to.be.an('array');
    pm.expect(response.length).to.be.above(0);
});

Advanced Test with Setup:

// Pre-request Script
const data = {
    name: "Test Threat Model",
    description: "Created by test"
};
pm.collectionVariables.set("threat_model_data", JSON.stringify(data));

// Test Script
pm.test("Threat model created", function () {
    pm.response.to.have.status(201);
    const response = pm.response.json();

    pm.expect(response).to.have.property('id');
    pm.expect(response.name).to.equal("Test Threat Model");

    // Save ID for subsequent tests
    pm.collectionVariables.set("threat_model_id", response.id);
});

End-to-End Testing

TMI-UX uses Cypress for E2E testing.

Running E2E Tests

# Run all E2E tests
pnpm run test:e2e

# Open Cypress GUI
pnpm run test:e2e:open

# Run specific spec
pnpm run test:e2e -- --spec="cypress/e2e/login.cy.ts"

Writing E2E Tests

Test File Naming: *.cy.ts

Example Login Test:

// cypress/e2e/login.cy.ts
describe('Login Flow', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  it('should display login page', () => {
    cy.contains('Sign In').should('be.visible');
  });

  it('should login with test provider', () => {
    cy.contains('Test Login').click();
    cy.url().should('include', '/dashboard');
    cy.contains('Threat Models').should('be.visible');
  });
});

Example Diagram Test:

// cypress/e2e/diagram.cy.ts
describe('Diagram Editor', () => {
  beforeEach(() => {
    cy.login(); // Custom command
    cy.visit('/threat-models/123/diagrams/456');
  });

  it('should add process to diagram', () => {
    // Open shape palette
    cy.get('[data-cy=shape-palette]').click();

    // Select process shape
    cy.get('[data-cy=shape-process]').click();

    // Click on canvas to add
    cy.get('[data-cy=diagram-canvas]').click(200, 200);

    // Verify process added
    cy.get('[data-shape=process]').should('exist');
  });

  it('should edit process label', () => {
    cy.get('[data-shape=process]').first().dblclick();
    cy.get('[data-cy=label-input]').clear().type('Authentication Service');
    cy.get('[data-cy=label-save]').click();

    cy.get('[data-shape=process]').first()
      .should('contain', 'Authentication Service');
  });
});

WebSocket Testing

Manual WebSocket Testing

TMI provides a WebSocket test harness for manual testing:

# Build test harness
make build-wstest

# Run 3-terminal test (alice as host, bob and charlie as participants)
make wstest

# Run monitor mode
make monitor-wstest

# Clean up
make clean-wstest

Automated WebSocket Testing

Test File: postman/collaboration-tests-collection.json

Tests WebSocket functionality:

Session creation and joining
Diagram operations broadcast
Presenter mode
Cursor sharing
User join/leave events

Coverage Reporting

Server Coverage

# Generate coverage reports
make test-coverage

# This creates:
# - coverage/unit.out
# - coverage/integration.out
# - coverage/combined.out
# - coverage_html/unit.html
# - coverage_html/integration.html
# - coverage_html/combined.html

View HTML Report:

open coverage_html/combined.html

Coverage Goals:

Unit Tests: 80%+ coverage
Integration Tests: 70%+ coverage
Combined: 85%+ coverage

Web App Coverage

# Generate coverage report
pnpm run test:coverage

# View report
open coverage/index.html

Coverage Configuration: vitest.config.ts

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8',
      reporter: ['text', 'html', 'lcov'],
      exclude: [
        'node_modules/',
        'src/**/*.spec.ts',
        'src/environments/'
      ]
    }
  }
});

Testing Best Practices

1. Test Organization

One test file per source file
Group related tests with describe blocks
Use clear, descriptive test names
Follow AAA pattern: Arrange, Act, Assert

2. Test Data

Use factories for test data
Create minimal test data
Clean up after tests
Use predictable test users (login hints)

3. Isolation

Tests should be independent
Don't rely on test order
Clean up between tests
Mock external dependencies

4. Assertions

Test one thing per test
Use specific assertions
Test both happy path and error cases
Verify side effects

5. Performance

Keep unit tests fast (<1s each)
Use before/after hooks efficiently
Parallelize tests when possible
Cache test fixtures

6. Maintainability

DRY - Don't Repeat Yourself
Use helper functions
Keep tests simple
Update tests with code

Continuous Integration

GitHub Actions

Tests run automatically on:

Pull requests
Pushes to main branch
Scheduled nightly builds

Workflow (.github/workflows/test.yml):

name: Tests

on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-go@v2
      - run: make test-unit

  integration-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-go@v2
      - run: make test-integration

  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - run: npm install -g newman
      - run: make test-api

Troubleshooting Tests

Integration Tests Fail

# Clean everything and retry
make clean-everything
make test-integration

# Check container logs
docker logs tmi-integration-postgres
docker logs tmi-integration-redis

# Verify ports are free
lsof -ti :5434  # PostgreSQL
lsof -ti :6381  # Redis

API Tests Fail

# Check server is running
curl http://localhost:8080/

# Check authentication
curl -H "Authorization: Bearer TOKEN" http://localhost:8080/threat_models

# Run specific collection
newman run postman/comprehensive-test-collection.json

E2E Tests Fail

# Clear Cypress cache
pnpm run test:e2e:clean

# Run in headed mode to see what's happening
pnpm run test:e2e:open

# Check screenshots
ls cypress/screenshots/

# Check videos
ls cypress/videos/

Testing

Testing

Table of Contents

Testing Philosophy

Test Pyramid

Testing Principles

Unit Testing

Server Unit Tests (Go)

Running Unit Tests

Writing Unit Tests

Test Patterns

Web App Unit Tests (Angular/TypeScript)

Running Unit Tests

Writing Unit Tests

Integration Testing

Server Integration Tests (Go)

Running Integration Tests

Test Configuration

Writing Integration Tests

Test Data Management

Test Patterns

API Testing

Running API Tests

Test Collections

Test Coverage

Writing Postman Tests

End-to-End Testing

Running E2E Tests

Writing E2E Tests

WebSocket Testing

Manual WebSocket Testing

Automated WebSocket Testing

Coverage Reporting

Server Coverage

Web App Coverage

Testing Best Practices

1. Test Organization

2. Test Data

3. Isolation

4. Assertions

5. Performance

6. Maintainability

Continuous Integration

GitHub Actions

Troubleshooting Tests

Integration Tests Fail

API Tests Fail

E2E Tests Fail

Next Steps

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!