taskplaner/.planning/phases/09-ci-pipeline/09-RESEARCH.md

Phase 9: CI Pipeline Hardening - Research

Researched: 2026-02-03 Domain: Testing infrastructure (Vitest, Playwright, svelte-check) + CI/CD (Gitea Actions) Confidence: HIGH

Summary

This phase establishes a comprehensive testing pipeline that blocks deployment on test or type failures. The research covers three testing layers: unit tests (Vitest), type checking (svelte-check), and E2E tests (Playwright), integrated into the existing Gitea Actions workflow.

The standard approach for SvelteKit testing in 2026 uses Vitest with browser mode for component tests (real browser via Playwright, not jsdom), traditional Vitest for server/utility tests, and standalone Playwright for full E2E tests. The user decisions lock in 80% coverage threshold, Playwright component tests for DOM interactions, and Slack notifications on failure.

Key finding: Vitest browser mode with vitest-browser-svelte is the modern approach for Svelte 5 component testing, replacing the older jsdom + @testing-library/svelte pattern. This provides real browser testing with runes support (.svelte.test.ts files).

Primary recommendation: Use multi-project Vitest configuration separating client (browser mode) and server (node) tests, with standalone Playwright for E2E, all gated before Docker build in CI.

Standard Stack

The established libraries/tools for this domain:

Core

Library	Version	Purpose	Why Standard
vitest	^3.x	Unit/component test runner	Official SvelteKit recommendation, Vite-native
@vitest/browser	^3.x	Browser mode for component tests	Real browser testing without jsdom limitations
vitest-browser-svelte	^0.x	Svelte component rendering in browser mode	Official Svelte 5 support with runes
@vitest/browser-playwright	^3.x	Playwright provider for Vitest browser mode	Real Chrome DevTools Protocol, not simulated events
@vitest/coverage-v8	^3.x	V8-based coverage collection	Fast native coverage, identical accuracy to Istanbul since v3.2
@playwright/test	^1.58	E2E test framework	Already installed, mature E2E solution
svelte-check	^4.x	TypeScript/Svelte type checking	Already installed, CI-compatible output

Supporting

Library	Version	Purpose	When to Use
@testing-library/svelte	^5.x	Alternative component testing	Only if not using browser mode (jsdom fallback)
drizzle-seed	^0.x	Database seeding for tests	E2E test fixtures with Drizzle ORM

Alternatives Considered

Instead of	Could Use	Tradeoff
Vitest browser mode	jsdom + @testing-library/svelte	jsdom simulates browser, misses real CSS/runes issues
v8 coverage	istanbul	istanbul 300% slower, v8 now equally accurate
Playwright for E2E	Cypress	Playwright already in project, better multi-browser support

Installation:

npm install -D vitest @vitest/browser vitest-browser-svelte @vitest/browser-playwright @vitest/coverage-v8 drizzle-seed
npx playwright install chromium

Architecture Patterns

Recommended Project Structure

src/
├── lib/
│   ├── components/
│   │   ├── Button.svelte
│   │   └── Button.svelte.test.ts    # Component tests (browser mode)
│   ├── utils/
│   │   ├── format.ts
│   │   └── format.test.ts           # Utility tests (node mode)
│   └── server/
│       ├── db/
│       │   └── queries.test.ts      # Server tests (node mode)
│       └── api.test.ts
├── routes/
│   └── +page.server.test.ts         # Server route tests (node mode)
tests/
├── e2e/                             # Playwright E2E tests
│   ├── fixtures/
│   │   └── db.ts                    # Database seeding fixture
│   ├── user-journeys.spec.ts
│   └── index.ts                     # Custom test with fixtures
├── docker-deployment.spec.ts        # Existing deployment tests
vitest-setup-client.ts               # Browser mode setup
vitest.config.ts                     # Multi-project config (or in vite.config.ts)
playwright.config.ts                 # E2E config (already exists)

Pattern 1: Multi-Project Vitest Configuration

What: Separate test projects for different environments (browser vs node) When to use: SvelteKit apps with both client components and server code Example:

// vite.config.ts
// Source: https://scottspence.com/posts/testing-with-vitest-browser-svelte-guide
import { sveltekit } from '@sveltejs/kit/vite';
import tailwindcss from '@tailwindcss/vite';
import { playwright } from '@vitest/browser-playwright';
import { defineConfig } from 'vite';

export default defineConfig({
  plugins: [tailwindcss(), sveltekit()],
  test: {
    coverage: {
      provider: 'v8',
      reporter: ['text', 'json', 'html'],
      include: ['src/**/*.{ts,svelte}'],
      exclude: ['src/**/*.test.ts', 'src/**/*.spec.ts'],
      thresholds: {
        global: {
          statements: 80,
          branches: 80,
          functions: 80,
          lines: 80,
        },
      },
    },
    projects: [
      {
        extends: true,
        test: {
          name: 'client',
          testTimeout: 5000,
          browser: {
            enabled: true,
            provider: playwright(),
            instances: [{ browser: 'chromium' }],
          },
          include: ['src/**/*.svelte.{test,spec}.{js,ts}'],
          setupFiles: ['./vitest-setup-client.ts'],
        },
      },
      {
        extends: true,
        test: {
          name: 'server',
          environment: 'node',
          include: ['src/**/*.{test,spec}.{js,ts}'],
          exclude: ['src/**/*.svelte.{test,spec}.{js,ts}'],
        },
      },
    ],
  },
});

Pattern 2: Component Test with Runes Support

What: Test Svelte 5 components with $state and $derived in real browser When to use: Any component using Svelte 5 runes Example:

// src/lib/components/Counter.svelte.test.ts
// Source: https://svelte.dev/docs/svelte/testing
import { render } from 'vitest-browser-svelte';
import { page } from '@vitest/browser/context';
import { describe, expect, it } from 'vitest';
import { flushSync } from 'svelte';
import Counter from './Counter.svelte';

describe('Counter Component', () => {
  it('increments count on click', async () => {
    render(Counter, { props: { initial: 0 } });

    const button = page.getByRole('button', { name: /increment/i });
    await button.click();

    // flushSync needed for external state changes
    flushSync();

    await expect.element(page.getByText('Count: 1')).toBeInTheDocument();
  });
});

Pattern 3: E2E Database Fixture with Drizzle

What: Seed database before tests, clean up after When to use: E2E tests requiring known data state Example:

// tests/e2e/fixtures/db.ts
// Source: https://mainmatter.com/blog/2025/08/21/mock-database-in-svelte-tests/
import { test as base } from '@playwright/test';
import { db } from '../../../src/lib/server/db/index.js';
import * as schema from '../../../src/lib/server/db/schema.js';
import { reset, seed } from 'drizzle-seed';

export const test = base.extend<{
  seededDb: typeof db;
}>({
  seededDb: async ({}, use) => {
    // Seed with known test data
    await seed(db, schema, { count: 10 });

    await use(db);

    // Clean up after test
    await reset(db, schema);
  },
});

export { expect } from '@playwright/test';

Pattern 4: SvelteKit Module Mocking

What: Mock $app/stores and $app/navigation in unit tests When to use: Testing components that use SvelteKit-specific imports Example:

// vitest-setup-client.ts
// Source: https://www.closingtags.com/blog/mocking-svelte-stores-in-vitest
/// <reference types="@vitest/browser/matchers" />

import { vi } from 'vitest';
import { writable } from 'svelte/store';

// Mock $app/navigation
vi.mock('$app/navigation', () => ({
  goto: vi.fn(() => Promise.resolve()),
  invalidate: vi.fn(() => Promise.resolve()),
  invalidateAll: vi.fn(() => Promise.resolve()),
  beforeNavigate: vi.fn(),
  afterNavigate: vi.fn(),
}));

// Mock $app/stores
vi.mock('$app/stores', () => ({
  page: writable({
    url: new URL('http://localhost'),
    params: {},
    route: { id: null },
    status: 200,
    error: null,
    data: {},
    form: null,
  }),
  navigating: writable(null),
  updated: { check: vi.fn(), subscribe: writable(false).subscribe },
}));

Anti-Patterns to Avoid

• Testing with jsdom for Svelte 5 components: jsdom cannot properly handle runes reactivity. Use browser mode instead.
• Parallel E2E tests with shared database: Will cause race conditions. Set workers: 1 in playwright.config.ts.
• Using deprecated @playwright/experimental-ct-svelte: Use vitest-browser-svelte instead for component tests.
• Mocking everything in E2E tests: E2E tests should test real integrations. Only mock external services if necessary.

Don't Hand-Roll

Problems that look simple but have existing solutions:

Problem	Don't Build	Use Instead	Why
Coverage collection	Custom instrumentation	@vitest/coverage-v8	Handles source maps, thresholds, reporters automatically
Database seeding	Manual INSERT statements	drizzle-seed	Generates consistent, seeded random data with schema awareness
Component mounting	Manual DOM manipulation	vitest-browser-svelte render()	Handles Svelte 5 lifecycle, context, and cleanup
Screenshot on failure	Custom error handlers	Playwright built-in screenshot: 'only-on-failure'	Integrated with test lifecycle and artifacts
CI test output parsing	Regex parsing	svelte-check --output machine	Structured, timestamp-prefixed output designed for CI

Key insight: The testing ecosystem has mature solutions for all common needs. Hand-rolling any of these leads to edge cases around cleanup, async timing, and framework integration that the official tools have already solved.

Common Pitfalls

Pitfall 1: jsdom Limitations with Svelte 5 Runes

What goes wrong: Tests pass locally but fail to detect reactivity issues, or throw cryptic errors about $state Why it happens: jsdom simulates browser APIs but doesn't actually run JavaScript in a browser context. Svelte 5 runes compile differently and expect real browser reactivity. How to avoid: Use Vitest browser mode with Playwright provider for all .svelte component tests Warning signs: Tests involving $state, $derived, or $effect behave inconsistently or require excessive await tick()

Pitfall 2: Missing flushSync for External State

What goes wrong: Assertions fail because DOM hasn't updated after state change Why it happens: Svelte batches updates. When state changes outside component (e.g., store update in test), DOM update is async. How to avoid: Call flushSync() from 'svelte' after modifying external state before asserting Warning signs: Tests that work with longer timeouts but fail with short ones

Pitfall 3: Parallel E2E with Shared Database

What goes wrong: Flaky tests that sometimes pass, sometimes fail with data conflicts Why it happens: Multiple test workers modify the same database simultaneously How to avoid: Set workers: 1 in playwright.config.ts for E2E tests. Use separate database per worker if parallelism is needed. Warning signs: Tests pass individually but fail in full suite runs

Pitfall 4: Coverage Threshold Breaking Existing Code

What goes wrong: CI fails immediately after enabling 80% threshold because existing code has 0% coverage Why it happens: Enabling coverage thresholds on existing codebase without tests How to avoid: Start with thresholds: { autoUpdate: true } to establish baseline, then incrementally raise thresholds as tests are added Warning signs: Immediate CI failure when coverage is first enabled

Pitfall 5: SvelteKit Module Import Errors

What goes wrong: Tests fail with "Cannot find module '$app/stores'" or similar Why it happens: $app/* modules are virtual modules provided by SvelteKit at build time, not available in test environment How to avoid: Mock all $app/* imports in vitest setup file. Keep mocks simple (don't use importOriginal with SvelteKit modules - causes SSR issues). Warning signs: Import errors mentioning $app, $env, or other SvelteKit virtual modules

Pitfall 6: Playwright Browsers Not Installed in CI

What goes wrong: CI fails with "browserType.launch: Executable doesn't exist" Why it happens: Playwright browsers need explicit installation, not included in npm install How to avoid: Add npx playwright install --with-deps chromium step before tests Warning signs: Works locally (where browsers are cached), fails in fresh CI environment

Code Examples

Verified patterns from official sources:

vitest-setup-client.ts

// Source: https://vitest.dev/guide/browser/
/// <reference types="@vitest/browser/matchers" />
/// <reference types="@vitest/browser/providers/playwright" />

Package.json Scripts

{
  "scripts": {
    "test": "vitest",
    "test:unit": "vitest run",
    "test:unit:watch": "vitest",
    "test:coverage": "vitest run --coverage",
    "test:e2e": "playwright test",
    "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json"
  }
}

CI Workflow (Gitea Actions)

# Source: https://docs.gitea.com/usage/actions/quickstart
name: Test and Build

on:
  push:
    branches: [master, main]
  pull_request:
    branches: [master, main]

env:
  REGISTRY: git.kube2.tricnet.de
  IMAGE_NAME: admin/taskplaner

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run type check
        run: npm run check -- --output machine

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run unit tests with coverage
        run: npm run test:coverage

      - name: Run E2E tests
        run: npm run test:e2e
        env:
          CI: true

      - name: Upload test artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: |
            coverage/
            playwright-report/
            test-results/

  build:
    needs: test
    runs-on: ubuntu-latest
    if: github.event_name != 'pull_request' || github.event.pull_request.merged == true
    steps:
      # ... existing build steps ...

  notify:
    needs: [test, build]
    runs-on: ubuntu-latest
    if: failure()
    steps:
      - name: Notify Slack on failure
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
        run: |
          curl -X POST -H 'Content-type: application/json' \
            --data '{"text":"Pipeline failed for ${{ gitea.repository }} on ${{ gitea.ref }}"}' \
            $SLACK_WEBHOOK_URL

Playwright Config for E2E with Screenshots

// playwright.config.ts
// Source: https://playwright.dev/docs/test-configuration
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  fullyParallel: false, // Sequential for shared database
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: 1, // Single worker for database tests
  reporter: [
    ['html', { open: 'never' }],
    ['github'], // GitHub/Gitea compatible annotations
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:5173',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'off', // Per user decision: screenshots only, no video
  },
  projects: [
    {
      name: 'chromium-desktop',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'chromium-mobile',
      use: { ...devices['Pixel 5'] },
    },
  ],
  webServer: {
    command: 'npm run build && npm run preview',
    port: 4173,
    reuseExistingServer: !process.env.CI,
  },
});

svelte-check CI Output Format

# Machine-readable output for CI parsing
# Source: https://svelte.dev/docs/cli/sv-check
npx svelte-check --output machine --tsconfig ./tsconfig.json

# Output format:
# 1590680326283 ERROR "/path/file.svelte" 10:5 "Type error message"
# 1590680326807 COMPLETED 50 FILES 2 ERRORS 0 WARNINGS

State of the Art

Old Approach	Current Approach	When Changed	Impact
jsdom + @testing-library/svelte	Vitest browser mode + vitest-browser-svelte	2025	Real browser testing, runes support
Istanbul coverage	V8 coverage with AST remapping	Vitest 3.2	10x faster, same accuracy
@playwright/experimental-ct-svelte	vitest-browser-svelte	2025	Better integration, official support
Jest with svelte-jester	Vitest	2024	Native Vite support, faster

Deprecated/outdated:

• vitest-svelte-kit package: Deprecated, no longer needed with modern Vitest
• @playwright/experimental-ct-svelte: Use vitest-browser-svelte for component tests instead
• jsdom for Svelte 5 components: Does not properly support runes reactivity

Open Questions

Things that couldn't be fully resolved:

1. Exact drizzle-seed API for this schema

   • What we know: drizzle-seed works with Drizzle ORM schemas
   • What's unclear: Specific configuration for the project's schema structure
   • Recommendation: Review drizzle-seed docs during implementation with actual schema

2. Gitea Actions Slack notification action availability

   • What we know: GitHub Actions Slack actions exist (rtCamp/action-slack-notify, etc.)
   • What's unclear: Whether these work identically in Gitea Actions
   • Recommendation: Use direct curl to Slack webhook (shown in examples) for maximum compatibility

3. Vitest browser mode stability

   • What we know: Vitest documents browser mode as "experimental" with stable core
   • What's unclear: Edge cases in production CI environments
   • Recommendation: Pin Vitest version, monitor for issues

Sources

Primary (HIGH confidence)

• Svelte Official Testing Docs - Official Vitest + browser mode recommendations
• Vitest Guide - Installation, configuration, browser mode
• Vitest Coverage Config - Threshold configuration
• Vitest Browser Mode - Playwright provider setup
• svelte-check CLI - CI output formats
• Gitea Actions Quickstart - Workflow syntax

Secondary (MEDIUM confidence)

• Scott Spence - Vitest Browser Mode Guide - Multi-project configuration
• Mainmatter - Database Fixtures - Drizzle seed pattern
• Roy Bakker - Playwright CI Guide - Artifact upload, caching
• @testing-library/svelte Setup - Alternative jsdom approach

Tertiary (LOW confidence)

• Slack webhook notification patterns from various blog posts - curl approach is safest

Metadata

Confidence breakdown:

• Standard stack: HIGH - Official Svelte docs explicitly recommend Vitest + browser mode
• Architecture: HIGH - Multi-project pattern documented in Vitest and community guides
• Pitfalls: HIGH - Common issues well-documented in GitHub issues and guides
• E2E fixtures: MEDIUM - Drizzle-seed pattern documented but specific schema integration untested

Research date: 2026-02-03 Valid until: 2026-03-03 (Vitest browser mode evolving, re-verify before major updates)