taskplaner/.planning/phases/03-images/03-RESEARCH.md

# Phase 3: Images - Research

**Researched:** 2026-01-29
**Domain:** Image attachments with file upload, camera capture, storage, and display
**Confidence:** HIGH

## Summary

This phase implements image attachments for entries with support for file upload (desktop), camera capture (mobile), thumbnail generation, and lightbox viewing. The research covers the complete image lifecycle: capture/upload on the client, transfer to server via form actions, storage on filesystem with thumbnail generation, and retrieval via API endpoints.

The established approach for SvelteKit image handling is:
1. **Client:** HTML5 File API for uploads, MediaDevices API for camera capture, canvas.toBlob() for photo capture
2. **Server:** Form actions with multipart/form-data, write files via Node.js fs/promises
3. **Processing:** Sharp for thumbnail generation (dramatically faster than alternatives)
4. **Storage:** Filesystem-based with API endpoints serving images (not static folder)
5. **Display:** svelte-lightbox for fullscreen viewing, native img tags for thumbnails

**Primary recommendation:** Use Sharp for server-side thumbnail generation, native browser APIs for camera/upload, and svelte-lightbox for the lightbox component. Store images outside static/ and serve via +server.ts endpoints for better control.

## Standard Stack

The established libraries/tools for this domain:

### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| sharp | latest | Server-side image processing, thumbnail generation | 4-5x faster than ImageMagick, 20x faster than Jimp. Native C++ bindings via libvips |
| svelte-lightbox | 1.1.x | Fullscreen image lightbox | Purpose-built for Svelte/SvelteKit, keyboard navigation, mobile support |

### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| nanoid | 5.1.x | Generate unique filenames | Already in project, use for image IDs |
| Node.js fs/promises | built-in | File system operations | Write uploaded images to disk |

### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| sharp | jimp | Pure JS (no native deps), but 20x slower - not recommended |
| svelte-lightbox | bigger-picture | Smaller bundle, but needs Svelte 4 compatibility config for Svelte 5 |
| Filesystem storage | SQLite blob | Simpler queries, but larger DB, slower, harder to backup |

**Installation:**
```bash
npm install sharp svelte-lightbox
```

## Architecture Patterns

### Recommended Project Structure
```
data/
  uploads/
    originals/         # Full-size images (user uploaded)
    thumbnails/        # Generated thumbnails (150x150)
src/
  lib/
    server/
      images/
        storage.ts     # File system operations
        thumbnails.ts  # Sharp thumbnail generation
    components/
      ImageUpload.svelte      # Upload button + drag-drop zone
      CameraCapture.svelte    # Camera modal with preview
      ImageGallery.svelte     # Horizontal scroll gallery
      ImageLightbox.svelte    # Fullscreen viewer wrapper
  routes/
    api/
      images/
        [id]/
          +server.ts          # Serve original image
          thumbnail/
            +server.ts        # Serve thumbnail
```

### Pattern 1: Form Action File Upload
**What:** Handle multipart form data in SvelteKit form actions
**When to use:** File uploads from forms
**Example:**
```typescript
// +page.server.ts
import { writeFile } from 'node:fs/promises';
import { nanoid } from 'nanoid';
import sharp from 'sharp';

export const actions = {
  uploadImage: async ({ request }) => {
    const formData = await request.formData();
    const file = formData.get('image') as File;
    const entryId = formData.get('entryId') as string;

    if (!file || file.size === 0) {
      return fail(400, { error: 'No file uploaded' });
    }

    const imageId = nanoid();
    const ext = file.name.split('.').pop() || 'jpg';
    const buffer = Buffer.from(await file.arrayBuffer());

    // Save original
    await writeFile(`data/uploads/originals/${imageId}.${ext}`, buffer);

    // Generate thumbnail with EXIF auto-rotation
    await sharp(buffer)
      .rotate()  // Auto-orient based on EXIF
      .resize(150, 150, { fit: 'cover', position: 'center' })
      .jpeg({ quality: 80 })
      .toFile(`data/uploads/thumbnails/${imageId}.jpg`);

    // Save to database
    // imageRepository.create({ id: imageId, entryId, ext });

    return { success: true, imageId };
  }
};
```

### Pattern 2: Camera Capture with getUserMedia
**What:** Access device camera directly in browser
**When to use:** Mobile photo capture, desktop webcam
**Example:**
```typescript
// CameraCapture.svelte
<script lang="ts">
  let videoElement: HTMLVideoElement;
  let canvasElement: HTMLCanvasElement;
  let stream: MediaStream | null = null;
  let capturedBlob: Blob | null = null;

  async function startCamera(facingMode: 'user' | 'environment' = 'environment') {
    // Stop existing stream first
    if (stream) {
      stream.getTracks().forEach(track => track.stop());
    }

    stream = await navigator.mediaDevices.getUserMedia({
      video: { facingMode },
      audio: false
    });
    videoElement.srcObject = stream;
    await videoElement.play();
  }

  async function capturePhoto(): Promise<Blob> {
    const ctx = canvasElement.getContext('2d')!;
    canvasElement.width = videoElement.videoWidth;
    canvasElement.height = videoElement.videoHeight;
    ctx.drawImage(videoElement, 0, 0);

    return new Promise(resolve => {
      canvasElement.toBlob(blob => resolve(blob!), 'image/jpeg', 0.9);
    });
  }

  async function handleCapture() {
    capturedBlob = await capturePhoto();
    // Show preview, allow retake or confirm
  }

  async function confirmAndUpload() {
    const formData = new FormData();
    formData.append('image', capturedBlob!, 'camera-capture.jpg');
    formData.append('entryId', entryId);

    await fetch('?/uploadImage', {
      method: 'POST',
      body: formData
    });
  }

  function stopCamera() {
    stream?.getTracks().forEach(track => track.stop());
    stream = null;
  }
</script>
```

### Pattern 3: API Endpoint for Serving Images
**What:** Serve uploaded images via +server.ts endpoints
**When to use:** Serving user-uploaded files not in static/
**Example:**
```typescript
// src/routes/api/images/[id]/+server.ts
import { readFile } from 'node:fs/promises';
import { error } from '@sveltejs/kit';

export async function GET({ params }) {
  const { id } = params;

  // Look up image metadata from database
  const image = imageRepository.getById(id);
  if (!image) {
    throw error(404, 'Image not found');
  }

  const filePath = `data/uploads/originals/${id}.${image.ext}`;

  try {
    const data = await readFile(filePath);
    return new Response(data, {
      headers: {
        'Content-Type': `image/${image.ext === 'jpg' ? 'jpeg' : image.ext}`,
        'Content-Length': data.length.toString(),
        'Cache-Control': 'public, max-age=31536000, immutable'
      }
    });
  } catch {
    throw error(404, 'Image file not found');
  }
}
```

### Pattern 4: Drag and Drop Zone
**What:** HTML5 drag-and-drop file upload
**When to use:** Desktop file upload experience
**Example:**
```typescript
// ImageUpload.svelte
<script lang="ts">
  let isDragging = $state(false);
  let fileInput: HTMLInputElement;

  function handleDragOver(e: DragEvent) {
    e.preventDefault();
    isDragging = true;
  }

  function handleDragLeave(e: DragEvent) {
    e.preventDefault();
    isDragging = false;
  }

  function handleDrop(e: DragEvent) {
    e.preventDefault();
    isDragging = false;

    const file = e.dataTransfer?.files[0];
    if (file && file.type.startsWith('image/')) {
      uploadFile(file);
    }
  }

  function handleFileSelect(e: Event) {
    const input = e.target as HTMLInputElement;
    const file = input.files?.[0];
    if (file) {
      uploadFile(file);
    }
  }

  async function uploadFile(file: File) {
    // Show optimistic preview immediately
    const previewUrl = URL.createObjectURL(file);
    // dispatch('preview', { url: previewUrl });

    // Upload in background
    const formData = new FormData();
    formData.append('image', file);
    formData.append('entryId', entryId);

    const response = await fetch('?/uploadImage', {
      method: 'POST',
      body: formData
    });

    // Handle response, revoke preview URL
    URL.revokeObjectURL(previewUrl);
  }
</script>

<div
  class="drop-zone {isDragging ? 'dragging' : ''}"
  ondragover={handleDragOver}
  ondragleave={handleDragLeave}
  ondrop={handleDrop}
  role="button"
  tabindex="0"
  onclick={() => fileInput.click()}
  onkeydown={(e) => e.key === 'Enter' && fileInput.click()}
>
  <input
    bind:this={fileInput}
    type="file"
    accept="image/*"
    onchange={handleFileSelect}
    class="hidden"
  />
  Drop image here or click to upload
</div>
```

### Anti-Patterns to Avoid
- **Storing images in SQLite:** Bloats database, slows queries, complicates backups
- **Storing in static/ folder:** Files overwritten on deploy, not recommended for user uploads
- **Blocking UI during upload:** Always show optimistic preview, upload in background
- **Missing EXIF rotation:** Images from phones will appear rotated; always use sharp.rotate()
- **Using jimp for thumbnails:** 20x slower than Sharp, poor choice for any volume

## Don't Hand-Roll

Problems that look simple but have existing solutions:

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Thumbnail generation | Canvas resize on client | Sharp on server | Server handles EXIF rotation, consistent quality, proper color management |
| Image EXIF orientation | Manual rotation logic | sharp.rotate() | Complex orientation matrix, mirroring cases, Sharp handles all 8 orientations |
| Lightbox gallery | Custom modal + gestures | svelte-lightbox | Keyboard navigation, pinch-zoom, swipe, accessibility baked in |
| Camera permissions | Custom prompts | getUserMedia native prompts | Browser handles permission UI, remembers choice, shows camera indicator |
| File type validation | Regex on extension | File API + accept attribute | MIME type checking, browser validates before upload |

**Key insight:** Image handling has many edge cases (EXIF, color profiles, format variations). Sharp and browser APIs handle these; custom solutions will miss them.

## Common Pitfalls

### Pitfall 1: Missing EXIF Auto-Rotation
**What goes wrong:** Photos from mobile devices appear rotated 90 or 180 degrees
**Why it happens:** Mobile cameras embed rotation in EXIF metadata rather than rotating pixels
**How to avoid:** Always call `sharp.rotate()` without arguments before any resize operation
**Warning signs:** Portrait photos appearing landscape, images "sideways"

### Pitfall 2: Blocking UI During Upload
**What goes wrong:** User clicks upload, UI freezes until complete
**Why it happens:** Not showing optimistic preview, waiting for server response
**How to avoid:** Use URL.createObjectURL() for instant preview, upload via background fetch
**Warning signs:** "Nothing happened" user feedback, perceived slowness

### Pitfall 3: Camera Not Available Handling
**What goes wrong:** App crashes or shows blank when camera unavailable
**Why it happens:** Not catching getUserMedia errors, not checking navigator.mediaDevices
**How to avoid:** Feature detection first, catch NotAllowedError and NotFoundError, show graceful fallback
**Warning signs:** White screen on desktop without webcam, permission denied with no feedback

### Pitfall 4: Memory Leaks from Object URLs
**What goes wrong:** Memory usage grows over time as user uploads images
**Why it happens:** Not calling URL.revokeObjectURL() after preview no longer needed
**How to avoid:** Revoke URLs when component unmounts or when upload completes
**Warning signs:** Browser tab memory increasing, eventual slowdown

### Pitfall 5: Form Without enctype
**What goes wrong:** File upload appears to work but server receives empty file
**Why it happens:** Missing `enctype="multipart/form-data"` on form element
**How to avoid:** Always set enctype for file upload forms, or use FormData with fetch directly
**Warning signs:** formData.get('file') returns null or 0-byte file

### Pitfall 6: Camera Stream Not Stopped
**What goes wrong:** Camera indicator stays on, battery drain, privacy concern
**Why it happens:** Not calling track.stop() when closing camera modal
**How to avoid:** Always stop all tracks when camera not in use: `stream.getTracks().forEach(t => t.stop())`
**Warning signs:** Browser camera indicator remains active after closing

## Code Examples

Verified patterns from official sources:

### Sharp Thumbnail Generation
```typescript
// Source: https://sharp.pixelplumbing.com/api-resize
import sharp from 'sharp';

async function generateThumbnail(
  inputBuffer: Buffer,
  outputPath: string,
  size = 150
): Promise<void> {
  await sharp(inputBuffer)
    .rotate()  // Auto-orient based on EXIF Orientation tag
    .resize(size, size, {
      fit: 'cover',           // Crop to fill dimensions
      position: 'center',     // Center the crop
      withoutEnlargement: true // Don't upscale small images
    })
    .jpeg({ quality: 80 })
    .toFile(outputPath);
}
```

### getUserMedia Camera Access
```typescript
// Source: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
async function startCamera(preferBack = true): Promise<MediaStream> {
  // Feature detection
  if (!navigator.mediaDevices?.getUserMedia) {
    throw new Error('Camera not supported in this browser');
  }

  try {
    return await navigator.mediaDevices.getUserMedia({
      video: {
        facingMode: preferBack ? 'environment' : 'user',
        width: { ideal: 1920 },
        height: { ideal: 1080 }
      },
      audio: false
    });
  } catch (err) {
    if (err instanceof DOMException) {
      if (err.name === 'NotAllowedError') {
        throw new Error('Camera permission denied');
      }
      if (err.name === 'NotFoundError') {
        throw new Error('No camera found');
      }
    }
    throw err;
  }
}
```

### Canvas to Blob for Photo Capture
```typescript
// Source: https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob
function canvasToBlob(
  canvas: HTMLCanvasElement,
  type = 'image/jpeg',
  quality = 0.9
): Promise<Blob> {
  return new Promise((resolve, reject) => {
    canvas.toBlob(
      (blob) => {
        if (blob) resolve(blob);
        else reject(new Error('Failed to create blob'));
      },
      type,
      quality
    );
  });
}
```

### SvelteKit Form Action File Upload
```typescript
// Source: https://travishorn.com/uploading-and-saving-files-with-sveltekit
import { writeFile, mkdir } from 'node:fs/promises';
import { existsSync } from 'node:fs';

export const actions = {
  upload: async ({ request }) => {
    const formData = await request.formData();
    const file = formData.get('file') as File;

    if (!file || file.size === 0) {
      return fail(400, { error: 'No file provided' });
    }

    // Ensure upload directory exists
    const uploadDir = 'data/uploads/originals';
    if (!existsSync(uploadDir)) {
      await mkdir(uploadDir, { recursive: true });
    }

    // Convert to buffer and write
    const buffer = Buffer.from(await file.arrayBuffer());
    const filename = `${crypto.randomUUID()}.${file.name.split('.').pop()}`;
    await writeFile(`${uploadDir}/${filename}`, buffer);

    return { success: true, filename };
  }
};
```

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| navigator.getUserMedia | navigator.mediaDevices.getUserMedia | 2017+ | Old API deprecated, use MediaDevices interface |
| readFileSync for serving | SvelteKit read() helper | SvelteKit 2.4 | Better streaming, works in serverless |
| Base64 data URLs | Blob + Object URLs | Always preferred | 33% smaller, faster parsing |
| jimp for Node.js images | Sharp | Ongoing | Sharp 20x faster, better format support |

**Deprecated/outdated:**
- `navigator.getUserMedia()`: Deprecated, use `navigator.mediaDevices.getUserMedia()`
- Canvas `toDataURL()` for uploads: Use `toBlob()` for better performance
- Storing files in static/: Build process overwrites; use data/ directory

## Open Questions

Things that couldn't be fully resolved:

1. **Thumbnail dimensions**
   - What we know: 150x150 is common for list thumbnails
   - What's unclear: Exact dimensions depend on UI design
   - Recommendation: Start with 150x150 cover crop, adjust based on visual testing

2. **Camera button visibility on desktop**
   - What we know: Can detect camera with enumerateDevices()
   - What's unclear: Whether to show camera button proactively or only after first use
   - Recommendation: Show camera button, handle no-camera gracefully with error message

3. **Upload progress indication**
   - What we know: fetch() doesn't support upload progress natively
   - What's unclear: Whether optimistic preview is sufficient UX
   - Recommendation: Show optimistic preview immediately; if progress needed, use XMLHttpRequest

## Sources

### Primary (HIGH confidence)
- [Sharp resize API](https://sharp.pixelplumbing.com/api-resize) - Thumbnail generation, fit modes
- [Sharp operations API](https://sharp.pixelplumbing.com/api-operation/) - EXIF auto-rotation
- [MDN getUserMedia](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia) - Camera access
- [MDN Taking still photos](https://developer.mozilla.org/en-US/docs/Web/API/Media_Capture_and_Streams_API/Taking_still_photos) - Canvas capture
- [MDN toBlob](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob) - Canvas to blob conversion
- [MDN File drag and drop](https://developer.mozilla.org/en-US/docs/Web/API/HTML_Drag_and_Drop_API/File_drag_and_drop) - Drag-drop API

### Secondary (MEDIUM confidence)
- [SvelteKit file upload tutorial](https://travishorn.com/uploading-and-saving-files-with-sveltekit) - Form actions pattern
- [svelte-lightbox docs](https://svelte-lightbox.js.org/docs/introduction) - Lightbox component usage
- [Sharp vs Jimp benchmark](https://www.peterbe.com/plog/sharp-vs-jimp) - Performance comparison

### Tertiary (LOW confidence)
- [SvelteKit serving uploads discussion](https://github.com/sveltejs/kit/discussions/10162) - Custom server pattern for uploads outside static/

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH - Sharp and browser APIs are well-documented, widely used
- Architecture: HIGH - SvelteKit patterns verified from official sources
- Pitfalls: HIGH - Based on MDN documentation and established best practices
- Lightbox choice: MEDIUM - svelte-lightbox is well-maintained but alternatives exist

**Research date:** 2026-01-29
**Valid until:** 2026-03-01 (Sharp and browser APIs are stable)