docs: complete project research
Files: - STACK.md - SvelteKit + SQLite + TypeScript stack recommendation - FEATURES.md - Feature landscape with MVP definition - ARCHITECTURE.md - Modular monolith architecture with repository pattern - PITFALLS.md - Critical pitfalls and prevention strategies - SUMMARY.md - Executive synthesis with roadmap implications Key findings: - Stack: SvelteKit 2.50.x + Svelte 5.49.x with SQLite and better-sqlite3 for single-user simplicity - Architecture: Modular monolith with content-addressable image storage, FTS5 for search - Critical pitfall: Store images on filesystem (not DB) from Phase 1 to avoid painful migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Thomas Richter
515
.planning/research/ARCHITECTURE.md
Normal file
515
.planning/research/ARCHITECTURE.md
Normal file
@@ -0,0 +1,515 @@
|
||||
# Architecture Research
|
||||
|
||||
**Domain:** Personal task/notes web application with image attachments
|
||||
**Researched:** 2026-01-29
|
||||
**Confidence:** HIGH
|
||||
|
||||
## Standard Architecture
|
||||
|
||||
### System Overview
|
||||
|
||||
```
|
||||
+------------------------------------------------------------------+
|
||||
| CLIENT LAYER |
|
||||
| +-------------------+ +-------------------+ +----------------+ |
|
||||
| | Desktop Browser | | Mobile Browser | | PWA (future) | |
|
||||
| +--------+----------+ +--------+----------+ +-------+--------+ |
|
||||
| | | | |
|
||||
+-----------+----------------------+---------------------+----------+
|
||||
| | |
|
||||
v v v
|
||||
+------------------------------------------------------------------+
|
||||
| PRESENTATION LAYER |
|
||||
| +------------------------------------------------------------+ |
|
||||
| | Web Frontend (SPA) | |
|
||||
| | +--------+ +--------+ +--------+ +--------+ +--------+ | |
|
||||
| | | Notes | | Tasks | | Search | | Tags | | Upload | | |
|
||||
| | | View | | View | | View | | View | | View | | |
|
||||
| | +--------+ +--------+ +--------+ +--------+ +--------+ | |
|
||||
| +------------------------------+-----------------------------+ |
|
||||
+---------------------------------|--------------------------------+
|
||||
| HTTP/REST
|
||||
v
|
||||
+------------------------------------------------------------------+
|
||||
| APPLICATION LAYER |
|
||||
| +------------------------------------------------------------+ |
|
||||
| | REST API (Monolith) | |
|
||||
| | +------------+ +------------+ +------------+ | |
|
||||
| | | Notes API | | Tasks API | | Search API | | |
|
||||
| | +------------+ +------------+ +------------+ | |
|
||||
| | +------------+ +------------+ +------------+ | |
|
||||
| | | Tags API | | Upload API | | Auth API | | |
|
||||
| | +------------+ +------------+ +------------+ | |
|
||||
| +------------------------------+-----------------------------+ |
|
||||
+---------------------------------|--------------------------------+
|
||||
|
|
||||
+---------------------+---------------------+
|
||||
| | |
|
||||
v v v
|
||||
+------------------------------------------------------------------+
|
||||
| DATA LAYER |
|
||||
| +----------------+ +----------------+ +------------------+ |
|
||||
| | SQLite | | File Storage | | FTS5 Index | |
|
||||
| | (primary) | | (images) | | (full-text) | |
|
||||
| +----------------+ +----------------+ +------------------+ |
|
||||
+------------------------------------------------------------------+
|
||||
```
|
||||
|
||||
### Component Responsibilities
|
||||
|
||||
| Component | Responsibility | Typical Implementation |
|
||||
|-----------|----------------|------------------------|
|
||||
| Web Frontend | UI rendering, user interaction, client-side state | React/Vue/Svelte SPA |
|
||||
| REST API | Business logic, validation, orchestration | Node.js/Go/Python monolith |
|
||||
| Notes API | CRUD operations for thoughts/notes | API route handler |
|
||||
| Tasks API | CRUD for tasks, status transitions | API route handler |
|
||||
| Search API | Full-text search across notes/tasks | Wraps FTS5 queries |
|
||||
| Tags API | Tag management, note-tag associations | API route handler |
|
||||
| Upload API | Image upload, validation, storage | Handles multipart forms |
|
||||
| Auth API | Session management (single user) | Simple token/session |
|
||||
| SQLite | Primary data persistence | Single file database |
|
||||
| File Storage | Binary file storage (images) | Docker volume mount |
|
||||
| FTS5 Index | Full-text search capabilities | SQLite virtual table |
|
||||
|
||||
## Recommended Project Structure
|
||||
|
||||
```
|
||||
project/
|
||||
+-- docker/
|
||||
| +-- Dockerfile # Multi-stage build for frontend + backend
|
||||
| +-- docker-compose.yml # Service orchestration
|
||||
| +-- nginx.conf # Reverse proxy config (optional)
|
||||
+-- backend/
|
||||
| +-- cmd/
|
||||
| | +-- server/
|
||||
| | +-- main.go # Entry point
|
||||
| +-- internal/
|
||||
| | +-- api/ # HTTP handlers
|
||||
| | | +-- notes.go
|
||||
| | | +-- tasks.go
|
||||
| | | +-- tags.go
|
||||
| | | +-- search.go
|
||||
| | | +-- upload.go
|
||||
| | +-- models/ # Domain entities
|
||||
| | | +-- note.go
|
||||
| | | +-- task.go
|
||||
| | | +-- tag.go
|
||||
| | | +-- attachment.go
|
||||
| | +-- repository/ # Data access
|
||||
| | | +-- sqlite.go
|
||||
| | | +-- notes_repo.go
|
||||
| | | +-- tasks_repo.go
|
||||
| | +-- service/ # Business logic
|
||||
| | | +-- notes_svc.go
|
||||
| | | +-- search_svc.go
|
||||
| | +-- storage/ # File storage abstraction
|
||||
| | +-- local.go
|
||||
| +-- migrations/ # Database migrations
|
||||
| +-- go.mod
|
||||
+-- frontend/
|
||||
| +-- src/
|
||||
| | +-- components/ # Reusable UI components
|
||||
| | +-- pages/ # Route-level views
|
||||
| | +-- stores/ # Client state management
|
||||
| | +-- api/ # Backend API client
|
||||
| | +-- utils/ # Helpers
|
||||
| +-- public/
|
||||
| +-- package.json
|
||||
+-- data/ # Mounted volume (gitignored)
|
||||
| +-- app.db # SQLite database
|
||||
| +-- uploads/ # Image storage
|
||||
+-- .planning/ # Project planning docs
|
||||
```
|
||||
|
||||
### Structure Rationale
|
||||
|
||||
- **Monorepo with backend/frontend split:** Keeps deployment simple (single container possible) while maintaining clear separation
|
||||
- **internal/ in Go:** Prevents external packages from importing internals; enforces encapsulation
|
||||
- **Repository pattern:** Abstracts SQLite access, enables future database swap if needed
|
||||
- **Service layer:** Business logic separated from HTTP handlers for testability
|
||||
- **data/ volume:** Single mount point for all persistent data (database + files)
|
||||
|
||||
## Architectural Patterns
|
||||
|
||||
### Pattern 1: Modular Monolith
|
||||
|
||||
**What:** Single deployable unit with clear internal module boundaries. Each domain (notes, tasks, tags, search) has its own package but shares the same database and process.
|
||||
|
||||
**When to use:** Single-user or small-team applications where operational simplicity matters more than independent scaling.
|
||||
|
||||
**Trade-offs:**
|
||||
- Pro: Simple deployment, easy debugging, no network overhead between modules
|
||||
- Pro: Single database transaction across domains when needed
|
||||
- Con: All modules must use same language/runtime
|
||||
- Con: Cannot scale modules independently (not needed for single user)
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
// internal/api/routes.go
|
||||
func SetupRoutes(r *mux.Router, services *Services) {
|
||||
// Each domain gets its own route group
|
||||
notes := r.PathPrefix("/api/notes").Subrouter()
|
||||
notes.HandleFunc("", services.Notes.List).Methods("GET")
|
||||
notes.HandleFunc("", services.Notes.Create).Methods("POST")
|
||||
|
||||
tasks := r.PathPrefix("/api/tasks").Subrouter()
|
||||
tasks.HandleFunc("", services.Tasks.List).Methods("GET")
|
||||
// Clear boundaries, but same process
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Repository Pattern for Data Access
|
||||
|
||||
**What:** Abstract data access behind interfaces. Repositories handle all database queries; services call repositories, not raw SQL.
|
||||
|
||||
**When to use:** Always for anything beyond trivial apps. Enables testing with mocks and future database changes.
|
||||
|
||||
**Trade-offs:**
|
||||
- Pro: Testable services (mock repositories)
|
||||
- Pro: Database-agnostic business logic
|
||||
- Pro: Query logic centralized
|
||||
- Con: Additional abstraction layer
|
||||
- Con: Can become overly complex if over-engineered
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
// internal/repository/notes_repo.go
|
||||
type NotesRepository interface {
|
||||
Create(ctx context.Context, note *models.Note) error
|
||||
GetByID(ctx context.Context, id string) (*models.Note, error)
|
||||
List(ctx context.Context, opts ListOptions) ([]*models.Note, error)
|
||||
Search(ctx context.Context, query string) ([]*models.Note, error)
|
||||
}
|
||||
|
||||
type sqliteNotesRepo struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func (r *sqliteNotesRepo) Search(ctx context.Context, query string) ([]*models.Note, error) {
|
||||
// FTS5 search query
|
||||
rows, err := r.db.QueryContext(ctx, `
|
||||
SELECT n.id, n.title, n.body, n.created_at
|
||||
FROM notes n
|
||||
JOIN notes_fts ON notes_fts.rowid = n.id
|
||||
WHERE notes_fts MATCH ?
|
||||
ORDER BY rank
|
||||
`, query)
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 3: Content-Addressable Image Storage
|
||||
|
||||
**What:** Store images using content hash (MD5/SHA256) as filename. Prevents duplicates and enables cache-forever headers.
|
||||
|
||||
**When to use:** Any app storing user-uploaded images where deduplication and caching matter.
|
||||
|
||||
**Trade-offs:**
|
||||
- Pro: Automatic deduplication
|
||||
- Pro: Cache-forever possible (hash changes if content changes)
|
||||
- Pro: Simple to verify integrity
|
||||
- Con: Need reference counting for deletion
|
||||
- Con: Slightly more complex upload logic
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
// internal/storage/local.go
|
||||
func (s *LocalStorage) Store(ctx context.Context, file io.Reader) (string, error) {
|
||||
// Hash while copying to temp file
|
||||
hasher := sha256.New()
|
||||
tmp, _ := os.CreateTemp(s.uploadDir, "upload-*")
|
||||
defer tmp.Close()
|
||||
|
||||
_, err := io.Copy(io.MultiWriter(tmp, hasher), file)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
hash := hex.EncodeToString(hasher.Sum(nil))
|
||||
finalPath := filepath.Join(s.uploadDir, hash[:2], hash)
|
||||
|
||||
// Move to final location (subdirs by first 2 chars prevent too many files in one dir)
|
||||
os.MkdirAll(filepath.Dir(finalPath), 0755)
|
||||
os.Rename(tmp.Name(), finalPath)
|
||||
|
||||
return hash, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Request Flow
|
||||
|
||||
```
|
||||
[User Action: Create Note]
|
||||
|
|
||||
v
|
||||
[Frontend Component] --HTTP POST /api/notes--> [Notes Handler]
|
||||
| |
|
||||
| (optimistic UI update) v
|
||||
| [Notes Service]
|
||||
| |
|
||||
| v
|
||||
| [Notes Repository]
|
||||
| |
|
||||
| v
|
||||
| [SQLite INSERT]
|
||||
| |
|
||||
| v
|
||||
| [FTS5 trigger auto-updates index]
|
||||
| |
|
||||
v v
|
||||
[UI shows new note] <--JSON response-- [Return created note]
|
||||
```
|
||||
|
||||
### Image Upload Flow
|
||||
|
||||
```
|
||||
[User: Attach Image]
|
||||
|
|
||||
v
|
||||
[Frontend: file input] --multipart POST /api/upload--> [Upload Handler]
|
||||
| |
|
||||
| (show progress) v
|
||||
| [Validate: type, size]
|
||||
| |
|
||||
| v
|
||||
| [Hash content]
|
||||
| |
|
||||
| v
|
||||
| [Store to /data/uploads/{hash}]
|
||||
| |
|
||||
| v
|
||||
| [Create attachment record in DB]
|
||||
| |
|
||||
v v
|
||||
[Insert image into note] <--{attachment_id, url}-- [Return attachment metadata]
|
||||
```
|
||||
|
||||
### Search Flow
|
||||
|
||||
```
|
||||
[User: Type search query]
|
||||
|
|
||||
v
|
||||
[Frontend: debounced input] --GET /api/search?q=...--> [Search Handler]
|
||||
| |
|
||||
| (show loading) v
|
||||
| [Search Service]
|
||||
| |
|
||||
| v
|
||||
| [Query FTS5 virtual table]
|
||||
| |
|
||||
| v
|
||||
| [JOIN with notes/tasks tables]
|
||||
| |
|
||||
| v
|
||||
| [Apply ranking (bm25)]
|
||||
| |
|
||||
v v
|
||||
[Display ranked results] <--JSON array-- [Return ranked results with snippets]
|
||||
```
|
||||
|
||||
### Key Data Flows
|
||||
|
||||
1. **Note/Task CRUD:** Frontend -> API Handler -> Service -> Repository -> SQLite. FTS5 index auto-updates via triggers.
|
||||
2. **Image Upload:** Frontend -> Upload Handler -> File Storage (hash-based) -> DB record. Returns URL for embedding.
|
||||
3. **Full-Text Search:** Frontend -> Search Handler -> FTS5 Query -> Ranked results with snippets.
|
||||
4. **Tag Association:** Many-to-many through junction table. Tag changes trigger re-index if needed.
|
||||
|
||||
## Scaling Considerations
|
||||
|
||||
| Scale | Architecture Adjustments |
|
||||
|-------|--------------------------|
|
||||
| 1 user (target) | Single SQLite file, local file storage, single container. Current design is perfect. |
|
||||
| 2-10 users | Still works fine. SQLite handles concurrent reads well. May want WAL mode for better write concurrency. |
|
||||
| 10-100 users | Consider PostgreSQL for better write concurrency. Move files to S3-compatible storage (MinIO or Garage for self-hosted). |
|
||||
| 100+ users | Out of scope for personal app. Would need auth system, PostgreSQL, object storage, potentially message queue for uploads. |
|
||||
|
||||
### Scaling Priorities (For Future)
|
||||
|
||||
1. **First bottleneck:** SQLite write contention (if ever). Fix: WAL mode (simple) or PostgreSQL (more complex).
|
||||
2. **Second bottleneck:** File storage if hosting many large images. Fix: Object storage with content-addressing.
|
||||
|
||||
**Note:** For a single-user personal app, these scaling considerations are theoretical. SQLite with WAL mode can handle thousands of notes and tasks without issue.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Anti-Pattern 1: Storing Images in Database
|
||||
|
||||
**What people do:** Store image bytes directly in SQLite as BLOBs.
|
||||
|
||||
**Why it's wrong:**
|
||||
- Bloats database file significantly
|
||||
- Slows down backups (entire DB must be copied for any change)
|
||||
- Cannot leverage filesystem caching
|
||||
- Makes database migrations more complex
|
||||
|
||||
**Do this instead:** Store images on filesystem, store path/hash reference in database. Use content-addressable storage (hash as filename) for deduplication.
|
||||
|
||||
### Anti-Pattern 2: No Full-Text Search Index
|
||||
|
||||
**What people do:** Use `LIKE '%query%'` for search.
|
||||
|
||||
**Why it's wrong:**
|
||||
- Full table scan for every search
|
||||
- Cannot rank by relevance
|
||||
- No word stemming or tokenization
|
||||
- Gets unusably slow with a few thousand notes
|
||||
|
||||
**Do this instead:** Use SQLite FTS5 from the start. It's built-in, requires no external dependencies, and handles relevance ranking.
|
||||
|
||||
### Anti-Pattern 3: Microservices for Single User
|
||||
|
||||
**What people do:** Split notes, tasks, search, auth into separate services "for scalability."
|
||||
|
||||
**Why it's wrong:**
|
||||
- Massive operational overhead for no benefit
|
||||
- Network latency between services
|
||||
- Distributed transactions become complex
|
||||
- Debugging across services is painful
|
||||
- 2024-2025 industry trend: many teams consolidating microservices back to monoliths
|
||||
|
||||
**Do this instead:** Build a well-structured modular monolith. Clear internal boundaries, single deployment. Extract services later only if needed (you won't need to for a personal app).
|
||||
|
||||
### Anti-Pattern 4: Overengineering Auth for Single User
|
||||
|
||||
**What people do:** Implement full OAuth2/OIDC, JWT refresh tokens, role-based access control.
|
||||
|
||||
**Why it's wrong:**
|
||||
- Single user doesn't need roles
|
||||
- Complexity adds attack surface
|
||||
- More code to maintain
|
||||
- Personal app accessible only on your network
|
||||
|
||||
**Do this instead:** Simple session-based auth with a password. Consider basic HTTP auth behind a reverse proxy, or even IP-based allowlisting if only accessible from home network.
|
||||
|
||||
## Integration Points
|
||||
|
||||
### External Services
|
||||
|
||||
| Service | Integration Pattern | Notes |
|
||||
|---------|---------------------|-------|
|
||||
| None required | N/A | Self-hosted, no external dependencies by design |
|
||||
| Optional: Reverse Proxy | HTTP | Nginx/Traefik for HTTPS termination if exposed to internet |
|
||||
| Optional: Backup | File copy | Simple rsync/backup of data/ directory contains everything |
|
||||
|
||||
### Internal Boundaries
|
||||
|
||||
| Boundary | Communication | Notes |
|
||||
|----------|---------------|-------|
|
||||
| Frontend <-> Backend | REST/JSON over HTTP | OpenAPI spec recommended for documentation |
|
||||
| API Handlers <-> Services | Direct function calls | Same process, no serialization |
|
||||
| Services <-> Repositories | Interface calls | Enables mocking in tests |
|
||||
| Services <-> File Storage | Interface calls | Abstracts local vs future S3 |
|
||||
|
||||
## Database Schema Overview
|
||||
|
||||
```sql
|
||||
-- Core entities
|
||||
CREATE TABLE notes (
|
||||
id TEXT PRIMARY KEY,
|
||||
title TEXT,
|
||||
body TEXT NOT NULL,
|
||||
type TEXT CHECK(type IN ('note', 'task')) NOT NULL,
|
||||
status TEXT CHECK(status IN ('open', 'done', 'archived')), -- for tasks
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Full-text search (virtual table synced via triggers)
|
||||
CREATE VIRTUAL TABLE notes_fts USING fts5(
|
||||
title,
|
||||
body,
|
||||
content='notes',
|
||||
content_rowid='rowid'
|
||||
);
|
||||
|
||||
-- Tags (many-to-many)
|
||||
CREATE TABLE tags (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT UNIQUE NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE note_tags (
|
||||
note_id TEXT REFERENCES notes(id) ON DELETE CASCADE,
|
||||
tag_id TEXT REFERENCES tags(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (note_id, tag_id)
|
||||
);
|
||||
|
||||
-- Attachments (images)
|
||||
CREATE TABLE attachments (
|
||||
id TEXT PRIMARY KEY,
|
||||
note_id TEXT REFERENCES notes(id) ON DELETE CASCADE,
|
||||
hash TEXT NOT NULL, -- content hash, also filename
|
||||
filename TEXT, -- original filename
|
||||
mime_type TEXT NOT NULL,
|
||||
size_bytes INTEGER,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indexes
|
||||
CREATE INDEX idx_notes_type ON notes(type);
|
||||
CREATE INDEX idx_notes_created ON notes(created_at DESC);
|
||||
CREATE INDEX idx_attachments_note ON attachments(note_id);
|
||||
CREATE INDEX idx_attachments_hash ON attachments(hash);
|
||||
```
|
||||
|
||||
## Build Order Implications
|
||||
|
||||
Based on component dependencies, suggested implementation order:
|
||||
|
||||
1. **Phase 1: Data Foundation**
|
||||
- SQLite setup with migrations
|
||||
- Basic schema (notes table)
|
||||
- Repository layer for notes
|
||||
- No FTS5 yet (add later)
|
||||
|
||||
2. **Phase 2: Core API**
|
||||
- REST API handlers for notes CRUD
|
||||
- Service layer
|
||||
- Basic frontend with note list/create/edit
|
||||
|
||||
3. **Phase 3: Tasks Differentiation**
|
||||
- Add type column (note vs task)
|
||||
- Task-specific status handling
|
||||
- Frontend task views
|
||||
|
||||
4. **Phase 4: Tags**
|
||||
- Tags table and junction table
|
||||
- Tag CRUD API
|
||||
- Tag filtering in frontend
|
||||
|
||||
5. **Phase 5: Image Attachments**
|
||||
- File storage abstraction
|
||||
- Upload API with validation
|
||||
- Attachment records in DB
|
||||
- Frontend image upload/display
|
||||
|
||||
6. **Phase 6: Search**
|
||||
- FTS5 virtual table and triggers
|
||||
- Search API with ranking
|
||||
- Search UI with highlighting
|
||||
|
||||
7. **Phase 7: Containerization**
|
||||
- Dockerfile (multi-stage)
|
||||
- docker-compose.yml
|
||||
- Volume mounts for data persistence
|
||||
|
||||
**Rationale:** This order ensures each phase builds on working foundation. Notes must work before tasks (which are just notes with extra fields). Tags and attachments can be added independently. Search comes later as it indexes existing content. Containerization last so development is simpler.
|
||||
|
||||
## Sources
|
||||
|
||||
- [Standard Notes Self-Hosting Architecture](https://standardnotes.com/help/self-hosting/getting-started) - API Gateway, Syncing Server patterns
|
||||
- [Flatnotes - Database-less Architecture](https://github.com/dullage/flatnotes) - Simple markdown file storage approach
|
||||
- [Evernote Data Structure](https://dev.evernote.com/doc/articles/data_structure.php) - Note/Resource/Attachment model
|
||||
- [Task Manager Database Schema](https://www.tutorials24x7.com/mysql/guide-to-design-database-for-task-manager-in-mysql) - Tags and task relationships
|
||||
- [SQLite FTS5 Extension](https://www.sqlite.org/fts5.html) - Full-text search implementation (HIGH confidence - official docs)
|
||||
- [Microservices vs Monoliths in 2026](https://www.javacodegeeks.com/2025/12/microservices-vs-monoliths-in-2026-when-each-architecture-wins.html) - Modular monolith recommendation
|
||||
- [MinIO Alternatives](https://rmoff.net/2026/01/14/alternatives-to-minio-for-single-node-local-s3/) - Self-hosted storage options
|
||||
- [Image Upload Architecture](https://medium.com/@jgefroh/software-architecture-image-uploading-67997101a034) - Content-addressable storage pattern
|
||||
- [Modern Web Application Architecture 2026](https://tech-stack.com/blog/modern-application-development/) - Container and deployment patterns
|
||||
|
||||
---
|
||||
*Architecture research for: Personal task/notes web application*
|
||||
*Researched: 2026-01-29*
|
||||
Reference in New Issue
Block a user