Kioku architecture

This document describes the current internal structure of the develop branch. It records operational component boundaries, not the historical alternatives or reasoning that produced them.

System boundary

MCP client -- stdio or Streamable HTTP --> Kioku host
Kioku host --> MCP adapters --> application services --> infrastructure
infrastructure --> Obsidian vault
infrastructure --> Ollama
infrastructure --> optional Obsidian bridge

The optional bridge is consumed by the independently versioned sandovaldavid/kioku-obsidian plugin. The plugin source and release workflow are not part of this repository.

Dependency direction

Dependencies point inward from protocol adapters to application contracts. Workflow services own domain decisions. Filesystem, indexing, bridge, embeddings, and process-hosting effects remain in infrastructure or hosting components.

The repository currently keeps these boundaries inside the single Kioku.Mcp.Server assembly and enforces them with architecture tests.

Storage and indexing

Markdown files and YAML frontmatter in the configured Obsidian vault are the durable source of truth.

VaultIndexService builds derived in-memory indexes for words, tags, links, document lengths, and graph operations. A bounded indexing pipeline processes startup and file-change work. Tools explicitly synchronize mutations with the index, while a file watcher covers external changes.

Embeddings are derived data cached at {vault}/.kioku/embeddings.bin. The cache can be rebuilt and is not a database or system of record.

See indexing-pipeline.md, vault-config.md, and threat-and-privacy-model.md.

The coordination slice persists immutable event files and rebuildable work-item projections under .kioku/coordination/. CoordinationEventStore validates schema versions, hashes, sequence numbers, idempotency, and state transitions before atomically writing an event. CoordinationClaimStore adds resource locks, lease projections, server-time expiry, and monotonic fencing. CoordinationConflictStore persists safe conflict records without rewriting event history. CoordinationService is the application boundary used by the gated MCP tools and resources. All four services use the pure CoordinationProjectionReducer and keep domain rules below the protocol layer. VaultMutationService adds canonical resource locking, optional revision/hash checks, claim fencing, atomic text mutations, and retry idempotency for writes. The architecture and supported-filesystem boundary are documented in durable-coordination.md.

Retrieval

• Keyword retrieval uses indexed full-text scoring.
• Semantic retrieval uses Ollama embeddings when available.
• Hybrid retrieval combines keyword and semantic ranked results.
• Keyword search remains available when Ollama is unavailable.
• Optional generation requires KIOKU_GEN_MODEL.

Exact public tool schemas and modes are defined in commands-reference.md. Retrieval evaluation methodology lives in retrieval-eval.md.

Application slices

Work sessions

SessionContextTools depends on IWorkSessionService. WorkSessionService owns session lifecycle and delegates filesystem operations to IWorkSessionFileSystem / WorkSessionFileSystem.

Architecture and integration tests enforce adapter shape, dependency injection, cancellation propagation, session ownership, collision-safe creation, and the absence of direct filesystem calls in the workflow service.

See work-sessions.md.

Project documents and engineering specs

EngineeringWorkflowTools and the focused engineering creation tools depend on the single application boundary IProjectDocumentService; MCP adapters do not construct engineering workflow services directly.

ProjectDocumentService owns the project-document facade: ADRs, bug logs, ordinary implementation plans, knowledge, backlog, project discovery/context, and engineering-template management. First-class specs and spec-linked plans are delegated through the same facade to EngineeringSpecService, which owns:

• the draft / approved / superseded / discarded spec lifecycle;
• canonical spec filename creation and exact-basename resolution;
• same-project SPEC → PLAN validation;
• rejection of malformed, traversal, wrong-project, superseded, or discarded references;
• final durable revision calculation after optional Templater evaluation;
• idempotent retry behavior that does not replay an already-applied Templater side effect.

Both paths use the existing project workspace and vault mutation boundaries. specs/ is a core/eager project folder; daily/ and tickets/ remain recognized optional/lazy workflow folders.

The generic create_project_doc surface remains a Deprecated compatibility wrapper and does not model first-class specs. New integrations should use the focused tools listed in focused-tool-migration.md and the workflow semantics in engineering-workflows.md.

Note queries

NoteQueryTools depends on INoteQueryService. NoteQueryService owns query outcomes and delegates response rendering to NoteResultPresenter.

The service reads current note content when required and otherwise works from the vault index, embeddings, and hybrid-search services. Architecture tests prevent the adapter from owning workflow or presentation logic.

Capability profiles

Core query, command, and utility tools are always registered. The default profile enables tasks, organization, sessions, workflows, graph, and engineering. The optional groups research, generation, css, assets, bridge, plugin, and coordination are disabled by default. Coordination tools and resources are absent until the vault explicitly enables coordination.

Vault-level capability configuration controls registration at startup. Exact profile counts and schemas are generated in commands-reference.md; configuration semantics are documented in vault-config.md.

Transports and hosting

stdio is the default transport for a client-spawned local process. Streamable HTTP is selected with KIOKU_TRANSPORT=http and adds listener validation, origin checks, bearer authentication, request limits, readiness, and trusted-proxy handling.

Streamable HTTP is configured explicitly as stateless at the MCP transport layer. Durable work sessions, coordination state, vault files, indexes, and the optional Obsidian bridge are application/runtime concerns rather than MCP HTTP-session storage.

The host validates configuration before starting runtime services. Cold vault reconciliation runs in the background; warm-up-safe status/project-context operations can respond while index-dependent operations wait on the explicit cold-index readiness gate. HTTP deployment guidance lives in deploy/auth-options.md.

Contract enforcement

The test suite covers:

• MCP tool names, schemas, annotations, prompts, and resources;
• typed result and protocol-error behavior;
• application/infrastructure dependency boundaries;
• first-class spec lifecycle, canonical basename resolution, SPEC → PLAN linking, revision, and idempotency behavior;
• project core/eager versus optional/lazy folder semantics;
• filesystem sandbox and permanent-delete policy;
• concurrent work-session ownership;
• frontmatter preservation;
• indexing synchronization, readiness, and recovery;
• canonical wikilink resolution and resolver-aware rewrite safety;
• bridge protocol fixtures;
• coordination event, claim, conflict, MCP-surface, and capability-gating tests;
• HTTP authentication, origins, limits, and readiness.

Generated contracts are verified by node scripts/generate-public-docs.mjs --check. See ci-quality-gates.md for the complete versioned gate.