--- { "n": 4, "title": "web-interface - a Python web server to browse journal state", "abstract": "Serve to-dos, docs, pins, rules and work across environments in a browser instead of the CLI", "refs": [], "seen": [ "agent" ], "data": {}, "created": 1789136243.0, "updated": 1789136243.0, "deleted": 0.0, "completed": 1789745298.550726, "outcome": "", "type": "doc" } --- User's request, verbatim intent: "I got a nice idea, and I want to add a web interface to the journals so we can start a Python web server. Then we have a web interface to view all the journals' stuff, like to-dos, the docs, etc." This is a NEW feature idea for the journal package itself (this repo IS the package — journal.py, docs.py, pins.py, todo.py, work.py, tools.py etc. at the project root; the project's own .journal/ is its dogfood instance, git-ignored per [[instance-untracked]]). No environment fit it (reminders, auto-no-questions, flags, leaks, listings, peer-findings are unrelated), so a new one, "web-interface", was prepared for it. What "the journals' stuff" means, concretely, per environment: - to-dos .journal/environments//todo/*.json (todo.py) - pins .journal/environments//pins.json (pins.py) - open work .journal/environments//work.json (work.py) - reminders .journal/environments//reminders.json (reminders.py) - rules project-wide, .journal/rules/ (binds every env) - docs .journal/docs// (docs.py) — catalogue, parts, attachments, status (draft/final) - tools .journal/tools// (tools.py) - the transcript / conversation history (transcript.py) Known constraints: - No web framework or third-party dependency exists in this project today — it is pure-stdlib Python (no pyproject.toml, no requirements.txt, no package.json). Whether to stay stdlib (http.server) or add a dependency (Flask/FastAPI) is an open decision for the plan, not assumed. - [[no-tests-in-consumers]]: whatever ships must not run this package's own test suite (test_*.py) inside a consumer project on install/upgrade. A web server module is a new shipped surface — it needs to be excluded the same way tests are, if it is added to the package itself rather than kept as a project-only dev tool. - [[tests-bounded]]: any test added for this needs a time limit and any subprocess (e.g. the server process itself, if tests spin it up) a timeout. - [[speed-over-ceremony]]: this user prefers shipping the small thing now over ceremony — but this request was explicitly to PLAN and PREPARE, not implement yet ("start planning or preparing the environment with all the to-dos, etc."), so this doc + to-do breakdown is the right amount of ceremony this time, not more. - There is prior, existing read logic for every one of these data kinds already in journal.py / docs.py / pins.py / todo.py / work.py / reminders.py / tools.py / transcript.py — the web server's job is to expose that, not to re-implement parsing of the JSON files directly. The plan should say which existing functions/CLI surfaces it reuses. Open questions the plan should resolve (not the user's answer yet — flag as to-dos with `todos ask` if genuinely undecided after research): - stdlib http.server vs. a minimal dependency, and why - read-only viewer to start, or read+act (e.g. closing a to-do from the browser) - single project (cwd's .journal/) or does it need to discover/serve multiple projects - how "all environments" is presented — one page per environment, or one dashboard - where the server process lives: a `journal serve` subcommand? a script under .journal/tools/ (a tool, per the tools catalogue) since [[workflows-self-updates]]- style "never hand-edit .journal, only ship through the package" may apply similarly here — the plan should say which. ## Plan (Opus) ## Findings that shape the design - Every noun already has a structured read layer, separate from its terminal renderer: todo.open_items/ready/blocked/_all, todo.show/render; docs._load/get/_parts/attachments, docs.catalogue/show; pins.listing(key=)/render; reminders.live/listing; work._all/open_work; tools._all/get. The viewer must consume the structured half, never re-parse JSON, never reimplement paging. - Do NOT import journal.py from a server — it has import-time side effects (scans sys.argv for --env=/--as=, calls tracks.override, state.use_track, fmt.cli, prints worktree notes; main() runs migrate.ensure). A server imports the leaf modules directly (todo, docs, pins, work, reminders, tools, tracks, state, fmt, entries) and computes root() itself. - Environment resolution is process-global: state.get(root, key) resolves TRACKED keys through state._TRACK, set once by use_track. A viewer showing several environments must use state.tracked(root, key, env) (already exists) per request, not flip use_track. This is the single biggest structural item: pins.listing/reminders.listing/entries.rows/ entries.all_of are currently current-environment-only and need a track-aware path first. - fmt is terminal-shaped but not hostile: bold/dim only under fmt._tty(), so calling docs.show() from a server yields plain text (a legitimate 
 fallback for day one). But
  fmt.room() reads terminal width — pass an explicit width= from the server.
- Anything under .journal/tools/ never ships to consumers (install.py DATA excludes
  "tools"/"docs"/"environments"/".journal"; tests excluded separately by _is_test). So a new
  serve.py at the package root ships automatically and correctly; a script under
  .journal/tools/ would be dogfood-only, forever.
- Rules are pins with key=pins.RULES, project-scoped. Docs carry Path objects in their dicts
  — not JSON-safe, and a path-traversal risk if attachments are served without a safe_path
  check.

## Recommendation 1 — stdlib http.server, no dependency
ThreadingHTTPServer + BaseHTTPRequestHandler, bound to 127.0.0.1. Not Flask/FastAPI/wsgiref.
This project has no dependency mechanism at all (no pyproject.toml/requirements.txt;
install.sh/install.py install by copying .py files). Adding Flask invents a dependency story
every consumer pays for, for a viewer most will never start — the same argument install.py
already makes for excluding the test suites. The MVP is ~a dozen GET routes, no forms, no
sessions, no auth, one local user: exactly what http.server is for. Keep handler functions
framework-agnostic (handler(request) -> (status, content_type, bytes) over a route table) so
a future dependency swap throws away only the ~60-line socket shim. Bind localhost only,
refuse --host in the MVP (http.server is not production-hardened; acceptable because the MVP
is read-only).

## Recommendation 2 — a `journal serve` subcommand, not a tool
New serve.py (socket/routing) + views.py (data -> view models) at the package root,
dispatched by a `journal serve` verb. .journal/tools/ is excluded from install.py's DATA, so
a script there can never reach a consumer — the tools catalogue is for this project's own
repeated scripts, not a viewer for every journal. Adding a verb is cheap and follows
convention: one row in journal.py's COMMANDS (~line 2349) and one help.py GROUPS entry.
Tests go in test_serve.py, auto-excluded from consumers by install._is_test and swept by
`pull`'s stale-file removal.

## Recommendation 3 — read-only MVP; writes are a later phase
Every write in this package is attributed (at, source, track, transcript line) and gated
(gate_writes_on_start, todo.block/after, asks, rule "closing a to-do is always explicit").
A browser click has none of that context and would either lie about source or invent a new
concept every renderer has to learn. http.server has no CSRF/origin checking, so
state-changing GET/POST on localhost is a bad combination a read-only GET-only server avoids.
Writes also raise concurrency questions (state.locked, record.json.lock) reading does not.
State this as a deliberate MVP scope line; file the write-mode question as a to-do, not a
phase-1 decision.

## Phases
1. Track-aware read layer (views.py): thread an optional track through entries.all_of and
   the pins/reminders/work read paths via state.tracked; add public accessors
   (docs.all_docs, todo.all_items, tools.all_tools) instead of reaching into privates;
   views.py functions per noun (environments/todos/todo_detail/docs/doc_detail/pins/rules/
   work/reminders/overview); a safe_path helper for attachment links.
   Risks: widening entries.rows/pins.listing signatures touches every numbered noun — needs
   test_state.py/test_tracks.py/test_todo.py green; decide JSON-safe dicts now (recommended)
   vs. view-shaped with private Paths; decide whether docs pages filter by scope per
   environment or always show all with a scope badge.
2. Server core (serve.py): ThreadingHTTPServer + route table, 127.0.0.1 only, --port= with a
   fixed default and a "port in use" message, GET/HEAD only (405 otherwise), one page()/CSS
   helper, html.escape everywhere, quiet/redirected request logging, clean Ctrl-C.
   Risk: threading is only safe once phase 1 removes reliance on process-global state
   (use_track/tracks.override/fmt tty-width) — if not fully done, use plain HTTPServer
   instead for the MVP and assert the choice with a test. No caching — read fresh per
   request.
3. The pages: / (overview: environments + counts, current marker, project-wide rules/docs
   counts); /env// dashboard; /env//todos[/n] (states, blocked/after, ask/
   answer); /docs, /docs/n[.p] (status/scope/abstract, parts, attachments, cited-by);
   /docs/n/files/ (via safe_path, mimetypes Content-Type); /env//pins, /rules,
   /work, /reminders (with bodies); cross-links (pin --doc -> doc, to-do after -> deps, doc
   cited_by -> back).
   Risks: doc/to-do/pin bodies are Markdown with no dependency available — MVP wraps escaped
   text in 
 (honest, instantly correct); a real renderer is a later ~40-line addition,
   don't let it grow into a Markdown implementation. "All environments" presentation:
   recommend overview-plus-per-environment, NOT a merged firehose — the environment is the
   unit the whole system is built around, merging pins across environments would contradict
   what a pin means; a project-wide view is justified only for rules/docs/tools (which are
   project-scoped). Cheap shortcut if time is short: 
-wrap docs.show()/todo.show()
   output directly for detail pages — acceptable for one iteration, but leaks `journal …`
   command hints into the browser; flag it as a known wart.
4. CLI wiring: cmd_serve(port, open_browser) + a COMMANDS row in journal.py; --port=/--open
   flag rows (never a new branch, per the dispatch docstring); a "serve" help.py group or
   ALIAS; house-voice refusals (no .journal here, port busy, unknown environment in a URL);
   README line, CHANGELOG entry, VERSION bump.
   Risks/decisions: no serve_port in settings.DEFAULTS for the MVP (a flag covers it;
   settings.py's own docstring discourages settings nobody reads); foreground-only
   (Ctrl-C to stop), NOT a daemon — a daemon needs pidfiles and a `journal serve stop`,
   which is its own phase.
5. Bounded tests (test_serve.py): use testkit.py's fixture project; start the server
   in-process on port 0 in a thread (no subprocess, no timeout problem) where possible; any
   test that does spawn the CLI needs an explicit subprocess timeout= and teardown kill,
   within the suite's overall SUITE_TIMEOUT (.journal/tools/suite already enforces one).
   Cases: every route 200 on a seeded fixture; unknown environment -> 404 not 500;
   attachment path traversal (.., absolute paths) refused; a second environment's pins
   really come from that environment (the state.tracked regression case); no ANSI escapes in
   output.
   Risk: confirm testkit.py can build a fixture with two environments, docs with parts and
   an attachment — if not, that fixture work belongs in this phase.
6. Deliberately later, file as to-dos not phases: write actions (close a to-do, strike a
   pin, mark a reminder done) — needs the attribution/gating design above first; transcript
   browsing (transcript.py is session-shaped and much larger, deserves its own "what is a
   conversation page" decision); multi-project mode (root() is currently a single resolved
   path, the whole CLI assumes it); auto-refresh/SSE; search across everything
   (docs.search_lines already exists); a JSON API for other tools.

## Critical files
journal.py (dispatch table ~line 2349, root()/project(), import-time side effects);
state.py (tracked, use_track, TRACKED, env_dir); entries.py (all_of, rows, listing — the
shared read loop that needs a track); docs.py (_load, get, attachments, cited_by, show);
todo.py (_all, open_items, states_of, show); install.py (DATA, _is_test, _package_files —
what ships and what cannot).

## Architecture pivot: JSON API + Vue SPA, not server-rendered HTML
Supersedes the rendering half of 4.1's plan (phases 2-3 as originally written assumed
serve.py renders HTML server-side with 
-wrapped markdown). The user redirected
mid-build (2026-09-11) to a cleaner split, and this is what actually shipped:

- serve.py is now a pure JSON API: every /api/... route returns views.py's response as
  json.dumps(...), nothing else. The only non-JSON routes are the static SPA shell
  (/ and /app.js) and the binary doc-attachment download (/docs//files/).
- static/index.html + static/app.js: a Vue 3 (pinned: 3.4.21, loaded from cdnjs, no
  npm/build step) single-page app does ALL rendering, client-side. A hash router
  (static/app.js's ROUTES table, deliberately the same shape as serve.py's own route
  table) maps URL fragments to components; browsers never send the fragment to the
  server, so the server only ever needs to answer `/` once per page load.
- This still satisfies pin 1 (no Python dependency): Vue runs only in the browser: the
  Python process never imports it.
- Still stdlib-only, still read-only, still 127.0.0.1-only — nothing about those pins
  changed, only where the HTML gets built.

Two real bugs found and fixed while wiring this up, worth knowing for anyone adding a
new Vue route param:
1. A route param/prop must never be named `ref` — Vue reserves that name for its own
   template-ref mechanism and silently drops it even when bound dynamically via
   `v-bind`. The doc-ref param is named `docref` for this reason.
2. `useFetch`'s url-building callback must return a falsy value (not build a URL with an
   undefined prop in it) until every prop it needs is actually present — otherwise a
   fresh load of a deep-linked route (e.g. opening `#/docs/4.1` directly rather than
   navigating there) can race and hit `/api/.../undefined` once before recovering, or
   worse, never recover if the guard swallows the request without ever retrying.

Doc 4's phase 4 (CLI wiring) and phase 5 (tests) are otherwise unaffected: `journal
serve` still starts the same process, and test_serve.py can test the /api/ JSON
contract directly without needing a browser.