# dsh plugins — plan (docs / summarize / web / images)

Goal: give each family dsh instance practical "plugin-style" capabilities:
handle documents, summarize web pages & docs, ingest images, output images,
and edit images from text. Decide how to deliver them on our lightweight
FastAPI+SSE dsh.

## Approach: extend our dsh (not adopt the official harness)
Our dsh is intentionally a small FastAPI+SSE chat (OmniRoute `auto/best-chat`).
The official DeepSeek Harness + its ~11k-plugin ecosystem is heavier and
prerelease. We keep our app and add plugin-style capabilities natively, phased.

Capabilities map:
| # | Capability | Mechanism | Phase |
|---|------------|-----------|-------|
| 1 | Summarize a web page | dsh fetches URL (httpx) → text → LLM summary. Add `/api/tool/web` or a "summarize URL" box. | 1 |
| 2 | Summarize a document | read a file from `/workspace` (txt/md; PDF via pypdf) → LLM summary. Add "Summarize file" button + `/api/tool/docs`. | 1 |
| 3 | Handle / attach docs | pass referenced workspace file contents into the prompt (file picker, cap size). | 1–2 |
| 4 | Ingest images (understand) | need a **vision** model on OmniRoute (e.g. qwen-vl / gpt-4o). Send image as data-URI in OpenAI vision content format. `DSH_LLM_VISION_MODEL`. | 2 |
| 5 | Output images (generate) | OmniRoute is LLM-only → wire an image API (OpenRouter image, or internal ComfyUI/SD). `/api/tool/image`. | 3 |
| 6 | Edit image from text | image-editing model (instruction-based) on the image API. | 3 |

## Phase 1 — text capabilities (no new infra)
- [ ] Web summarize: `POST /api/tool/web {url}` → httpx fetch → strip HTML → LLM summary (stream). UI: a "Paste URL to summarize" box.
- [ ] Docs summarize: `POST /api/tool/docs {path}` (relative to /workspace) → read txt/md (pypdf for PDFs) → LLM summary.
- [ ] Chat context: pick a workspace file → prepend its contents to the message (cap ~8k tokens).

## Phase 2 — image ingest (vision)
- [ ] Find/configure a vision model on OmniRoute (qwen2.5-vl, gpt-4o, or similar). Set `DSH_LLM_VISION_MODEL`.
- [ ] Image upload in chat.html (`<input type=file accept=image/*>` + base64) → send OpenAI vision `content` array with `image_url` data-URI.
- [ ] Handle docs as images (scan/photo) → describe/OCR via the vision model.

## Phase 3 — image output & edit
- [ ] Wire an image-generation API (OpenRouter `gpt-image-1`/`flux`, or internal ComfyUI/SD webui).
- [ ] `/api/tool/image` — text→image; stream/provide a URL or return a base64 image to display in chat.
- [ ] Image edit — instruction/ref edit endpoint (gpt-image edit or SD img2img) from an uploaded image + text.
- [ ] Save images into the user's S3 bucket (`image/…`) so they're kept.

## Open questions
- Vision + image models on OmniRoute: confirm availability/ids (check `/v1/models`).
- Kids' instances: should image output be gated (cost/appropriateness)? Probably gate image-gen to sam initially.
- Marker: keep everything streaming + iframe-friendly CSP.

## Sources
- docs/dsh-plugins.md — shortlist from github topic: WeKnora (docs→RAG), modlens (vision), open-design.