Add project documentation
This commit is contained in:
201
docs/developer-operations.md
Normal file
201
docs/developer-operations.md
Normal file
@@ -0,0 +1,201 @@
|
||||
# Developer and Operations Guide
|
||||
|
||||
## Local Stack
|
||||
|
||||
Agentic OS runs with Docker Compose.
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Default service ports:
|
||||
|
||||
| Service | Port | Purpose |
|
||||
|---|---:|---|
|
||||
| Frontend | `3000` | Operator UI |
|
||||
| Backend | `8000` | FastAPI API |
|
||||
| LiteLLM | `4000` | LLM gateway |
|
||||
| Postgres | `5432` | Application and LiteLLM databases |
|
||||
|
||||
Ollama runs on the macOS host, not inside Docker, so it can use Apple Silicon acceleration. Containers reach it through `host.docker.internal:11434`.
|
||||
|
||||
## Configuration
|
||||
|
||||
Use `.env.example` as the source of truth for configuration names. Copy it to `.env` and fill real values locally.
|
||||
|
||||
Never commit `.env`.
|
||||
|
||||
Important groups:
|
||||
|
||||
- **Security**: `SECRET_KEY`, `CREDENTIAL_ENCRYPTION_KEY`, first admin credentials.
|
||||
- **Database/cache**: `DATABASE_URL`, `REDIS_URL`.
|
||||
- **CORS**: `CORS_ALLOW_ORIGINS`.
|
||||
- **LLM routing**: local, Gemini, DeepSeek, OpenRouter, LiteLLM settings.
|
||||
- **MCP development**: enable/approval/push policy.
|
||||
- **Gitea**: clone/push base URL, username, token, MCP repos.
|
||||
- **Obsidian**: vault repo, task folder, auto-push.
|
||||
- **Project memory**: HTTP MCP URL, token, project IDs.
|
||||
- **Rules/skills/templates**: mounted paths for agent guidance and baselines.
|
||||
- **Device defaults**: per-device credentials and endpoint defaults.
|
||||
|
||||
## Database
|
||||
|
||||
Models live in `backend/app/models/`.
|
||||
|
||||
Alembic migrations live in `backend/alembic/`.
|
||||
|
||||
The backend initializes/upgrades the database on startup. Avoid ad-hoc schema changes in application startup code; prefer migrations.
|
||||
|
||||
## Tests
|
||||
|
||||
Run backend tests inside the container:
|
||||
|
||||
```bash
|
||||
docker compose exec -T backend pytest -q
|
||||
```
|
||||
|
||||
Run a focused subset:
|
||||
|
||||
```bash
|
||||
docker compose exec -T backend pytest tests/test_troubleshooting_rules.py tests/test_agent.py -q
|
||||
```
|
||||
|
||||
Build the frontend to validate TypeScript and production rendering:
|
||||
|
||||
```bash
|
||||
docker compose build frontend
|
||||
```
|
||||
|
||||
Known caveat: some environment-sensitive tests can fail when the Compose `.env` overrides test-specific environment expectations. Keep test configuration isolated when adding new tests.
|
||||
|
||||
## Key Extension Points
|
||||
|
||||
### Add a Device Type
|
||||
|
||||
1. Add or update a catalog entry in `backend/app/inventory_catalog.py`.
|
||||
2. Add environment defaults in `backend/app/services/device_defaults.py`.
|
||||
3. Add an MCP server or select an existing one.
|
||||
4. Add routing keywords/rules in `rules/troubleshooting.yaml`.
|
||||
5. Add a device profile or playbook under `backend/app/agent/`.
|
||||
6. Add report formatting in `backend/app/services/diagnostic_format.py`.
|
||||
7. Add tests.
|
||||
|
||||
### Add an MCP Server
|
||||
|
||||
MCP servers can be local folders under `mcp-servers/` or cloned from Gitea by name.
|
||||
|
||||
Configuration:
|
||||
|
||||
- `MCP_REPOS`: comma-separated repo names.
|
||||
- `MCP_LOCAL_DIR`: local mounted MCP folder.
|
||||
- `MCP_START_<REPO_NAME>`: optional custom launch command.
|
||||
|
||||
The worker starts servers and exposes tool status through the MCP manager.
|
||||
|
||||
### Add a Troubleshooting Rule
|
||||
|
||||
Edit `rules/troubleshooting.yaml`.
|
||||
|
||||
Rules should define:
|
||||
|
||||
- matching keywords and optional `all_of`;
|
||||
- target devices;
|
||||
- diagnostic order;
|
||||
- severity hint;
|
||||
- human-readable hints and steps.
|
||||
|
||||
Rules should be specific enough to avoid accidental full-stack sweeps.
|
||||
|
||||
### Add a Skill
|
||||
|
||||
Create or edit `skills/<id>/SKILL.md`.
|
||||
|
||||
Skills add deep procedural guidance. They should not decide device scope; rules do that. A good skill describes the investigation method, known pitfalls, commands/tools to prefer, and what evidence proves or disproves the cause.
|
||||
|
||||
### Add Diagnostic Formatting
|
||||
|
||||
Raw MCP output is noisy. Prefer compact formatted summaries in `backend/app/services/diagnostic_format.py`.
|
||||
|
||||
For each new diagnostic output:
|
||||
|
||||
- parse JSON/text using structured logic where possible;
|
||||
- render short tables or bullet summaries;
|
||||
- preserve metadata such as `output_chars`, `raw_available`, and `truncated`;
|
||||
- keep raw output behind evidence expansion, not in the default report view.
|
||||
|
||||
## Task Agent Development
|
||||
|
||||
The main workflow is in `backend/app/agent/graph.py`.
|
||||
|
||||
Be careful with changes there:
|
||||
|
||||
- `context_node` affects memory/Obsidian reuse.
|
||||
- `triage_node` affects device scope and severity.
|
||||
- `diagnose_node` affects MCP calls and approval risk.
|
||||
- `reason_node` affects model cost and report interpretation.
|
||||
- `report_node` affects persistence, approvals, Memory, and Obsidian.
|
||||
|
||||
For device behavior, prefer smaller profile modules such as:
|
||||
|
||||
- `pfsense_profiles.py`
|
||||
- `proxmox_profiles.py`
|
||||
- `asterisk_profiles.py`
|
||||
- generic `playbooks.py`
|
||||
|
||||
## Frontend Development
|
||||
|
||||
Frontend pages live under `frontend/app/`.
|
||||
|
||||
Important shared pieces:
|
||||
|
||||
- `frontend/lib/api.ts`: REST and WebSocket helpers.
|
||||
- `frontend/lib/auth.tsx`: auth context and roles.
|
||||
- `frontend/lib/task-types.ts`: shared task and event helpers.
|
||||
- `frontend/components/TaskLiveStatus.tsx`: phase/timeline rendering.
|
||||
- `frontend/components/task-detail-panels.tsx`: report, scope, timeline, artifacts, cost, history.
|
||||
- `frontend/components/ui.tsx`: common UI primitives.
|
||||
|
||||
Design principle: default screens should show summaries and attention items; raw evidence should be expandable.
|
||||
|
||||
## Operational Checks
|
||||
|
||||
After changing backend or worker code:
|
||||
|
||||
```bash
|
||||
docker compose build backend worker
|
||||
docker compose up -d backend worker
|
||||
```
|
||||
|
||||
After changing frontend code:
|
||||
|
||||
```bash
|
||||
docker compose build frontend
|
||||
docker compose up -d frontend
|
||||
```
|
||||
|
||||
After changing MCP servers:
|
||||
|
||||
1. Use the MCP page to reload/upgrade when possible.
|
||||
2. Check worker logs if server status is `error`.
|
||||
3. Confirm the tool list changed as expected.
|
||||
|
||||
## Troubleshooting the Platform
|
||||
|
||||
| Symptom | First checks |
|
||||
|---|---|
|
||||
| Task stuck queued | Worker running, Redis healthy, no stuck in-flight task. |
|
||||
| No MCP tools | Worker logs, MCP repo cloned/local path exists, start command works. |
|
||||
| Empty Proxmox VM list | API token has `VM.Audit` on `/vms`, node permissions, SSL verify setting. |
|
||||
| Obsidian note missing | Task `obsidian_path`, backend logs, vault git status/push, local Obsidian pull. |
|
||||
| Balance missing | `/api/balance`, provider keys, DNS from backend container. |
|
||||
| Report too noisy | Add/adjust diagnostic formatter and report fields instead of dumping raw output. |
|
||||
|
||||
## Security Practices
|
||||
|
||||
- Do not commit `.env`, tokens, keys, or device credentials.
|
||||
- Reference secret names, not values, in docs and memory.
|
||||
- Keep MCP write flags disabled unless intentionally testing writes.
|
||||
- Keep MCP source patch approval enabled for normal operation.
|
||||
- Avoid broad CORS origins in deployments.
|
||||
- Treat task reports and Obsidian notes as operational records; do not include full credentials.
|
||||
|
||||
Reference in New Issue
Block a user