# Developer and Operations Guide

## Local Stack

Agentic OS runs with Docker Compose.

```bash
docker compose up -d --build
```

Default service ports:

| Service | Port | Purpose |
|---|---:|---|
| Frontend | `3000` | Operator UI |
| Backend | `8000` | FastAPI API |
| LiteLLM | `4000` | LLM gateway |
| Postgres | `5432` | Application and LiteLLM databases |

Ollama runs on the macOS host, not inside Docker, so it can use Apple Silicon acceleration. Containers reach it through `host.docker.internal:11434`.

## Configuration

Use `.env.example` as the source of truth for configuration names. Copy it to `.env` and fill real values locally.

Never commit `.env`.

Important groups:

- **Security**: `SECRET_KEY`, `CREDENTIAL_ENCRYPTION_KEY`, first admin credentials.
- **Database/cache**: `DATABASE_URL`, `REDIS_URL`.
- **CORS**: `CORS_ALLOW_ORIGINS`.
- **LLM routing**: local, Gemini, DeepSeek, OpenRouter, LiteLLM settings.
- **MCP development**: enable/approval/push policy.
- **Gitea**: clone/push base URL, username, token, MCP repos.
- **Obsidian**: vault repo, task folder, auto-push.
- **Project memory**: HTTP MCP URL, token, project IDs.
- **Rules/skills/templates**: mounted paths for agent guidance and baselines.
- **Device defaults**: per-device credentials and endpoint defaults.

## Database

Models live in `backend/app/models/`.

Alembic migrations live in `backend/alembic/`.

The backend initializes/upgrades the database on startup. Avoid ad-hoc schema changes in application startup code; prefer migrations.

## Tests

Run backend tests inside the container:

```bash
docker compose exec -T backend pytest -q
```

Run a focused subset:

```bash
docker compose exec -T backend pytest tests/test_troubleshooting_rules.py tests/test_agent.py -q
```

Build the frontend to validate TypeScript and production rendering:

```bash
docker compose build frontend
```

Known caveat: some environment-sensitive tests can fail when the Compose `.env` overrides test-specific environment expectations. Keep test configuration isolated when adding new tests.

## Key Extension Points

### Add a Device Type

1. Add or update a catalog entry in `backend/app/inventory_catalog.py`.
2. Add environment defaults in `backend/app/services/device_defaults.py`.
3. Add an MCP server or select an existing one.
4. Add routing keywords/rules in `rules/troubleshooting.yaml`.
5. Add a device profile or playbook under `backend/app/agent/`.
6. Add report formatting in `backend/app/services/diagnostic_format.py`.
7. Add tests.

### Add an MCP Server

MCP servers can be local folders under `mcp-servers/` or cloned from Gitea by name.

Configuration:

- `MCP_REPOS`: comma-separated repo names.
- `MCP_LOCAL_DIR`: local mounted MCP folder.
- `MCP_START_<REPO_NAME>`: optional custom launch command.

The worker starts servers and exposes tool status through the MCP manager.

### Add a Troubleshooting Rule

Edit `rules/troubleshooting.yaml`.

Rules should define:

- matching keywords and optional `all_of`;
- target devices;
- diagnostic order;
- severity hint;
- human-readable hints and steps.

Rules should be specific enough to avoid accidental full-stack sweeps.

### Add a Skill

Create or edit `skills/<id>/SKILL.md`.

Skills add deep procedural guidance. They should not decide device scope; rules do that. A good skill describes the investigation method, known pitfalls, commands/tools to prefer, and what evidence proves or disproves the cause.

### Add Diagnostic Formatting

Raw MCP output is noisy. Prefer compact formatted summaries in `backend/app/services/diagnostic_format.py`.

For each new diagnostic output:

- parse JSON/text using structured logic where possible;
- render short tables or bullet summaries;
- preserve metadata such as `output_chars`, `raw_available`, and `truncated`;
- keep raw output behind evidence expansion, not in the default report view.

## Task Agent Development

The main workflow is in `backend/app/agent/graph.py`.

Be careful with changes there:

- `context_node` affects memory/Obsidian reuse.
- `triage_node` affects device scope and severity.
- `diagnose_node` affects MCP calls and approval risk.
- `reason_node` affects model cost and report interpretation.
- `report_node` affects persistence, approvals, Memory, and Obsidian.

For device behavior, prefer smaller profile modules such as:

- `pfsense_profiles.py`
- `proxmox_profiles.py`
- `asterisk_profiles.py`
- generic `playbooks.py`

## Frontend Development

Frontend pages live under `frontend/app/`.

Important shared pieces:

- `frontend/lib/api.ts`: REST and WebSocket helpers.
- `frontend/lib/auth.tsx`: auth context and roles.
- `frontend/lib/task-types.ts`: shared task and event helpers.
- `frontend/components/TaskLiveStatus.tsx`: phase/timeline rendering.
- `frontend/components/task-detail-panels.tsx`: report, scope, timeline, artifacts, cost, history.
- `frontend/components/ui.tsx`: common UI primitives.

Design principle: default screens should show summaries and attention items; raw evidence should be expandable.

## Operational Checks

After changing backend or worker code:

```bash
docker compose build backend worker
docker compose up -d backend worker
```

After changing frontend code:

```bash
docker compose build frontend
docker compose up -d frontend
```

After changing MCP servers:

1. Use the MCP page to reload/upgrade when possible.
2. Check worker logs if server status is `error`.
3. Confirm the tool list changed as expected.

## Troubleshooting the Platform

| Symptom | First checks |
|---|---|
| Task stuck queued | Worker running, Redis healthy, no stuck in-flight task. |
| No MCP tools | Worker logs, MCP repo cloned/local path exists, start command works. |
| Empty Proxmox VM list | API token has `VM.Audit` on `/vms`, node permissions, SSL verify setting. |
| Obsidian note missing | Task `obsidian_path`, backend logs, vault git status/push, local Obsidian pull. |
| Balance missing | `/api/balance`, provider keys, DNS from backend container. |
| Report too noisy | Add/adjust diagnostic formatter and report fields instead of dumping raw output. |

## Security Practices

- Do not commit `.env`, tokens, keys, or device credentials.
- Reference secret names, not values, in docs and memory.
- Keep MCP write flags disabled unless intentionally testing writes.
- Keep MCP source patch approval enabled for normal operation.
- Avoid broad CORS origins in deployments.
- Treat task reports and Obsidian notes as operational records; do not include full credentials.