Fleet Operations Playbook
Monitor, diagnose, and maintain your agent fleet — health checks, job audits, cost monitoring, and incident response.
Why this playbook
An agent fleet is a production system — it needs monitoring, maintenance, and incident response just like any other service. The difference is that agent failures are often subtle: a stuck heartbeat, a silently failing cron job, or a gradual cost increase that doesn't trigger alerts until the bill arrives.
This playbook chains four operations prompts that cover the full ops lifecycle: check fleet health, audit scheduled jobs, monitor spend, and write incident reports when things go wrong. Run the first three weekly as preventive maintenance.
Built from AGNT's own internal ops runbook. These are the exact prompts our Sentinel agent uses to monitor the fleet.
Prerequisites
- Claude Code with filesystem access to fleet logs
- SSH access to the deployment environment (Railway, VPS, or similar)
- Access to the scheduler's job list (APScheduler config or crontab)
- Billing API access or cost dashboard credentials
Input requirements
| Input | Type | Required | Description |
|---|---|---|---|
| Fleet log directory | directory path | Yes | Path to heartbeat logs, error logs, and agent activity logs. Typically: /var/log/agnt/ or the Railway log stream. |
| Scheduler config path | file path | Yes | Path to the APScheduler job configuration or crontab. Used to audit job health and detect stuck/orphaned jobs. |
| Billing timeframe | string | No | Period to analyze for spend anomalies. Default: last 7 days. Format: '7d', '30d', or ISO date range. |
Step-by-step workflow
Review fleet health and heartbeat status
Open promptStart with the big picture. This prompt analyzes heartbeat logs to identify stuck agents, degraded performance, and communication failures. It produces a fleet health matrix: green/yellow/red per agent.
Pay special attention to yellow-status agents — they're still running but showing early signs of degradation (increasing latency, higher error rates, or missed heartbeats).
Audit scheduled jobs for failures and drift
Open promptScheduled jobs are the backbone of fleet automation — if they drift or fail silently, your agents lose critical maintenance routines. This prompt audits every scheduled job: last run time, success rate, execution duration, and schedule drift.
The output flags jobs that have silently stopped, jobs running significantly longer than their historical average, and jobs whose schedules have drifted from their intended cadence.
Monitor and detect cost anomalies
Open promptThis prompt analyzes your billing data to detect unusual spend patterns: sudden spikes, gradual creep, and per-agent cost outliers. It compares current spend against a rolling 30-day baseline.
Cost anomalies often signal operational issues before health checks catch them — a stuck retry loop, an agent making redundant API calls, or an unexpected traffic spike hitting your LLM budget.
Document incidents and generate postmortems
Open promptIf any of the previous steps found critical issues, this prompt generates a structured incident report: timeline, root cause, impact, resolution, and follow-up actions.
Even if nothing is broken, run this step to document the review itself — it creates a 'weekly ops review' note that feeds into your knowledge base and helps track fleet health trends over time.
Expected outputs
Fleet health matrix
Per-agent health status (green/yellow/red) with latency, error rate, and heartbeat metrics.
Produced by step 1Job audit report
Status of every scheduled job: last run, success rate, duration trend, schedule drift analysis.
Produced by step 2Spend anomaly report
Cost analysis with per-agent breakdown, anomaly flags, and trend comparison against 30-day baseline.
Produced by step 3Incident report / ops review
Structured incident report or weekly ops review with timeline, root cause, and follow-up actions.
Produced by step 4Tool requirements
- Claude Code with filesystem + SSH access
- Access to fleet scheduler (APScheduler / cron)
- Access to spend/billing dashboard or API
Troubleshooting
Heartbeat logs are empty or missing
Spend API returns 403 Forbidden
Cron doctor reports all jobs healthy but fleet is degraded
Share as social post
Fleet Operations Playbook 4 steps, 60-90 minutes. Monitor, diagnose, and maintain your agent fleet — health checks, job audits, cost monitoring, and incident response. https://agntdot.com/playbooks/fleet-operations-playbook
225 / 280 chars