Quiet Inbox

Build a ai agent called Quiet Inbox: an email agent that drafts replies and only pings you for the ones that matter.

What it does

Quiet Inbox reads new email, sorts it into ‘needs you’, ‘handled’ and ‘can wait’, and drafts replies in your voice for the second pile. You get one notification a day instead of forty.

Stack

Use Python, FastAPI, Postgres, Fly.io, Anthropic API. The original was built with Claude Code and ChatGPT in 6 weeks, for about $210 in AI usage.

How it fits together

• An agent loop that reads input, decides and acts through a few tools
• A store for what it did, so a person can review and undo
• An evaluation set to measure it after every prompt change

Build it in this order

1. Write down the agent's job in one paragraph and the cases where it must ask a person
2. Build the loop with one tool and a review screen
3. Collect 50 to 200 labelled examples and build an evaluation harness
4. Add the remaining tools one at a time, re-running the evaluation each time
5. Add limits: cost per run, rate limits and a stop button

Tools to connect

• Gmail MCP: read and draft without storing full message bodies

A prompt that worked: Grade your own agent

Build an evaluation harness for the triage step. Input: 200 labelled emails in evals/inbox.jsonl. For each, run the triage prompt and compare against the label. Output a table of precision and recall per bucket and the 10 worst misses with the model's reasoning. Make it runnable with one command so I can rerun it after every prompt change.

Watch out for

• Measure before you tune the prompt; otherwise you can't tell if a change helped
• Log every action with the reason the model gave

Done when

• The evaluation runs with one command
• A person can see and undo every action
• Costs per run are capped

Before writing code, ask me up to five questions about who this productivity ai agent is for and what the first version must do. Then work in small steps: after each one, tell me how to try it and wait for my feedback.