Matheus Castelliano
← All work

Built it2026 · Voice AI

Teaching an AI interviewer to talk less.

A real-time voice agent that runs 70-question research interviews for an organizational research startup.

Role
Defined and built
Stack
LiveKit, LLMs, Claude Code
Scope
70-question interviews

Measure, then fix

The first measurement was humbling.

Agent time per question68.9s
Full 70-question interview110.7 min
Scores captured correctly3/7
01

Problem

Interviews ran over video calls. Someone shared the screen, read 70+ questions aloud and typed every answer into a spreadsheet. One person, one interview at a time.

02

Decision

Keep the LLM out of the driver’s seat. A deterministic state machine runs the interview: question order, steps, and saving every score. The LLM only handles what’s open-ended: answering a doubt, a follow-up question, keeping it natural.

I tried the free-form version first. It confirmed what the best voice agents already do: let the model talk, not run the process.

03

What I built

A real-time voice agent where every question follows the same fixed protocol. Built entirely with Claude Code: I specified, measured and reviewed every step; the AI wrote the code.

04

Measure, then fix

For every minute a participant spoke, the agent spoke almost five. And it captured only 3 of 7 scores: the transcription turned “sete” (seven) into “Sčet”.

I tested transcription options against my own recordings and switched to the one that got 7 of 7. Then I studied recordings of human interviewers and cut everything they never do: re-reading the whole question, asking “did you understand?”, chasing three separate justifications.

05

Trade-off

The more accurate transcription added about a second to each turn. I took it: fewer repeated questions saved far more time than it cost.

06

Result

A full 70-question interview now runs end to end in 45 minutes, down from a projected 110.

07

What I’d do differently

Measure from day one. I tuned by ear for weeks; the real gains only came once I could see where every second went.

Let’s talk.

A role, a project or just a conversation about AI products. Send me a line; I answer every message.

matheus_castelliano@outlook.com

Open to JobsFreelanceConsulting