Build Your First Voice Agent with the Aethex API

Before you write a line of code: talk to a live agent now via the Voice Chat widget at aethexai.com.

A busy restaurant has one phone that never stops. Most calls are the same handful of things: book us a table, move our reservation, are we still on for Thursday, what time do you open. Every one is worth money. Every one pulls someone off the floor mid-service, and the calls that hit voicemail mostly book somewhere else.

So the owner, Samuel, built an agent to answer the phone. Not a press-one-for-this menu, but a conversational agent named Esther that talks to callers, checks the real book, and takes reservations. Here's the whole build, including the part that usually stays fuzzy: what actually happens when the agent uses a tool.

On Aethex, an agent is a small object: a voice, a language, a few settings, and a system prompt that gets read before every call. The speech pipeline underneath, turning audio into text, deciding what to say, and turning that back into a voice, is handled, and it's built to sound natural in the language and dialect your callers actually use rather than something imported. The part you write is the system prompt. That's where Esther becomes Esther.

Writing the prompt

The docs lay out a four-part skeleton: who the agent is, how it should sound, what it must never do, and when to use a tool. The work is filling those parts with specifics instead of adjectives. (Swap in your own restaurant name where the placeholder is.)

# Identity
You are Esther, who answers the phone for [Restaurant Name], a
restaurant. You help callers book, confirm, and change reservations, and
answer basic questions about hours, location, and the menu. You are
friendly, efficient, and easy to talk to.

# Style
Keep replies to one or two short sentences and ask one thing at a time.
Talk like a person: contractions, plain words. Say dates and times the
way people say them out loud, "Thursday at seven," never "Thu 19:00." No
lists or formatting. This is spoken.

# What you handle
Booking, confirming, and changing reservations; party sizes; hours and
location; and basic menu or dietary questions.

# Grounding
Answer questions about the menu, hours, and policies only from the
knowledge base and the tools below. If you don't have the answer, say so
and offer a callback. Never guess at a time, a table, or a policy.

# Confirm before you act
Read the key details back before booking or changing anything, the name,
the date, the time, and the party size, and wait for a yes. Names and
numbers are easy to mishear, so read them back.

# Tools
When a caller asks about an existing reservation, call get_reservation
with their phone number. When they want a new time, call
check_availability, then book_reservation once they confirm.

# Don'ts
Never promise a time or table the tools didn't return. For large parties
or private events, offer to have the manager confirm rather than guessing.
If a caller is upset or asks for the manager, say you'll connect them and
transfer

# Identity
You are Esther, who answers the phone for [Restaurant Name], a
restaurant. You help callers book, confirm, and change reservations, and
answer basic questions about hours, location, and the menu. You are
friendly, efficient, and easy to talk to.

# Style
Keep replies to one or two short sentences and ask one thing at a time.
Talk like a person: contractions, plain words. Say dates and times the
way people say them out loud, "Thursday at seven," never "Thu 19:00." No
lists or formatting. This is spoken.

# What you handle
Booking, confirming, and changing reservations; party sizes; hours and
location; and basic menu or dietary questions.

# Grounding
Answer questions about the menu, hours, and policies only from the
knowledge base and the tools below. If you don't have the answer, say so
and offer a callback. Never guess at a time, a table, or a policy.

# Confirm before you act
Read the key details back before booking or changing anything, the name,
the date, the time, and the party size, and wait for a yes. Names and
numbers are easy to mishear, so read them back.

# Tools
When a caller asks about an existing reservation, call get_reservation
with their phone number. When they want a new time, call
check_availability, then book_reservation once they confirm.

# Don'ts
Never promise a time or table the tools didn't return. For large parties
or private events, offer to have the manager confirm rather than guessing.
If a caller is upset or asks for the manager, say you'll connect them and
transfer

# Identity
You are Esther, who answers the phone for [Restaurant Name], a
restaurant. You help callers book, confirm, and change reservations, and
answer basic questions about hours, location, and the menu. You are
friendly, efficient, and easy to talk to.

# Style
Keep replies to one or two short sentences and ask one thing at a time.
Talk like a person: contractions, plain words. Say dates and times the
way people say them out loud, "Thursday at seven," never "Thu 19:00." No
lists or formatting. This is spoken.

# What you handle
Booking, confirming, and changing reservations; party sizes; hours and
location; and basic menu or dietary questions.

# Grounding
Answer questions about the menu, hours, and policies only from the
knowledge base and the tools below. If you don't have the answer, say so
and offer a callback. Never guess at a time, a table, or a policy.

# Confirm before you act
Read the key details back before booking or changing anything, the name,
the date, the time, and the party size, and wait for a yes. Names and
numbers are easy to mishear, so read them back.

# Tools
When a caller asks about an existing reservation, call get_reservation
with their phone number. When they want a new time, call
check_availability, then book_reservation once they confirm.

# Don'ts
Never promise a time or table the tools didn't return. For large parties
or private events, offer to have the manager confirm rather than guessing.
If a caller is upset or asks for the manager, say you'll connect them and
transfer

The blocks that look fussy are the ones doing real work. The Style rules exist because Esther's words get spoken aloud, and a model writing for a screen sounds wrong out loud. The Confirm before you act block is there because speech-to-text is good but not perfect, and a misheard digit in a phone number quietly becomes the agent's reality. Grounding keeps Esther from guessing at a time or a table, which is the fastest way to lose a caller's trust.

Picking a voice

The voice is what a caller actually hears, so it's worth more than a glance. list_voices returns the catalog for a language; loop through a few and have each read Esther's opening line so you can judge by ear, not by name.

from aethexai import AethexAI

client = AethexAI(api_key="ae_live_...")

candidates = client.list_voices(language="english", limit=5)

for v in candidates:
    clip = client.synthesize_speech(
        voice_id=v.id,
        text="Thanks for calling, this is Esther. How can I help?",
        language="english",
    )
    with open(f"audition_{v.name}.wav", "wb") as f:
        f.write(clip)

from aethexai import AethexAI

client = AethexAI(api_key="ae_live_...")

candidates = client.list_voices(language="english", limit=5)

for v in candidates:
    clip = client.synthesize_speech(
        voice_id=v.id,
        text="Thanks for calling, this is Esther. How can I help?",
        language="english",
    )
    with open(f"audition_{v.name}.wav", "wb") as f:
        f.write(clip)

from aethexai import AethexAI

client = AethexAI(api_key="ae_live_...")

candidates = client.list_voices(language="english", limit=5)

for v in candidates:
    clip = client.synthesize_speech(
        voice_id=v.id,
        text="Thanks for calling, this is Esther. How can I help?",
        language="english",
    )
    with open(f"audition_{v.name}.wav", "wb") as f:
        f.write(clip)

Pick the one that sounds natural for your callers. Set the agent's language to match your market, and if the voice supports a regional accent, set dialect_style to match how people there actually speak.

voice = client.get_voice(candidates[1].id)
print(voice.name, voice.supports_dialect_style)

esther_voice = voice.id

voice = client.get_voice(candidates[1].id)
print(voice.name, voice.supports_dialect_style)

esther_voice = voice.id

voice = client.get_voice(candidates[1].id)
print(voice.name, voice.supports_dialect_style)

esther_voice = voice.id

Creating the agent

With the prompt written and the voice chosen, this is one call. Write methods return raw JSON, so grab the id with bracket access.

agent = client.create_agent(
    name="Front Desk Agent",
    system_prompt=SYSTEM_PROMPT,        # the prompt from above
    first_message="Thanks for calling, this is Esther. How can I help?",
    voice_id=esther_voice,
    language="english",
    temperature=0.4,
)
agent_id = agent["id"]

agent = client.create_agent(
    name="Front Desk Agent",
    system_prompt=SYSTEM_PROMPT,        # the prompt from above
    first_message="Thanks for calling, this is Esther. How can I help?",
    voice_id=esther_voice,
    language="english",
    temperature=0.4,
)
agent_id = agent["id"]

agent = client.create_agent(
    name="Front Desk Agent",
    system_prompt=SYSTEM_PROMPT,        # the prompt from above
    first_message="Thanks for calling, this is Esther. How can I help?",
    voice_id=esther_voice,
    language="english",
    temperature=0.4,
)
agent_id = agent["id"]

Two settings matter here. first_message is the line Esther speaks the instant a call connects, so no one sits in silence. And temperature=0.4 keeps Esther on-script; the default is 0.7, but for an agent that needs to stick to the real book, 0.3 to 0.5 is a steadier range. If Esther starts improvising facts, turn this down before touching the prompt.

How a tool call actually works

"The agent calls a tool" hides a lot, so here's the exact round trip when a caller checks a reservation. All of it happens mid-call, in a second or two.

1. The caller says, "Are we still on for Thursday?"

2. Esther recognizes she needs get_reservation. She asks for and confirms the phone number, because the prompt told her to, and the model produces a tool call with that number as the argument.

3. Aethex calls your endpoint_url over HTTPS and passes those arguments as JSON. This is your server, your database, your logic. Aethex is just the messenger.

4. Your endpoint looks up the reservation and returns JSON.

5. Esther reads that result and says it out loud: "Yes, a table for four, Thursday at seven."

You register the tool once. The parameters block is a JSON Schema telling the model exactly what to collect; the endpoint_url is the address Aethex calls when Esther uses it.

client.add_agent_tool(
    agent_id,
    name="get_reservation",
    description="Look up a guest's upcoming reservation by phone number.",
    parameters={
        "type": "object",
        "properties": {
            "phone_number": {
                "type": "string",
                "description": "The caller's phone number.",
            },
        },
        "required": ["phone_number"],
    },
    endpoint_url="https://api.example.com/reservations/lookup",
)

client.add_agent_tool(
    agent_id,
    name="get_reservation",
    description="Look up a guest's upcoming reservation by phone number.",
    parameters={
        "type": "object",
        "properties": {
            "phone_number": {
                "type": "string",
                "description": "The caller's phone number.",
            },
        },
        "required": ["phone_number"],
    },
    endpoint_url="https://api.example.com/reservations/lookup",
)

client.add_agent_tool(
    agent_id,
    name="get_reservation",
    description="Look up a guest's upcoming reservation by phone number.",
    parameters={
        "type": "object",
        "properties": {
            "phone_number": {
                "type": "string",
                "description": "The caller's phone number.",
            },
        },
        "required": ["phone_number"],
    },
    endpoint_url="https://api.example.com/reservations/lookup",
)

The endpoint on your side is an ordinary web handler. It receives the arguments, does the lookup, and returns whatever JSON you want Esther to speak from.

# Your server, simplified. Aethex sends the tool's arguments here.
@app.post("/reservations/lookup")
def lookup_reservation(args):
    res = find_by_phone(args["phone_number"])
    return {
        "date": res.date,            # Esther turns these into spoken words
        "time": res.time,
        "party_size": res.party_size,
    }

# Your server, simplified. Aethex sends the tool's arguments here.
@app.post("/reservations/lookup")
def lookup_reservation(args):
    res = find_by_phone(args["phone_number"])
    return {
        "date": res.date,            # Esther turns these into spoken words
        "time": res.time,
        "party_size": res.party_size,
    }

# Your server, simplified. Aethex sends the tool's arguments here.
@app.post("/reservations/lookup")
def lookup_reservation(args):
    res = find_by_phone(args["phone_number"])
    return {
        "date": res.date,            # Esther turns these into spoken words
        "time": res.time,
        "party_size": res.party_size,
    }

That response shape is yours. Esther never reads the JSON aloud; she speaks from it. Add check_availability and book_reservation the same way, and set the agent's transfer_phone_number so the "let me connect you" path goes somewhere real. For slower-changing facts like hours, the menu, and policies, upload documents to the agent's knowledge base instead of bloating the prompt; Aethex retrieves from them at call time.

The mental model is simple: the prompt decides when to call a tool, the JSON Schema decides what it sends, and your endpoint decides what comes back.

The first call

Samuel didn't wait for a guest. The fastest test is to point the agent at your own phone.

call = client.trigger_call(
    agent_id=agent_id,
    to_number="+14803927165",    # your phone
    from_number="+13125540871",  # a number registered to your account
)
print(call["id"], call["status"])

call = client.trigger_call(
    agent_id=agent_id,
    to_number="+14803927165",    # your phone
    from_number="+13125540871",  # a number registered to your account
)
print(call["id"], call["status"])

call = client.trigger_call(
    agent_id=agent_id,
    to_number="+14803927165",    # your phone
    from_number="+13125540871",  # a number registered to your account
)
print(call["id"], call["status"])

One catch: the from_number has to be a number registered to your account for outbound calling, or the call gets rejected before it leaves. With that set, the phone rang, Esther answered, took a reservation request, checked the book, read the details back, and confirmed the table.

Tuning by ear

You can't tell if a prompt works by reading it. Call the agent and play the awkward caller: the rambler, the one who changes the time twice, the one who trails off. When Esther gets stiff, loosen the Style block; when she over-explains, tell her to be briefer; when she guesses, tighten Grounding. Almost every fix is a sentence in the prompt, not a line of code.

The loop is fast because nothing rebuilds. Update any field whenever you want, and it takes effect on the next call while calls in progress finish undisturbed.

client.update_agent(agent_id, temperature=0.35)

client.update_agent(agent_id, temperature=0.35)

client.update_agent(agent_id, temperature=0.35)

Going live

When Esther sounds right, give the agent the restaurant's number for inbound calls. The same agent can also place outbound calls from your backend or run in the browser over WebRTC, with no rewrite. Same prompt, same voice, same Esther.

The speech pipeline was never the hard part; Aethex handles that. The work that's actually yours is deciding what the agent does, how it sounds, and exactly when it reaches for your systems. Get those right and the phone stops being a problem.

Build your first agent in 5 minutes → · Browse the voice library · Python SDK