Build a Local Voice Assistant with ESPHome Satellites and Home Assistant Assist

Why Local Voice Matters in a Smart Home

Voice control is the most natural remote you can own, but cloud assistants often fail right when you need them most: during an outage, when the internet is slow, or when you want to avoid sending your home’s audio to somebody else’s servers. A local voice assistant flips that script. It runs on your network, responds quickly, and keeps your audio at home. For many families, this is the missing piece between smart gadgets and truly smart living.

In this guide, you’ll build a room-aware voice system using ESPHome voice satellites and Home Assistant Assist. Each “satellite” is a small ESP32 device with a microphone and (optionally) a speaker. It listens for a wake word locally, streams the command to Home Assistant for transcription and intent handling, and then speaks a reply back through the nearest speaker. The result feels like a modern, privacy-first voice assistant—without the subscription or data trail.

We’ll walk through what you need, how to wire the pieces together, how to make commands room-scoped (so “turn on the lights” in the kitchen only affects the kitchen), and how to polish the experience with smart responses and audio routing. The goal is clear, practical steps you can complete in an afternoon—then expand room by room.

Before we dive in, here’s the philosophy behind this build: keep it local, keep it robust, and keep it simple enough that everyone in the house can use it. You don’t need a closet full of servers or a degree in machine learning. You just need a handful of affordable devices, a good Wi‑Fi signal, and the patience to iterate once or twice.

Newsletter

Never miss out on great articles — just enter your email and receive a weekly update!

Go

I have read and accept the Terms & Policy

Hardware and Network Setup

The best part about ESPHome satellites is how flexible they are. You can start with one device on your desk and expand to every room over time. Pick hardware that fits each room’s acoustics and power options. A bedroom might be fine with a tiny puck by the nightstand, while an open-plan kitchen benefits from a slightly stronger mic and a visible status LED.

Below are three common device types that work well today. Choose the blend that fits your space and budget. You only need one to get started.

On the server side, you’ll need Home Assistant running on a device with some headroom—an x86 mini PC, a Raspberry Pi 4/5 with SSD, or a NAS. The Assist feature handles speech-to-text (STT), intent parsing, and text-to-speech (TTS). You can keep both STT and TTS entirely local by choosing offline engines. If you opt for hybrid cloud engines, you’ll typically gain accuracy but lose a bit of privacy; this guide assumes you’ll stay local or at least be able to switch later.

Network quality matters. Voice streaming is light on bandwidth but sensitive to latency. Make sure your satellites have strong Wi‑Fi and minimal interference. If you have a mesh system, place access points so each room gets consistent coverage. Disable overly aggressive “band steering” or sticky roaming if you notice dropouts; voice satellites prefer stable connections over constant optimization games.

Power is simple: most satellites run over USB-C or micro-USB. If you’re planning a clean install (e.g., under a cabinet or behind a frame), consider short right-angle cables and adhesive cable guides. Label each satellite by room in both your wiring and your Home Assistant entity names. Clear naming saves hours later, especially when you scale to six or more rooms.

Finally, plan your audio output. Satellites with built-in speakers are great for chimes and short replies. For fuller responses or media announcements, you can route TTS to a nearby smart speaker or soundbar using integrations like media_player in Home Assistant. The trick is making sure the reply plays in the same room where you spoke—more on that in a moment.

Configure Room-Aware Commands and Replies

With hardware in place, you’re ready to stitch everything together. The secret to a natural voice experience is “room awareness.” Instead of saying “Turn on the kitchen lights,” you should be able to say “Turn on the lights,” and have it understand you’re in the kitchen because the kitchen satellite heard you. Likewise, the spoken reply should play back in that same room.

Start by flashing ESPHome firmware to each satellite. In your ESPHome configuration, enable a wake-word model and the voice assistant component. Many setups use local wake-word detection via lightweight models that run on the ESP32 itself. You can pick a neutral phrase (“Hey Home”) or your own custom model if supported. Keep sensitivity conservative at first to avoid false triggers. You can always nudge it up once you’ve tested daytime and nighttime noise.

Next, add the satellites to Home Assistant. Confirm that each appears with a friendly name like “Voice Satellite – Kitchen.” Assign an Area to each device in Home Assistant that matches the real-world room. Areas are the backbone of room awareness; they let automations infer the target entities without every command needing a room name.

Create an Assist pipeline that specifies your STT engine, language, and TTS voice. If you’re going fully local, choose offline STT and TTS providers. If you’re hybrid, start with a local default and swap to cloud only when the local engine struggles with a name or accent. Keep logs open during your first trial runs so you can see exactly where the pipeline succeeds or stumbles.

Now wire up the intents. Home Assistant can map natural language to actions using built-in intents for lights, switches, scenes, climate, and media. The key is telling your automations how to discover the correct room. A reliable pattern is: “When a command comes from Satellite X, limit targets to entities in Area X.” This is simple, effective, and scalable.

For example, a simple lights-on intent can resolve like this: Identify the satellite entity that captured audio, read its Area assignment, filter lights to that Area, and execute the requested action. You can implement this logic using scripts or intent handlers that accept a “source area” context passed by Assist.

If you want more precise control, pair satellites with a room “anchor” entity, like a unique helper or a binary_sensor representing voice activity in that room. Then target automations can reference that anchor to route replies and actions. This approach helps when rooms overlap or share entities, like a great room that includes both living and dining zones.

Let’s make replies room-aware. Create a small mapping between Areas and preferred speakers (media_player entities). In the living room, send replies to the TV soundbar; in the bedroom, use the bedside speaker; in the kitchen, reply from the satellite’s own speaker unless the smart display is active. A short script can choose the best output based on availability: if the room’s primary speaker is busy, fall back to the satellite speaker; if it’s late at night, whisper mode; if a timer is running, keep responses short.

That last point—tone and content—matters. Shorter replies make the system feel snappy. Save longer responses for tasks like “What’s my shopping list?” or “Read the morning briefing.” For timed tasks (e.g., “Set a pasta timer for 8 minutes”), play a soft confirmation chime instead of a spoken reply. When the timer ends, play a distinctive tone in the same room.

Once the backbone is working, sprinkle in everyday automations that generate delight. Define a “Goodnight” intent that turns off common lights, locks the door, arms security, and sets your thermostat. Make sure it only triggers fully when spoken from the bedroom satellite; if spoken elsewhere, it can offer a confirmation. Add a “Where is my phone?” intent that makes a phone ring using a siren scene or a notification service. Kids especially love “Lights to movie mode,” which dims lamps, closes blinds, and powers on the projector.

Test, iterate, and keep a short notebook of your household’s favorite phrases. Tune the wake word sensitivity room by room. Kitchens need lower sensitivity during blender time; bedrooms need higher sensitivity for quiet voice. Adjust noise suppression and AGC (automatic gain control) options in ESPHome to improve recognition in echo-prone spaces.

When you’re comfortable, expand to intercom-style features. You can push a text-to-speech message to multiple rooms—“Dinner is ready!”—or create a “drop-in” routine that temporarily opens a one-way voice path to a child’s room satellite. Keep privacy first: require a button press to open live room audio, and indicate status with an LED.

Finally, back up your configuration. Export ESPHome YAMLs, Home Assistant automations, and your Assist pipeline settings. If you add a new satellite next month, you’ll be able to clone a known-good template and only tweak the device name and Area.

Here’s a concise checklist you can keep open during setup:

• Flash first satellite with ESPHome voice assistant and wake word.
• Add to Home Assistant, assign Area, confirm mic input and speaker output.
• Create an Assist pipeline with local STT/TTS where possible.
• Implement intents that resolve targets by the satellite’s Area.
• Map Areas to preferred media players for room-aware replies.
• Tune wake word sensitivity, AGC, and noise suppression per room.
• Document your phrases; iterate on natural language that your household uses.

Advanced tips can take your system from good to great. First, reduce command round-trip time by keeping your Assist pipeline lean. Avoid heavy post-processing during peak hours; schedule nightly model updates if needed. Second, build a per-room “quiet hours” helper. After bedtime, automatically set TTS volume to a gentle level in bedrooms and use chimes instead of words for confirmations. Third, teach the assistant simple follow-up questions. If you say “Set the thermostat,” it can ask “What temperature?” and a second quick utterance finishes the job without repeating the wake phrase.

Security deserves special care. While the system runs locally, you still don’t want sensitive actions to trigger too easily. Require a confirmation phrase for door unlocks or alarm disarms, and restrict those intents to specific satellites. A bedside satellite could arm the system fully, but a kitchen satellite might only arm in home mode. Use LEDs or a small display to show when the mic is listening after wake and when it’s idle again.

For resilience, design a graceful degradation path. If your main TTS engine fails, the system should fall back to a simpler voice or chimes. If STT stalls, your satellite can play a soft error tone and flash an LED so you’re not left wondering. Log failures with timestamps and room names, then review weekly to spot patterns—maybe the laundry room sits on a weak Wi‑Fi channel during dryer cycles and needs a small reposition.

On the hardware front, experiment with placement. Microphones like to be slightly away from walls to reduce reflections. Avoid direct airflow from vents and keep satellites away from high-vibration appliances. If the room is echoey, a small rug or a fabric wall hanging can improve recognition more than any software tweak.

Don’t forget discoverability. New visitors should be able to see or intuit how to talk to your home. A tiny tent card next to the living room satellite with a wake phrase and two example commands makes adoption instant. A subtle LED pattern—one color for listening, another for thinking, a third for replying—helps everyone understand what’s happening.

If you’re keen on deeper customization, you can define custom intents for complex routines. For example, “Get ready to cook” can turn on under-cabinet lights, start a playlist, and set the oven’s smart plug. Add slot values like cuisine type or music mood, and parse them in your automation script. Keep responses fun: “Let’s turn this kitchen into a stage. Lights, playlist, action.”

Over time, you’ll develop a style that fits your household. Some homes prefer ultra-minimal replies; others love playful banter. The beauty of a local voice stack is the freedom to adapt. You can change voices, phrases, and behaviors without waiting for a firmware update from a vendor. And if a new satellite device hits the market, you can drop it in by cloning your template and assigning its Area.

Finally, remember that small wins build trust. Start with three high-impact voice actions: lights in the main living space, kitchen timers, and a home/away scene. Make them fast, reliable, and obvious. Once those feel automatic, add media controls, thermostat tweaks, and routine scheduling. When your family stops reaching for a phone to control the home, you’ll know you’ve crossed the threshold from “smart gadgets” to a truly helpful home.