Project - A voice companion that remembers you

Overview

Sahm is a voice companion that lives on a small screen-and-microphone device on my desk. The idea came from watching Her: not an assistant that answers questions, but something that gets to know you.

It runs firmware I wrote for an ESP32, with 8MB of memory to work in, so most of the job was fitting a personality into a very small space. It listens for the wake word on the device itself, using a model I trained, then opens a voice session through my server to OpenAI's realtime model. The device never sees the API key.

Memory is the part that makes it feel different. There are two kinds: lasting facts about you, and passing notes about your mood or your week that expire after a few days. When a conversation ends, a second model reads the transcript and decides what is worth keeping, what has changed and what should be dropped. Her character lives in a versioned text file on the server, so I can change who she is without touching the firmware.

There's a web app alongside it: timers and reminders, voice notes and pictures you can send to the screen, and a kids mode with three age bands that changes how much she explains. Parents see the questions their child asked rather than a transcript.

It's still a prototype, one device on a desk, but the unglamorous parts are done: firmware updates over the air, remote wipe, and a way to move a personality onto a replacement board if the hardware dies.

What I built