basically alexa. except it's me talking to myself.
Billy Bot is basically a mini Alexa — a local voice assistant that uses keyword extraction instead of AI. It runs on a Raspberry Pi with Python 3. There are two core output functions: play(), which plays a pre-recorded voice clip of me saying an answer (because I'm a narcissist), and say(), which falls back to text-to-speech if you'd rather not talk to yourself.
Billy waits for the wake word "billy" before listening for commands. Then it runs speech-to-text through Google (with Vosk as an offline backup) and converts what you said into a string. From there it scans for keywords. On the surface it sort of looks like AI — but it's actually just a really long list of if/else statements checking whether a word is in the sentence.
Because of the modular design, the keywords and functionality can be extended as far as you want. Write the function, add it to the if/else chain, done. Easy to expand — but it was a huge pain to build from scratch. Here's everything that went into it.
Full headless setup from a blank microSD card. You will never plug a monitor into this Pi — everything happens over SSH from your computer. If that sounds scary, it isn't. It's just a terminal window that talks to the Pi over WiFi.
Download Raspberry Pi Imager from raspberrypi.com/software and install it on your computer. Open it and make these selections:
Before you click Write, click the cog icon (Edit Settings). This is critical — without it you can't connect headlessly:
billybot[username] for the rest of this guideClick Save → Write. Wait until it fully finishes. Eject, plug into the Pi, connect the power.
sdcard.org, format the card, then try Imager againGive the Pi about 90 seconds to boot — it's doing first-time setup in the background. Log into your home router (usually 192.168.0.1 or 192.168.1.1 in a browser) and look at the connected devices list. Find billybot and write down its IP address — it'll look like 192.168.1.42.
If you can't find it in the router, try from your computer's terminal:
ping billybot.localping billybot.local times out on WindowsWindows doesn't always support .local hostnames. Download Angry IP Scanner and scan your network range to find the IP directlyOpen a terminal on your computer. On Windows that's Command Prompt or PowerShell, on Mac/Linux it's Terminal.
ssh [username]@192.168.1.42Replace the IP with yours. First time it'll ask if you want to continue connecting — type yes and hit Enter. Then enter your password. You won't see anything appear while you type — that's normal.
If you see [username]@billybot:~$ you're in. Everything from here runs in this window.
ssh-keygen -R 192.168.1.42 (your actual IP), then SSH again normallysudo apt updatesudo apt upgrade -yThe first fetches the list of available updates. The second installs them. The -y skips every confirmation prompt. Takes 5–10 minutes on first run — let it finish completely before moving on.
ping google.com. If that fails, the Pi isn't on the internet — recheck your WiFi credentialssudo dpkg --configure -a, then retry sudo apt upgrade -ysudo apt install python3-pip espeak espeak-ng espeak-ng-data portaudio19-dev flac sox git -yespeak + espeak-ng + espeak-ng-data — the voice engine pyttsx3 needs. You need all three. Installing only espeak causes pyttsx3 to appear to work but produce zero sound, with no useful error to tell you whyportaudio19-dev — lets Python communicate with the microphoneflac — required by speech_recognition. Without it, Google STT silently does nothingsox — for creating the keep-alive silence file in Step 14git — useful for moving files aroundsudo apt update first to refresh the package list, then retry the installsudo apt install -f to auto-fix, then retrypip install SpeechRecognition pygame pyttsx3 requests vosk word2number --break-system-packagesThe --break-system-packages flag is required on current Raspberry Pi OS — newer versions block pip from installing to system Python by default. It sounds alarming. It's fine. Wait for it to finish — as long as it ends with Successfully installed and no red ERROR: lines, you're good.
--break-system-packages flag. Copy and paste the full command exactly as writtensudo apt install python3-pygame -ysudo apt install ca-certificates -y, then retry the pip installGo to gnews.io → sign up for a free account → copy your API key. You'll paste it into the code in Step 13.
Install Bluetooth audio packages:
sudo apt install pulseaudio pulseaudio-module-bluetooth bluez -ysudo rebootSSH back in after about 60 seconds. Start PulseAudio:
pulseaudio --startOpen the Bluetooth controller:
bluetoothctlYou'll see a [bluetooth]# prompt. Run these one at a time:
power on
agent on
default-agent
scan onPut your speaker into pairing mode now. Devices will appear with their MAC addresses (XX:XX:XX:XX:XX:XX). When you see your speaker's name:
pair XX:XX:XX:XX:XX:XX
trust XX:XX:XX:XX:XX:XX
connect XX:XX:XX:XX:XX:XXIf it says Connection successful, type exit. Now set it as the default audio output:
pactl list short sinksFind the entry with bluez in the name. Then (colons in MAC become underscores):
pactl set-default-sink bluez_sink.XX_XX_XX_XX_XX_XX.a2dp_sinkTest it:
paplay /usr/share/sounds/alsa/Front_Left.wavIf you hear something, audio is working. Now create a reconnect script so Bluetooth re-pairs automatically on every boot:
nano /home/[username]/bt-connect.shPaste this in (replace MAC with yours):
#!/bin/bash
sleep 20
bluetoothctl connect XX:XX:XX:XX:XX:XXSave: Ctrl+X → Y → Enter. Make it executable:
chmod +x /home/[username]/bt-connect.shscan onNot in pairing mode, or too far from the Pi. Move it closer and try againremove XX:XX:XX:XX:XX:XX inside bluetoothctl to clear the attempt, then try pair againpaplay produces no soundDefault sink wasn't set. Redo the pactl set-default-sink stepbluez not appearing in pactl list short sinksRun pulseaudio --kill then pulseaudio --start, then reconnect the speaker in bluetoothctlpulseaudio --kill then pulseaudio --starttrust command. Run bluetoothctl trust XX:XX:XX:XX:XX:XXTest it works straight away:
aplay /usr/share/sounds/alsa/Front_Left.wavIf sound comes out the wrong output, run raspi-config → System Options → Audio → select the right one.
You can delete the keep_alive() function from the code entirely — it only exists to stop Bluetooth speakers from auto-shutting off. A wired speaker doesn't need it.
aplay -l to list output devices and confirm your speaker is detected. If it's not listed, try a different USB portraspi-config → System Options → Audio → manually select your output devicePlug in your USB microphone. Check the Pi can see it:
arecord -lYou should see something like:
card 1: Device [USB Audio Device], device 0: USB Audio [USB Audio]Note the card number. Record a 3-second test (replace 1,0 with your card number if different):
arecord -D hw:1,0 -f cd -t wav -d 3 test.wavTalk into the mic. Then play it back:
paplay test.wavIf you hear yourself, the mic works. Any USB mic that shows up in arecord -l will work — just use its card number.
arecord -l shows nothingMic isn't being detected. Unplug and replug it, then run arecord -l again. If still nothing, try a different USB portarecord -l carefully — the card number is the digit after "card"paplay plays through the wrong outputPulseAudio default sink reset. Redo the pactl set-default-sink step from Step 8mkdir -p /home/[username]/billy/VoicesThe -p flag creates both folders at once. Your project lives here:
whoami to confirm your actual usernamecd /home/[username]/billywget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zipunzip vosk-model-small-en-us-0.15.ziprm vosk-model-small-en-us-0.15.zipCheck it worked — run ls and confirm vosk-model-small-en-us-0.15 is there as a folder. Don't rename it. The code looks for it by that exact name.
unzip: command not foundsudo apt install unzip -y, then run the unzip command againCtrl+C and retry the wget commandmv [wrong-name] vosk-model-small-en-us-0.15Run these from your computer's terminal — open a new window for this, separate from the SSH session.
Transfer assistant.py:
scp /path/to/assistant.py [username]@192.168.1.42:/home/[username]/billy/Transfer your Voices folder:
scp -r /path/to/Voices/ [username]@192.168.1.42:/home/[username]/billy/Voices/Back in the SSH session, check everything arrived:
ls /home/[username]/billy/Voices/scp not recognised on WindowsUse PowerShell, not Command Prompt — Windows 10 and 11 have scp built into PowerShellmv /home/[username]/billy/Voices/Voices/* /home/[username]/billy/Voices/nano /home/[username]/billy/assistant.pyUse Ctrl+W to search. There are no hardcoded file paths to fix — the code uses os.getcwd() so it works for anyone as long as you run it from the right directory. You only need to update these:
GNEWS_KEY = "YOUR_KEY_HERE" and replace the value with your own key from Step 7latlong.netbirthday() function and update the target dates to your own, or delete the whole function if you don't want itSave: Ctrl+X → Y → Enter. Three separate keystrokes.
cd /home/[username]/billy/Voicessox -n -r 44100 -c 1 silence.wav trim 0.0 3.0ffmpeg -i silence.wav silence.mp3rm silence.wavThis creates a 3-second silent .mp3 file. The code looks for silence.mp3 specifically — sox creates wav by default so the ffmpeg conversion step is required. If ffmpeg isn't installed: sudo apt install ffmpeg -y.
sox: command not foundYou missed it in Step 5. Run sudo apt install sox -y, then try the sox command againcd command first, then soxcd /home/[username]/billypython3 assistant.pyThe cd is required — the code uses os.getcwd() to find the Voices folder and the Vosk model, so Python needs to be running from inside the billy directory. Running python3 /home/[username]/billy/assistant.py from anywhere else will fail to find both.
Vosk model loaded successfully.
Billy is online. Say 'billy' to wake.And hear your startup clip through the speaker. Say "billy hello" — if it responds, everything works. Hit Ctrl+C to stop.
FileNotFoundError for Voices or VoskYou didn't cd into the billy folder first. The code uses os.getcwd() — it only finds the files if Python is running from inside /home/[username]/billy/No module named XA pip install from Step 6 failed. Run pip install [module-name] --break-system-packages for just that packagepactl set-default-sink step from Step 8 and try againls /home/[username]/billy/ and confirm the folder is there and named exactly vosk-model-small-en-us-0.15arecord -l confirms the mic is still detected, then check speech_recognition is finding the right deviceALSA lib errors flooding the terminalHarmless warnings from pygame initialising audio. Ignore them — the bot still worksOSError: [Errno -9996] Invalid input devicePython can't find the microphone. Confirm the mic is plugged in and visible in arecord -lping google.com — if that fails, your WiFi dropped. Vosk will take over automatically but check your connectioncrontab -eIf it asks which editor, pick 1 (nano). Scroll to the very bottom and add your lines.
@reboot /home/[username]/bt-connect.sh
@reboot sleep 30 && pulseaudio --start && sleep 10 && cd /home/[username]/billy && python3 assistant.py >> /home/[username]/billy/billy.log 2>&1 &@reboot sleep 15 && cd /home/[username]/billy && python3 assistant.py >> /home/[username]/billy/billy.log 2>&1 &The sleep gives the Pi time to connect to WiFi before the script starts. The >> billy.log saves all output to a log file — if it breaks on boot, that's where you'll find out why.
Save: Ctrl+X → Y → Enter. Then reboot:
sudo rebootWait 60 seconds. Without touching anything — say "billy hello." If it responds, you're done.
If it doesn't, SSH back in and check the log:
cat /home/[username]/billy/billy.logcrontab -e again and check your lines are actually there at the bottomchmod +x /home/[username]/bt-connect.sh then reboot againsleep 30 isn't long enough. Change it to sleep 45 in crontab and rebootbluetoothctl trust XX:XX:XX:XX:XX:XX then rebootThis is a full breakdown of the code. It's all in one file, split into clear sections so you can paste it into a Python file and load it onto the Pi with the setup instructions. It looks simple from the outside but it's actually a carefully balanced system of audio playback, speech recognition, and controlled chaos. To view the full code, click here.
Every module the project needs, and what it actually does:
pip install SpeechRecognitionplay() function and basically what gives Billy its "personality".pip install pygamepip install requestssay() when there's no recorded clip, or when I don't want to hear my own voice again.pip install pyttsx3vosk-model-small-en-us-0.15.pip install voskpip install word2numberimport speech_recognition as sr
import os, random, pygame, requests
import pyttsx3, datetime, json, threading, time
from vosk import Model, KaldiRecognizer
from datetime import timedelta
from word2number import w2nEverything in Billy Bot ends up in one of two outputs. play() loads and plays a pre-recorded .mp3 clip from the Voices directory using pygame. say() is the text-to-speech fallback for dynamic data — temperature readings, timer durations, calculation results. Most responses are hybrid: a clip plays first for structure, then say() handles the variable part.
Both functions share an audio_lock — a reentrant lock (RLock) that prevents two things playing at once. It's reentrant specifically because if a clip is missing, play() calls say() internally. A regular Lock would deadlock at that point since the same thread can't acquire it twice. RLock allows it.
Clip filenames don't need the .mp3 extension — play() appends it automatically. VOICE_PATH uses os.getcwd() so the path is portable — no hardcoded username anywhere.
VOICE_PATH = os.path.join(os.getcwd(), "Voices")
audio_lock = threading.RLock()
def play(filename):
with audio_lock:
if not filename.endswith(".mp3"):
filename += ".mp3"
path = os.path.join(VOICE_PATH, filename)
if os.path.exists(path):
pygame.mixer.music.load(path)
pygame.mixer.music.play()
while pygame.mixer.music.get_busy():
time.sleep(0.05)
else:
say("Missing audio file")
def say(text):
with audio_lock:
pygame.mixer.music.stop()
speaker.say(text)
speaker.runAndWait()Billy has a set of functional systems that make it feel like an assistant instead of just a soundboard. Weather pulls live data from Open-Meteo (no API key needed — I value convenience over bureaucracy) and reads back temperature, conditions, and wind speed for Brisbane. Time and date come straight from the system clock. News uses the GNews API to fetch a real article on whatever topic you ask about. The calculator converts natural speech into evaluatable math expressions using word-to-operator replacement and eval(). All rule-based logic — but structured to feel responsive and "intelligent" without actually using AI.
The one I'm most proud of. It understands any combination of hours, minutes, and seconds in natural language — "set a timer for 2 hours 15 minutes and 30 seconds", "remind me in 45 minutes", "timer for an hour and a half" — all of it works. It scans the word list, converts number-words to integers using word2number, then accumulates total seconds based on whatever unit word follows each number. threading.Timer fires the buzzer clip when time's up, without blocking the rest of the program.
def timer(speech):
words = speech.lower().split()
total_seconds = 0
last_number = None
for word in words:
try:
last_number = w2n.word_to_num(word)
except ValueError:
if last_number is not None:
if word in ["second", "seconds", "sec", "secs"]:
total_seconds += last_number
last_number = None
elif word in ["minute", "minutes", "min", "mins"]:
total_seconds += last_number * 60
last_number = None
elif word in ["hour", "hours", "hr", "hrs"]:
total_seconds += last_number * 3600
last_number = None
threading.Timer(total_seconds, play, args=["buzzer.wav"]).start()Billy supports interactive modes like number guessing and truth or dare — games that need multiple back-and-forth turns. Instead of nested loops (which would've locked the whole program), it uses a single global state variable: current_game. When a game starts, current_game gets set to "number_guess" or "truth_or_dare", and the main listening loop checks it before calling respond() — routing input to the right game handler instead. The system stays responsive and keeps running normally. When the game ends, current_game resets to None.
Billy prioritises Google Speech-to-Text because it's fast and accurate — but it needs internet and isn't always reliable. So there's a fallback: if Google throws an sr.RequestError, the system switches to Vosk, which runs entirely on the Pi with no internet required. If both fail, input is ignored and the loop continues.
Two things worth knowing about the main loop structure. First, the entire loop runs inside a single with sr.Microphone() block — opening and closing the USB mic device every iteration causes unnecessary overhead and can throw "device busy" errors on the Pi. One block, open for the whole session. Second, sr.WaitTimeoutError is caught separately from the speech errors — if the mic hears nothing for 5 seconds it raises this, and without catching it the whole program would crash silently.
Full feature spec. Say "billy" then any of these.
The honest log.
Placeholder — your words here.
Placeholder — your words here.
Placeholder — your words here.
Placeholder — your words here.