Here's how to build your own real-time Jarvis voice assistant

Here's how to build your own real-time Jarvis voice assistant using the ChatGPT API. We create three functions: speak, listen, and ask_ai. Together they capture your voice, transcribe it, send the text to the LLM, and play back the response as speech using a text to speech model on the response. By passing a simple instruction to act like an intelligent assistant named Jarvis and picking a de First, we begin with the imports. For this project, we need to install and import speech recognition, sound device, sound file, numpy, and OpenAI. client = OpenAI(api_keys="sk-proj-FRhtlhdgrS1MthwUjZ3kbadmaXt-HL-P This provides us with the chat model and the text to speech API. It takes three functions to make this project work. Speak, listen, and ask AI. We start with the speak function. So in here, we pass the text which will be the API response into OpenAI's speech endpoint and we specify the model, which should be a fast real-time one, the voice, which we set to onyx. Then we can add a custom instruction to make the model sound like a calm, intelligent AI assistant named Jarvis. The response comes back in raw audio bytes. We can't play that directly. So we wrap it in a BytesIO object. Data, samplerate = sf.read(audio_bytes). sd.play(data, samplerate). sd.wait(). The wait function at the end makes sure the audio will be fully done before continuing. Next, we define the listen function. We tell sound device to record for a duration of 5 seconds at a 16000 hertz sample rate. In here, we use the rec function to pass these inputs. We set the channels to 1 and store the result in 16-bit integers. That combination is the standard format expected by speech recognition. But here's the thing, sound device provides us with a NumPy array, not a file. So we write that array as a WAV file. audio_bytes.seek(0). r = sr.Recognizer(). with sr.AudioFile(audio_bytes) as source: audio = r.record(source). return r.recognize_google(audio). If it gets silent or something it can't parse, we could add a simple error handler here. Let me know if you want to see this. Now to the ask AI feature. We then send the transcribed text to the LLM through the responses.create endpoint. We provide the model and input text. And return response.output_text. Nice and clean. And the main loop. It runs forever while true. In each iteration, we call listen. And if we get None back, we skip everything with continue. print("You said: " {text}) reply = ask_ai(text) print("Jarvis: " {reply}) speak(reply). This is just a basic script. The listen time isn't perfect and the text to speech can take some time by design. But here's the explanation of how Python passes variables to functions. Jarvis: Python uses a mechanism often referred to as "pass by object reference" or "pass by assignment." This means that when you pass a variable to a function, you are passing a reference to the object in memory, not the actual object itself. Here's a breakdown: -Mutable objects (like lists and dictionaries): If you modify the object within the function, those changes will be reflected outside the function since both the original variable and the function parameter point to the same object. -Immutable objects (like integers, strings, and tuples): If you try to modify an immutable object within a function, a new object is created, and the function parameter will point to this new object. The original variable outside the function remains unchanged.