OpenAI Open Sources Plant Talk: Connect a Microphone and Sensors to Chat with Your Potted Plants in Real Time

According to Dynamic Beating monitoring, the OpenAI open-source education project Plant Talk integrates Realtime API, multimodal vision models, and Arduino microcontrollers to enable indoor plants to have environmental awareness and voice interaction capabilities.

The project supports low-latency full-duplex voice interaction with plants through a computer camera and microphone, and can interface with Arduino for capacitive soil moisture and LM393 light sensor readings. The plant's AI character (such as the default character George) retrieves hardware data through function calling. When users inquire about water or light exposure, the AI calls tools to retrieve real-time physical metrics, estimates plant moisture level, branch count, and maximum span from the captured images (supporting the use of a 25-cent coin as a size reference), and responds in a human-like plant voice.

Regarding security and interface design, the backend Express server securely delegates WebRTC connection to the frontend by issuing ephemeral client keys, avoiding exposure of the API master key on the client side. Visual recognition and timed observations are combined with the Responses API using the Zod pattern to achieve structured output. The codebase also includes an operation guide AGENTS.md for AI agents, guiding the AI assistant in assisting users step-by-step through software environment configuration and hardware assembly.