LLM-based Conversational AI Knowledge Assistant for MyBuddy Humanoid Robot

Summary

This paper presents a system-level integration of an LLM-based voice assistant on the MyBuddy 280, a small dual-arm humanoid robot driven by a Raspberry Pi with 13 actuated axes. The pipeline is a cascaded speech-to-speech architecture: Silero VAD performs voice activity detection, Whisper-Fast transcribes speech, Llama 3.1 8B runs locally via Ollama and is orchestrated by the Agno agent framework with tool calls to Google Search, Wikipedia, and arXiv, and Kokoro-82M synthesizes the spoken reply. A keyword layer maps five spoken keywords ('hello', 'love', 'time', 'talk', 'bye') to facial-expression videos and switches into assistant mode. Evidence consists of demonstration dialogues on topics such as travel, finance, and science, a dialogue screenshot figure, and one supplementary video; the reported LLM execution times for individual queries range from about 12 to 42 seconds. The authors argue that this cascaded design is more accurate, reliable, and tool-extensible than direct speech-to-speech models, but they provide no comparative experiment, baseline, user study, or evaluation metrics. Main limitations: the evaluation is anecdotal, the 12-42 s latencies are long for natural conversation yet framed as efficient, all heavy inference runs on an external GPU PC rather than the robot, and no code or data are released.