Qlippy: A Retrieval-Augmented GenAI Assistant for Reproducible Quantum Workflows and Experiment Tracking

Summary

The paper presents Qlippy, a Visual Studio Code extension embedding a retrieval-augmented generation (RAG) assistant into quantum software development to promote experiment tracking and provenance practices. A FastAPI/LangChain service backed by a ChromaDB vector store holds a small curated corpus — five research papers, two synthetic reference documents, and a subset of the MLflow documentation — and pairs with a real MLflow tracking server. The authors demonstrate two scenarios: an advisory mode explaining tracking concepts in context, and an augmentation mode that rewrites Qiskit programs to insert MLflow logging aligned to the QProv provenance schema. A preliminary evaluation applies the RAGAS LLM-as-a-judge framework (GPT-5.4-mini as judge) to 20 question-answer pairs spanning five subdomains and four question types, comparing four generators: GPT-4o-mini and GPT-5.4 via the OpenAI API, plus Gemma4:e4b and Qwen3.5:9b served locally via Ollama. Locally hosted Qwen3.5:9b achieved the best answer relevancy (0.917), while GPT-5.4 achieved the best factual correctness (0.908 versus Qwen3.5:9b's 0.814). The authors conclude that grounded local models can approach commercial ones, motivating low-cost, privacy-preserving deployment. Main limitations: no statistical testing or confidence intervals on the small question set; the augmentation scenario is not quantitatively evaluated; no ungrounded-model or general-assistant baselines; no user study; and no public artifact or evaluation data. The authors themselves call the results preliminary and relative.