If you\u2019re one of the cool kids building AI-based products you\u2019ve probably heard of \u2014 or are already doing \u2014 RAG. If you\u2019re not, let me tell you what RAG is before telling you one weird fact about it.<\/p>\n\n\n\n
\u201cRetrieval-Augmented Generation\u201d is nothing more than a fancy way of saying \u201cincluding helpful information in your LLM prompt.\u201d Still, there are many ways to do it and many questions to answer when building a RAG pipeline. I mentioned some of these things in my recent article about how to build reliable systems using unreliable agents<\/a>.<\/p>\n\n\n\n If you\u2019ve looked into it, you might\u2019ve realized that setting up a RAG pipeline introduces a bunch of complexity to your already-complex AI system. The one weird trick and the good news I\u2019m bringing you is that your data warehouse provider already has a bunch of recently-added goodies to help you, which could significantly simplify your system.<\/p>\n\n\n\n Besides the fact that most databases offer built-in full-text-search, there are now also utilities to compute embeddings and do vector search \u2014 and that\u2019s pretty much all you need for your RAG pipeline. Some even offer multimodal embeddings, so you can search for semantically-similar items across text and images.<\/p>\n\n\n\n In this post I\u2019ll walk through the high-level steps of building a RAG pipeline, illustrate it by describing how we built ours, and then show how we improved it by relying more on built-in BigQuery features. (For context, we\u2019re building an AI agent system to write and update automated software test scripts<\/a>.)<\/p>\n\n\n\n I\u2019ll focus on BigQuery, since that\u2019s what we use and have the most experience with. But the same general techniques are applicable to other products, too.<\/p>\n\n\n\n