The world of AI is extremely fast moving. New models, new technologies and new jargons seem to crop up everyday.

This article seeks to explain what RAG - retrieval augmented generation - is in simple terms, why it is emerging as a game-changer, and provide a side by side comparison of RAG and ChatGPT in action, so readers can better understand the merits of both.

Read more on RAG-as-a-Service here.

How LLMs like ChatGPT works

Without getting into too much technical details, LLMs like ChatGPT work by learning from millions and billions of data points within its training data, and outputting a response that it predicts as the most plausible based on its knowledge.

LLMs work extremely well on certain conditions: having been trained on a massive volume of written words, ChatGPT can produce high-quality writing in professional, business-appropriate tone, or Shakespearean prose if you prefer. ChatGPT can also ace tests on subjects that it has been trained on.

However, LLMs also have some glaring weaknesses: given they are trained on a static data set, they lack access to the latest, or private/ proprietary, facts and knowledge. They are also known to “hallucinate”, generating plausible responses that are not based on accurate information, or even outright fabricating facts.

How RAG works

Enters retrieval augmented generation (RAG). RAG is a relatively new technology that started entering discussions in the AI space around 2024. It combines the prowess of large language models (LLMs) with external knowledge sources to generate more accurate and contextual relevant responses.

It does so by adding a few more steps in its architecture:

1. Knowledge base and indexing:

• The first step is constructing a database and converting that knowledge base - which could contain text, images, audio, and more - into numerical representations (vectors) and storing in a vector database.

2. Retrieval:

• The second step is retrieval. Whenever a RAG-enabled AI model, agent or application receives a query or request, it will first search for and retrieve the relevant blocks of information from the vector database.

3. Generation:

• Finally, it will generate a response based on the retrieved information.

In short, think of RAG as a helpful librarian who goes to find the most relevant information before generating a summary in their response.

Benefits of RAG

In theory, RAG’s more nuanced approach brings a few key benefits:

1. More relevant, contextual response

• LLMs are trained on billions of data points. Due to the sheer size of their knowledge, when you try to ask a domain-specific question, the LLM may return a response that is too general or irrelevant to the specific niche you’re looking for.
• In contrast, with RAG, you can guide the LLM to only retrieve relevant information from a selected body of knowledge, which helps to hone the focus of its response.

2. Higher reliability, mitigates hallucination

• With LLMs, reliability of their outputs can be hit and miss.
• With RAG, you have control over the pool of data that the LLM is drawing from, and you can also trace back the sources of its response through citations.

3. Access to private, specialized, or dynamic data

• Finally, LLMs like ChatGPT are trained on publicly available data or data that the company (e.g. OpenAI or Meta) has permission to train on. Their training data set is also static and usually lags real-time data.
• RAG implementations, in contrast, allows the LLM to be connected to up-to-date knowledge sources, ensuring access to current information. Organizations can also rely on RAG methods to safely and securely connect proprietary, specialized knowledge with LLMs to reap benefits.

With Ragdoll AI, you can enjoy these features out-of-the-box, no code needed.

RAG vs ChatGPT In Action

Well, theory is theory. How does RAG and ChatGPT match up in action?

While there are many studies and literature out there that compares benchmark data between the two and confirm the benefits of RAG, we at Ragdoll AI decided to run a quick test for fun anyway. Here we go.

The Task:

We asked both Ragdoll and ChatGPT to generate a blog article based on three inputs: target keywords, reader intent, and writing style.

The goal is to create a blog article that articulates the difference between LightRAG and GraphRAG - two advanced retrieval-augmented-generation techniques - in language that is easy to understand for a non-technical audience.

What we fed Ragdoll AI:

In the Ragdoll AI knowledge base, we have previously connected a few authoritative sources on GraphRAG and LightRAG, including this original research paper on LightRAG.

The Result:

Here’s a snippet of the two articles produced:

Analysis:

In the ChatGPT version, it gave an elaborate introduction of a “cutting-edge, cloud-based data analytics platform” that utilizes “a blend of machine learning algorithms and data visualization techniques” to make complex data understandable.

It went on for 1200 words describing this “analytics platform”, but makes no mention whatsoever of the keyword, “retrieval-augmented-generation”.

In short - the ChatGPT version is a highly plausible article based on totally fabricated facts.

Meanwhile, the article produced by Ragdoll AI successfully retrieved the relevant definitions of LightRAG: an advanced RAG method with two primary features, graph-based indexing and dual-level retrieval framework. It even provided linked sources if you want to check out where it got the information from.

Conclusion

Our exercise once again demonstrated the advantage of RAG technology over a pure LLM workflow, showing how the combination of curated knowledge and LLM can truly unleash the maximum potential of generative AI in business applications.

Want to give RAG a try? Ragdoll AI provides an easy-to-use platform where you can simply upload or connect your data to create and chat with your private knowledge base in minutes.

‍