Large pre-trained language foundation models (LLMs), such as ChatGPT, that are pre-trained offline have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream natural language processing (NLP) tasks. However, their ability to access and precisely manipulate knowledge, especially with up-to-date knowledge, is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures.

To overcome such limitations, Retrieval Augmentation Generation (RAG) is a technique that can be utilized to retrieve data from outside a foundation model, and as such, it can be used for augmenting the prompts by injecting the relevant retrieved data into the context. RAG has proven to be more cost-effective and efficient than pre-training or fine-tuning foundation models, and can help to reduce hallucinations in LLMs.

We’ll take a look at how utilizing an event-driven streaming approach, by using a new open source library, LangStream, can quickly integrate your existing data in motion into generative AI applications such as with prompt engineering and the RAG pattern. LangStream can be used with Apache Kafka or Apache Pulsar underneath as the event streaming and message broker platform.