https://blog.streamlit.io/build-a-real-time-rag-chatbot-google-drive-sharepoint/

In this post, we explore how to build a real-time RAG app with up-to-date information from your files stored in Google Drive or Sharepoint. This means that your chatbot will always have access to the most recent version of your knowledge base—no manual pipeline reruns needed. By the end of this tutorial, you’ll use Pathway and LlamaIndex to build a RAG chatbot that instantly updates.

Why Pathway?

Pathway is an open data processing framework. It allows you to easily develop data transformation pipelines and ML apps that work with live data sources. Pathway listens to your documents for changes, additions, and removals. It handles loading and indexing without ETL.

Pathway offers an indexing solution that is always up-to-date without the need for traditional ETL pipelines. It can monitor several data sources (such as files, S3 folders, and cloud storage) and provide the latest info to your LLM app.

This means you don’t need to worry about:

• Checking files to see if there are any changes
• Parsing PDFs, Word documents, or other text files
• Transforming, embedding documents, and loading them into a vector database

Once updates are made to the files that make up your knowledge base, the updated content is immediately re-indexed — you don’t have to deal with rerunning the pipeline.

App overview

This demo consists of three parts.

1. For up-to-date knowledge and information retrieval from the knowledge base’s documents, Pathway’s vector store is used.
2. LlamaIndex creates the RAG pipeline and offers chat memory.
3. Streamlit powers the easy-to-navigate user interface.

Tutorial: Creating a real-time RAG app with Pathway + LlamaIndex

🏂

Want to jump right in? Check out the app to see how it works or explore the code in the GitHub repository.