Building LLM Applications with LangChain and Streamlit

The field of artificial intelligence (AI) and large language models (LLMs) is evolving rapidly. These technologies have the ability to completely transform the way we understand and interpret large volumes of data. With the likes of ChatGPT, it’s already shown huge advancements in tasks such as text summarization, question answering, document classification, content generation, and much more.

If you are fascinated by the transformative capabilities of generative AI and LLMs, you will greatly benefit from this how-to guide to bootstrap your journey using LangChain and Streamlit. Throughout the article, we will unravel the myth around these new libraries and concepts, how they fit together, and finish with a step-by-step guide to build an application, like chatpdf.com which allows you to chat with any PDF file, from scratch.

For the inpatient, this is what you can expect at the end of this tutorial.

Disclaimer: This article is intended to be a quick starter guide rather than a detailed walkthrough.If you want more detailed examples and usages of LangChain, check out my other notebooks. Without further delay, let’s get started.

What is LangChain

LangChain is an open-source framework for developing applications powered by large language models (LLMs). Its recent and humble start doesn’t stop it from being some of the most popular open source projects on GitHub, including mentions from the likes of Google, Amazon and Microsoft.

At its core, LangChain serves as a bridge between LLMs like OpenAI and HuggingFace, and external data sources including raw files (PDFs, presentations, spreadsheets, etc), APIs (Salesforce, Slack, Discord and more), databases (SQL and vector stores), using an agentic approach. LangChain offers a suite of modules, such as prompt templates, memory management, document loaders and output parsers, in order to achieve its goal. These modules build on top of seven fundamental components:

• Models: Supports both a range of LLMs.

• Prompts: Offers prompt templates for accepting user input and output parsers to format the LLM model's responses.

• Indexes: Structures and prepares data to interact with LLMs.

• Memory: Equips chains or agents with short-term and long-term memory, allowing them to recall previous user interactions.

• Chains: Provides a mechanism to combine multiple components or chains into a single pipeline, streamlining the application's workflow.

• Agents: Makes decisions on a course of actions based on input and the available tools and data at their disposal.

• Callbacks: Triggers specific functions at defined points during LLM execution.

You can also refer to this tutorial for a more detailed walk through of each component.

Architecture

To bootstrap a working application, you will be using OpenAI as the LLM, LangChain being the orchestrator of user input and OpenAI, Steamlit to provide a user friendly web frontend, and we will also need FAISS as our local data storage. And this is what it looks like.

Create OpenAI API key

First, you need to create your OpenAI API key:

1. Visit the OpenAI API key management page at https://platform.openai.com/account/api-keys

2. Click on the "+ Create new secret key" button.

3. Optionally, provide an identifier name for your key.

4. Click on the "Create secret key" button to generate your API key.

5. Copy the API key for use in this tutorial. 

Note: please ensure to keep your API key secure and do not share it with unauthorised individuals.

Set up local Environment

Insert the OpenAI API key into .env file using the following command

echo "OPENAI_API_KEY='your-api-key'" >> ~/.env

Install Python and dependencies

You can find out if you have Python installed and which version is installed via python -V.

The simplest way to install python on a Mac is run brew install python@3.10

 in the terminal. Python version 3.10 is the version used when this instruction is written, if you are an advanced user, feel free to use pyenv to manage your own versioning.

Now, we need to install the libraries mentioned.

pip install openai langchain streamlit

TL;DR

To give you a sneak peek, this is all the code needed to build the application, which is less than 50 lines of code. You can also find this on my GitHub.

import dotenv

import os

import streamlit as st

import tempfile

from langchain import OpenAI

from langchain.text_splitter import RecursiveCharacterTextSplitter

from langchain.embeddings.openai import OpenAIEmbeddings

from langchain.vectorstores import FAISS

from langchain.document_loaders import PyPDFLoader

from langchain.chains import RetrievalQA

dotenv.load_dotenv()

openai_api_key = os.getenv("OPENAI_API_KEY")

st.header("Chat PDF")

file = st.file_uploader("Upload a PDF file", type=["pdf"])

if file is not None:

with tempfile.NamedTemporaryFile(delete=False) as tf:

tf.write(file.getbuffer())

file_path = tf.name

doc = PyPDFLoader(file_path).load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)

docs = text_splitter.split_documents(doc)

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)

knowledge_base = FAISS.from_documents(docs, embeddings)

question = st.text_input("Ask your question here:")

if question:

llm = OpenAI(temperature=0, openai_api_key=openai_api_key)

chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=knowledge_base.as_retriever())

response = chain.run(question)

st.success("Completed question.")

st.write("Answer: ", response)

Step-by-step walk through

Now, let me actually walk you through the code and help you understand better. Following the local environment setup from the previous step, you need to load up the OpenAI API key programmatically.

import dotenv

import os

dotenv.load_dotenv()

openai_api_key = os.getenv("OPENAI_API_KEY")

Because we are using a low-code approach so that you don’t have to deal with a lot of frontend boilerplate. Simply import Streamlit and get started.

import streamlit as st

file = st.file_uploader("Upload a PDF file", type=["pdf"])

Coming up next is the main action. It may look a bit succinct at first, however, allow me to elaborate. 

Without much surprise, you need to import a bunch of dependencies from LangChain first.

import tempfile

from langchain import OpenAI

from langchain.text_splitter import RecursiveCharacterTextSplitter

from langchain.embeddings.openai import OpenAIEmbeddings

from langchain.vectorstores import FAISS

from langchain.document_loaders import PyPDFLoader

from langchain.chains import RetrievalQA

Once we can confirm a PDF file has been uploaded, we need to extract the bytes to a commonly used data object called Document in LangChain.

with tempfile.NamedTemporaryFile(delete=False) as tf:

tf.write(file.getbuffer())

file_path = tf.name

doc = PyPDFLoader(file_path).load()

Typically you will be using LLM to query a large file which can be thousands of lines, tens of thousands of words. In which case, you will exceed the LLM context window limit, unless you have the luxury of using claude-instant-100k, then forget what I said. For those who are using OpenAI, this process of breaking a large file into chunks and creating embeddings for each chunk is called vectorisation.

The large body of text is split into smaller chunks by a fixed length, which is under the token limit. You can also do the chunking by token, follow this insightful tutorial if you want to learn how.

text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)

docs = text_splitter.split_documents(doc)

To create embeddings for the text chunks, you need to invoke OpenAI. And the result is stored in a local vector store, which we use FAISS in this case. There are plenty more other choices available for you to integrate with, all via LangChain.

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)

knowledge_base = FAISS.from_documents(docs, embeddings)

Once everything is set, we can now query the file directly using natural language.

llm = OpenAI(temperature=0, openai_api_key=openai_api_key)

chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=knowledge_base.as_retriever())

response = chain.run(question)

Looks pretty straightforward, right? Let’s run up the application and see it in action.

Running the App

Create a file called “app.py”, and save all the code. Then run this command in the terminal to start Streamlit frontend.

streamlit run app.py

Your own ChatPDF is now deployed and ready to use! Have fun!

Final words

Now you understand what’s involved in building a basic LLM application with LangChain and Streamlit, I would like to encourage you to play with it more, and build other amazing LLM applications. Just bear in mind, the only limit is your own imagination.

You can find the complete solution and other resources in my GitHub. If you’d like to explore other LangChain use cases, be sure to check out my other tutorial. I hope you enjoy the immersive learning experience.

If you have any questions or feedback, or are keen to get involved in building, don’t hesitate to reach out to me on Twitter or LinkedIn, let’s have a chat!