LangChain Tutorial: Showcasing LangChain Use Cases
This tutorial dives straight into implementing LangChain use case using a step-by-step approach, and hopefully inspire you to build.
Summarization¶
Langchain summarization is one of the most common usage of LangChain and LLMs. It can summarize any amount of text or documentations, including the ones that exceeds the token limit (currently set at 4096 tokens for gpt-35-turbo), which roughly equates to 3000 words.
The technique used is similar to the concept of sliding window, which a fixed size window, usually under the max context window limit, is used to chunk the the long document into smaller pieces, then summarize the content recursively to produce the final summary using mapreduce.
You can use it to summarize not only text, books, documents, audios, social media threads, and etc. Some use cases including: articles, research pagers, legal and financial documents, transcripts, chat history, customer interactions, product reviews, podcasts, twitter threads and more.
import dotenv
import os
import pdfplumber
from langchain.llms import OpenAI
from langchain.chains.summarize import load_summarize_chain
from langchain.text_splitter import RecursiveCharacterTextSplitter
dotenv.load_dotenv()
openai_api_key = os.getenv("OPENAI_API_KEY")
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
Now, let's load up The Tragedy of Hamlet
with pdfplumber.open("hamlet.pdf") as pdf:
text = ""
for page in pdf.pages:
text += page.extract_text()
print(text[:300])
The Tragedy of Hamlet, Prince of Denmark ASCIItextplacedinthepublicdomainbyMobyLexicalTools,1992.SGMLmarkupbyJonBosak, 1992-1994.XMLversionbyJonBosak,1996-1999.SimplifiedXMLversionbyMaxFroumentin,2001.The XMLmarkupinthisversionisCopyright©1999JonBosak.Thisworkmayfreelybedistributedoncondition thatit
By outputting the total number of tokens of the book with get_num_tokens, you can clearly see it exceeds the context window limit.
n_tokens = llm.get_num_tokens(text)
print(f"total number of tokens: {n_tokens}")
total number of tokens: 53929
The process starts by splitting the text into multiple parts by chunk size using RecursiveCharacterTextSplitter, with a coverlap of each adjacent chunk of text.
text_splitter = RecursiveCharacterTextSplitter(separators=["\n\n", "\n"], chunk_size=2000, chunk_overlap=200, length_function = len)
docs = text_splitter.create_documents([text])
print(f"number of docs: {len(docs)}")
number of docs: 100
Now the docs are ready, we can then load up a chain to do produce the sum of sums.
chain = load_summarize_chain(llm = llm, chain_type = 'map_reduce')
result = chain.run(docs)
print(result)
In The Tragedy of Hamlet, Prince of Denmark, Hamlet is on a quest for revenge against his uncle, Claudius, who has usurped the throne. After encountering a ghostly figure, Hamlet learns that his father was murdered by his uncle and is determined to avenge his death. King Claudius and Lord Polonius devise a plan to spy on Hamlet and Ophelia, and Hamlet kills Polonius. Laertes returns from France and challenges Hamlet to a fencing match, during which Laertes wounds Hamlet with a poisoned sword. Hamlet stabs King Claudius with a poisoned sword and forces him to drink a poisoned potion. Prince Fortinbras arrives and orders that the bodies be placed on a stage for all to see, and then claims his rights of memory in the kingdom.
When instantiating the chain, you can also add Verbose = True to see the steps LangChain took before producing the final summary, i.e. generating a summary per each chunk of text, and produce the final summary by aggregating the 100 summaries with map reduce.
This is a powerful technique to overcome the token limitation. Even though more powerful models will be released in the future to support more tokens, such as Claude now supports 100K tokens as of 11 May 2023, biggest of the its kind. There is not guaranteed better performance, accuracy or cost-effectiveness.
Moreover, you can parallelise the calls to super charge summarization using batch_size when instantiating LLMs.
Question Answering Over Documents¶
Similar to how humans answer questions, first you need to provide some context to LLMs. The context can be in many different formats, such as text documents, SQL database, APIs, etc. For the purpose of this exercise, we will focus on dealing with text documents input as context.
However, as the context grows, it naturally exceeds the token limit (again, just like in the case of summarization), and it gets harder and harder to give accurate answer quickly. The problem is solved by converting the context and your question into embeddings, basically a list of vector value representation of information; then find out a few results that are closely related to your question before giving out a final answer. The process to do the comparison is via an algorithm called cosine similarity. If you are getting confused by the terminologies, don't worry. The formula is as following:
llm(context + question) = your answer.
With this capability, you can chat with your documents, ask questions to papers, create study guides, reference medical information and more.
from langchain import OpenAI
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.document_loaders import PyMuPDFLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
Now, let's load The Tragedy of Hamlet again.
loader = PyMuPDFLoader("hamlet.pdf")
doc = loader.load()
print(f"number of pages in the doc: {len(doc)}")
total_chars = sum([len(page.page_content) for page in doc])
print(f"number of characters in the doc: {total_chars}")
number of pages in the doc: 142 number of characters in the doc: 179843
Now, we can split the PDF into chunks based on our definition.
text_splitter = RecursiveCharacterTextSplitter(chunk_size = 2000, chunk_overlap = 200)
docs = text_splitter.split_documents(doc)
print(f"number of splitted pages: {len(docs)}")
print(f"average number of characters in each page: {total_chars / len(docs):,.0f}")
number of splitted pages: 143 average number of characters in each page: 1,258
Then convert the documents into embeddings by calling OpenAI API Embedding engine, and store the result in a local vector store called FAISS. You can choose from a number of supported vector stores (local or remote).
embeddings = OpenAIEmbeddings(openai_api_key = openai_api_key)
knowledge_base = FAISS.from_documents(docs, embeddings)
Create a retrieval QA engine to perform queries.
qa = RetrievalQA.from_chain_type(llm = llm, chain_type = "refine", retriever = knowledge_base.as_retriever())
query = """What is the subject of Hamlet's second soliloquy, the famous "To be or not to be" speech?"""
qa.run(query)
'\n\nThe subject of Hamlet\'s second soliloquy, the famous "To be or not to be" speech, is the contemplation of life and death and the decision of whether to take action against the troubles of life or to accept them. Hamlet is reflecting on the consequences of his inaction in the face of his father\'s death and his mother\'s remarriage, and is considering the idea of suicide as a way to escape his suffering. He is also questioning the value of life and death, and whether it is better to endure the pain of life or to end it. He is further considering the implications of his actions, and whether his life is worth the risk of taking action against his troubles. He is also questioning the power of death, and whether it is better to accept death or to fight against it. Additionally, Hamlet is questioning the nature of ambition and the idea of being "bounded in a nut shell" and whether it is possible to be content with one\'s life despite the troubles and suffering that come with it.'
By now, you should have a working example of QA over documents. You should also notice there are a number of similarities comparing to summarization, such as the way the context was loaded, splitted, before passing onto the chain.
You may also have noticed different chain types specification used, and there is a good reason for that. Before explaining further, understand that LangChain chain types are basically different ways you can connect LLMs. Here are the supported chain types:
map_reduce: It separates texts into batches, feeds each batch with the question to LLM separately, and comes up with the final answer based on the answers from each batch.map_rerank: It separates texts into batches, feeds each batch to LLM, returns a score of how fully it answers the question, and comes up with the final answer based on the high-scored answers from each batch.refine: It separates texts into batches, feeds the first batch to LLM, and feeds the answer and the second batch to LLM. It refines the answer by going through all the batches.stuff: The default chain type that uses ALL of the text from the documents in the prompt.
Because the chain type will actually affect the result, I highly recommend you try different types, and figure out which works best for your scenario.
Extraction¶
LLMs are very good at extracting structured information out of unstructured text, whether that's extracting one or multiple structured row from text to insert into database from a sentence, or extracting the correct API parameters from a user query.
In essence, it is about working with OutputParsers, which is responsible for specifying the schema a LLM should respond in, and then parsing their raw-text output into that structured format.
Without further ado, let's dive straight into an example.
from langchain.schema import HumanMessage
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
chat_model = ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo", openai_api_key = openai_api_key)
This step is to define the schema, or more specifically, format instructions, to be added to the prompt, so that LLMs can act accordingly.
response_schemas = [
ResponseSchema(name="author", description="The name of the author"),
ResponseSchema(name="books", description="The books published")
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
print(format_instructions)
The output should be a markdown code snippet formatted in the following schema:
```json
{
"author": string // The name of the author
"books": string // The books published
}
```
prompt = ChatPromptTemplate(
messages=[
HumanMessagePromptTemplate.from_template(
"""Extract the author and books.\n\n{format_instructions}\n{user_prompt}""")
],
input_variables=["user_prompt"],
partial_variables={"format_instructions": format_instructions}
)
query = prompt.format_prompt(user_prompt="I recently read the book The Tragedy of Hamlet written by William Shakespeare")
print (query.messages[0].content)
Extract the author and books.
The output should be a markdown code snippet formatted in the following schema:
```json
{
"author": string // The name of the author
"books": string // The books published
}
```
I recently read the book The Tragedy of Hamlet written by William Shakespeare
output = chat_model(query.to_messages())
result = output_parser.parse(output.content)
print (result)
print (type(result))
{'author': 'William Shakespeare', 'books': 'The Tragedy of Hamlet'}
<class 'dict'>
This is pretty cool example of data extraction and sanitisation, however, it is also quite basic. In real life scenario, you are likely to be dealing with more complex data and schema, such as nested structures. In which case, make sure to check out Kor, which provides the next level functionalities.
Querying Tabular Data¶
Aside from unstructured data, there is also a lot of structured data sitting in CSVs, SQL, whether that's financial data, report or models or gazillions of day-to-day data that every company has. LLMs can deal with that as well.
It requires a bit of prep in order to demonstrate this:
- To start with, I'll be using a
SQLitedatabase which basically storesGoodreads Top 100 Classics, this data comes from kaggle. Make sure you have SQLite installed or runbrew install sqlite3. - Use this to convert CSV data into a SQL INSERT script, results are kept in
goodreads.sql. - Run the command to insert the data into a database called
goodreads.db.
sqlite3 goodreads.db
sqlite> .read goodreads.db
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
This step will load up the data for the chain.
sqlite_db_path = "goodreads.db"
db = SQLDatabase.from_uri(f"sqlite:///{sqlite_db_path}")
Then we can query the database using natural language, and I'll enable verbose mode so that you can see the chain of thoughts there.
db_chain = SQLDatabaseChain(llm = llm, database = db, verbose = True)
db_chain.run("How many books achieved an average rating above 4?")
> Entering new SQLDatabaseChain chain... How many books achieved an average rating above 4? SQLQuery: SELECT COUNT(*) FROM goodreads WHERE avg_rating > 4; SQLResult: [(100,)] Answer: 100 books achieved an average rating above 4. > Finished chain.
' 100 books achieved an average rating above 4.'
Amazing! By using verbose mode, you can actually obverse the SQL query being constructed to extract the data based on our question, and the output in natural language as well.
Let's run the query using pandas to verify the result, in case any hallucination in the LLM.
import sqlite3
import pandas as pd
conn = sqlite3.connect(sqlite_db_path)
query = "SELECT COUNT(*) FROM goodreads WHERE avg_rating > 4;"
df = pd.read_sql_query(query, conn)
conn.close()
print(df.iloc[0,0])
100
And the result checks out! But that's not it. To query larger databases and more complex schema, we will need to introduce the use of agents, which typically involves running multiple sequential queries or error recovery. You can checkout the agent_toolkits for more details or I'll cover this concept when introducing Agents.
Code Understanding¶
This is probably one of the most sought after features every developer or anyone who's interested in building has been dreaming for. LangChain can parse GitHub repositories and generate new code using the mechanisms demonstrated so far, including embeddings, vector stores, conversational retriever chain and LLMs.
Without paying for GitHub Copilot, why not see what you can do with LLMs first?
import os
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.vectorstores import FAISS
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA
llm = ChatOpenAI(model="gpt-3.5-turbo", openai_api_key=openai_api_key)
Here we do the embeddings again in preparation for loading the project as context.
embeddings = OpenAIEmbeddings(openai_api_key = openai_api_key)
project = "pandas-ai"
docs = []
for path, dirs, files in os.walk(project):
for file in files:
try:
loader = TextLoader(os.path.join(path, file), encoding='utf-8')
docs.extend(loader.load_and_split())
except Exception as e:
pass
print(f"number of docs loaded: {len(docs)}")
number of docs loaded: 133
Now we create the embeddings and store them locally.
knowledge_base = FAISS.from_documents(docs, embeddings)
qa = RetrievalQA.from_chain_type(llm = llm, chain_type = "stuff", retriever = knowledge_base.as_retriever())
Now the fun begins, we gonna test whether LLM really understands the code and can help us code.
query = "How can I use Google PaLM2 as the LLM for queries?"
output = qa.run(query)
print(output)
To use Google PaLM2 as the LLM for queries, you can use the `GooglePalm` class provided in the code. Here's an example of how you can use it: ``` from pandasai.llm.google_palm import GooglePalm # Replace YOUR_API_KEY with your Google Cloud API key llm = GooglePalm(api_key='YOUR_API_KEY', model='models/text_bison_001') response = llm.call(instruction='Hello', value='world') print(response) ``` You need to replace `YOUR_API_KEY` with your own Google Cloud API key. You can also change the value of the `model` parameter to use a different model if necessary.
query = "Please write the code to instantiate Google PaLM2 as LLM, and only respond with code."
output = qa.run(query)
print(output)
``` from pandasai.llm.google_palm import GooglePalm llm = GooglePalm(api_key="your_api_key") ```
As you can see from the output, the result is pretty staggering. LLM plus LangChain can truly understand code, explain the functionalities, and it's able to produce some sensible code as a result of that.
However, this is a very small sample in terms of its coding ability, you can use it to bootstrap work, but don't rely on it to code for you! Until we reach AGI, we still need a sound logical-minded person to give directions.
Interacting with APIs¶
There are tons of information and operations sitting behind APIs, and be able to interact with these APIs can take us even further. LangChain can read and understand API documents, and act upon that knowledge based on user instructions.
from langchain.llms import OpenAI
from langchain.chains import APIChain
from langchain.chains.api import open_meteo_docs
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
In this case, I'll load up an API doc called OPEN_METEO_DOCS, which is used to query weather information, without needing to sign up or an api key, and use natural language to query the API.
api_chain = APIChain.from_llm_and_api_docs(llm, open_meteo_docs.OPEN_METEO_DOCS, verbose=True)
api_chain.run("What is the weather like right now in Edingburgh in degrees celsius?")
> Entering new APIChain chain... https://api.open-meteo.com/v1/forecast?latitude=55.953251&longitude=-3.188267&hourly=temperature_2m¤t_weather=true&temperature_unit=celsius {"latitude":55.96,"longitude":-3.18,"generationtime_ms":0.3180503845214844,"utc_offset_seconds":0,"timezone":"GMT","timezone_abbreviation":"GMT","elevation":69.0,"current_weather":{"temperature":13.8,"windspeed":13.5,"winddirection":74.0,"weathercode":0,"is_day":1,"time":"2023-05-22T14:00"},"hourly_units":{"time":"iso8601","temperature_2m":"°C"},"hourly":{"time":["2023-05-22T00:00","2023-05-22T01:00","2023-05-22T02:00","2023-05-22T03:00","2023-05-22T04:00","2023-05-22T05:00","2023-05-22T06:00","2023-05-22T07:00","2023-05-22T08:00","2023-05-22T09:00","2023-05-22T10:00","2023-05-22T11:00","2023-05-22T12:00","2023-05-22T13:00","2023-05-22T14:00","2023-05-22T15:00","2023-05-22T16:00","2023-05-22T17:00","2023-05-22T18:00","2023-05-22T19:00","2023-05-22T20:00","2023-05-22T21:00","2023-05-22T22:00","2023-05-22T23:00","2023-05-23T00:00","2023-05-23T01:00","2023-05-23T02:00","2023-05-23T03:00","2023-05-23T04:00","2023-05-23T05:00","2023-05-23T06:00","2023-05-23T07:00","2023-05-23T08:00","2023-05-23T09:00","2023-05-23T10:00","2023-05-23T11:00","2023-05-23T12:00","2023-05-23T13:00","2023-05-23T14:00","2023-05-23T15:00","2023-05-23T16:00","2023-05-23T17:00","2023-05-23T18:00","2023-05-23T19:00","2023-05-23T20:00","2023-05-23T21:00","2023-05-23T22:00","2023-05-23T23:00","2023-05-24T00:00","2023-05-24T01:00","2023-05-24T02:00","2023-05-24T03:00","2023-05-24T04:00","2023-05-24T05:00","2023-05-24T06:00","2023-05-24T07:00","2023-05-24T08:00","2023-05-24T09:00","2023-05-24T10:00","2023-05-24T11:00","2023-05-24T12:00","2023-05-24T13:00","2023-05-24T14:00","2023-05-24T15:00","2023-05-24T16:00","2023-05-24T17:00","2023-05-24T18:00","2023-05-24T19:00","2023-05-24T20:00","2023-05-24T21:00","2023-05-24T22:00","2023-05-24T23:00","2023-05-25T00:00","2023-05-25T01:00","2023-05-25T02:00","2023-05-25T03:00","2023-05-25T04:00","2023-05-25T05:00","2023-05-25T06:00","2023-05-25T07:00","2023-05-25T08:00","2023-05-25T09:00","2023-05-25T10:00","2023-05-25T11:00","2023-05-25T12:00","2023-05-25T13:00","2023-05-25T14:00","2023-05-25T15:00","2023-05-25T16:00","2023-05-25T17:00","2023-05-25T18:00","2023-05-25T19:00","2023-05-25T20:00","2023-05-25T21:00","2023-05-25T22:00","2023-05-25T23:00","2023-05-26T00:00","2023-05-26T01:00","2023-05-26T02:00","2023-05-26T03:00","2023-05-26T04:00","2023-05-26T05:00","2023-05-26T06:00","2023-05-26T07:00","2023-05-26T08:00","2023-05-26T09:00","2023-05-26T10:00","2023-05-26T11:00","2023-05-26T12:00","2023-05-26T13:00","2023-05-26T14:00","2023-05-26T15:00","2023-05-26T16:00","2023-05-26T17:00","2023-05-26T18:00","2023-05-26T19:00","2023-05-26T20:00","2023-05-26T21:00","2023-05-26T22:00","2023-05-26T23:00","2023-05-27T00:00","2023-05-27T01:00","2023-05-27T02:00","2023-05-27T03:00","2023-05-27T04:00","2023-05-27T05:00","2023-05-27T06:00","2023-05-27T07:00","2023-05-27T08:00","2023-05-27T09:00","2023-05-27T10:00","2023-05-27T11:00","2023-05-27T12:00","2023-05-27T13:00","2023-05-27T14:00","2023-05-27T15:00","2023-05-27T16:00","2023-05-27T17:00","2023-05-27T18:00","2023-05-27T19:00","2023-05-27T20:00","2023-05-27T21:00","2023-05-27T22:00","2023-05-27T23:00","2023-05-28T00:00","2023-05-28T01:00","2023-05-28T02:00","2023-05-28T03:00","2023-05-28T04:00","2023-05-28T05:00","2023-05-28T06:00","2023-05-28T07:00","2023-05-28T08:00","2023-05-28T09:00","2023-05-28T10:00","2023-05-28T11:00","2023-05-28T12:00","2023-05-28T13:00","2023-05-28T14:00","2023-05-28T15:00","2023-05-28T16:00","2023-05-28T17:00","2023-05-28T18:00","2023-05-28T19:00","2023-05-28T20:00","2023-05-28T21:00","2023-05-28T22:00","2023-05-28T23:00"],"temperature_2m":[10.1,10.0,10.0,10.0,10.0,10.3,10.5,11.5,12.2,13.2,14.4,15.3,16.0,13.7,13.8,14.5,14.8,14.8,14.2,13.3,11.9,11.5,11.5,11.1,10.8,10.7,10.4,10.0,9.7,9.6,10.0,10.9,12.0,13.2,14.3,14.9,15.7,16.3,16.5,16.4,16.0,15.5,14.7,13.7,12.8,12.1,11.6,11.3,10.9,10.2,9.8,9.4,9.2,9.3,10.2,11.6,13.0,14.4,15.3,15.4,15.6,15.9,15.7,15.6,15.7,15.8,15.8,15.4,14.5,13.4,12.5,11.7,11.0,10.2,9.4,8.6,8.1,8.2,9.9,11.2,12.6,14.2,15.6,16.8,18.0,18.8,19.5,19.9,20.0,19.8,19.3,18.2,16.8,15.5,14.4,13.5,12.7,12.0,11.4,11.2,11.2,11.5,12.1,13.1,14.5,15.9,17.3,18.6,19.7,20.5,21.1,21.3,21.1,20.5,19.6,18.3,16.7,15.3,14.2,13.2,12.5,11.9,11.6,11.6,12.2,13.1,13.8,13.6,15.0,16.2,16.9,17.4,17.6,17.4,16.9,16.4,16.0,15.6,15.0,14.0,12.8,11.7,10.6,9.6,8.8,8.2,7.8,7.7,7.8,8.2,8.8,9.7,10.7,11.6,12.3,13.0,13.6,14.2,14.7,14.9,14.8,14.3,13.5,12.3,10.9,9.6,8.8,8.3]}} > Finished chain.
' The current temperature in Edinburgh is 13.8°C.'
Let's try another question to see how that works.
api_chain.run("Can you give me an hourly weather forecast for Edingburgh in degrees celsius?")
> Entering new APIChain chain... https://api.open-meteo.com/v1/forecast?latitude=55.953251&longitude=-3.188267&hourly=temperature_2m&temperature_unit=celsius {"latitude":55.96,"longitude":-3.18,"generationtime_ms":0.25403499603271484,"utc_offset_seconds":0,"timezone":"GMT","timezone_abbreviation":"GMT","elevation":69.0,"hourly_units":{"time":"iso8601","temperature_2m":"°C"},"hourly":{"time":["2023-05-22T00:00","2023-05-22T01:00","2023-05-22T02:00","2023-05-22T03:00","2023-05-22T04:00","2023-05-22T05:00","2023-05-22T06:00","2023-05-22T07:00","2023-05-22T08:00","2023-05-22T09:00","2023-05-22T10:00","2023-05-22T11:00","2023-05-22T12:00","2023-05-22T13:00","2023-05-22T14:00","2023-05-22T15:00","2023-05-22T16:00","2023-05-22T17:00","2023-05-22T18:00","2023-05-22T19:00","2023-05-22T20:00","2023-05-22T21:00","2023-05-22T22:00","2023-05-22T23:00","2023-05-23T00:00","2023-05-23T01:00","2023-05-23T02:00","2023-05-23T03:00","2023-05-23T04:00","2023-05-23T05:00","2023-05-23T06:00","2023-05-23T07:00","2023-05-23T08:00","2023-05-23T09:00","2023-05-23T10:00","2023-05-23T11:00","2023-05-23T12:00","2023-05-23T13:00","2023-05-23T14:00","2023-05-23T15:00","2023-05-23T16:00","2023-05-23T17:00","2023-05-23T18:00","2023-05-23T19:00","2023-05-23T20:00","2023-05-23T21:00","2023-05-23T22:00","2023-05-23T23:00","2023-05-24T00:00","2023-05-24T01:00","2023-05-24T02:00","2023-05-24T03:00","2023-05-24T04:00","2023-05-24T05:00","2023-05-24T06:00","2023-05-24T07:00","2023-05-24T08:00","2023-05-24T09:00","2023-05-24T10:00","2023-05-24T11:00","2023-05-24T12:00","2023-05-24T13:00","2023-05-24T14:00","2023-05-24T15:00","2023-05-24T16:00","2023-05-24T17:00","2023-05-24T18:00","2023-05-24T19:00","2023-05-24T20:00","2023-05-24T21:00","2023-05-24T22:00","2023-05-24T23:00","2023-05-25T00:00","2023-05-25T01:00","2023-05-25T02:00","2023-05-25T03:00","2023-05-25T04:00","2023-05-25T05:00","2023-05-25T06:00","2023-05-25T07:00","2023-05-25T08:00","2023-05-25T09:00","2023-05-25T10:00","2023-05-25T11:00","2023-05-25T12:00","2023-05-25T13:00","2023-05-25T14:00","2023-05-25T15:00","2023-05-25T16:00","2023-05-25T17:00","2023-05-25T18:00","2023-05-25T19:00","2023-05-25T20:00","2023-05-25T21:00","2023-05-25T22:00","2023-05-25T23:00","2023-05-26T00:00","2023-05-26T01:00","2023-05-26T02:00","2023-05-26T03:00","2023-05-26T04:00","2023-05-26T05:00","2023-05-26T06:00","2023-05-26T07:00","2023-05-26T08:00","2023-05-26T09:00","2023-05-26T10:00","2023-05-26T11:00","2023-05-26T12:00","2023-05-26T13:00","2023-05-26T14:00","2023-05-26T15:00","2023-05-26T16:00","2023-05-26T17:00","2023-05-26T18:00","2023-05-26T19:00","2023-05-26T20:00","2023-05-26T21:00","2023-05-26T22:00","2023-05-26T23:00","2023-05-27T00:00","2023-05-27T01:00","2023-05-27T02:00","2023-05-27T03:00","2023-05-27T04:00","2023-05-27T05:00","2023-05-27T06:00","2023-05-27T07:00","2023-05-27T08:00","2023-05-27T09:00","2023-05-27T10:00","2023-05-27T11:00","2023-05-27T12:00","2023-05-27T13:00","2023-05-27T14:00","2023-05-27T15:00","2023-05-27T16:00","2023-05-27T17:00","2023-05-27T18:00","2023-05-27T19:00","2023-05-27T20:00","2023-05-27T21:00","2023-05-27T22:00","2023-05-27T23:00","2023-05-28T00:00","2023-05-28T01:00","2023-05-28T02:00","2023-05-28T03:00","2023-05-28T04:00","2023-05-28T05:00","2023-05-28T06:00","2023-05-28T07:00","2023-05-28T08:00","2023-05-28T09:00","2023-05-28T10:00","2023-05-28T11:00","2023-05-28T12:00","2023-05-28T13:00","2023-05-28T14:00","2023-05-28T15:00","2023-05-28T16:00","2023-05-28T17:00","2023-05-28T18:00","2023-05-28T19:00","2023-05-28T20:00","2023-05-28T21:00","2023-05-28T22:00","2023-05-28T23:00"],"temperature_2m":[10.1,10.0,10.0,10.0,10.0,10.3,10.5,11.5,12.2,13.2,14.4,15.3,16.0,13.7,13.8,14.5,14.8,14.8,14.2,13.3,11.9,11.5,11.5,11.1,10.8,10.7,10.4,10.0,9.7,9.6,10.0,10.9,12.0,13.2,14.3,14.9,15.7,16.3,16.5,16.4,16.0,15.5,14.7,13.7,12.8,12.1,11.6,11.3,10.9,10.2,9.8,9.4,9.2,9.3,10.2,11.6,13.0,14.4,15.3,15.4,15.6,15.9,15.7,15.6,15.7,15.8,15.8,15.4,14.5,13.4,12.5,11.7,11.0,10.2,9.4,8.6,8.1,8.2,9.9,11.2,12.6,14.2,15.6,16.8,18.0,18.8,19.5,19.9,20.0,19.8,19.3,18.2,16.8,15.5,14.4,13.5,12.7,12.0,11.4,11.2,11.2,11.5,12.1,13.1,14.5,15.9,17.3,18.6,19.7,20.5,21.1,21.3,21.1,20.5,19.6,18.3,16.7,15.3,14.2,13.2,12.5,11.9,11.6,11.6,12.2,13.1,13.8,13.6,15.0,16.2,16.9,17.4,17.6,17.4,16.9,16.4,16.0,15.6,15.0,14.0,12.8,11.7,10.6,9.6,8.8,8.2,7.8,7.7,7.8,8.2,8.8,9.7,10.7,11.6,12.3,13.0,13.6,14.2,14.7,14.9,14.8,14.3,13.5,12.3,10.9,9.6,8.8,8.3]}} > Finished chain.
' The hourly weather forecast for Edinburgh for the next 7 days is 10°C at 00:00, 10°C at 01:00, 10°C at 02:00, 10°C at 03:00, 10°C at 04:00, 10.3°C at 05:00, 10.5°C at 06:00, 11.5°C at 07:00, 12.2°C at 08:00, 13.2°C at 09:00, 14.4°C at 10:00, 15.3°C at 11:00, 16.0°C at 12:00, 13.7°C at 13:00, 13.8°C at 14:00, 14.5°C at 15:00, 14.8°C at 16:00, 14.8°C at 17:00, 14.2°C at 18:00, 13.3°C at 19:00, 11.9°C at 20:00, 11.5°C at 21:00, 11.5°C at 22:00, 11.1°C at 23:00, 10.8°C at 00:00 the next day, and'
From the exampls, you can see LangChain is able to extract meaning from my question, create API calls to retrieve the data, and display the result, whilst the whole interaction is completed purely with natural language.
Evaluation¶
By now, you probably have noticed that the output of LLMs can vary dramatically. Even with the same LLM, you can still expect different results for the same question every time you rerun. Lowering the temperature setting is one way to have more controlled and predictable output, however, that's not quite the point here.
Evaluation is the mechanism to quality control the pipeline, i.e. the output of applications, and make sure it does not suffer from any regression.
from langchain import OpenAI
from langchain.document_loaders import PyMuPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.evaluation.qa import QAEvalChain
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
We need to prepare the data just like what we did in Question Answering Over Documents.
loader = PyMuPDFLoader("hamlet.pdf")
doc = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size = 2000, chunk_overlap = 200)
docs = text_splitter.split_documents(doc)
embeddings = OpenAIEmbeddings(openai_api_key = openai_api_key)
knowledge_base = FAISS.from_documents(docs, embeddings)
Then prepare a list of questions and answers as test cases, which I picked a few from this.
question_answers = [
{"question" : "Where does the play take place?", "answer" : "Denmark"},
{"question" : "What is the name of the castle?", "answer" : "Elsinore"},
{"question" : "What are the first words spoken in the play?", "answer" : "Who's there?"},
{"question" : "How has Ophelia died?", "answer" : "She has supposedly drowned (ambiguity surrounds her death)."},
{"question" : "When was Hamlet written?", "answer" : "1600-1601"},
]
Create a RetrievalQA chain, the input_key tells the chain to look for the question key in the dict above, as the input prompt / query to the LLM.
chain = RetrievalQA.from_chain_type(
llm = llm,
chain_type = "stuff",
retriever = knowledge_base.as_retriever(),
input_key = "question"
)
This step is like executing the unit tests, it not only outputs the test assertion, but also the results for comparison.
predications = chain.apply(question_answers)
predications
[{'question': 'Where does the play take place?',
'answer': 'Denmark',
'result': ' The play takes place in Denmark.'},
{'question': 'What is the name of the castle?',
'answer': 'Elsinore',
'result': ' The castle is Elsinore.'},
{'question': 'What are the first words spoken in the play?',
'answer': "Who's there?",
'result': ' The first words spoken in the play are "Who\'s there?" by Francisco.'},
{'question': 'How has Ophelia died?',
'answer': 'She has supposedly drowned (ambiguity surrounds her death).',
'result': ' Ophelia has drowned.'},
{'question': 'When was Hamlet written?',
'answer': '1600-1601',
'result': ' Hamlet was written by William Shakespeare in the early 1600s.'}]
Once the execution result is produced, we call on QAEvalChain to evaluate the result.
eval_chain = QAEvalChain.from_llm(llm)
evaluation = eval_chain.evaluate(
question_answers,
predications,
question_key = "question",
answer_key = "answer",
prediction_key = "result"
)
evaluation
[{'text': ' CORRECT'},
{'text': ' CORRECT'},
{'text': ' CORRECT'},
{'text': ' CORRECT'},
{'text': ' CORRECT'}]
Now the results are here, all answered correctly. Despite the exceptional outcome, I was expecting it to make some tiny mistakes, which would probably seem more human. I almost tempted to create some convoluted mind-twisting assertion, I am limited by my own creativity.
Another thing you might find interesting is: having QAEvalChain marking it own homework. And I am not entirely sure how that mechanism works underneath the hood. So either you should dig into it if you are interested or take it with a pinch of salt (at least for now, until I can find a self-consistent answer).
Chatbots¶
Chatbot is a very powerful and interesting application of LLMs, however, what sets humans apart from AI chat bots is its memory. And LangChain has a number of ways to make LLMs remember the conversation.
With memory, LLMs can have a real time interaction with users, providing customised useful advice to people without sounding robotic.
from langchain.llms import OpenAI
from langchain import LLMChain
from langchain.prompts.prompt import PromptTemplate
from langchain.memory import ConversationBufferMemory
llm = OpenAI(temperature = 0, openai_api_key=openai_api_key)
template = """
You are a helpful personal assistant.
Your goal is to try to help answer the questions best you can. Do not make things up.
{chat_history}
Human: {question}
Chatbot:"""
prompt = PromptTemplate(
input_variables = ["chat_history", "question"],
template = template
)
memory = ConversationBufferMemory(memory_key = "chat_history")
llm_chain = LLMChain(llm = llm, prompt = prompt, memory = memory, verbose = True)
llm_chain.predict(question = "Who is the current president of united states?")
> Entering new LLMChain chain... Prompt after formatting: You are a helpful personal assistant. Your goal is to try to help answer the questions best you can. Do not make things up. Human: Who is the current president of united states? Chatbot: > Finished chain.
' The current President of the United States is Joe Biden.'
llm_chain.predict(question = "Who is the last president of united states?")
> Entering new LLMChain chain... Prompt after formatting: You are a helpful personal assistant. Your goal is to try to help answer the questions best you can. Do not make things up. Human: Who is the current president of united states? AI: The current President of the United States is Joe Biden. Human: Who is the last president of united states? Chatbot: > Finished chain.
' The last President of the United States was Donald Trump.'
llm_chain.predict(question = "What was the first question I asked you?")
> Entering new LLMChain chain... Prompt after formatting: You are a helpful personal assistant. Your goal is to try to help answer the questions best you can. Do not make things up. Human: Who is the current president of united states? AI: The current President of the United States is Joe Biden. Human: Who is the last president of united states? AI: The last President of the United States was Donald Trump. Human: What was the first question I asked you? Chatbot: > Finished chain.
' You asked me who is the current President of the United States.'
The final question shows how memory works for chat bots. That said, there are more than one way of preserving the memory, you can find out more here.
Agents¶
Last but not least, Agents. Agent is one of the most exciting part of LangChain that can be used for a variety of tasks. Agent is the decision maker that looks at data, understand what the next action should be, and execute and implement solutions on your behalf, by invoking a series of tools it has access.
import dotenv
import os
import json
from langchain.llms import OpenAI
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import Tool
from langchain.utilities import GoogleSearchAPIWrapper
from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
dotenv.load_dotenv()
llm = OpenAI(temperature = 0, openai_api_key = openai_api_key)
In this example, we will set up an agent that's capable of doing google search and performing basic calculations, which ChatGPT is notoriously bad at.
To search google, you need to setup and retrieve api keys from Google Services for Custom Search API and Programmable Search Engine.
To perform calculation, we will use a tool called Wolfram Alpha, you can register for free, and follow the developer portal to create an app. Be sure to make a note of the APPID.
Keep all the api keys in the ~/.env file with appropriate names, refer to the code below for examples.
After the keys have been setup, you may need to install the following packages if you don't have them locally.
pip install google-api-python-client
pip install wolframalpha
Now the laborious steps are out of the way, let the fun begin.
google_api_key = os.getenv("GOOGLE_API_KEY")
google_cse_id = os.getenv("GOOGLE_CSE_ID")
wolfram_alpha_appid = os.getenv("WOLFRAM_ALPHA_APPID")
google = GoogleSearchAPIWrapper(google_api_key = google_api_key, google_cse_id = google_cse_id)
wolfram = WolframAlphaAPIWrapper(wolfram_alpha_appid = wolfram_alpha_appid)
toolkit = [
Tool(
name = "Google Search",
func = google.run,
description= "Google Search is a web search engine that helps you find information on the web."
),
Tool(
name = "Wolfram Alpha",
func = wolfram.run,
description = "Wolfram Alpha is a computational knowledge engine that answers your questions in a natural language format"
),
]
agent = initialize_agent(toolkit, llm, agent="zero-shot-react-description", verbose=True, return_intermediate_steps=True)
response = agent({"input":"What is the capital city of France?"})
response['output']
> Entering new AgentExecutor chain... I should look up the answer online. Action: Google Search Action Input: "capital city of France" Observation: As the capital of France, Paris is the seat of France's national government. For the executive, the two chief officers each have their own official residences, ... The capital and by far the most important city of France is Paris, one of the world's preeminent cultural and commercial centres. ... Paris is now a sprawling ... Paris is the capital of France, the largest country of Europe with 550 000 km2 (65 millions inhabitants). Paris has 2.234 million inhabitants end 2011. She is ... Paris, France - Intercultural City ... Paris is the capital and most populous city of France. Situated on the Seine River, in the north of the country, it is in ... 4 days ago ... Paris, city and capital of France, situated in the north-central part of the country. People were living on the site of the present-day city ... French Translation of “capital city” | The official Collins English-French Dictionary online. Over 100000 French translations of English words and phrases. In 1699, French explorers discovered the area where Baton Rouge is now located. ... Baton Rouge, Louisiana's capital city is now 74.74 square miles in size ... Nov 24, 2009 ... On July 8, 1951, Paris, the capital city of France, celebrates turning 2,000 years old. In fact, a few more candles would've technically ... The city of Paris is built along a bend in the River Seine, between the confluence of ... A large number of major monuments of the French capital are built ... Sep 25, 2021 ... GRASSE, France — The town of Grasse sits in the hills above the more famous French Riviera city of Cannes, and it doesn't have the ... Thought: I now know the final answer. Final Answer: Paris is the capital city of France. > Finished chain.
'Paris is the capital city of France.'
response = agent({"input":"Calculate how many days until the next Solar exclipse?"})
response['output']
> Entering new AgentExecutor chain... I need to find out when the next Solar eclipse is Action: Google Search Action Input: "next Solar eclipse" Observation: Apr 8, 2024 ... A total solar eclipse happens when the Moon passes between the Sun and Earth, completely blocking the face of the Sun. The sky will darken as if ... Featured Eclipses in Coming Years · 1Oct 14, 2023Annular Solar Eclipse · 2Apr 8, 2024Total Solar Eclipse · 3Oct 2, 2024Annular Solar Eclipse · 4Feb 17, 2026Annular ... Apr 8, 2024 ... The total solar eclipse will begin over the South Pacific Ocean. Weather permitting, the first location in continental North America that will ... On April 8, 2024, the shadow of the Moon will cross Mexico, the United States, and Canada. This spectacular total solar eclipse will amaze many millions of ... Get Ready for These Upcoming Eclipses in the United States! · October 14, 2023 · October 14, 2023 · April 8, 2024 · April 8, 2024 · October 14, 2023 · April 8, 2024. Apr 20, 2023 ... The next solar eclipse will be an annular or "ring of fire" solar eclipse on Oct. 14, 2023 and will be visible across North America, ... Apr 19, 2023 ... On April 19 (in the US) / April 20 (Australia), the Moon will pass between the Sun and Earth, creating a total solar eclipse visible from ... Feb 20, 2023 ... A total solar eclipse occurs when the moon passes between the sun and Earth, completely obscuring the face of the sun. These solar eclipses ... Aug 27, 2016 ... NASA Solar Eclipse Publications Online · NASA TP2008-14171: Annular and Total Solar Eclipses of 2010 · NASA TP2008-14169: Total Solar Eclipse of ... Apr 29, 2022 ... After the total solar eclipse of August 21, 2017, the next total solar eclipse to cross the North American continent will be on April 8, ... Thought: I need to calculate the number of days between now and the next Solar eclipse Action: Wolfram Alpha Action Input: "days between now and April 8, 2024" Observation: Assumption: days from | current time to Monday, April 8, 2024 Answer: 321.1 days Thought: I now know the final answer Final Answer: 321.1 days > Finished chain.
'321.1 days'
You can see the simple question which can be done with simply google search, the agent can complete the task perfectly. On the more complex task that require more than google search, but also calculations that's outside of what google does, the agent then invokes Wolfram Alpha as you can see from the chain of thoughts.
This is truly a reflection of what LangChain is, i.e. a chain of tasks. With this ability, you can create tools like BabyAGI or AutoGPT, the only limit is the number of tokens, the OpenAI quota and your imagination!
Final Thoughts¶
Congratulations on making it all the way to the end, I hope you enjoy this exercise and learn about the fundamental principles and capabilities of LangChain. And ready to crack on building!!!
Checkout other learnings and resources I shared in my GitHub.
If you'd like to share your questions and feedback, or keen to get involved in building, you can reach out to me directly on Twitter and LinkedIn!
Comments
Post a Comment