How to use Embedchain with AWS Bedrock
LangChain is a framework that streamlines developing LLM applications. In a nutshell, it provides abstractions that help build out common steps and flows for working with LLMs.
In this post, we’ll go over how to integrate Embedchain, an open source RAG framework, with AWS Bedrock!
What is Embedchain
Embedchain is an open source RAG framework aimed at allowing developers to quickly gather necessary knowledge for LLM context when answering user prompts.
In particular, it shines with its easy-to-use API for adding unstructured data into manageable and optimized chunks for retrieval. In addition, it has synchronization options to ensure data sources are frequently updated.
Getting started with Embedchain
To showcase the power of Embedchain, let’s spin up a quick demo about Ada Lovelace - “The world’s first programmer”.
To begin, install the necessary dependencies:
1pip install embedchainFor the sake of this example, I’m using OpenAI’s API. Embedchain supports many other models as well.
1export OPENAI_API_KEY="..."Embedchain example RAG
The following code is a basic example of RAG using Embedchain.
1from embedchain import App
2
3bot = App()
4
5bot.add("https://en.wikipedia.org/wiki/Ada_Lovelace")
6
7print(bot.query("Who is Ada Lovelace?"))Output:
1Inserting batches in chromadb: 100% 1/1 [00:00<00:00, 1.26it/s]
2Successfully saved https://en.wikipedia.org/wiki/Ada_Lovelace (DataType.WEB_PAGE). New chunks count: 33
3Ada Lovelace was an English mathematician and writer known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine. She was the first to recognize that the machine had applications beyond pure calculation. Ada Lovelace was the only legitimate child of poet Lord Byron and reformer Lady Byron. She made significant contributions to the field of mathematics and is considered to be the world's first computer programmer.Steps:
- Instantiate a
Pipelinefromembedchain - Insert new documents into the pipeline
- Automatically chunk and batch the documents into
chromadb - Query the
chromadbstore for relevant documents and answer the question using OpenAI’s API
For more information on ChromaDB as a vector store, checkout my guide.
Using Embedchain with AWS Bedrock and Claude
Fortunately, Embedchain directly supports AWS Bedrock as an LLM.
Install the dependencies
1pip install embedchain[aws-bedrock]We must install the aws-bedrock extension for embedchain.
Create a config.yaml file
Add the following config.yaml file:
1llm:
2 provider: aws_bedrock
3 config:
4 model: anthropic.claude-v2
5 model_kwargs:
6 temperature: 0.5
7 top_p: 1
8 top_k: 250
9 max_tokens_to_sample: 200The model_kwargs in this configuration are the defaults from the model’s arguments specified in the AWS documentation.
Create main.py file
Before running the code, ensure you have AWS access. In my case, I’m using an AWS profile, but you can use any of the supported boto3 authentication methods.
1from embedchain import App
2
3bot = App.from_config(config_path="config.yaml")
4
5bot.add("https://en.wikipedia.org/wiki/Ada_Lovelace")
6
7print(bot.query("Who is Ada Lovelace?"))Output:
1Ada Lovelace was an English mathematician and writer. Some key facts about her:
2
3- She was the only legitimate child of poet Lord Byron and his wife Lady Byron.
4
5- She is chiefly known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine. She recognized the machine had applications beyond pure calculation and published the first algorithm intended to be carried out by such a machine.
6
7- She was the first to recognize the full potential of a "computing machine" and of "programming". For this reason she is often considered the first computer programmer.
8
9- She was interested in mathematics and logic from an early age, and pursued studies assiduously despite often being ill in her childhood.
10
11- She married William King in 1835. He was made Earl of Lovelace in 1838, making her Countess of Lovelace.
12
13- She died of uterine cancer in 1852 at age