how.wtf

How to use Embedchain with AWS Bedrock

· Thomas Taylor

LangChain is a framework that streamlines developing LLM applications. In a nutshell, it provides abstractions that help build out common steps and flows for working with LLMs.

In this post, we’ll go over how to integrate Embedchain, an open source RAG framework, with AWS Bedrock!

What is Embedchain

Embedchain is an open source RAG framework aimed at allowing developers to quickly gather necessary knowledge for LLM context when answering user prompts.

In particular, it shines with its easy-to-use API for adding unstructured data into manageable and optimized chunks for retrieval. In addition, it has synchronization options to ensure data sources are frequently updated.

Getting started with Embedchain

To showcase the power of Embedchain, let’s spin up a quick demo about Ada Lovelace - “The world’s first programmer”.

To begin, install the necessary dependencies:

1pip install embedchain

For the sake of this example, I’m using OpenAI’s API. Embedchain supports many other models as well.

1export OPENAI_API_KEY="..."

Embedchain example RAG

The following code is a basic example of RAG using Embedchain.

1from embedchain import App
2
3bot = App()
4
5bot.add("https://en.wikipedia.org/wiki/Ada_Lovelace")
6
7print(bot.query("Who is Ada Lovelace?"))

Output:

1Inserting batches in chromadb: 100% 1/1 [00:00<00:00,  1.26it/s]
2Successfully saved https://en.wikipedia.org/wiki/Ada_Lovelace (DataType.WEB_PAGE). New chunks count: 33
3Ada Lovelace was an English mathematician and writer known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine. She was the first to recognize that the machine had applications beyond pure calculation. Ada Lovelace was the only legitimate child of poet Lord Byron and reformer Lady Byron. She made significant contributions to the field of mathematics and is considered to be the world's first computer programmer.

Steps:

  1. Instantiate a Pipeline from embedchain
  2. Insert new documents into the pipeline
  3. Automatically chunk and batch the documents into chromadb
  4. Query the chromadb store for relevant documents and answer the question using OpenAI’s API

For more information on ChromaDB as a vector store, checkout my guide.

Using Embedchain with AWS Bedrock and Claude

Fortunately, Embedchain directly supports AWS Bedrock as an LLM.

Install the dependencies

1pip install embedchain[aws-bedrock]

We must install the aws-bedrock extension for embedchain.

Create a config.yaml file

Add the following config.yaml file:

1llm:
2  provider: aws_bedrock
3  config:
4    model: anthropic.claude-v2
5    model_kwargs:
6      temperature: 0.5
7      top_p: 1
8      top_k: 250
9      max_tokens_to_sample: 200

The model_kwargs in this configuration are the defaults from the model’s arguments specified in the AWS documentation.

Create main.py file

Before running the code, ensure you have AWS access. In my case, I’m using an AWS profile, but you can use any of the supported boto3 authentication methods.

1from embedchain import App
2
3bot = App.from_config(config_path="config.yaml")
4
5bot.add("https://en.wikipedia.org/wiki/Ada_Lovelace")
6
7print(bot.query("Who is Ada Lovelace?"))

Output:

 1Ada Lovelace was an English mathematician and writer. Some key facts about her:
 2
 3- She was the only legitimate child of poet Lord Byron and his wife Lady Byron.
 4
 5- She is chiefly known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine. She recognized the machine had applications beyond pure calculation and published the first algorithm intended to be carried out by such a machine.
 6
 7- She was the first to recognize the full potential of a "computing machine" and of "programming". For this reason she is often considered the first computer programmer.
 8
 9- She was interested in mathematics and logic from an early age, and pursued studies assiduously despite often being ill in her childhood.
10
11- She married William King in 1835. He was made Earl of Lovelace in 1838, making her Countess of Lovelace.
12
13- She died of uterine cancer in 1852 at age

#Chroma-Db   #Generative-Ai   #Python  

Reply to this post by email ↪