使用 Atlas Vector Search 进行检索增强生成 (RAG)

设置环境。

通过保存扩展名为 .ipynb的文件来创建交互式 Python 笔记本，然后在笔记本中运行以下代码以安装依赖项：

pip install --quiet pymongo langchain langchain_community langchain_mongodb langchain_huggingface pypdf sentence_transformers

将数据导入 Atlas。

在本部分中，您将把法学硕士无权访问的样本数据导入到 Atlas 中。以下代码使用LangChain 集成和PyMongo 驱动程序执行以下操作：

• 加载包含MongoDB 最新收益报告的 PDF。

• 将数据拆分为数据段，并指定数据段大小（字符数）和数据段重叠（连续数据段之间的重叠字符数）。

• 加载 nomic-embed-text-v1 嵌入来自 Huugging Face 模型中心的模型。

• 从数据创建向量嵌入，并将这些嵌入存储在 Atlas 集群的rag_db.test集合中。

在笔记本中粘贴并运行以下代码，将<connection-string>替换为您的 Atlas连接字符串：

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_mongodb import MongoDBAtlasVectorSearch
from pymongo import MongoClient
# Load the PDF
loader = PyPDFLoader("https://investors.mongodb.com/node/12236/pdf")
data = loader.load()
# Split the data into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=20)
docs = text_splitter.split_documents(data)
# Load the embedding model (https://huggingface.co/nomic-ai/nomic-embed-text-v1")
model = HuggingFaceEmbeddings(model_name="nomic-ai/nomic-embed-text-v1", model_kwargs={ "trust_remote_code": True })
# Connect to your Atlas cluster
client = MongoClient("<connection-string>")
collection = client["rag_db"]["test"]
# Store the data as vector embeddings in Atlas
vector_store = MongoDBAtlasVectorSearch.from_documents(
    documents = docs,
    embedding = model,
    collection = collection,
    index_name = "vector_index"
)

使用 Atlas Vector Search 检索文档。

在本部分中，您将设置 Atlas Vector Search 以从向量数据库中检索文档。完成以下步骤：

1. 在向量嵌入上创建 Atlas Vector Search 索引。

   免费集群

   专用集群

   对于免费和共享集群，请按照步骤通过 Atlas 用户界面创建索引。将索引命名为vector_index并使用以下索引定义：

   {
      "fields": [
         {
            "type": "vector",
            "path": "embedding",
            "numDimensions": 768,
            "similarity": "euclidean"
         }
      ]
   }

   对于专用集群，您可以使用 PyMongo 驱动程序直接从应用程序创建索引。在笔记本中粘贴并运行以下代码：

   pymongo.operations import SearchIndexModel
   # Create your index model, then create the search index
   search_index_model = SearchIndexModel(
     definition = {
       "fields": [
         {
           "type": "vector",
           "numDimensions": 768,
           "path": "embedding",
           "similarity": "cosine"
         }
       ]
     },
     name = "vector_index",
     type = "vectorSearch"
   )
   collection.create_search_index(model=search_index_model)

2. 将 Atlas Vector Search 配置为检索器。

   在您的笔记本中，运行以下代码以设置检索系统，并使用LangChain 集成运行示例语义搜索查询：

   # Instantiate Atlas Vector Search as a retriever
   retriever = vector_store.as_retriever(
      search_type = "similarity"
   )
   # Run a sample query in order of relevance
   retriever.invoke("AI technology")

   隐藏输出

   [Document(metadata={'_id': '66a910ba7f78f7ec6760ceba', 'source': 'https://investors.mongodb.com/node/12236/pdf', 'page': 0}, page_content="more of our customers. We also see a tremendous opportunity to win more legacy workloads, as AI has now become a catalyst to modernize these\napplications. MongoDB's  document-based architecture is particularly well-suited for the variety and scale of data required by AI-powered applications."),
    Document(metadata={'_id': '66a910ba7f78f7ec6760ced6', 'source': 'https://investors.mongodb.com/node/12236/pdf', 'page': 1}, page_content='artificial intelligence, in our offerings or partnerships; the growth and expansion of the market for database products and our ability to penetrate that\nmarket; our ability to integrate acquired businesses and technologies successfully or achieve the expected benefits of such acquisitions; our ability to'),
    Document(metadata={'_id': '66a910ba7f78f7ec6760cec3', 'source': 'https://investors.mongodb.com/node/12236/pdf', 'page': 0}, page_content='MongoDB  continues to expand its AI ecosystem with the announcement of the MongoDB AI Applications Program (MAAP),'),
    Document(metadata={'_id': '66a910ba7f78f7ec6760cec4', 'source': 'https://investors.mongodb.com/node/12236/pdf', 'page': 1}, page_content='which provides customers with reference architectures, pre-built partner integrations, and professional services to help\nthem quickly build AI-powered applications. Accenture will establish a center of excellence focused on MongoDB  projects,\nand is the first global systems integrator to join MAAP.')]

使用 LLM 生成回复。

在本部分中，您将通过提示 LLM 使用检索到的文档作为上下文来 生成 响应。以下代码使用 LangChain 执行以下操作：

• 访问 Mistral7 B 指令 来自 Huging Face 模型中心的模型。

• 使用 提示模板 指示 法学硕士 在提示中包含用户的问题和检索到的文档 和 链。

• 向法学硕士提示 MongoDB 的最新 AI 公告。

在笔记本中粘贴并运行以下代码，将<token>替换为您的 Huging Face 访问令牌。生成的响应可能会有所不同。

from langchain_huggingface import HuggingFaceEndpoint
from langchain.prompts import PromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
import os
# Authenticate to your Hugging Face account
os.environ["HF_TOKEN"] = "<token>"
# Access the LLM (https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
llm = HuggingFaceEndpoint(repo_id="mistralai/Mistral-7B-Instruct-v0.2")
# Create prompt and RAG workflow
prompt = PromptTemplate.from_template("""
   Answer the following question based on the given context.
   Question: {question}
   Context: {context}
""")
rag_chain = (
   { "context": retriever, "question": RunnablePassthrough()}
   | prompt
   | llm
   | StrOutputParser()
)
# Prompt the LLM
question = "In a few sentences, what are MongoDB's latest AI announcements?"
answer = rag_chain.invoke(question)
print(answer)

Answer: MongoDB recently announced the MongoDB AI Applications Program
(MAAP) as part of their efforts to expand their AI ecosystem.
The document-based architecture of MongoDB is particularly well-suited
for AI-powered applications, offering an opportunity to win more legacy
workloads. These announcements were made at MongoDB.local NYC.