github star gitee star atomgit star PyPI Downloads AI编程 AI交流群

大家好,这里是程序员晚枫,正在all in AI编程实战

第15讲:RAG入门——让AI读取你的文档

什么是RAG?

RAG = Retrieval-Augmented Generation(检索增强生成)

简单说:先从你的文档里找到相关内容,再让AI基于这些内容回答问题。

为什么需要RAG?因为AI模型的知识有截止日期,而且不知道你公司的内部文档。RAG让AI"读过"你的资料再回答。

1、RAG的工作流程

1
用户提问 → 检索相关文档 → 把文档+问题发给AI → AI基于文档回答

2、简单实现:直接把文档塞给AI

文档不大时,最简单的方式:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from openai import OpenAI

client = OpenAI(api_key="你的Key", base_url="https://api.deepseek.com")

with open('产品说明书.txt', 'r', encoding='utf-8') as f:
doc_content = f.read()

question = "这个产品的保修期是多久?"

response = client.chat.completions.create(
model="deepseek-chat",
messages=[
{"role": "system", "content": f"请根据以下文档内容回答用户问题。如果文档中没有相关信息,请说'文档中未找到相关信息'。\n\n文档内容:\n{doc_content}"},
{"role": "user", "content": question}
]
)

print(response.choices[0].message.content)

3、处理大文档:分段检索

文档太大怎么办?分成小段,只检索相关的:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
def split_text(text, chunk_size=500):
chunks = []
for i in range(0, len(text), chunk_size):
chunks.append(text[i:i+chunk_size])
return chunks

def simple_search(chunks, keyword):
results = []
for i, chunk in enumerate(chunks):
if keyword in chunk:
results.append((i, chunk))
return results

with open('大文档.txt', 'r', encoding='utf-8') as f:
text = f.read()

chunks = split_text(text)
question = "退款流程是什么?"
results = simple_search(chunks, "退款")

context = "\n".join([chunk for _, chunk in results[:3]])

response = client.chat.completions.create(
model="deepseek-chat",
messages=[
{"role": "system", "content": f"根据以下内容回答:\n{context}"},
{"role": "user", "content": question}
]
)

4、用向量检索提升效果

关键词搜索太简单,用向量搜索更智能:

1
pip install chromadb sentence-transformers
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import chromadb
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('shibing624/text2vec-base-chinese')
client_db = chromadb.Client()
collection = client_db.create_collection("my_docs")

with open('文档.txt', 'r', encoding='utf-8') as f:
text = f.read()

chunks = split_text(text, 500)

for i, chunk in enumerate(chunks):
collection.add(documents=[chunk], ids=[f"chunk_{i}"])

results = collection.query(query_texts=["退款流程是什么?"], n_results=3)
context = "\n".join(results['documents'][0])

5、实战:做一个文档问答助手

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from openai import OpenAI
import chromadb

ai_client = OpenAI(api_key="你的Key", base_url="https://api.deepseek.com")
db_client = chromadb.Client()
collection = db_client.get_or_create_collection("docs")

def add_document(file_path):
with open(file_path, 'r', encoding='utf-8') as f:
text = f.read()
chunks = split_text(text, 500)
for i, chunk in enumerate(chunks):
collection.add(documents=[chunk], ids=[f"{file_path}_{i}"])
print(f"已添加 {len(chunks)} 个文档片段")

def ask(question):
results = collection.query(query_texts=[question], n_results=3)
context = "\n".join(results['documents'][0])
response = ai_client.chat.completions.create(
model="deepseek-chat",
messages=[
{"role": "system", "content": f"根据以下内容回答问题:\n{context}"},
{"role": "user", "content": question}
]
)
return response.choices[0].message.content

add_document('产品手册.txt')
add_document('常见问题.txt')
print(ask("如何退换货?"))

下讲预告

学会了RAG,下一讲我们学 AI聊天机器人开发——从零开始做一个完整的聊天应用。

敬请期待!


程序员晚枫专注AI编程培训,小白看完他和图灵社区合作的教程《30讲 · AI编程训练营》就能上手做AI项目。

前3讲可以试听,试听链接:https://www.bilibili.com/cheese/play/ss982042944