bacow/happy-llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(chapter5): 更新LLaMA2注意力机制图示

Browse Source
This commit is contained in:
KMnO4-zx
2025-06-18 16:32:07 +08:00
parent ce535629ca
commit 2fca30c239
3 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/chapter5/第五章 动手搭建大模型.md
View File

@@ -114,7 +114,7 @@ torch.Size([1, 50, 768])
在 LLaMA2 模型中，虽然只有 LLaMA2-70B模型使用了分组查询注意力机制（Grouped-Query Attention，GQA），但我们依然选择使用 GQA 来构建我们的 LLaMA Attention 模块，它可以提高模型的效率，并节省一些显存占用。
<div align='center'>
<img src="https://raw.githubusercontent.com/datawhalechina/happy-llm/main/docs/images/5-images/Attention.png" alt="alt text" width="100%">
<img src="https://raw.githubusercontent.com/datawhalechina/happy-llm/main/docs/images/5-images/llama2-attention" alt="alt text" width="70%">
<p>图 5.2 LLaMA2 Attention 结构</p>
</div>