大模型LLMS¶

来源 PDF：大模型LLMS.pdf
主题：训练 / 微调
页数：103
字符数：56377
索引片段数：142

摘要¶

大模型 LLMs面试宝典目录一、LLMs Tokenizer 篇 ....................................................................................................... 4 Byte-Pair Encoding(BPE)篇 ..............................................................................................

代表性原文片段¶

片段 1¶

大模型 LLMs面试宝典目录一、LLMs Tokenizer 篇 ....................................................................................................... 4 Byte-Pair Encoding(BPE)篇 ...............................................................................................................4 1 Byte-Pair Encoding(BPE) 如何构建词典？................................................................................ 4

片段 2¶

如何构建词典？................................................................................ 4 WordPiece 篇.................................................................................................................................... 4 SentencePiece 篇 ..............................................................................................................................4

片段 3¶

.........................................................................................4 对比篇.......................................................................................................................................................5 1 举例介绍一下不同大模型LLMs 的分词方式？.............................................................................5 2 介绍一下不同大模型LLMs 的分词方式的区别？ .................................................................... 5

片段 4¶

不同大模型LLMs 的分词方式的区别？ .................................................................... 5 二、大模型（LLMs）分布式训练面 ..........................................................................................5 1. 理论篇.....................................................................................................................................................5 2. 实践篇 8 3. 并行化策略选择篇 ..........9 4. 问题篇 10

片段 5¶

..............................................5 2. 实践篇 8 3. 并行化策略选择篇 ..........9 4. 问题篇 10 3. 推理性能方面: ................................................................................................................................ 12 1. 解决方法：....................................................................................................................................... 13 三、【大模型 LLMs 基础面 Plus】 ..................................................................................... 14

片段 6¶

】 ..................................................................................... 14 Layer normalization-方法篇 ........................................................................................................... 14 1、Layer Norm 篇 .......................................................................................................................... 14 2、RMS Norm 篇（均方根 Norm） ....................................................................................... 14

片段 7¶

....................................................................................... 14 3、Deep Norm 篇 .......................................................................................................................... 15 Layer normalization-位置篇 ........................................................................................................... 15 Layer normalization 对比篇 .........................................................................................................16

片段 8¶

........................................................................................16 LLMs 激活函数篇............................................................................................................................. 17 LLMs 注意力机制优化篇 ............................................................................................................... 18 1 传统 Attention 存在哪些问题？............................................................................................... 18

片段 9¶

....................................................................................... 18 2 Attention 优化方向？..................................................................................................................... 18 3 Attention 变体有哪些？................................................................................................................. 18 4 Multi-Query Attention 篇 ........................................................................................................19

片段 10¶

........................................................................................19 5 Grouped-query Attention 篇 ....................................................................................................20 6 Flash Attention 篇 ........................................................................................................................ 20 7 并行 transformer block ............................................................................................................20

片段 11¶

........................................................................................20 LLMs 损失函数篇............................................................................................................................. 21 1 介绍一下 KL 散度？...................................................................................................................... 21 2 交叉熵损失函数写一下，物理意义是什么？..............................................................................21

片段 12¶

下，物理意义是什么？..............................................................................21 3 KL 散度与交叉熵的区别？.......................................................................................................... 21 4 多任务学习各loss 差异过大怎样处理？...................................................................................... 22 5 分类问题为什么用交叉熵损失函数不用均方误差（MSE）？..................................................22

使用建议¶

这些内容来自授权公开 PDF 的文本抽取结果，网页端会基于同一份静态索引做关键词检索。
面试复习时建议先用网页 PDF 原文检索定位题目，再回到主 notes 学系统化答案。
如果片段出现排版噪声，以原 PDF 语义为准，并优先整理成自己的回答模板。