第1章 什么是数据科学 ························1

1.1 数据科学的定义 ···························1

1.1.1 数据科学的背景 ···················1

1.1.2 数据科学的定义 ···················1

1.2 数据科学的关键技术 ·····················3

1.2.1 数据存储计算 ·····················5

1.2.2 数据治理 ························· 12

1.2.3 结构化数据分析 ················ 28

1.2.4 语音分析 ························· 44

1.2.5 视觉分析 ··························55

1.2.6 文本分析 ··························61

1.2.7 知识图谱 ························· 65

1.3 本章小结 ·································· 65

1.4 习题 ········································ 66

1.5 本章参考文献 ···························· 66

第2章 文本预训练模型······················ 68

2.1 文本分析技术的发展史 ················ 68

2.2 Transformer模型结构 ·················· 70

2.3 预训练模型的结构和变种 ··············75

2.4 加速处理器GPU和TPU ················ 79

2.4.1 GPU的介绍 ······················ 79

2.4.2 GPU产品命名 ··················· 80

2.4.3 TPU和GPU的区别 ·············· 83

2.4.4 TPU的使用总结 ················· 84

2.5 预训练模型的常见问题 ················· 87

2.5.1 模型输入的常见问题 ··········· 87

2.5.2 模型原理的常见问题 ··········· 90

2.5.3 模型进化的常见问题 ··········· 94

2.6 预训练模型的源码解读 ················ 96

2.6.1 模型架构 ························· 96

2.6.2 BertModel ························ 96

2.6.3 BERT预训练任务 ·············· 107

2.6.4 BERT 微调 ······················ 112

2.7 本章小结 ································· 114

2.8 习题 ······································· 114

2.9 本章参考文献 ··························· 115

第3章 多语种文本分析 ·····················116

3.1 多语种文本分析背景介绍 ············· 116

3.2 多语种文本分析技术 ··················· 116

3.2.1 Polyglot技术 ···················· 116

3.2.2 Multilingual BERT ············ 117

3.2.3 XLM多语言模型 ··············· 117

3.2.4 XLMR多语言模型 ············· 119

3.2.5 模型实验效果 ·················· 120

3.3 多语种文本分析源码解读 ············· 121

3.4 本章小结 ································· 125

3.5 习题 ······································· 126

3.6 本章参考文献 ··························· 126

第4章 文本情感分析 ························127

4.1 情感分析背景介绍 ····················· 127

4.2 情感分析技术 ··························· 127

4.2.1 目标和挑战 ····················· 127

4.2.2 技术发展历程 ·················· 129

4.2.3 情感分析的需求分析 ·········· 133

4.2.4 情感分析的落地实践 ·········· 134

4.2.5 模型开发平台的构建 ·········· 137

4.3 情感分析比赛和方案 ·················· 144

4.3.1 背景介绍 ························ 144

4.3.2 方案介绍 ························ 146

4.3.3 数据清洗和增广 ··············· 147

4.3.4 多模态融合 ····················· 147

4.3.5 机器学习技巧 ·················· 148

4.4 情感分析源码解读 ····················· 151

4.4.1 F1值适应优化技巧代码 ······· 151

4.4.2 对抗训练代码 ·················· 152

4.5 本章小结 ································· 154

4.6 习题 ······································· 154

4.7 本章参考文献 ··························· 155

第5章 文本机器翻译 ·······················156

5.1 机器翻译背景介绍 ····················· 156

5.2 机器翻译技术 ··························· 157

5.2.1 基于规则的机器翻译 ·········· 157

5.2.2 统计机器翻译 ·················· 158

5.2.3 神经网络机器翻译 ············ 159

5.2.4 Encoder-Decoder模型 ········· 161

5.2.5 注意力机制模型 ··············· 162

5.2.6 工业级神经网络实践 ·········· 164

5.3 机器翻译比赛和方案 ·················· 167

5.3.1 WMT21翻译任务 ·············· 167

5.3.2 WMT22 翻译任务 ············· 168

5.4 机器翻译源码解读 ····················· 169

5.4.1 通用框架介绍 ·················· 169

5.4.2 翻译模型实现 ·················· 170

5.5 本章小结 ································· 180

5.6 习题 ······································· 181

5.7 本章参考文献 ··························· 181

第6章 文本智能纠错 ·······················183

6.1 文本纠错背景介绍 ····················· 183

6.2 文本智能纠错技术 ····················· 184

6.2.1 智能纠错的意义和难点 ······· 185

6.2.2 智能纠错解决的问题 ·········· 185

6.2.3 业界主流解决方案 ············ 186

6.2.4 技术方案实践 ·················· 190

 6.3 文本智能纠错技术 ···················· 193

6.3.1 比赛介绍 ························ 193

6.3.2 校对问题思考 ·················· 194

6.4 纠错方案和源码解读 ·················· 195

6.4.1 GECToR原理解读 ············· 195

6.4.2 MacBERT原理解读 ··········· 199

6.4.3 PERT原理解读 ·················200

6.4.4 PLOME原理解读 ··············202

6.4.5 比赛方案 ························ 203

6.5 本章小结 ·································204

6.6 习题 ······································· 205

6.7 本章参考文献 ··························· 205

第7章 知识图谱构建 ······················ 206

7.1 知识图谱背景介绍 ·····················206

7.1.1 知识和知识图谱 ················206

7.1.2 知识获取、知识抽取与信息抽取

 的区别····························207

7.1.3 知识图谱构建范式 ·············208

7.2 非结构化信息抽取技术 ··············· 211

7.2.1 信息抽取框架 ··················· 211

7.2.2 命名实体识别··················· 212

7.2.3 关系识别 ························ 213

7.2.4 事件抽取 ························ 215

7.3 生成式统一模型抽取技术 ············ 216

7.4 模型源码解读 ···························220

7.5 本章小结 ·································224

7.6 习题 ·······································224

7.7 本章参考文献 ··························· 225

第8章 知识图谱问答 ······················ 226

8.1 背景介绍 ································· 226

8.2 知识图谱问答技术 ····················· 229

8.2.1 信息检索方法 ·················· 229

8.2.2 语义解析方法 ·················· 231

8.3 方案和源码解读 ························ 233

8.3.1 NL2SPARQL ··················· 233

8.3.2 NL2SPARQL语义解析方案 ··· 234

8.3.3 T5、BART、UniLM模型简介 ··· 234

8.3.4 T5、BART、UniLM方案 ······ 236

8.3.5 训练T5、BART、UniLM

 生成模型 ······················· 237

8.3.6 语义排序方案和代码 ·········· 239

8.3.7 SPARQL修正代码 ············· 241

8.4 本章小结 ································· 245

8.5 习题 ······································· 245

第9章 结构化知识NL2SQL问答 ·········246

9.1 NL2SQL背景介绍 ······················246

9.2 NL2SQL技术 ··························· 249

9.2.1 NL2SQL技术路线 ············· 249

9.2.2 NL2SQL项目实践 ············· 255

9.3 NL2SQL比赛和方案 ··················· 256

9.4 NL2SQL源码解读 ······················ 259

9.5 本章小结 ································· 269

9.6 习题 ······································· 269

9.7 本章参考文献 ··························· 270

第10章 ChatGPT大语言模型 ·············271

10.1 ChatGPT介绍 ·························· 271

10.1.1 ChatGPT的定义和背景 ······ 271

10.1.2 ChatGPT的发展历程 ········· 272

10.2 GPT模型概述·························· 272

10.2.1 GPT-1模型的原理 ············ 272

10.2.2 GPT-2模型的原理 ············ 273

10.2.3 GPT-3模型的原理 ············ 275

10.3 ChatGPT的实现原理 ················· 277

10.3.1 大模型的微调技术 ··········· 277

10.3.2 ChatGPT的能力来源 ········ 278

10.3.3 ChatGPT的预训练和微调 ··· 279

10.4 ChatGPT的应用 ······················· 282

10.4.1 ChatGPT提示工程 ··········· 282

10.4.2 ChatGPT应用场景 ··········· 283

10.4.3 ChatGPT的优缺点 ···········284

10.5 开源大模型 ···························· 285

10.5.1 ChatGLM大模型 ············· 285

10.5.2 LLaMA大模型 ················ 288

10.6 本章小结································ 294

10.7 习题······································ 294

10.8 本章参考文献·························· 295

第11章 行业实践案例 ····················· 296

11.1 智慧政务实践案例 ···················· 296

11.1.1 案例背景 ······················· 296

11.1.2 解决方案 ······················· 297

11.1.3 系统架构和实现 ·············· 299

11.1.4 案例总结 ······················· 307

11.2 公共安全实践案例 ····················308

11.2.1 案例背景 ·······················308

11.2.2 解决方案 ·······················309

11.2.3 系统架构及实现 ·············· 311

11.2.4 案例总结 ······················· 317

11.3 智能应急实践案例 ···················· 318

11.3.1 案例背景 ······················· 319

11.3.2 解决方案 ······················· 320

11.3.3 系统架构及实现 ·············· 321

11.3.4 案例总结 ······················· 332

11.4 本章小结 ································ 334

11.5 习题······································ 334