《深度自然语言处理》课程教学课件（Natural language processing with deep learning）04 Language Model & Distributed Representation（1/6）

西安交通大学Natural languageprocessingwith deeplearningXIANHAOTONGUNIVERSITYLanguage Model&Distributed Representation (1)交通大学ChenLicli@xjtu.edu.cn2023

Chen Li cli@xjtu.edu.cn 2023 Language Model & Distributed Representation (1) Natural language processing with deep learning

Outlines1. Traditional Language Model (LM)2.Distributed Representation交道大学

Outlines 1. Traditional Language Model (LM) 2. Distributed Representation

Outlines1.Traditional Language Model (LM)2.Distributed Representation交道大学

Outlines 1. Traditional Language Model (LM) 2. Distributed Representation

OverviewAnewNLPtaskLanguage modelingRelated conceptsMarkov chainn-gram modelMaximumLikelihoodEstimation(MLELaplace smoothing, additive smoothingGood Turing

• Language modeling A new NLP task Overview • Markov chain • n-gram model • Maximum Likelihood Estimation (MLE) • Laplace smoothing, additive smoothing • Good Turing Related concepts

LanguageModelDefinition: compute probability distribution over the words in a sentence交道大学

• Definition: compute probability distribution over the words in a sentence Language Model