LanguageModelDefinition: compute probability distribution over the words in a sentenceApplication: predicting the next word in a sequence given the sequence ofwordsalreadypresentbooklaptopthe girl opened herdoormind交通大学
• Definition: compute probability distribution over the words in a sentence • Application: predicting the next word in a sequence given the sequence of words already present Language Model the girl opened her book laptop door mind
LanguageModelDefinition: compute probability distribution overthe words in a sentenceApplication: predicting the next word in a sequence given the sequence ofwords already presentbooklaptopthe girl opened herdoormindFormal Definition:Given a sequence of words x(1), x(2),., x(t), estimate the probabilitydistribution of the next word x(t+1)P(x(t+1)|x(t), .. (1)x(t+1) could be any word in the vocabulary V = (W1, .., Wivi} Language Model is a system like this
Language Model the girl opened her book laptop door • Formal Definition: Given a sequence of words � (1) , � (2) ,., � (�) , estimate the probability distribution of the next word � (t+1) 。 � � (�+1)|� (�) , ., � (1) � (t+1) could be any word in the vocabulary V = {�1, ., �|�|} 。 • Language Model is a system like this mind • Definition: compute probability distribution over the words in a sentence • Application: predicting the next word in a sequence given the sequence of words already present
LanguageModelLanguage Model could also be regarded as a system assigning a probabilitytoatext.交道大学
• Language Model could also be regarded as a system assigning a probability to a text. Language Model
LanguageModelLanguage Model could also be regarded as a system assigning a probabilityto a text.For example, the probability of a text x(1), x(2) ,., x(T) (Language Model)couldbedenotedasP(x(1), . (T) = P(x(1) × P(x(2)|x(1) × . × P(x(T)|x(T-1),.. (1)= I/= P(x() |x(t-1), ., (1)ThisiswhatLanguageModelcomputes
• Language Model could also be regarded as a system assigning a probability to a text. • For example, the probability of a text � (1) , � (2) ,., � (�) (Language Model) could be denoted as P � (1) , ., � (�) = � � (1) × � � (2) � (1) × ∙∙∙ × � � (�) � (�−1) , ., � (1) = �=1 � � � (�) � (�−1) , ., � (1) Language Model This is what Language Model computes
LanguageModelHave youeverused Language Model?交通大学
• Have you ever used Language Model？ Language Model