Neural Network Language Models交通大学thegirlheropenedabandonFixedWindow
Before her mum arrives the girl opened her _ abandon Fixed window Neural Network Language Models
Neural Network Language Models交通大学Word vector (one-hot,distributedthegirlheropenedrepresentation......)x(2)x(3)x(4)x(1)x(1), x(2),x(3), x(4)
the girl opened her _ � (1) , � (2) ,� (3) , � (4) � (1) � (2) � (3) � (4) Word vector（one-hot, distributed representation.） Neural Network Language Models
Neural Network Language ModelsConcatenate word vectorse = [e(1); e(2); e(3); e(4)]零电电餐2囍6606Word vector (one-hot,distributedthegirlheropenedrepresentation......)x(2)x(3)x(4)x(1)x(1), x(2),x(3), x(4)
� = [� (1) ; � (2) ; � (3) ; � (4)] Concatenate word vectors Neural Network Language Models the girl opened her _ � (1) , � (2) ,� (3) , � (4) � (1) � (2) � (3) � (4) Word vector（one-hot, distributed representation.）
Neural Network Language ModelsHiddenLayerh = f(We+b)WConcatenate word vectorse = [e(1); e(2); e(3); e(4)]Q相电#06Word vector (one-hot,distributedthegirlheropenedrepresentation......x(2)x(3)x(4)x(1)x(1), x(2),x(3), x(4)
� � = [� (1) ; � (2) ; � (3) ; � (4)] Concatenate word vectors ℎ = �(�� + �1) Hidden Layer Neural Network Language Models the girl opened her _ � (1) , � (2) ,� (3) , � (4) � (1) � (2) � (3) � (4) Word vector（one-hot, distributed representation.）
Neural Network Language ModelsbookslaptopsOutput Layer = softmax(Uh + b2) E RIVIZOOUHiddenLayerh = f(We+b)WConcatenate word vectorse = [e(1); e(2); e(3); e(4)]Q0QQ：#606Word vector (one-hot, distributedthegirlheropenedrepresentation......x(1)x(2)x(3)x(4)x(1), x(2),x(3), x(4)
� � books laptops a zoo � = [� (1) ; � (2) ; � (3) ; � (4)] Concatenate word vectors ℎ = �(�� + �1) Hidden Layer � = 𝑠��𝑓�(�ℎ + �2) ∈ ℝ|�| Output Layer Neural Network Language Models the girl opened her _ � (1) , � (2) ,� (3) , � (4) � (1) � (2) � (3) � (4) Word vector（one-hot, distributed representation.）