西安交通大学Natural languageprocessingwith deeplearningXIANHAOTONGUNIVERSITYLanguage Model&Distributed Representation (4)交通大学ChenLicli@xjtu.edu.cn2023
Chen Li cli@xjtu.edu.cn 2023 Language Model & Distributed Representation (4) Natural language processing with deep learning
Outlines1.RNN based LM2. Seq2seq Model3.AttentionMechanism
Outlines 1. RNN based LM 2. Seq2seq Model 3. Attention Mechanism
Outlines1.RNNbasedLM2.Seq2seqModel3.AttentionMechanism
Outlines 1. RNN based LM 2. Seq2seq Model 3. Attention Mechanism
RNNLMRecurrent Neural Networks(RNN)Main idea:use the same W(1)(2)(3)(4)Output-(optional)h(1)h(2)h(3)h(4)自具wWHidden layer-福OInput sequencer(1)x(2)(3)x(4)(variable-length)
RNN LM � (1) � (2) � (3) � (4) � Recurrent Neural Networks (RNN) • Main idea: use the same � � � � � (1) � (2) � (3) � (4) ℎ (1) ℎ (2) ℎ (3) ℎ (4) Output (optional) Hidden layer Input sequence (variable-length) . .
RNNLM交通大学Word vector (one-hot, distributedthegirlheropenedrepresentation......)x(1)x(2)x(3)x(4)x(t) E IRIVI
RNN LM � (1) � (2) � (3) � (4) � (�) ∈ ℝ|�| Word vector（one-hot, distributed representation.） the girl opened her