Bag-of-wordsWhat is bag-of-words?交通大学
Bag-of-words l What is bag-of-words？
CBOWWhatis bag-of-words?Continuous bag-of-words (CBOw): Predict keywords based oncontext麦通大学
CBOW l What is bag-of-words？ l Continuous bag-of-words (CBOW)： Predict keywords based on context
CBOWWhat is bag-of-words?Continuous bag-of-words (CBOw): Predict keywords based oncontextINPUTPROJECTIONOUTPUTQInput layerXIAw(t-2)W0Output layerHiddenlayerw(t-1)OOOSUMWyNW'NxBhX2yOw(t)CN-dimV-dimw(t+1)WVXAXckw(t+2)CxV-dim
CBOW l What is bag-of-words？ l Continuous bag-of-words (CBOW)： Predict keywords based on context
CBOWProcess1.Generateone-hotwordvectorsforthe inputcontextofsizem:(xc-m, .. xc-1, xc+1,., xc+m E Rlvl).InputlayerXIkWyxNOutputlayeHiddenlayerWyxNW'NxBhX2ky大N-dimV-dimWyxNXCACxV-dim3X
CBOW l Process 1. Generate one-hot word vectors for the input context of size m: (� �−�, ., � �−1 , � �+1 , ., � �+� ∈ � |�|). x � z
CBOWProcess1.Generateone-hotwordvectorsforthe inputcontextofsizem:(xc-m, .., xc-1, xc+1, ., xc+m E Rlvl).Inputlayer2.Getembeddedwordvectorsforthecontext.XIkVc-m = xc-m . W...Vc+m = xc+m.WE RNWyxNOutput layerHiddenlayerQWyxNW'NxBhX2ky通大N-dimV-dimWyxNXCKCxV-dim3XZ
CBOW l Process 1. Generate one-hot word vectors for the input context of size m: (� �−�, ., � �−1 , � �+1 , ., � �+� ∈ � |�|). 2. Get embedded word vectors for the context. ��−� = � �−� ∙ �,.,��+� = � �+� ∙ � ∈ � � x � z