Self-attention Hung-yi Lee 李宏毅
Hung -yi Lee 李宏毅 1
Sophisticated Input 。Input is a vector Model →Scalar or Class Input is a set of vectors 000- Model →Scalars or Classes (may change length)
Sophisticated Input • Input is a vector • Input is a set of vectors Model Scalar or Class Model Scalars or Classes (may change length) 2
this is a cat Vector Set as Input 1000 One-hot Encoding Word Embedding apple=[10000…] bag=[01000.] dog rabbit orun ●● cat=[00100…] ojump cat dog=[00010…] otree o flower elephant =[0000 1......] To learn more:https://youtu.be/X7PH3NuYWOQ(in Mandarin)
Vector Set as Input this is a cat dog cat rabbit jump run flower tree apple = [ 1 0 0 0 0 …… ] bag = [ 0 1 0 0 0 …… ] cat = [ 0 0 1 0 0 …… ] dog = [ 0 0 0 1 0 …… ] elephant = [ 0 0 0 0 1 …… ] One-hot Encoding Word Embedding To learn more: https://youtu.be/X7PH3NuYW0Q (in Mandarin) 3
Vector Set as Input 10ms 1s→100 frames 25ms 400 sample points(16KHz) frame 39-dim MFCC 80-dim filter bank output
Vector Set as Input 10ms 25ms 400 sample points (16KHz) 39-dim MFCC 80-dim filter bank output frame 1s → 100 frames 4
https://medium.com/analytics-vidhya/social- network-analytics-f082f4e21b16 Vector Set as Input Graph is also a set of vectors(consider each node as a vector) Each profile is a vector 5
Vector Set as Input • Graph is also a set of vectors (consider each node as a vector) https://medium.com/analytics-vidhya/socialnetwork-analytics-f082f4e21b16 Each profile is a vector 5