Deep cnn in AlphaGo Policy network Value network Policy network nput:19×19,48 p,(als) vo(s) input channels ayer 1: 5X5 kernel 192 filters ayer 2 to 12: 3X3 kernel. 192 filters ayer 13: 1X1 kernel 1 filter Value network has similar architecture to policy network (Silver et al, 2016)
Deep CNN in AlphaGO Policy network: • Input: 19x19, 48 input channels • Layer 1: 5x5 kernel, 192 filters • Layer 2 to 12: 3x3 kernel, 192 filters • Layer 13: 1x1 kernel, 1 filter Value network has similar architecture to policy network (Silver et al, 2016)
Sequence Modelling Why do we need RNN? What are rnns? RNN EXtensions What can rnns can do?
Sequence Modelling • Why do we need RNN? • What are RNNs? • RNN Extensions • What can RNNs can do?
Why do we need RNNS? The limitations of the Neural network(CNNS) Rely on the assumption of independence among the (training and test) examples After each data point is processed, the entire state of the network is lost Rely on examples being vectors of fixed length We need to model the data with temporal or sequential structures and varying length of inputs and outputs Frames from video Snippets of audio Words pulled from sentences
Why do we need RNNs? The limitations of the Neural network (CNNs) • Rely on the assumption of independence among the (training and test) examples. – After each data point is processed, the entire state of the network is lost • Rely on examples being vectors of fixed length We need to model the data with temporal or sequential structures and varying length of inputs and outputs – Frames from video – Snippets of audio – Words pulled from sentences
What are rnns? Recurrent neural networks(RNNs )are connectionist models with the ability to selectively pass information across sequence steps, while processing sequential data one element at a time Outputs Allow a'memory' of previous inputs to persist in h(t) the network's internal state, and thereby h(t) influence the network output Hidden units Delay h(t)=fH WIHx(t)+ Whhh(t-1)) x y(t)=fo(WHoh(t)) Inputs fH and fo are the activation function for hidden and output unit; WIH, WHH, and The simplest form of fully recurrent WHo are connection weight matrices which neural network is an mlp with the are learnt by training previous set of hidden unit activations feeding back into the network along with the inputs
Recurrent neural networks (RNNs) are connectionist models with the ability to selectively pass information across sequence steps, while processing sequential data one element at a time. The simplest form of fully recurrent neural network is an MLP with the previous set of hidden unit activations feeding back into the network along with the inputs ℎ 𝑡 = 𝑓𝐻 𝑊𝐼𝐻𝑥 𝑡 + 𝑊𝐻𝐻ℎ(𝑡 − 1) 𝑦 𝑡 = 𝑓𝑂(𝑊𝐻𝑂ℎ(𝑡)) 𝑓𝐻 and 𝑓𝑂 are the activation function for hidden and output unit; 𝑊𝐼𝐻, 𝑊𝐻𝐻, and 𝑊𝐻𝑂 are connection weight matrices which are learnt by training Allow a ‘memory’ of previous inputs to persist in the network’s internal state, and thereby influence the network output What are RNNs?
What are rnns? The recurrent network can be converted into a feed-forward network by unfolding over time W ○-○-○ An unfolded recurrent network. each node represents a layer of network units at a single time step. The weighted connections from the input layer to hidden layer are labelled'wl, those from the hidden layer to itself (i.e. the recurrent weights) are labelled w2 and the hidden to output weights are labelled" w3. Note that the same weights are reused at every time step. Bias weights are omitted for clarity
An unfolded recurrent network. Each node represents a layer of network units at a single time step. The weighted connections from the input layer to hidden layer are labelled ‘w1’, those from the hidden layer to itself (i.e. the recurrent weights) are labelled ‘w2’ and the hidden to output weights are labelled‘w3’. Note that the same weights are reused at every time step. Bias weights are omitted for clarity. What are RNNs? • The recurrent network can be converted into a feed-forward network by unfolding over time