NEURONS AS FUNCTIONS Swould transduce the four-neuron vector of activations (-6 350 49 -689)to the four- dimensional bit vector of signal(0 1 1 0) Zero activations to unity,zero,or the previous signal
NEURONS AS FUNCTIONS S would transduce the four-neuron vector of activations (-6 350 49 –689) to the fourdimensional bit vector of signal (0 1 1 0) Zero activations to unity,zero,or the previous signal
SIGNAL MONOTONICITY In general,signal functions are monotone nondecreasing S'>=0. S(x) X -00 0 +00 This means signal functions have an upper bound or saturation value
SIGNAL MONOTONICITY In general, signal functions are monotone nondecreasing S’>=0. This means signal functions have an upper bound or saturation value. S(x) x -∞ - 0 + +∞
SIGNAL MONOTONICITY An important exception:bell-shaped signal function or Gaussian signal functions S(x)=e-cx2 c>0 S=-2cxe Soc-x The sign of the signal-activation derivation s'is opposite the sign of the activation x.We shall assume signal functions are monotone nondecreasing unless stated otherwise
SIGNAL MONOTONICITY An important exception: bell-shaped signal function or Gaussian signal functions 0 2 = − S x e c cx ( ) S cxe S x cx = − − − ' 2 , ' 2 The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise
SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function S,(x): SU-ml2a2,门 input activation vector::x=(xi,…,xn)∈R" variance: mean vector: 4,=(4,,4) we shall consider only scalar-input signal functions:S,(x)
SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function : ( ) ] 2 1 ( ) exp[ 2 = − 2 − n j i j j i i S x x n x = (x1 , , xn )R ( , , ) i n i i = 1 input activation vector: variance: mean vector: 2 i S (x) i we shall consider only scalar-input signal functions: ( ) i i S x
SIGNAL MONOTONICITY A property of signal monotonicity:semi-linearity Comparation: a.Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b.Nonlinear signal functions: increases a network's computational richness and facilitates noise suppression; risks computational and analytical intractability;
SIGNAL MONOTONICITY A property of signal monotonicity: semi-linearity Comparation: a. Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b. Nonlinear signal functions: increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;