NEURONSASFUNCTIONSS(x)X++808Fig.1 s(x) is a bounded monotone-nondecreasing function of xIf c-→+oo, we get threshold signal function (dash line)Which is piecewise differentiable
NEURONS AS FUNCTIONS S(x) x -∞ - + +∞ Fig.1 s(x) is a bounded monotone-nondecreasing function of x If c→+∞，we get threshold signal function (dash line), Which is piecewise differentiable
NEURONS ASFUNCTIONSSwould transduce the four-neuron vector ofactivations (-6 350 49 -689) to the four-dimensional bit vector of signal (0 1 1 0)Zero activations to unity,zero,or the previoussignal
NEURONS AS FUNCTIONS S would transduce the four-neuron vector of activations (-6 350 49 –689) to the fourdimensional bit vector of signal (0 1 1 0) Zero activations to unity,zero,or the previous signal
SIGNALMONOTONICITYIn general, signal functions are monotonenondecreasing S'>=0S(x)x++88This means signal functions have an upper boundor saturation value
SIGNAL MONOTONICITY In general, signal functions are monotone nondecreasing S’>=0. This means signal functions have an upper bound or saturation value. S(x) x -∞ - + +∞
SIGNALMONOTONICITYAn important exception: bell-shaped signal function orGaussian signalfunctionsS(x) = e-cr2c>0S'= -2cxe-ax?.S'o-xThe sign of the signal-activation derivation s' is oppositethe sign of the activation x. We shall assume signalfunctions are monotone nondecreasing unless statedotherwise
SIGNAL MONOTONICITY An important exception: bell-shaped signal function or Gaussian signal functions 0 2 = − S x e c cx ( ) S cxe S x cx = − − − ' 2 , ' 2 The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise
SIGNALMONOTONICITYGeneralized Gaussian signal function define potential orradial basis function S,(x) :1Z(x,-μi)1S,(x) = exp[20'ix=(xi1,,xn)eR"input activation vector:o?variance:u, =(ui,..", uh)mean vector:we shall consider only scalar-input signal functions: S,(x,)
SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function : ( ) ] 2 1 ( ) exp[ 2 = − 2 − n j i j j i i S x x n x = (x1 , , xn )R ( , , ) i n i i = 1 input activation vector: variance: mean vector: 2 i S (x) i we shall consider only scalar-input signal functions: ( ) i i S x