x SEined hailer Figure 4.2 multilayer perceptron models The neurons in the first layer of the multilayer perceptron perform computations, and the outputs of these neurons are x=f(∑wx)-0") withj=1, 2..n The neurons in the second layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by f2(∑v2x”) with j=1,2,,n2. The neurons in the third layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by y=/(∑x2)-日) withj=1,2,…,m The parameters(scalar real numbers)w are called the weights of the first hidden layer. The w(2)are called the weights of the second hidden layer. The w are called the weights of the output layer. The parameters 0, are called the biases of the first hidden layer. The parameters 0, are called the biases of the second hidden layer, and thee, are the biases of the output layer. The functions f(for the output layer), f2)(for the second hidden layer), and fo(for the first hidden layer) represent the activation functions. The activation functions can be different for each neuron in the multilayer perception(e.g, the first layer could have one type of sigmoid, while the next two layers could have different sigmoid functions or threshold functions) This completes the definition of the multilayer perception. Next, we will introduce the radial basis function neural network. After that we explain how both of these neural networks relate to the other topics covered in this book. 4.3.2 Radial basis function Neural Networks A locally tuned, overlapping receptive field is found in parts of the cerebral cortex, in the visual cortex, and in other par of the brain. The radial basis function neural network model is based on these biological systems A radial basis function neural network is shown in Figure 4.3. There, the inputs are x i=1, 2,,n, and the output is PDF文件使用" pdffactory Pro"试用版本创建ww, fineprint,com,cn
Figure 4.2 multilayer perceptron models. The neurons in the first layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by (1) (1) (1) (1) 1 (( ) ) n j j ij i j i x f w x q = = - å with j = 1,2.....n1. The neurons in the second layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by 1 (2) (2) (2) (1) (2) 1 (( ) ) n j j ij i j i x f w x q = = - å with j = 1,2,..., n2. The neurons in the third layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by 2 (2) 1 (( ) ) n j j ij i j i y f w x q = = - å with j = 1,2, ...,m. The parameters (scalar real numbers) (1) ij w are called the weights of the first hidden layer. The (2) ij w are called the weights of the second hidden layer. The wij are called the weights of the output layer. The parameters (1) qj are called the biases of the first hidden layer. The parameters (2) q j are called the biases of the second hidden layer, and the j q are the biases of the output layer. The functions fj (for the output layer), (2) j f (for the second hidden layer), and (1) j f (for the first hidden layer) represent the activation functions. The activation functions can be different for each neuron in the multilayer perception (e.g., the first layer could have one type of sigmoid, while the next two layers could have different sigmoid functions or threshold functions). This completes the definition of the multilayer perception. Next, we will introduce the radial basis function neural network. After that we explain how both of these neural networks relate to the other topics covered in this book. 4.3.2 Radial Basis Function Neural Networks A locally tuned, overlapping receptive field is found in parts of the cerebral cortex, in the visual cortex, and in other parts of the brain. The radial basis function neural network model is based on these biological systems. A radial basis function neural network is shown in Figure 4.3. There, the inputs are xi , i = 1,2,..., n, and the output is PDF 文件使用 "pdfFactory Pro" 试用版本创建 www.fineprint.com.cn
y=f(x)where f represents the processing by the entire radial basis function neural network. Let x=[=,,x2,x].The input to the i receptive field unit is x, and its output is denoted with R;(x). It has what is called a"strength"which we denote by y,. Assume that there are M receptive field units. Hence, from Figure 4.3, y=f(x)=∑R(x) is the output of the radial basis function neural network y Figure 4.3 Radial basis function neural network model There are several possible choices for the"receptive field units"Ri(x) R(x) Wherec-[ci ci.ciI, o, is a scalar, and if: is a vector then==V== 2. We could choose R(x) where c and o. are defined in choice 1 There are also alternatives to how to compute the output of the radial basis function neural network. For instance rather than computing the simple sum as in Equation(4.3), you could compute a weighted average y, R() y=f(x)=M (44) ∑R(x) It is also possible to define multilayer radial basis function neural networks This completes the definition of the radial basis function neural network. Next, we explain the relationships between multilayer perceptions and radial basis function neural networks and fuzzy systems 4.3.3 Relations hips between Fuzzy Systems and Neural Networks There are two ways in which there are relationships between fuzzy systems and neural networks. First, techniques from one area can be used in the other. Second, in some cases the functionality(i.e, the nonlinear function that they PDF文件使用" pdffactory Pro"试用版本创建ww. fineprint,com,cn
y =f(x) where f represents the processing by the entire radial basis function neural network. Let [ 1 2 , , ] T n x = x x x . The input to the i th receptive field unit is x, and its output is denoted with Ri (x). lt has what is called a "strength" which we denote by i y . Assume that there are M receptive field units. Hence, from Figure 4.3, ( ) ( ) 1 M i i i y f x y R x = = = å (4.3) is the output of the radial basis function neural network. Figure 4.3 Radial basis function neural network model. There are several possible choices for the "receptive field units" R i (x): 1. We could choose ( ) 2 2 exp i i i x c R x s æ ö - = ç- ÷ ç ÷ è ø Where 1 2 , , T i i i i n c = é ù c c c ë û ,si is a scalar, and if z is a vector then T z = z z . 2. We could choose ( ) 2 2 1 1 exp i i i R x x c s = æ ö - + -ç ÷ ç ÷ è ø where i i c and s are defined in choice 1. There are also alternatives to how to compute the output of the radial basis function neural network. For instance, rather than computing the simple sum as in Equation (4.3), you could compute a weighted average ( ) ( ) ( ) 1 1 M i i i M i i y R x y f x R x = = = = å å （4.4） It is also possible to define multilayer radial basis function neural networks. This completes the definition of the radial basis function neural network. Next, we explain the relationships between multilayer perceptions and radial basis function neural networks and fuzzy systems. 4.3.3 Relationships Between Fuzzy Systems and Neural Networks There are two ways in which there are relationships between fuzzy systems and neural networks. First, techniques from one area can be used in the other. Second, in some cases the functionality (i.e., the nonlinear function that they PDF 文件使用 "pdfFactory Pro" 试用版本创建 www.fineprint.com.cn
implement) is identical. Some label the intersection between fuzzy systems and neural networks with the term fuzzy-neural"or "neuro-fuzzy"to highlight that techniques from both fields are being used. Here, we avoid this terminology and simply highlight the basic relationships between the two fields Multilayer Perceptrons The multilayer perceptron should be viewed as a nonlinear network whose nonlinearity can be tuned by changing the weights, biases, and parameters of the activation functions. The fuzzy system is also a tunable nonlinearity whose shape can be changed by tuning, for example, the membership functions. Since both are tunable nonlinearities, the following approaches are possible Gradient methods can be used for training neural networks to perform system identification or to act estimators or predictors in the same way as fuzzy systems were trained. Indeed, the gradient training of neural networks, called"back-propagation training, " was introduced well before the gradient training of fuzzy systems, and the idea for training fuzzy systems this way came from the field of neural networks a Hybrid methods for training can also be used for neural networks. For instance, gradient methods may be used in conjunction with clustering methods applied to neural networks a Indirect adaptive control can also be achieved with a multilayer perceptron. To do this we use two multilayer perceptrons as the tunable nonlinearities in the certainty equivalence control law and the gradient method for tuning a Gain scheduled control may be achieved by training a multilayer perceptron to map the associations between operating conditions and controller parameters This list is by no means exhaustive. It simply shows that multilayer perceptron networks can take on a similar role to that of a fuzzy system in performing the function of being a tunable nonlinearity. An advantage that the fuzzy system may have, however, is that it often facilitates the incorporation of heuristic knowledge into the solution to the problem, which can, at times, have a significant impact on the quality of the solution Radial basis function neural Networks Some radial basis function neural networks are equivalent to some standard fuzzy systems in the sense that they are functionally equivalent (i.e, given the same inputs, they will produce the same outputs). To see this, suppose that in Equation(4.4)we let M=R(i.e, the number of receptive field units equal to the number of rules),)i=Di(i.e,the receptive field unit strengths equal to the output membership function centers), and choose the receptive field units as R(x)=1(x) (.e, choose the receptive field units to be the same as the premise membership functions). In this case we see that the radial basis function neural network is identical to a certain fuzzy system that uses center-average defuzzification. Thi fuzzy system is then given by y=/(x)=2=9(x) ∑A1(x) It is also interesting to note that the functional fuzzy system( the more general version of the Takagi-Sugeno fuzzy system) is equivalent to a class of two-layer neural networks 2001 The equivalence between this type of fuzzy system and a radial basis function neural network shows that all the hniques in this book for the above type of fuzzy system work in the same way for the above type of radial basis function neural network(or, using[2001, the techniques for the Takagi-Sugeno fuzzy system can be used for a type of multilayer radial basis function neural network) PDF文件使用" pdffactory Pro"试用版本创建ww. fineprint,com,cn
implement) is identical. Some label the intersection between fuzzy systems and neural networks with the term "fuzzy-neural" or "neuro-fuzzy" to highlight that techniques from both fields are being used. Here, we avoid this terminology and simply highlight the basic relationships between the two fields. Multilayer Perceptrons The multilayer perceptron should be viewed as a nonlinear network whose nonlinearity can be tuned by changing the weights, biases, and parameters of the activation functions. The fuzzy system is also a tunable nonlinearity whose shape can be changed by tuning, for example, the membership functions. Since both are tunable nonlinearities, the following approaches are possible: ¡ Gradient methods can be used for training neural networks to perform system identification or to act as estimators or predictors in the same way as fuzzy systems were trained. Indeed, the gradient training of neural networks, called "back-propagation training," was introduced well before the gradient training of fuzzy systems, and the idea for training fuzzy systems this way came from the field of neural networks. ¡ Hybrid methods for training can also be used for neural networks. For instance, gradient methods may be used in conjunction with clustering methods applied to neural networks. ¡ Indirect adaptive control can also be achieved with a multilayer perceptron. To do this we use two multilayer perceptrons as the tunable nonlinearities in the certainty equivalence control law and the gradient method for tuning. ¡ Gain scheduled control may be achieved by training a multilayer perceptron to map the associations between operating conditions and controller parameters. This list is by no means exhaustive. It simply shows that multilayer perceptron networks can take on a similar role to that of a fuzzy system in performing the function of being a tunable nonlinearity. An advantage that the fuzzy system may have, however, is that it often facilitates the incorporation of heuristic knowledge into the solution to the problem, which can, at times, have a significant impact on the quality of the solution. Radial Basis Function Neural Networks Some radial basis function neural networks are equivalent to some standard fuzzy systems in the sense that they are functionally equivalent (i.e., given the same inputs, they will produce the same outputs). To see this, suppose that in Equation (4.4) we let M =R (i.e., the number of receptive field units equal to the number of rules), i i y b = (i.e., the receptive field unit strengths equal to the output membership function centers), and choose the receptive field units as ( ) ( ) Ri i x x = m (i.e., choose the receptive field units to be the same as the premise membership functions). In this case we see that the radial basis function neural network is identical to a certain fuzzy system that uses center-average defuzzification. This fuzzy system is then given by 1 1 ( ) ( ) ( ) R i i i R i i b x y f x x m m = = = = å å It is also interesting to note that the functional fuzzy system (the more general version of the Takagi-Sugeno fuzzy system) is equivalent to a class of two-layer neural networks [200]. The equivalence between this type of fuzzy system and a radial basis function neural network shows that all the techniques in this book for the above type of fuzzy system work in the same way for the above type of radial basis function neural network (or, using [200], the techniques for the Takagi-Sugeno fuzzy system can be used for a type of multilayer radial basis function neural network). PDF 文件使用 "pdfFactory Pro" 试用版本创建 www.fineprint.com.cn
Due to the above relationships between fuzzy systems and neural networks, some would like to view fuzzy systems and neural networks as identical areas This is. however not the case for the following reasons a There are classes of neural networks(e.g, dynamic neural networks )that may have a fuzzy system analog, but if so it would have to include not only standard fuzzy components but some form of a differential equation There are certain fuzzy systems that have no clear neural analog. Consider, for example, certain"fuzzy dynamic systems"[48, 167. We can, however, envision how you could go about"designing a neural analog to such fuzzy systems a The neural network has traditionally been a"black box"approach where the weights and biases are trained (e.g using gradient methods like back-propagation)using data, often without using extra heuristic knowledge we often have In fuzzy systems you can incorporate heuristic information and use data to train them. This last difference is often quoted as being one of the advantages of fuzzy systems over neural networks, at least for some applications Regardless of the differences, it is important to note that many methods in neural control (i.e, when we use a neural network for the control of a system) are quite similar to those in adaptive fuzzy control. For instance, since the fuzzy system and radial basis function neural network can be linearly parameterized, we can use them as the identifier structures in direct or indirect adaptive control schemes and use gradient or least squares methods to update the parameters. Indeed, we could have used neural networks as the structure that we trained for all of the identification methods. In this sense we can use neural networks in system identification, estimation, and prediction, and as a direct ( fixed) controller that is trained with input-output data. Basically, to be fluent with the methods of adaptive fuzz systems and control, you must know the methods of neural controland vice versa 4.4 Genetic Algorithms A genetic algorithm(GA)uses the principles of evolution, natural selection, and genetics from natural biological systems in a computer algorithm to simulate evolution. Essentially, the genetic algorithm is an optimization technique that performs a parallel, stochastic, but directed search to evolve the most fit population. In this section we will introduce the genetic algorithm and explain how it can be used for design and tuning of fuzzy systems 4.4.1 Genetic Algorithms: A Tutorial The genetic algorithm borrows ideas from and attempts to simulate Darwin's theory on natural selection and Mendel's work in genetics on inheritance. The genetic algorithm is ah optimization technique that evaluates more than one area of the search space and can discover more than one solution to a problem. In particular, it provides a stochastic optimization method where if it"gets stuck"at a local optimum, it tries to simultaneously find other parts of the search space and jump out"of the local optimum to a global one Representation and the Population of Individuals The "fitness function ures the fitness of an individual to survive in a population of individuals. The genetic algorithm will seek to maximize the fitness function Je)by selecting the individuals that we represent with 6. To represent the genetic algorithm in a computer, we make 6 a string. In particular, we show such a string in Figure 4. 4.A tring is a chromosome in a biological system. It is a string of "genes"that can take on different"alleles. In a computer PDF文件使用" pdffactory Pro"试用版本创建ww, fineprint,com,cn
Due to the above relationships between fuzzy systems and neural networks, some would like to view fuzzy systems and neural networks as identical areas. This is, however, not the case for the following reasons: ¡ There are classes of neural networks (e.g., dynamic neural networks) that may have a fuzzy system analog, but if so it would have to include not only standard fuzzy components but some form of a differential equation component. ¡ There are certain fuzzy systems that have no clear neural analog. Consider, for example, certain "fuzzy dynamic systems" [48, 167]. We can, however, envision how you could go about "designing a neural analog to such fuzzy systems. ¡ The neural network has traditionally been a "black box" approach where the weights and biases are trained (e.g., using gradient methods like back-propagation) using data, often without using extra heuristic knowledge we often have. In fuzzy systems you can incorporate heuristic information and use data to train them. This last difference is often quoted as being one of the advantages of fuzzy systems over neural networks, at least for some applications. Regardless of the differences, it is important to note that many methods in neural control (i.e., when we use a neural network for the control of a system) are quite similar to those in adaptive fuzzy control. For instance, since the fuzzy system and radial basis function neural network can be linearly parameterized, we can use them as the identifier structures in direct or indirect adaptive control schemes and use gradient or least squares methods to update the parameters. Indeed, we could have used neural networks as the structure that we trained for all of the identification methods. In this sense we can use neural networks in system identification, estimation, and prediction, and as a direct (fixed) controller that is trained with input-output data. Basically, to be fluent with the methods of adaptive fuzzy systems and control, you must know the methods of neural control—and vice versa. 4.4 Genetic Algorithms A genetic algorithm (GA) uses the principles of evolution, natural selection, and genetics from natural biological systems in a computer algorithm to simulate evolution. Essentially, the genetic algorithm is an optimization technique that performs a parallel, stochastic, but directed search to evolve the most fit population. In this section we will introduce the genetic algorithm and explain how it can be used for design and tuning of fuzzy systems. 4.4.1 Genetic Algorithms: A Tutorial The genetic algorithm borrows ideas from and attempts to simulate Darwin's theory on natural selection and Mendel's work in genetics on inheritance. The genetic algorithm is ah optimization technique that evaluates more than one area of the search space and can discover more than one solution to a problem. In particular, it provides a stochastic optimization method where if it "gets stuck" at a local optimum, it tries to simultaneously find other parts of the search space and "jump out" of the local optimum to a global one. Representation and the Population of Individuals The "fitness function" measures the fitness of an individual to survive in a population of individuals. The genetic algorithm will seek to maximize the fitness function J(θ) by selecting the individuals that we represent with θ . To represent the genetic algorithm in a computer, we make θ a string. In particular, we show such a string in Figure 4.4. A string is a chromosome in a biological system. It is a string of "genes" that can take on different "alleles." In a computer PDF 文件使用 "pdfFactory Pro" 试用版本创建 www.fineprint.com.cn