Chapter One-Foundations an organism is called the genotype.In evolutionary computation,the collection of patterns or strings needed to completely specify a system is known as a structure. Most of the systems considered in this text are specified by one pattern,or string,or state vector;the terms chromosome and structure are thus generally interchangeable. In the biological world,chromosomes are made up of genes,each of which is identified by its location (locus)and its function,such as a person's hair color gene. In other words,genes are specific segments of chromosomes associated with specific functions.Individual values a gene may assume are called alleles;a hair color allele value may be "brown hair."In the artificial chromosomes of evolutionary compu- tation systems,the chromosome patterns or strings are made up of parameters,or features,that can vary over a specified range of values.A given parameter or feature occupies a fixed location in the artificial chromosome.The chromosome therefore is encoded to represent a set of parameters. Biological versus Artificial Chromosomes Just as artificial neural networks are only roughly analogous to collections of bio- logical neurons,so artificial chromosomes are only approximately modeled after biological ones. Composition Biological chromosomes contain linear threads of DNA,nucleic acids that make up an extremely complex double helix structure.Artificial chromosomes are typically strings ofbinary and/or real values.Each occurrence ofthe string typically represents a system state vector. Length The biological chromosomes that define an organism vary in length,although a spe- cific chromosome is generally the same length from one organism to another.Each artificial chromosome in a population is the same length,that is,contains the same number of bits. Reproduction Biological chromosomes duplicate themselves during cell division,which occurs during a normal cell's lifetime.Many cell divisions(duplications)occur within an organism for every event of sexual reproduction.During reproduction,the egg and the sperm each contribute one chromosome for each homologous pair.In evolu- tionary computation,the duplication of chromosomes analogous to what occurs during biological cell division is generally called"reproduction."Also,the synthe- sis of new chromosomes from two "parents"is called crossover,or recombination, in evolutionary computation.Furthermore,during crossover (or recombination), any number of bits or real values can be exchanged between two parent artificial
Chapter OnemFoundations an organism is called the genotype. In evolutionary computation, the collection of patterns or strings needed to completely specify a system is known as a structure. Most of the systems considered in this text are specified by one pattern, or string, or state vector; the terms chromosome and structure are thus generally interchangeable. In the biological world, chromosomes are made up of genes, each of which is identified by its location (locus) and its function, such as a person's hair color gene. In other words, genes are specific segments of chromosomes associated with specific functions. Individual values a gene may assume are called alleles; a hair color allele value may be "brown hair." In the artificial chromosomes of evolutionary computation systems, the chromosome patterns or strings are made up of parameters, or features, that can vary over a specified range of values. A given parameter or feature occupies a fixed location in the artificial chromosome. The chromosome therefore is encoded to represent a set of parameters. Biological versus Artificial Chromosomes Just as artificial neural networks are only roughly analogous to collections of biological neurons, so artificial chromosomes are only approximately modeled after biological ones. Composition Biological chromosomes contain linear threads of DNA, nucleic acids that make up an extremely complex double helix structure. Artificial chromosomes are typically strings of binary and/or real values. Each occurrence of the string typically represents a system state vector. Length The biological chromosomes that define an organism vary in length, although a specific chromosome is generally the same length from one organism to another. Each artificial chromosome in a population is the same length, that is, contains the same number of bits. Reproduction Biological chromosomes duplicate themselves during cell division, which occurs during a normal cell's lifetime. Many cell divisions (duplications) occur within an organism for every event of sexual reproduction. During reproduction, the egg and the sperm each contribute one chromosome for each homologous pair. In evolutionary computation, the duplication of chromosomes analogous to what occurs during biological cell division is generally called "reproduction." Also, the synthesis of new chromosomes from two "parents" is called crossover, or recombination, in evolutionary computation. Furthermore, during crossover (or recombination), any number of bits or real values can be exchanged between two parent artificial
Behavioral Motivations for Fuzzy Logic chromosomes,as compared with the fixed 50 percent contribution of chromosomes by each parent in human reproduction.3 This section has primarily discussed the biological basis of evolutionary compu- tation from a genetics point of view.Concepts such as survival of the fittest,associ- ated with Darwinian evolution,also play an important role in CI and are discussed in Chapter 3. Behavioral Motivations for Fuzzy Logic The biological motivation or basis for fuzzy logic does not originate at the cellular and subcellular level,as is the case with neural networks and evolutionary compu- tation,respectively.It is reflected at the behavioral level of the organism,that is, in the ways the organism interacts with its environment.While the previous two methods are deeply rooted in biology,fuzzy logic deals mainly with uncertainty and vagueness.We do not live in a world of ones and zeros,black and white,true and false,or other absolutes.Our observations,communications,and experiences almost always include a large measure of uncertainty.For example,a statement such as"Next year I will visit Hawaii"cannot be categorized in terms of truth and falsehood.It is uncertain. Two main types of uncertainty exist.One is statistical,based on the laws of prob- ability.An example of statistical uncertainty is the outcome of the toss of a coin. Observations or measurements can be used to resolve statistical uncertainty.For example,once the coin is tossed,no statistical uncertainty remains.The other type of uncertainty is nonstatistical and is based on vagueness,imprecision,and/or ambi- guity.Nonstatistical uncertainty is illustrated by statements such as"Go to bed pretty soon”and“Jim is very tall”and“That car is going around75 kilometers per hour. The concept of fuzziness is associated with nonstatistical uncertainty. Those of you who are experts in the English language may have noticed that, particularly in the first statement,the imperative state does not mesh very well with the vague qualifier"pretty soon."This,however,is exactly the kind of vague,messy English we often use for communication.One of the primary attributes of fuzzy logic is its ability to efficiently capture and manipulate these vague,messy concepts. Fuzziness is an inherent property of a system.It is not resolved or altered by observation or measurement.Allowing uncertainty in the description of a complex system makes it more tractable to analysis.Fuzzy logic thus provides a framework within which nonstatistical uncertainty can be defined,described,and analyzed A similar perspective on fuzzy logic is articulated by George Klir(Klir and Folger 1988),who refers to fuzziness as arising from what he calls"linguistic imprecision." 3 For more information on natural genetics,refer to a genetics text.A good choice is one written by Mange and Mange(1998)
Behavioral Motivations for Fuzzy Logic chromosomes, as compared with the fixed 50 percent contribution of chromosomes by each parent in human reproduction. 3 This section has primarily discussed the biological basis of evolutionary computation from a genetics point of view. Concepts such as survival of the fittest, associated with Darwinian evolution, also play an important role in CI and are discussed in Chapter 3. Behavioral Motivations for Fuzzy Logic The biological motivation or basis for fuzzy logic does not originate at the cellular and subcellular level, as is the case with neural networks and evolutionary computation, respectively. It is reflected at the behavioral level of the organism, that is, in the ways the organism interacts with its environment. While the previous two methods are deeply rooted in biology, fuzzy logic deals mainly with uncertainty and vagueness. We do not live in a world of ones and zeros, black and white, true and false, or other absolutes. Our observations, communications, and experiences almost always include a large measure of uncertainty. For example, a statement such as "Next year I will visit Hawaii" cannot be categorized in terms of truth and falsehood. It is uncertain. Two main types of uncertainty exist. One is statistical, based on the laws of probability. An example of statistical uncertainty is the outcome of the toss of a coin. Observations or measurements can be used to resolve statistical uncertainty. For example, once the coin is tossed, no statistical uncertainty remains. The other type of uncertainty is nonstatistical and is based on vagueness, imprecision, and/or ambiguity. Nonstatistical uncertainty is illustrated by statements such as "Go to bed pretty soon" and "Jim is very tall" and "That car is going around 75 kilometers per hour." The concept of fuzziness is associated with nonstatistical uncertainty. Those of you who are experts in the English language may have noticed that, particularly in the first statement, the imperative state does not mesh very well with the vague qualifier "pretty soon." This, however, is exactly the kind of vague, messy English we often use for communication. One of the primary attributes of fuzzy logic is its ability to efficiently capture and manipulate these vague, messy concepts. Fuzziness is an inherent property of a system. It is not resolved or altered by observation or measurement. Allowing uncertainty in the description of a complex system makes it more tractable to analysis. Fuzzy logic thus provides a framework within which nonstatistical uncertainty can be defined, described, and analyzed. A similar perspective on fuzzy logic is articulated by George Klir (Klir and Folger 1988), who refers to fuzziness as arising from what he calls "linguistic imprecision." 3 For more information on natural genetics, refer to a genetics text. A good choice is one written by Mange and Mange (1998)
Chapter One-Foundations Myths about Computational Intelligence There are a number of myths regarding computational intelligence.First,it is a myth that the only way to achieve results with CI tools is with a vast sum of money,a supercomputer,and an interdisciplinary team of Nobel laureates,as some commer- cial vendors imply.Having a supercomputer or a parallel processing machine isn't required to do something useful with CI tools.It's not even necessary to have a Sun workstation.A personal computer is a perfectly adequate hardware base for most implementation and application projects.So,with relatively simple hardware and software tools,it is possible to solve problems that are otherwise impossible or impractical.Computational intelligence tools do offer solutions to some problems that aren't feasible to solve in any other way known to the authors.That isn't a myth! What is a myth is that some combination of CI tools can solve all difficult engi- neering or computer science problems faster and cheaper than anything previously available.It is also a myth that CI tools can solve most problems single-handedly. They are often inappropriate for problems requiring precise calculations.For exam- ple,it is unlikely that anyone will ever successfully balance a checkbook with a neural network. Another statement that qualifies as mostly myth is that no programming is needed to use artificial neural networks.This is at best misleading.It is true that a neural network trains(adapts)and runs on input data and according to a set of rules that update the weights that connect the processing elements,or nodes,and that the learning of the network is not,strictly speaking,programmed.It is also true that computer-aided software engineering(CASE)tools are becoming more available and that little or no programming expertise may be required to use these tools to generate executable neural network code. It is also true,however,that in the real world of neural network applications, some programming is required to get from the specification of the problem to a solution.Neural network applications significantly reduce the requirement for reprogramming.Once the problem is specified,it is not unusual to reuse the net- work code repeatedly,making changes in data preprocessing and network runtime parameters. Furthermore,although it is accurate to say that computational intelligence tools such as neural networks can play a key role in the solution of several classes of problems that are difficult if not impossible to solve any other way currently known, it is almost always true that the CI portion of the solution is only a relatively small part of the overall system.For example,in terms of the total amount of computer code in a neural network-based solution,the network often accounts for only about 10 percent of the total solution.It is an absolutely indispensable 10 percent,and success would not be possible without it,but it is important to keep it in perspective.Preprocessing and further manipulation of the data to form
Chapter OnemFoundations Myths about Computational Intelligence There are a number of myths regarding computational intelligence. First, it is a myth that the only way to achieve results with CI tools is with a vast sum of money, a supercomputer, and an interdisciplinary team of Nobel laureates, as some commercial vendors imply. Having a supercomputer or a parallel processing machine isn't required to do something useful with CI tools. It's not even necessary to have a Sun workstation. A personal computer is a perfectly adequate hardware base for most implementation and application projects. So, with relatively simple hardware and software tools, it is possible to solve problems that are otherwise impossible or impractical. Computational intelligence tools do offer solutions to some problems that aren't feasible to solve in any other way known to the authors. That isn't a myth! What is a myth is that some combination of CI tools can solve all difficult engineering or computer science problems faster and cheaper than anything previously available. It is also a myth that CI tools can solve most problems single-handedly. They are often inappropriate for problems requiring precise calculations. For example, it is unlikely that anyone will ever successfully balance a checkbook with a neural network. Another statement that qualifies as mostly myth is that no programming is needed to use artificial neural networks. This is at best misleading. It is true that a neural network trains (adapts) and runs on input data and according to a set of rules that update the weights that connect the processing elements, or nodes, and that the learning of the network is not, strictly speaking, programmed. It is also true that computer-aided software engineering (CASE) tools are becoming more available and that little or no programming expertise may be required to use these tools to generate executable neural network code. It is also true, however, that in the real world of neural network applications, some programming is required to get from the specification of the problem to a solution. Neural network applications significantly reduce the requirement for reprogramming. Once the problem is specified, it is not unusual to reuse the network code repeatedly, making changes in data preprocessing and network runtime parameters. Furthermore, although it is accurate to say that computational intelligence tools such as neural networks can play a key role in the solution of several classes of problems that are difficult if not impossible to solve any other way currently known, it is almost always true that the CI portion of the solution is only a relatively small part of the overall system. For example, in terms of the total amount of computer code in a neural network-based solution, the network often accounts for only about 10 percent of the total solution. It is an absolutely indispensable 10 percent, and success would not be possible without it, but it is important to keep it in perspective. Preprocessing and further manipulation of the data to form
Computational Intelligence Application Areas pattern files for presentation to the network typically involve much of the code (although we'll show you a way to develop a neural network that eliminates much of the preprocessing).Interpreting and displaying the results often account for another large portion.4 Another myth about neural network and evolutionary computation applications is that it is necessary to know something about neural biology or biological genetics, respectively,to understand them.Nothing could be further from the truth.In fact, for most engineers and computer scientists,neural network and evolutionary com- putation tools can be considered just another(powerful)set of resources in the CI analysis toolkit.Furthermore,a good case can be made for the argument that neural networks are technical descendants of analog computing just as much as they are descended from biology or neurology. A myth about fuzzy logic is that it is really fuzzy,or imprecise.It is not.The inputs to a fuzzy system are precise values for input parameters.Likewise,outputs from a fuzzy system are "crisp"(exact)values,capable,for instance,of being used as precise inputs to control systems. Another myth about fuzzy logic is that it is just another version of probability.It isn't.Probability deals with statistical uncertainty,whereas fuzzy logic is related to nonstatistical uncertainty,as we discussed previously. Finally,it is a myth that optimization exists.This is being said somewhat with tongue in cheek,but it is important to realize that very seldom does a real-world CI implementation find the absolute optimum ofanything.It is almost always sufficient to get within a specified region of the optimum,if it is known.Often,in fact,the optimum value is not even known.Note that we use the term optimization in its puredictionary definition"sense:Optimization is the identification of the very best solution,or,in the case in which multiple optima exist,the identification of all of the multiple optima. Computational Intelligence Application Areas Each component methodology of computational intelligence has application areas for which it is particularly well suited.We briefly review these areas in this section. Keep in mind that application areas may overlap;that is,a given problem may be solvable by either a neural network or a fuzzy system,albeit with different levels of performance.In later chapters we examine combinations of the methodologies that can produce different results.This compilation of application areas is not meant 4The 10 percent of the code typically represented by the neural network often takes a dispropor- tionately large percentage of the development effort,perhaps 20 percent,but that effort associated directly with neural network application development is usually still a relatively small portion of the total project
Computational Intelligence Application Areas pattern files for presentation to the network typically involve much of the code (although we'll show you a way to develop a neural network that eliminates much of the preprocessing). Interpreting and displaying the results often account for another large portion. 4 Another myth about neural network and evolutionary computation applications is that it is necessary to know something about neural biology or biological genetics, respectively, to understand them. Nothing could be further from the truth. In fact, for most engineers and computer scientists, neural network and evolutionary computation tools can be considered just another (powerful) set of resources in the CI analysis toolkit. Furthermore, a good case can be made for the argument that neural networks are technical descendants of analog computing just as much as they are descended from biology or neurology. A myth about fuzzy logic is that it is really fuzzy, or imprecise. It is not. The inputs to a fuzzy system are precise values for input parameters. Likewise, outputs from a fuzzy system are "crisp" (exact) values, capable, for instance, of being used as precise inputs to control systems. Another myth about fuzzy logic is that it is just another version of probability. It isn't. Probability deals with statistical uncertainty, whereas fuzzy logic is related to nonstatistical uncertainty, as we discussed previously. Finally, it is a myth that optimization exists. This is being said somewhat with tongue in cheek, but it is important to realize that very seldom does a real-world CI implementation find the absolute optimum of anything. It is almost always sufficient to get within a specified region of the optimum, if it is known. Often, in fact, the optimum value is not even known. Note that we use the term optimization in its pure "dictionary definition" sense: Optimization is the identification of the very best solution, or, in the case in which multiple optima exist, the identification of all of the multiple optima. Computational Intelligence Application Areas Each component methodology of computational intelligence has application areas for which it is particularly well suited. We briefly review these areas in this section. Keep in mind that application areas may overlap; that is, a given problem may be solvable by either a neural network or a fuzzy system, albeit with different levels of performance. In later chapters we examine combinations of the methodologies that can produce different results. This compilation of application areas is not meant 4 The 10 percent of the code typically represented by the neural network often takes a disproportionately large percentage of the development effort, perhaps 20 percent, but that effort associated directly with neural network application development is usually still a relatively small portion of the total project
Chapter One-Foundations to be complete.It is not necessarily even representative of all of the major areas of applications.It is meant to convey some sense of the range of problems to which CI's component methodologies have been applied. Neural Networks There are five application areas for which neural networks are generally considered to be best suited.The first three are related. Classification This area analyzes which of several predefined classes best reflects an input pattern. The number of classes is typically small compared with the number of inputs.One example is a decision whether or not a given segment of EEG data represents an epileptiform spike waveform.Neural networks'ability to construct nonlinear map- pings between high-dimensional spaces is another type of classification analysis. Some types of video image processing by neural networks(such as diagnoses of tumors)are examples of this application area. Content Addressable Memory or Associative Memory A typical example is obtaining the complete version of a pattern at the output of the network by providing a partial version at the input.(The input and output nodes of the network may sometimes be the same nodes.)This process is sometimes described as obtaining an exemplar pattern from a noisy and/or incomplete one. Clustering or Compression This area involves classification but can also be considered a form of encoding.An example is the significant reduction of the dimensionality of an input,as in the case of speech recognition.Another is the reduction of the number of bits that must be stored or transmitted to represent,within some allowed error margin,a block of data;in other words,the original block of data can be reconstructed within the allowed error with fewer bits than were in the original data. Generation of Sequences or Patterns This fourth area is somewhat different from the first three in that no classification is involved.This generation of patterns is done by a network trained to examples. For instance,if a network is trained to reproduce a certain style of musical sequence, then it is possible for the network to compose"original"versions of that type of music.Or a neural network may be trained to model,or simulate,something.Grow- ing numbers of applications in the financial world,becoming known as"financial engineering"applications,are being reported.Because of inherent randomness in the process being simulated,there may be no"right"answers,but the system can perhaps be described statistically.The network simulation may then be designed
Chapter OnemFoundations to be complete. It is not necessarily even representative of all of the major areas of applications. It is meant to convey some sense of the range of problems to which CI's component methodologies have been applied. Neural Networks There are five application areas for which neural networks are generally considered to be best suited. The first three are related. Classification This area analyzes which of several predefined classes best reflects an input pattern. The number of classes is typically small compared with the number of inputs. One example is a decision whether or not a given segment of EEG data represents an epileptiform spike waveform. Neural networks' ability to construct nonlinear mappings between high-dimensional spaces is another type of classification analysis. Some types of video image processing by neural networks (such as diagnoses of tumors) are examples of this application area. Content Addressable Memory or Associative Memory A typical example is obtaining the complete version of a pattern at the output of the network by providing a partial version at the input. (The input and output nodes of the network may sometimes be the same nodes.) This process is sometimes described as obtaining an exemplar pattern from a noisy and/or incomplete one. Clustering or Compression This area involves classification but can also be considered a form of encoding. An example is the significant reduction of the dimensionality of an input, as in the case of speech recognition. Another is the reduction of the number of bits that must be stored or transmitted to represent, within some allowed error margin, a block of data; in other words, the original block of data can be reconstructed within the allowed error with fewer bits than were in the original data. Generation of Sequences or Patterns This fourth area is somewhat different from the first three in that no classification is involved. This generation of patterns is done by a network trained to examples. For instance, if a network is trained to reproduce a certain style of musical sequence, then it is possible for the network to compose "original" versions of that type of music. Or a neural network may be trained to model, or simulate, something. Growing numbers of applications in the financial world, becoming known as "financial engineering" applications, are being reported. Because of inherent randomness in the process being simulated, there may be no "right" answers, but the system can perhaps be described statistically. The network simulation may then be designed