Preface This book is about self-organization,which is closely related to emergent computation.Self-organization involves simple processes that lead to complex results,and the whole being greater than the sum of its parts.As Stephen Wolfram (1994)said,"It is possible to make things of great complexity out of things that are very simple.There is no conservation of simplicity." It is about complex adaptive systems,a term that describes nonlinear systems com- prising the interaction of numerous adaptive elements,or entities.The concepts of self-organization and complexity are related,as we discuss later. This book is not an exhaustive treatise on all permutations and variations ofcom- putational intelligence and its constituent methodologies.If you want an exhaustive discussion of artificial neural network paradigms,for instance,you'll need to turn to another book.We present only those paradigms we believe provide the most useful tools for someone solving practical problems. It is not a compendium of mathematical derivations and proofs.We present only those few we believe are essential to gaining a working-level understanding of how and why the computational tools work. This book is not about agents.Most of our computational intelligence tools do not qualify as"agents"because they lack the required autonomy and specialization. They can,however,be incorporated into intelligent agents and agent systems. It is not about life.We nip around the edges of artificial life in a few places,but we don't address the question"What is alive?"(We do,however,share some pre- liminary thoughts on that subject.)We also do not address the search for artificial intelligence (whatever that is)or even for a computational intelligence tool from which intelligent behavior will emerge.Our focus is on solving problems. Throughout the text,additional aspects of our approach and philosophy should become evident,perhaps a little bit at a time.First,when considering computa- tional intelligence tools and systems,traditional distinctions between hardware and software get a bit blurred;distinctions between data and program are often almost nonexistent.Second,our emphasis is on problem solving and applications rather than physiological,biological,or behavioral plausibility.We do not pay too much attention to whether the CI tools reflect what actually goes on in the brain or any other part of a biological organism.Third,we believe that the activities of a com- putational intelligence application developer and user are often somewhat different from those in other technical areas. Developing computational intelligence applications requires the developer to play two roles.The first is the hands-on active design,develop,test,and debug role that is fairly common in other technical areas.The second,as important as the first, is a more passive observation and analytical thinking role.Results from a compu- tational intelligence tool are often not what was expected.Most of the time,if the developer takes the time to observe and think,rather than "bash to fit and paint to match,"something very useful can be learned
Preface This book is about self-organization, which is closely related to emergent computation. Self-organization involves simple processes that lead to complex results, and the whole being greater than the sum of its parts. As Stephen Wolfram (1994) said, "It is possible to make things of great complexity out of things that are very simple. There is no conservation of simplicity." It is about complex adaptive systems, a term that describes nonlinear systems comprising the interaction of numerous adaptive elements, or entities. The concepts of self-organization and complexity are related, as we discuss later. This book is not an exhaustive treatise on all permutations and variations of computational intelligence and its constituent methodologies. If you want an exhaustive discussion of artificial neural network paradigms, for instance, you'll need to turn to another book. We present only those paradigms we believe provide the most useful tools for someone solving practical problems. It is not a compendium of mathematical derivations and proofs. We present only those few we believe are essential to gaining a working-level understanding of how and why the computational tools work. This book is not about agents. Most of our computational intelligence tools do not qualify as "agents" because they lack the required autonomy and specialization. They can, however, be incorporated into intelligent agents and agent systems. It is not about life. We nip around the edges of artificial life in a few places, but we don't address the question "What is alive?" (We do, however, share some preliminary thoughts on that subject.) We also do not address the search for artificial intelligence (whatever that is) or even for a computational intelligence tool from which intelligent behavior will emerge. Our focus is on solving problems. Throughout the text, additional aspects of our approach and philosophy should become evident, perhaps a little bit at a time. First, when considering computational intelligence tools and systems, traditional distinctions between hardware and software get a bit blurred; distinctions between data and program are often almost nonexistent. Second, our emphasis is on problem solving and applications rather than physiological, biological, or behavioral plausibility. We do not pay too much attention to whether the CI tools reflect what actually goes on in the brain or any other part of a biological organism. Third, we believe that the activities of a computational intelligence application developer and user are often somewhat different from those in other technical areas. Developing computational intelligence applications requires the developer to play two roles. The first is the hands-on active design, develop, test, and debug role that is fairly common in other technical areas. The second, as important as the first, is a more passive observation and analytical thinking role. Results from a computational intelligence tool are often not what was expected. Most of the time, if the developer takes the time to observe and think, rather than "bash to fit and paint to match," something very useful can be learned
Preface Web Site Details The authors'web site for this book is http://www.computelligence.org/issue/CICI/ CICI.html.(There is a link to this site from the publisher's web site.)Software imple- mentations are written for the Windows and/or Java environment,and executable versions of software described in the implementation chapters are located and main- tained on the web site.Included as part of each implementation are the ancillary files-a run file and a data file-needed to run the implementation.In addition,out- put(results)files,obtained by the authors using the executable and ancillary files, are provided.You may want to rename these output files,or move them to another directory,so that you can compare your results with those of the authors. We'd like to emphasize that the software is not just for demonstration;you can use it for many real-world applications.The Cand C++source code has been written using the Borland C++4.5 development environment.The Java code will run on any computer that supports the Java Virtual Machine;this includes machines running Windows,Unix,and Macintosh operating systems. Of special note are the recent variations of particle swarm optimization that have been integrated into the EC theory and paradigms chapter and the EC implementa- tions chapter.Source code is provided on the web site for some of the implementa- tions so that you can modify the software for specific applications. Some of our software can be run using a web browser.Other software,including source code,is useful only after downloading it from the book's web site.Approx- imately 600 slides that cover the material in this book are available to instructors (or anyone else)at no cost.These slides,configured as Word files,are downloadable from the web site.The site also contains hyperlinks to other resource information on the Internet related to subjects in this book. A significant amount of source code is also on the web site.A total of eight software modules are available,both as executables and as source code: ■Genetic algorithm Particle swarm optimization(including multiple swarms) ■ Back-propagation neural network Learning vector quantization neural network Self-organizing feature map neural network Evolutionary back-propagation neural network ■Fuzzy rule system Evolutionary fuzzy rule system We ask that you send the authors a payment of US $25 per software module of source code($150 for all of the source code)if you find it useful.We are relying on your honesty.(The address is on the web site with the software.)
Preface ~.,~~ Web Site Details The authors' web site for this book is http://www.computelligence.org/issue/CICI/ CICI.html. (There is a link to this site from the publisher's web site.) Software implementations are written for the Windows and/or lava environment, and executable versions of software described in the implementation chapters are located and maintained on the web site. Included as part of each implementation are the ancillary files~a run file and a data file~needed to run the implementation. In addition, output (results) files, obtained by the authors using the executable and ancillary files, are provided. You may want to rename these output files, or move them to another directory, so that you can compare your results with those of the authors. We'd like to emphasize that the software is not just for demonstration; you can use it for many real-world applications. The C and C++ source code has been written using the Borland C++ 4.5 development environment. The lava code will run on any computer that supports the lava Virtual Machine; this includes machines running Windows, Unix, and Macintosh operating systems. Of special note are the recent variations of particle swarm optimization that have been integrated into the EC theory and paradigms chapter and the EC implementations chapter. Source code is provided on the web site for some of the implementations so that you can modify the software for specific applications. Some of our software can be run using a web browser. Other software, including source code, is useful only after downloading it from the book's web site. Approximately 600 slides that cover the material in this book are available to instructors (or anyone else) at no cost. These slides, configured as Word files, are downloadable from the web site. The site also contains hyperlinks to other resource information on the Internet related to subjects in this book. A significant amount of source code is also on the web site. A total of eight software modules are available, both as executables and as source code: m Genetic algorithm m Particle swarm optimization (including multiple swarms) m Back-propagation neural network m Learning vector quantization neural network [] Self-organizing feature map neural network [] Evolutionary back-propagation neural network [] Fuzzy rule system . Evolutionary fuzzy rule system We ask that you send the authors a payment of US $25 per software module of source code ($150 for all of the source code) if you find it useful. We are relying on your honesty. (The address is on the web site with the software.)
Preface Finally,as described previously,Chapter 12,Case Studies,is available on the web site. Acknowledgments Each of us has numerous people who should be acknowledged;we mention only a few. Russ Eberhart:First,I want to acknowledge my wife Francie and son Sean who put up with a higher than usual absence rate of their spouse and father,respectively. I also want to acknowledge my son Mark,a three-time cancer survivor,who has taught me what courage is.Special thanks go to my students in ECE 536,Intro- duction to Computational Intelligence.They were the guinea pigs.Sometimes,just from their eyes glazing over,I knew that a section needed to be rewritten(or deleted). Their patience is appreciated,and their input has been invaluable. Yuhui Shi:I would like to thank my parents and parents-in-law for taking good care of my daughter Melissa Xueyin Shi and my son Nicholas Yuge Shi so that I had plenty of quality time to work on this book.My thanks also go to profes- sors Zhenya He of Southeast University,M.N.S.Swamy and M.Omair Ahmad of Concordia University,Xin Yao ofthe University of Birmingham,Jinhyung Kim ofthe Korean Advanced Institute of Science and Technology,and to Russell C.Eberhart, who are my mentors and have paved the way for me in my career development. Both of us acknowledge the contributions of our technical reviewers.Their insights resulted in improvements in both the organization and content of this book. Finally,we are grateful to the team at Morgan Kaufmann Publishers who worked dili- gently with us throughout the process of writing,editing,and production.Working with Denise Penrose,Diane Cerra,Emilia Thiuri,Marilyn Rash,and Mary James has been a pleasure and a learning experience
Preface Finally, as described previously, Chapter 12, Case Studies, is available on the web site. Acknowledgments Each of us has numerous people who should be acknowledged; we mention only a few. Russ Eberhart:. First, I want to acknowledge my wife Francie and son Sean who put up with a higher than usual absence rate of their spouse and father, respectively. I also want to acknowledge my son Mark, a three-time cancer survivor, who has taught me what courage is. Special thanks go to my students in ECE 536, Introduction to Computational Intelligence. They were the guinea pigs. Sometimes, just from their eyes glazing over, I knew that a section needed to be rewritten (or deleted). Their patience is appreciated, and their input has been invaluable. Yuhui Shi: I would like to thank my parents and parents-in-law for taking good care of my daughter Melissa Xueyin Shi and my son Nicholas Yuge Shi so that I had plenty of quality time to work on this book. My thanks also go to professors Zhenya He of Southeast University, M. N. S. Swamy and M. Omair Ahmad of Concordia University, Xin Yao of the University of Birmingham, Jinhyung Kim ofthe Korean Advanced Institute of Science and Technology, and to Russell C. Eberhart, who are my mentors and have paved the way for me in my career development. Both of us acknowledge the contributions of our technical reviewers. Their insights resulted in improvements in both the organization and content of this book. Finally, we are grateful to the team at Morgan Kaufmann Publishers who worked diligently with us throughout the process of writing, editing, and production. Working with Denise Penrose, Diane Cerra, Emilia Thiuri, Marilyn Rash, and Mary James has been a pleasure and a learning experience
chapter one Foundations This chapter introduces general terms the differences between biological struc- used to discuss computational intelligence tures and these analysis tools,and we as well as component methodologies- discuss the behavioral motivations for fuzzy computational intelligence (CI),including systems.The chapter ends with a review of artificial neural networks,fuzzy logic,and myths related to implementations and appli- evolutionarycomputation-asthey are used cations of CI and its component technolo- in this text.We review the biological bases gies,and a review of major application areas for artificial neural network and evolution- for each of the three main computational ary computation analysis tools,including intelligence methodologies. ◆ 1
chapter one Foundations This chapter introduces general terms used to discuss computational intelligence as well as component methodologies~ computational intelligence (Cl), including artificial neural networks, fuzzy logic, and evolutionarycomputation~as they are used in this text. We review the biological bases for artificial neural network and evolutionary computation analysis tools, including the differences between biological structures and these analysis tools, and we discuss the behavioral motivations for fuzzy systems. The chapter ends with a review of myths related to implementations and applications of Cl and its component technologies, and a review of major application areas for each of the three main computational intelligence methodologies, m
Chapter One-Foundations Definitions This section defines some of the most important terms used in this book.These definitions set the stage for more detailed analyses;more comprehensive definitions appear in subsequent chapters.Often,the first time a term is used in the book,it is in italics.In addition,whenever a term is italicized,you can find its definition in the glossary. We begin with a general definition of intelligence and then focus on the issues relevant to computational intelligence.A standard dictionary(Webster's New Colle- giate Dictionary,1975)definition of intelligence is:"1 a(1):The ability to learn or understand or to deal with new or trying situations:REASON;also:the skilled use of reason(2):the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria(as tests)." "Intelligence is the capability ofa system to adapt its behavior to meet its goals in a range ofenvironments.It is a property of all purpose-driven decision-makers."This definition,perhaps more relevant to the subject matter of this book,was published by David Fogel (1995). An artificial neural network(ANN)is an analysis paradigm that is roughly mod- eled after the massively parallel structure of the brain.It simulates a highly inter- connected,parallel computational structure with many relatively simple individual processing elements(PEs).Henceforth in this text the terms artificial neural network and neural network are used interchangeably. As used in this text,fuzziness refers to nonstatistical imprecision and vagueness in information and data.Most concepts dealt with or described in the"real world" are fuzzy.For example,"It is kind of foggy outside now,but it should be fairly sunny before too long"is an example of a statement that incorporates three fuzzy concepts: “kind of,”“fairly,”and“before too long..”(It could even be argued that the word now"is imprecise and vague enough to be fuzzy.) Fuzzy sets model the properties of imprecision,approximation,or vagueness.In conventionallogic,known as crisp logic,an element either is or is not a member ofthe set.It can be said,therefore,that each element has a membership value of either 1 or 0 in the set.In a fuzzy set,fuzzy membership values reflect the membership extents (or grades)of the elements in the set.It will be shown that a membership function is the basic idea in fuzzy set theory;a fuzzy membership function is identical to a fuzzy set. Fuzzy logic is the logic of"approximate reasoning."It comprises operations on fuzzy sets including equality,containment,complementation,intersection,and union;it is a generalization of conventional (two-valued,or crisp)logic. Evolutionary computation comprises machine learning optimization and clas- sification paradigms roughly based on mechanisms of evolution such as biolog- ical genetics and natural selection.The evolutionary computation field includes genetic algorithms,evolutionary programming,genetic programming,evolution
Chapter One--Foundations Definitions This section defines some of the most important terms used in this book. These definitions set the stage for more detailed analyses; more comprehensive definitions appear in subsequent chapters. Often, the first time a term is used in the book, it is in italics. In addition, whenever a term is italicized, you can find its definition in the glossary. We begin with a general definition of intelligence and then focus on the issues relevant to computational intelligence. A standard dictionary (Webster's New Collegiate Dictionary, 1975) definition of intelligence is: "1 a (1): The ability to learn or understand or to deal with new or trying situations : REASON; also : the skilled use of reason (2): the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (as tests)." "Intelligence is the capability of a system to adapt its behavior to meet its goals in a range of environments. It is a property of all purpose-driven decision-makers." This definition, perhaps more relevant to the subject matter of this book, was published by David Fogel (1995). An artificial neural network (ANN) is an analysis paradigm that is roughly modeled after the massively parallel structure of the brain. It simulates a highly interconnected, parallel computational structure with many relatively simple individual processing elements (PEs). Henceforth in this text the terms artificial neural network and neural network are used interchangeably. As used in this text, fuzziness refers to nonstatistical imprecision and vagueness in information and data. Most concepts dealt with or described in the "real world" are fuzzy. For example, "It is kind of foggy outside now, but it should be fairly sunny before too long" is an example of a statement that incorporates three fuzzy concepts: "kind of," "fairly," and "before too long." (It could even be argued that the word "now" is imprecise and vague enough to be fuzzy.) Fuzzy sets model the properties of imprecision, approximation, or vagueness. In conventional logic, known as crisp logic, an element either is or is not a member ofthe set. It can be said, therefore, that each element has a membership value of either 1 or 0 in the set. In a fuzzy set, fuzzy membership values reflect the membership extents (or grades) of the elements in the set. It will be shown that a membership function is the basic idea in fuzzy set theory; a fuzzy membership function is identical to a fuzzy set. Fuzzy logic is the logic of "approximate reasoning." It comprises operations on fuzzy sets including equality, containment, complementation, intersection, and union; it is a generalization of conventional (two-valued, or crisp) logic. Evolutionary computation comprises machine learning optimization and classification paradigms roughly based on mechanisms of evolution such as biological genetics and natural selection. The evolutionary computation field includes genetic algorithms, evolutionary programming, genetic programming, evolution