1.2 Passage Between Conventional Approach to Pattern Classification k,图叫缴 (SI.M? M1M2) Fuzzy s1S2) (B1.S2) Smal Small -s Hedm·M Fig. 1.5 features are not fuzzy. But we can fuzzify them(measurable features) as we have done in case of unmeasurable features(see Fig. 1. 1)to represent our perception of classification(recognition) of different objects(patterns). For instance Rs: if first formant frequency is small and second formant frequency is medium then (m1/cI, m2 c2,.. mk/ck) where Ci, i =l,.,k, and m,j=1,., kare number of classes and membership values of the said classes respectively(see Fig Let us now consider Fig. 1.6 to provide further clarity about the process of fuzzification of the universe of feature axes and the pattern space All the values in [a, b] R-real line is the universe of feature axis F1. All measurable (i.e. sensed by transducers) values in [a, b] are the elements of the niverse. We discretize [a, b] by line segments to have finite number of generic elements of [a, b]. The primary terms S1, MI, Bi are represented by fuzzy sets defined over the said discrete universe of [a: b. Same things are true for the feature axis F2 Let us now define the pattern space as the cover of the classes CI: C2:g and h and we consider them to be the generic elements of the pattern space which is the se. From Fig. 1.6 it is obvious that CI and C2 are the classes of patterns Whereas classes like g and h represent the empty portions of the pattern space as shown in Fig. 1. 6. Classes like these are considered to be the total cover of the pattern space. In all pattern classification(recognition) problems we may consider such classes like g and h. The possibility of occurrence of different classes patterns under a particular Ff is represented by a fuzzy set which is defined over the pattern space and which is the consequent part of a MI
features are not fuzzy. But we can fuzzify them (measurable features) as we have done in case of unmeasurable features (see Fig. 1.1) to represent our perception of classification (recognition) of different objects (patterns). For instance, R5: if first formant frequency is small and second formant frequency is medium then mf g 1=~c1; m2=~c2; ::mk=~ck where ~ci; i ¼ 1; : : :; k, and mj, j = 1,…, k are number of classes and membership values of the said classes respectively (see Fig. A.1 and Example A.1). Let us now consider Fig. 1.6 to provide further clarity about the process of fuzzification of the universe of feature axes and the pattern space. All the values in [a, b] , R-real line is the universe of feature axis F1. All measurable (i.e. sensed by transducers) values in [a, b] are the elements of the universe. We discretize [a, b] by line segments to have finite number of generic elements of [a, b]. The primary terms S1, M1, B1 are represented by fuzzy sets defined over the said discrete universe of [a; b]. Same things are true for the feature axis F2. Let us now define the pattern space as the cover of the classes C~1; C~2; g and h and we consider them to be the generic elements of the pattern space which is the universe. From Fig. 1.6 it is obvious that C~1 and C~2 are the classes of patterns. Whereas classes like g and h represent the empty portions of the pattern space as shown in Fig. 1.6. Classes like these are considered to be the total cover of the pattern space. In all pattern classification (recognition) problems we may consider such classes like g and h. The possibility of occurrence of different classes of patterns under a particular F !f is represented by a fuzzy set which is defined over the pattern space and which is the consequent part of a MFI. Fig. 1.5 Quantized pattern space 1.2 Passage Between Conventional Approach to Pattern Classification 11
I Soft Computing Approach to Pattern Classification hord portion between classes overtopped zone produces fuzzy tition between discretization of [o, b] by line Fig. 1.6 Fuzzy sets represented over the discrete universe of feature axes and pattern space F1≡{S1,M1,B1},F2≡{S2,M2B2},C≡{C1;C2;g,h} I be the closed unit interval [0, 1], and /c be the collection of all mappings of C into According to the usual interpretation of MFl (i.e. Eq. A I of Appendix-A), we have to estimate the relation r such that RF1×F2→ and according to the new interpretation of MFI (i.e. Eq. A4 of Appendix-A)we have to estimate two relations RI and R as follows R1:F1→ and R Note that for any decision making based on fuzzy reasoning a relation/set of lations is constructed based on any suitable choice(which is adhoc in nature)of logical operator(e.g. Mamdanis min operator, Zadeh's arithmatic rule)between e antecedent clause(s)and consequent clause of fuzzy If-Then rule/rules. But rather we go by estimating the said relation/relations by soft compu g lool this monograph we do not go by any such adhoc choice of said logical operator From the given If? Then rule/rules for representing knowledge for supervised pattern classification or model based object recognition, once the fuzzy relation/ relations (i.e. either R or R, and r2) is/are estimated(Pedrycz 1990), we can use it/ them further for inferencing for classification(recognition) of patterns(objects)as
Let, F1 fS1; M1; B1g; F2 fS2; M2; B2g; C~ fC~1 ; C~2 ; g; hg; I be the closed unit interval [0, 1], and IC~ be the collection of all mappings of C~ into I. According to the usual interpretation of MFI (i.e. Eq. A.1 of Appendix-A), we have to estimate the relation R such that R:F1 F2!I C~ ; and according to the new interpretation of MFI (i.e. Eq. A4 of Appendix-A) we have to estimate two relations R1 and R2 as follows: R1:F1 ! I C~ and R2:F2 ! I C~ : Note that for any decision making based on fuzzy reasoning a relation/set of relations is constructed based on any suitable choice (which is adhoc in nature) of logical operator (e.g. Mamdani’s min operator, Zadeh’s arithmatic rule) between the antecedent clause(s) and consequent clause of fuzzy If–Then rule/rules. But in this monograph we do not go by any such adhoc choice of said logical operators; rather we go by estimating the said relation/relations by soft computing tools. From the given If?Then rule/rules for representing knowledge for supervised pattern classification or model based object recognition, once the fuzzy relation/ relations (i.e. either R or R1 and R2) is/are estimated (Pedrycz 1990), we can use it/ them further for inferencing for classification (recognition) of patterns (objects) as Fig. 1.6 Fuzzy sets represented over the discrete universe of feature axes and pattern space 12 1 Soft Computing Approach to Pattern Classification
1.2 Passage Between Conventional Approach to Pattern Classification Process of Fuzricabon Passage to pass with an a nitive process of human reasoni Conrentional Soft comun approach to MFI=(fr-c)= two different different soft computing interpretations methodologies for main9/9:ⅵ preces - firnfeaNn Fig. 1.7 Passage between conventional approach to pattern classification(object recognition and soft computing approach to pattern classification(object recognition) discussed in the following chapters. The fuzzy relation/relations as stated above is/ are the core of the classifier(recognizer) In Fig. 1.7, we depict the passage to pass from conventional approach to soft omputing approach and vice versa. Now we state the specific advantages in adopting soft computing approach to pattern classification(object recognition) 1. We obtain the description of the pattern space(formed by the features of the training patterns or model objects)in terms of few quantized zones. Depending upon the need of the problem we may increase/decrease the granularity of our description of the pattern space by smaller/bigger quantized zones 2. For estimating the relation R/R, Vi of a classifier(recognizer) we do not need to select the representative data set from the given set of data(patterns). Instead, we use the gross property of few populations of given data(patterns) spread over the pattern space by using few fuzzy pattern vectors which describe the overall distribution of patterns in pattern space 3. We obtain multiple classification which is very natural in case of overlapped classes of patterns. 4. If we try to classify the patterns on R, n>2 our new interpretation for MFI ven by equation(A4 of Appendix-A)does not suffer from the curse of large feature dimension. For instance, according to the conventional interpretation of MFI i. e, Eq(A I of Appendix-A), the antecedent clauses of fuzzy if-Then rules will produce a large dimension of relation when"n"becomes large, say given 3, 4, 5 etc. Whereas, according to our new interpretation, i.e. Eq.(A4 of Appendix-A)we may assume any large value of"n"which does not affect the 5. Using GA/neural net(Goldberg 1989: Michalewicz 1994: Pao 1989) approach we can avoid the computation of the derivative of fuzzy max- function and min-
discussed in the following chapters. The fuzzy relation/relations as stated above is/ are the core of the classifier (recognizer). In Fig. 1.7, we depict the passage to pass from conventional approach to soft computing approach and vice versa. Now we state the specific advantages in adopting soft computing approach to pattern classification (object recognition). 1. We obtain the description of the pattern space (formed by the features of the training patterns or model objects) in terms of few quantized zones. Depending upon the need of the problem we may increase/decrease the granularity of our description of the pattern space by smaller/bigger quantized zones. 2. For estimating the relation </<i Vi of a classifier (recognizer) we do not need to select the representative data set from the given set of data (patterns). Instead, we use the gross property of few populations of given data (patterns) spread over the pattern space by using few fuzzy pattern vectors which describe the overall distribution of patterns in pattern space. 3. We obtain multiple classification which is very natural in case of overlapped classes of patterns. 4. If we try to classify the patterns on Rn; n[ 2 our new interpretation for MFI given by equation (A4 of Appendix-A) does not suffer from the curse of large feature dimension. For instance, according to the conventional interpretation of MFI i.e., Eq. (A.1 of Appendix-A), the antecedent clauses of fuzzy if–Then rules will produce a large dimension of relation when ‘‘n’’ becomes large, say given 3,4,5 etc. Whereas, according to our new interpretation, i.e. Eq. (A4 of Appendix-A) we may assume any large value of ‘‘n’’ which does not affect the above said problem of dimensionality. 5. Using GA/neural net (Goldberg 1989; Michalewicz 1994; Pao 1989) approach we can avoid the computation of the derivative of fuzzy max-function and minfunction. Fig. 1.7 Passage between conventional approach to pattern classification (object recognition) and soft computing approach to pattern classification (object recognition) 1.2 Passage Between Conventional Approach to Pattern Classification 13
I Soft Computing Approach to Pattern Classification 6. Classifier(recognizer) based on fuzzy if-then rules learns faster than crisp classifier system(Schulz 1995). The reason for this is that fuzzy rules are high level languages which keep the cardinality of the term set smaller than crisp rules. Thus, we can describe the decision strategy over the pattern space by a ew number of rules References D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning(Addison Wesley, Reading, Bonston, 1989) M.w.Koch, R.L. Kashyap. Using to recognize and locate partially occluded objects IEEE Trans. Pattern Anal. Mach Z Michalewicz, Genetic Algorithm Data Structures Evolution Programs (Springer, New M. Mizumoto, Extended Fuzzy Reasoning in Approximate Reasoning in Expert Systems, ed by q pp 7 gupta, A.Kandel, W. Bandler. J.B. Kiszka(North-Holland, Amsterdam, 1985). MM G Newell, H.A. Simon, Human problem solving(Prentice-Hall, Englewood Cliffs, 1972) Y. H. Pao, Adaptive Pattern Recognition and Neural Networks(Addison Wesley Publishing Company, Reading, Boston. 1989) Irycz, Applications of fuzzy relational equations for methods of reasoning in presence of fuzzy data. Fuzzy Sets Syst. 16, 163-174(1985) w. Pedrycz, Fuzzy sets in pattern recognition methodology and methods. Pattern Recogn. 23 K. S. Ray, D. Dutta Mazumder, Application of differential geometry to recognize and locate partially occluded objects. Pattern Recogn. Lett. 9, 351-360(1989) G. Schulz, Fuzzy Rule Based Expert Systems and Genetic Machine Learning(Physica-verlag, M. Sugeno, T. Takagi, Multidimensional fuzzy reasoning. Fuzzy Sets Syst. 9, 313-325(1983) J. T. Tou, R C. Gonzalez, Pattern Recognition Principles(Addison-Wesley, Reading, 1974) Y. Tsukamoto, An approach to Fuzzy Reasoning Method, in Advance in Fuzzy Set Theory and Applications, ed by M. M. Gupta, R. K Ragade, R. R. Yager(North-Holland, Amsterdam, 1979,pp.I37-149 L. A. Zadeh, Theory of Approximate Reasoning, in Machine Intelligence, ed by Hayes J. E Donald Michie, L. I. Mikulich(Ellis Horwood, Chichester York.1970,pp.149194 L. A. Zadeh, Fuzzy Sets and their Applications to Classification and Clustering, in Classification d Clustering, ed by J. Van Ryzin(Academic, New York, 1977), Pp. 251-299 L. A. Zadeh, K.S. Fu, K. Tanaka, M. Shimura(eds ) Fuzzy Sets and their Applicatio Cognitive and Decision Processes(Academic, New York, 1975)
6. Classifier (recognizer) based on fuzzy if–then rules learns faster than crisp classifier system (Schulz 1995). The reason for this is that fuzzy rules are high level languages which keep the cardinality of the term set smaller than crisp rules. Thus, we can describe the decision strategy over the pattern space by a few number of rules. References D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison Wesley, Reading, Bonston, 1989) M.W. Koch, R. L. Kashyap, Using polygons to recognize and locate partially occluded objects. IEEE Trans. Pattern Anal. Mach. lntell. 9(4), 483–494 (1987) Z. Michalewicz, Genetic Algorithm ? Data Structures = Evolution Programs (Springer, New York, 1994) M. Mizumoto, Extended Fuzzy Reasoning, in Approximate Reasoning in Expert Systems, ed. by M.M. Gupta, A. Kandel, W. Bandler, J. B. Kiszka (North-Holland, Amsterdam, 1985), pp. 71–85 A. Newell, H.A. Simon, Human problem solving (Prentice-Hall, Englewood Cliffs, 1972) Y. H. Pao, Adaptive Pattern Recognition and Neural Networks (Addison Wesley Publishing Company, Reading, Boston, 1989) W. Pedrycz, Applications of fuzzy relational equations for methods of reasoning in presence of fuzzy data. Fuzzy Sets Syst. 16, 163–174 (1985) W. Pedrycz, Fuzzy sets in pattern recognition methodology and methods. Pattern Recogn. 23 (1/2), 121–146 (1990) K. S. Ray, D. Dutta Mazumder, Application of differential geometry to recognize and locate partially occluded objects. Pattern Recogn. Lett. 9, 351–360 (1989) G. Schulz, Fuzzy Rule Based Expert Systems and Genetic Machine Learning (Physica-verlag, Germany, 1995) M. Sugeno, T. Takagi, Multidimensional fuzzy reasoning. Fuzzy Sets Syst. 9, 313–325 (1983) J. T. Tou, R. C. Gonzalez, Pattern Recognition Principles (Addison-Wesley, Reading, 1974) Y. Tsukamoto, An approach to Fuzzy Reasoning Method, in Advance in Fuzzy Set Theory and Applications, ed. by M. M. Gupta, R. K. Ragade, R. R. Yager (North-Holland, Amsterdam, 1979), pp. 137–149 L. A. Zadeh, Theory of Approximate Reasoning, in Machine Intelligence, ed. by Hayes J. E., Donald Michie, L. I. Mikulich (Ellis Horwood, Chichester, New York, 1970), pp. 149–194 L. A. Zadeh, Fuzzy Sets and their Applications to Classification and Clustering, in Classification and Clustering, ed. by J. Van Ryzin (Academic, New York, 1977), pp. 251–299 L. A. Zadeh, K. S. Fu, K. Tanaka, M. Shimura (eds.), Fuzzy Sets and their Applications to Cognitive and Decision Processes (Academic, New York, 1975) 14 1 Soft Computing Approach to Pattern Classification
Chapter 2 Pattern classification based on Conventional Interpretation of MFI Abstract Our aim is to design a pattern classifier using fuzzy relational (FRC) which was initially proposed by Pedrycz(Pattern Recognition 23 (1/2). 21-146, 1990). In the course of doing so, we first consider a particular inter- pretation of the multidimensional fuzzy implication (MFI)to represent ou knowledge about the training data set. Subsequently, we introduce the notion of a uzzy pattern vector to represent a population of training patterns in the pattern space and to denote the antecedent part of the said particular interpretation of the MFI. We introduce a new approach to the computation of the derivative of the fuzzy max-function and min-function using the concept of a generalized function During the construction of the classifier based on FRC, we use fuzzy linguistic statements(or fuzzy membership function to represent the linguistic statement)to represent the values of features (e. g, feature FI is small and F2 is big)for population of patterns. Note that the construction of the classifier essentially depends on the estimate of a fuzzy relation 3 between the input (fuzzy set)and output(fuzzy set)of the classifier. Once the classifier is constructed, the nonfuzz features of a pattern can be classified. At the time of classification of the nonfuzzy features of the test patterns, we use the concept of fuzzy masking to fuzzify the lonfuzzy feature values of the test patterns. The performance of the proposed scheme is tested on synthetic data. Finally, we use the proposed scheme for the vowel classification problem of an Indian language 2.1 Introduction In real world pattern classification problems, fuzziness is connected with diverse facets of cognitive activity of human being. The sources of fuzziness are to labels expressed in pattern space, as well as, labels of classes taken K. S. Ray, Soft Computing Approach to Pattern Classificatie and Object Recognition, DOI: 10.1007/978-1-4614-5348-22 a Springer Science+Business Media New York 2012
Chapter 2 Pattern Classification Based on Conventional Interpretation of MFI Abstract Our aim is to design a pattern classifier using fuzzy relational calculus (FRC) which was initially proposed by Pedrycz (Pattern Recognition 23 (1/2), 121–146, 1990). In the course of doing so, we first consider a particular interpretation of the multidimensional fuzzy implication (MFI) to represent our knowledge about the training data set. Subsequently, we introduce the notion of a fuzzy pattern vector to represent a population of training patterns in the pattern space and to denote the antecedent part of the said particular interpretation of the MFI. We introduce a new approach to the computation of the derivative of the fuzzy max-function and min-function using the concept of a generalized function. During the construction of the classifier based on FRC, we use fuzzy linguistic statements (or fuzzy membership function to represent the linguistic statement) to represent the values of features (e.g., feature F1 is small and F2 is big) for a population of patterns. Note that the construction of the classifier essentially depends on the estimate of a fuzzy relation < between the input (fuzzy set) and output (fuzzy set) of the classifier. Once the classifier is constructed, the nonfuzzy features of a pattern can be classified. At the time of classification of the nonfuzzy features of the test patterns, we use the concept of fuzzy masking to fuzzify the nonfuzzy feature values of the test patterns. The performance of the proposed scheme is tested on synthetic data. Finally, we use the proposed scheme for the vowel classification problem of an Indian language. 2.1 Introduction In real world pattern classification problems, fuzziness is connected with diverse facets of cognitive activity of human being. The sources of fuzziness are related to labels expressed in pattern space, as well as, labels of classes taken into K. S. Ray, Soft Computing Approach to Pattern Classification and Object Recognition, DOI: 10.1007/978-1-4614-5348-2_2, Springer Science+Business Media New York 2012 15