2Fs-s u OvE 2EEE7Xs LE0 AEF 1ii8 Error Rate (% 0 5 3 Test .Training 2 16 20 Training set Iterations Fig.5.Training and test error of LeNet-5 as a function of the num- ber of passes through the 60,000 pattern training set Rvithout distortions&The average training error is measured on-the-fly as 8 8 8 training proceeds.This eplains why the training error appears to be larger than the test error.Convergence is attained after 10 99 9 9 to 12 passes through the training set. Fig.7.Eamples of distortions of ten training patterns. Error Rate ( 1.8 1.6 Test error(no distortions) ,4B。正，。3，品，品品 8克女3思。名名人 8品可32息。乌如 0.8 Test error (with distortions) 0.e 予，4总克总88。 0.4 0、 Training error(no distortions) 002030400070动01d0 。子2子2日 可o Training Set size (x1000) Fig.6.Training and test errors of LeNet-5 achieved using training sets of various sizes.This graph suggests that a larger training Fig.8.The 82 test patterns misclassiged by LeNet-5.Below each set could improve the performance of LeNet-5.The hollow square image is displayed the correct answers Heft8 and the network an- show the test error when more training patterns are artigcially swer Right&These errors are mostly caused either by genuinely generated using random distortions.The test patterns are not ambiguous patterns,or by digits written in a style that are under- distorted. represented in the training set. distorted patterns with randomly picked distortion param- perfectly identifiable by humans,although they are writ- eters.The distortions were combinations of the follow- ten in an under-represented style.This shows that further ing planar affine transformations:horizontal and verti- improvements are to be expected with more training data. cal translations,scaling,squeezing (simultaneous horizon- tal compression and vertical elongation,or the reverse), S.Somparon wYh Other Slass Yiers and horizontal shearing.Figure 7 shows examples of dis- For the sake of comparison,a variety of other trainable torted patterns used for training.When distorted data was classifiers was trained and tested on the same database.An used for training,the test error rate dropped to 0.i%(from early subset of these results was presented in 51.The error 0.95%without deformation).The same training parame- rates on the test set for the various methods are shown in ters were used as without deformations.The total length of figure 9. the training session was left unchanged(20 passes of p0,000 patterns each).It is interesting to note that the network C.1 Linear Classifier,and Xairwise Linear Classifier effectively sees each individual sample only twice over the Xossibly the simplest classifier that one might consider is course of these 20 passes. a linear classifier.Each input pixel value contributes to a Figure i shows all i2 misclassified test examples.some weighted sum for each output unit.The output unit with of those examples are genuinely ambiguous,but several are the highest sum (including the contribution of a bias con-
✂✁☎✄✝✆✟✞✠✄☛✡✌☞✎✍✟✏✒✑✓✏✂✏✂✏✎✔✖✕☛✄☎✗☛✏✙✘✛✚✙✏✂✁✢✜✤✣✥✣✧✦ ✜✥✜ 0 4 8 12 16 20 4% 2% 0% Test Training Error Rate (%) 1% 3% 5% Training set Iterations ✁❼✿▲❍✪❦ ✁★❦➵✮✦✸✶✱✴✿▲❈✪✿▲❈✪❍✎✱✴❈✪❉④✵✻✰❅✺✻✵❀✰❅✸✶✸✶✷✹✸❢✷✴⑥❼P❩✰❺❤❀✰r✵✻❑✂✁✎✱✴✺❜✱❹⑥✠✳✪❈★❆❅✵✻✿▲✷✹❈✞✷✴⑥✦✵✻✯✪✰❷❈✄✳✪❋✔❑ ◗✑✰❅✸✔✷✴⑥❜❃♦✱✴✺✶✺✻✰❅✺✎✵✶✯★✸✶✷✹✳✪❍✹✯➵✵✻✯✪✰t❽✹❪★❚ ❪✹❪✹❪➓❃♦✱❘✵ ✵✶✰❅✸✶❈➣✵✻✸❖✱✴✿▲❈✪✿▲❈✪❍➓✺✻✰❅✵✁✙✽❀✿ ✵✻✯✪✷✹✳★✵ ❉✪✿▲✺✻✵✻✷✹✸✻✵✻✿▲✷✹❈✪✺✄✂❖❦✗✮✏✯✪✰❀✱❇▼❯✰❅✸❖✱✴❍✹✰❀✵✶✸✶✱✴✿▲❈✪✿▲❈✪❍❹✰❅✸✶✸✶✷✹✸✗✿▲✺✈❋✩✰❇✱✴✺✻✳✪✸✶✰❅❉✔✷✹❈★❑✠✵✶✯✪✰r❑✆☎★❙✩✱✴✺ ✵✶✸✶✱✴✿▲❈✪✿▲❈✪❍✛❃✪✸✶✷✄❆❅✰❅✰❇❉★✺❺❦❷✮✏✯✪✿▲✺❳✰☎✒❃✪❴➂✱✴✿▲❈✪✺❳✽❳✯❯❙✞✵✻✯✪✰➜✵✶✸✶✱✴✿▲❈✪✿▲❈✪❍✛✰❅✸✶✸✶✷✹✸❜✱✴❃✪❃❄✰❇✱✴✸✶✺ ✵✶✷➃◗✑✰❳❴➂✱✴✸✶❍✹✰❅✸✇✵✻✯♦✱✴❈✬✵✻✯✪✰❜✵✻✰❅✺✻✵✏✰❅✸✶✸✻✷✹✸❺❦✗❻✇✷✹❈❯▼❯✰❅✸✶❍✹✰❅❈✪❆❅✰❷✿▲✺✏✱❘✵✻✵✶✱✴✿▲❈✪✰❅❉t✱❘⑥➟✵✻✰❇✸❨❬❇❪ ✵✶✷✞❬❇❾➃❃♦✱✴✺✶✺✻✰❇✺✏✵✻✯✪✸✶✷✹✳✪❍✹✯④✵✶✯✪✰❨✵✻✸❖✱✴✿▲❈★✿▲❈✪❍✔✺✶✰r✵❇❦ 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Training error (no distortions) Test error (no distortions) Test error (with distortions) Training Set Size (x1000) 10 20 30 40 50 60 70 80 90 100 Error Rate (%) ✁❼✿▲❍✪❦➎❽★❦⑧✮✦✸✶✱✴✿▲❈✪✿▲❈✪❍④✱✴❈✪❉✌✵✶✰❅✺✵❷✰❅✸✶✸✶✷✹✸✻✺❨✷✴⑥✇P❩✰❺❤❀✰r✵✻❑✂✁✬✱✴❆❖✯✪✿▲✰r▼❯✰❅❉➉✳✪✺✶✿▲❈✪❍✛✵✻✸❖✱✴✿▲❈✪✿▲❈✪❍ ✺✶✰r✵✶✺❹✷✴⑥✇▼✴✱✴✸✶✿▲✷✹✳✪✺❷✺✻✿▲❵❅✰❇✺❇❦✔✮✏✯✪✿▲✺❷❍✹✸✶✱✴❃✪✯➓✺✶✳✪❍✹❍✹✰❅✺✻✵✻✺❹✵✶✯♦✱❘✵➃✱✛❴➂✱✴✸✶❍✹✰❅✸❨✵✻✸❖✱✴✿▲❈✪✿▲❈✪❍ ✺✶✰r✵❼❆❅✷✹✳✪❴▲❉✎✿▲❋✩❃✪✸✻✷❺▼❯✰✗✵✶✯✪✰✈❃✑✰❅✸⑥✠✷✹✸✻❋✛✱✴❈✪❆❅✰✈✷✴⑥❄P☛✰❇❤❳✰r✵ ❑✂✁★❦❼✮✏✯✪✰✇✯✪✷✹❴▲❴▲✷❇✽➵✺✶⑦✄✳♦✱✴✸✻✰ ✺✶✯✪✷❇✽①✵✶✯✪✰✔✵✻✰❅✺✻✵➃✰❅✸✶✸✶✷✹✸❨✽❀✯✪✰❅❈➑❋✩✷✹✸✶✰✎✵✻✸❖✱✴✿▲❈✪✿▲❈★❍✞❃♦✱❘✵ ✵✶✰❅✸✻❈✪✺✔✱✴✸✻✰✩✱✴✸✻✵✻✿✂♦❆❅✿➂✱✴❴▲❴❙ ❍✹✰❇❈★✰❇✸✶✱❘✵✶✰❅❉➣✳✪✺✻✿▲❈✪❍✌✸❖✱✴❈✪❉★✷✹❋✟❉✪✿▲✺✻✵✻✷✹✸✻✵✻✿▲✷✹❈✪✺❇❦➓✮❀✯★✰✛✵✻✰❅✺✻✵✎❃♦✱❘✵✻✵✻✰❅✸✶❈✪✺✔✱✴✸✶✰✛❈✪✷✴✵ ❉✪✿▲✺✻✵✻✷✹✸✻✵✻✰❅❉❩❦ ù✂ø✓ê④è✥ì➈ä✧è✧ï❞ù➻æ☎å✠è✧è✧ï➊ä✥é☎ê❫ó❨ø➩è✥ç➓ä✥å➈é☎ù✂ì➈î➻û✠✘❙æ☎ø✠❿ô❛ï✖ù➻ù✂ø✓ê④è✥ì➈ä✧è✧ø➀ì➈é➓æ☎å➈ä✥å➈î➑✝ ï❶è✥ï➊ä❿ê➊ðPã✛ç✙ï ù✂ø➀ê✤è✧ì❛ä✤è✥ø➑ì❛é☎ê✢ó✇ï✖ä✧ï ❶ì❛î➉✡✙ø➀é☎å✠è✥ø➑ì❛é☎ê✢ì➄ë➞è✥ç✙ï ëíì➈û➀û➑ì✠ó✔✝ ø➀é❼✂✻æ✙û✓å➄é☎å➈ä❺å❆✯➻é☎ï è✧ä❿å➄é☎ê✤ëíì➈ä✥î➓å✠è✥ø➑ì❛é☎ê✱✰✾ç✙ì➈ä✥ø✠➽➊ì➈é✐è❿å➄û✒å➈é☎ù ú➈ï✖ä✤è✥ø➟✝ ➊å➈û❀è✥ä✥å➈é☎ê✧û➀å➄è✧ø➀ì➈é☎ê✒➌◆ê❘➊å➈û➑ø➀é❼✂✥➌◆ê❘➍✐ÿ✙ï➊ï✒➽➊ø➀é❼✂➛➪üê✤ø➀î✒ÿ✙û➑è✥å➈é✙ï➊ì❛ÿ☎ê♣ç✙ì➈ä✥ø✙➽✖ì➈é✦✝ è✥å➈û✞➊ì➈î➻æ✙ä✥ï✖ê✥ê✤ø➀ì➈é✻å➈é☎ù➘ú➈ï✖ä✤è✥ø✠✖å➄û♣ï➊û➀ì➈é❼✂✐å✠è✧ø➀ì➈é❀➌❨ì➈ä➽è✥ç✙ï ä✧ï✖ú➈ï➊ä❿ê✧ï★➶✹➌ å➄é✎ù❺ç☎ì➈ä✥ø✙➽✖ì➈é✐è✥å➈û❯ê✧ç✙ï✖å➈ä✧ø➀é❼✂✎ð➝✜❯ø✠✂➈ÿ✙ä✥ï✙✔✢ê✤ç✙ì✠ó➔ê➳ï✄↔✂å➈î➻æ✙û➑ï❞ê➤ì➄ë❨ù✂ø✓ê❅✝ è✧ì❛ä✤è✥ï✖ù❙æ☎å✠è✧è✧ï✖ä✧é☎ê❯ÿ☎ê✧ï✖ù✒ëíì➈ä❦è✥ä✥å➈ø➑é☎ø➑é❼✂✎ð ✎ç✙ï✖é❭ù✙ø➀ê✤è✧ì❛ä✤è✥ï✖ù❭ù✙å✠è❿å➉ó✛å➈ê ÿ☎ê✧ï✖ù❙ëíì➈ä❯è✥ä✥å➈ø➑é✙ø➀é❼✂✥➌✠è✧ç✙ï✛è✥ï✖ê✤è➏ï➊ä✥ä✧ì❛ä❇ä❿å✠è✥ï❨ù✂ä✧ì❛æ✙æ◆ï✖ù✒è✧ì✛✘✙ð ✺✞✶ ➪íëíä✧ì❛î ✘✙ð ❀ ✙✞✶▼ó❨ø➑è✧ç✙ì❛ÿ✂è✒ù✂ï❶ëíì❛ä✧î➓å➄è✧ø➀ì➈é✥➶❶ð✶ã✛ç✙ï➽ê✥å➄î➻ï❭è✧ä❿å➄ø➀é✙ø➑é✗✂✫æ☎å➄ä❿å➄î➻ï❯✝ è✧ï✖ä✥ê❇ó➵ï➊ä✥ï❫ÿ☎ê✧ï✖ù➞å❛ê❀ó❨ø➑è✧ç☎ì➈ÿ✂è❣ù✂ï❶ëíì❛ä✧î➓å➄è✧ø➀ì➈é☎ê✖ð❀ã✛ç✙ï✇è✧ì➈è✥å➄û✐û➀ï➊é❼✂➈è✧ç➞ì➈ë è✧ç☎ï✇è✥ä✥å➈ø➑é☎ø➑é❼✂➤ê✧ï✖ê✥ê✤ø➀ì➈é❙ó➵å❛ê❀û➀ï❶ë➺è➏ÿ✙é✥❿ç☎å➄é❼✂❛ï✖ù➣➪❩✑❆✘♣æ☎å➈ê✥ê✧ï✖ê❯ì➄ë☞✓✻✘❼➌ ✘✻✘✗✘ æ☎å➄è✤è✧ï✖ä✧é✎ê➤ï✖å✑❿ç✈➶↔ð➝➏⑥è➞ø➀ê➤ø➀é✐è✧ï✖ä✧ï❞ê④è✥ø➑é❼✂✿è✥ì✫é✙ì➄è✥ï❙è✥ç☎å✠è✌è✥ç✙ï➓é✙ï➊è④ó✇ì❛ä✧ô ï❯➘✟ï✒❶è✧ø➀ú➈ï✖û✙✘✺ê✤ï✖ï✖ê➉ï✖å✑❿ç✫ø➀é☎ù✂ø➀ú➇ø➀ù✙ÿ☎å➄û❦ê✧å➈î➻æ✙û➑ï✒ì➈é✙û✠✘✶è④ó❨ø✁❶ï✒ì✠ú➈ï➊ä➔è✧ç✙ï ❶ì❛ÿ✙ä❿ê✤ï➤ì➄ë❇è✧ç✙ï❞ê✤ï ✑✻✘✒æ✎å➈ê✥ê✤ï❞ê➊ð ✜❇ø✠✂➈ÿ☎ä✧ï ✺✶ê✧ç✙ì✠ó➔ê♣å➈û➑û ✺ ✑➽î➻ø➀ê❘❶û✓å➈ê✥ê✤ø✙➞☎ï❞ù✺è✧ï❞ê④è➤ï❯↔✙å➄î➻æ✙û➀ï✖ê✖ð➤ê✤ì❛î❭ï ì➄ë◆è✧ç☎ì❛ê✧ï❨ï❯↔✙å➄î➻æ✙û➀ï✖ê❣å➈ä✧ï✔✂➈ï✖é➇ÿ✙ø➑é☎ï➊û✠✘✒å➄î➑✡✙ø✙✂❛ÿ✙ì➈ÿ✎ê✄➌❄✡☎ÿ✂è❫ê✤ï✖ú➈ï✖ä✥å➈û➇å➈ä✧ï ✁✗✿▲❍✪❦♦❥✒❦➣❧✴☎★✱✴❋✩❃✪❴▲✰❅✺✏✷✴⑥❼❉✪✿▲✺✻✵✻✷✹✸✻✵✻✿▲✷✹❈✪✺❀✷✴⑥✗✵✻✰❅❈④✵✻✸❖✱✴✿▲❈✪✿▲❈★❍✔❃♦✱❘✵ ✵✶✰❅✸✶❈★✺❺❦ 4−>6 3−>5 8−>2 2−>1 5−>3 4−>8 2−>8 3−>5 6−>5 7−>3 9−>4 8−>0 7−>8 5−>3 8−>7 0−>6 3−>7 2−>7 8−>3 9−>4 8−>2 5−>3 4−>8 3−>9 6−>0 9−>8 4−>9 6−>1 9−>4 9−>1 9−>4 2−>0 6−>1 3−>5 3−>2 9−>5 6−>0 6−>0 6−>0 6−>8 4−>6 7−>3 9−>4 4−>6 2−>7 9−>7 4−>3 9−>4 9−>4 9−>4 8−>7 4−>2 8−>4 3−>5 8−>4 6−>5 8−>5 3−>8 3−>8 9−>8 1−>5 9−>8 6−>3 0−>2 6−>5 9−>5 0−>7 1−>6 4−>9 2−>1 2−>8 8−>5 4−>9 7−>2 7−>2 6−>5 9−>7 6−>1 5−>6 5−>0 4−>9 2−>8 ✁❼✿▲❍✪❦☛❿★❦ ✮✏✯✪✰✩❿✹❾④✵✻✰❅✺✻✵➜❃♦✱❘✵ ✵✶✰❅✸✻❈✪✺➃❋✩✿▲✺✻❆❅❴➂✱✴✺✶✺✶✿✂✪✰❇❉✌◗❯❙➓P☛✰❇❤❳✰r✵ ❑✂✁★❦✬❞✇✰❅❴▲✷❺✽①✰❇✱✴❆❖✯ ✿▲❋✛✱✴❍✹✰❳✿▲✺✇❉★✿▲✺✶❃✪❴➂✱❅❙❯✰❇❉✩✵✻✯✪✰❨❆❅✷✹✸✶✸✻✰❅❆❅✵❢✱✴❈✪✺✻✽❢✰❅✸✻✺✝✠❴▲✰r⑥➟✵✄✂✈✱✴❈★❉✬✵✻✯✪✰❜❈★✰❅✵●✽✇✷✹✸✶❣✛✱✴❈★❑ ✺✻✽❢✰❅✸✞✠✸✻✿▲❍✹✯✄✵✟✂r❦✈✮✏✯✪✰❅✺✶✰❹✰❅✸✶✸✶✷✹✸✻✺❀✱✴✸✶✰❷❋✩✷✹✺✻✵✻❴❙✛❆❇✱✴✳✪✺✶✰❅❉✌✰❅✿ ✵✻✯✪✰❅✸❳◗❯❙✞❍✹✰❅❈✄✳✪✿▲❈✪✰❅❴❙ ✱✴❋✎◗✪✿▲❍✹✳✪✷✹✳★✺✦❃♦✱❘✵✻✵✻✰❅✸✶❈✪✺❇❚❯✷✹✸✦◗✄❙❹❉★✿▲❍✹✿ ✵✶✺✦✽❀✸✶✿ ✵ ✵✶✰❅❈➜✿▲❈➜✱❳✺✻✵●❙✒❴▲✰✈✵✻✯♦✱❘✵✥✱✴✸✶✰✇✳★❈✪❉✪✰❅✸✻❑ ✸✶✰❅❃✪✸✶✰❅✺✻✰❇❈❯✵✻✰❇❉✞✿▲❈✬✵✻✯✪✰❷✵✶✸✶✱✴✿▲❈✪✿▲❈✪❍➃✺✶✰r✵❺❦ æ◆ï➊ä✧ëíï✒↔è✥û✙✘➷ø✓ù✂ï➊é✐è✥ø➟➞✎å✑✡✙û➀ï➣✡☛✘ ç➇ÿ✙î➓å➈é☎ê✄➌❦å➄û➑è✧ç☎ì➈ÿ❼✂❛ç➷è✧ç✙ï✒✘➷å➈ä✧ï➓ó❨ä✥ø➑è❇✝ è✧ï✖é✢ø➀é✺å➄é✿ÿ✙é☎ù✂ï✖ä❇✝⑥ä✧ï✖æ✙ä✥ï✖ê✧ï➊é✐è✧ï❞ù➽ê④è❅✘➇û➀ï➈ð❣ã✛ç✙ø✓ê❨ê✤ç☎ì✠ó➔ê❫è✥ç☎å✠è✛ëíÿ✙ä✧è✧ç☎ï➊ä ø➀î❭æ☎ä✧ì✠ú❛ï➊î➻ï➊é✐è✥ê➵å➄ä✥ï➔è✥ì➑✡◆ï➳ï❯↔✂æ✎ï★↔è✥ï✖ù➽ó❨ø➑è✧ç✿î➻ì➈ä✥ï➔è✥ä✥å➈ø➑é✙ø➀é❼✂❭ù✙å✠è❿å✙ð ❃✛ ❃ ➯❄➲✛➤✗➢♦➡▼✣✁➩✄➯♦➨ ➻✣●➺➭➥✠☞✔➺➭➥❼➳✄➡ ❃ ✲✠➢✪➩✴➩▼✣✁➃➳✄➡✴➩ ✜✙ì➈ä➉è✧ç✙ï➻ê✥å➄ô➈ï✒ì➄ë➜❶ì❛î➻æ☎å➄ä✥ø➀ê✧ì➈é❀➌✎å➓ú✠å➄ä✥ø➀ï❶è❅✘✿ì➄ë➏ì➄è✥ç✙ï➊ä♣è✧ä❿å➄ø➀é☎å✑✡✙û➑ï ❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê❀ó➵å❛ê✝è✥ä✥å➈ø➑é☎ï✖ù➞å➈é☎ù➤è✧ï❞ê④è✥ï✖ù✒ì➈é✌è✥ç✙ï➵ê✥å➄î➻ï➵ù✙å✠è❿å❄✡☎å❛ê✤ï❛ð❳✕➔é ï✖å➈ä✧û✠✘♣ê✧ÿ❼✡☎ê✧ï❶è✝ì➈ë➇è✧ç✙ï❞ê✤ï➏ä✥ï✖ê✧ÿ✙û➑è✥ê☛ó✛å➈ê✝æ✙ä✧ï❞ê✤ï✖é✐è✧ï✖ù➳ø➀é❵✞✙❼➾✡✠⑥ð❀ã✛ç✙ï➏ï✖ä✧ä✥ì➈ä ä❿å✠è✧ï❞ê❨ì➈é✿è✥ç✙ï✌è✧ï❞ê④è➳ê✤ï➊è❨ëíì➈ä❨è✥ç✙ï➞ú✠å➄ä✥ø➑ì❛ÿ☎ê❨î➻ï❶è✥ç✙ì✂ù✙ê➔å➈ä✧ï➞ê✧ç✙ì✠ó❨é✺ø➑é ➞✗✂❛ÿ✙ä✧ï ❀☎ð ☞♣ð✠➾ ✗❀ø➑é✙ï❞å➄ä④☞✇û✓å➈ê✥ê✤ø✙➞☎ï✖ä✒➌✂å➄é✎ù➐✓❣å➈ø➑ä✥ó❨ø✓ê✤ï❊✗✝ø➀é✙ï✖å➈ä④☞✇û➀å❛ê✧ê✧ø➟➞✎ï➊ä ✓❦ì❛ê✥ê✤ø✠✡✙û✠✘♣è✥ç✙ï➔ê✧ø➑î➻æ✙û➀ï✖ê✤è❹❶û✓å➈ê✥ê✤ø✙➞☎ï➊ä❯è✥ç☎å✠è❣ì➈é☎ï➵î➻ø✠✂➈ç✐è❹❶ì❛é☎ê✤ø✓ù✂ï✖ä❯ø✓ê å➽û➀ø➀é✙ï✖å➈ä④❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✖ð ✭❫å➎❿ç✫ø➑é✙æ☎ÿ✂è➳æ☎ø➟↔✂ï➊û❦ú✠å➄û➀ÿ✙ï➚❶ì➈é✐è✥ä✧ø✠✡✙ÿ✂è✥ï✖ê❨è✥ì✢å ó➵ï➊ø✠✂➈ç✐è✧ï❞ù✫ê✤ÿ✙îPëíì➈ä♣ï✖å➎❿ç✿ì❛ÿ✂è✧æ☎ÿ✂è♣ÿ✙é☎ø➩è❞ð➉ã✛ç✙ï➞ì❛ÿ✂è✧æ✙ÿ✙è♣ÿ✙é✙ø➑è♣ó❨ø➑è✧ç è✧ç☎ï➞ç✙ø✠✂➈ç✙ï❞ê④è♣ê✧ÿ✙î✤➪íø➀é✗❶û➀ÿ☎ù✂ø➀é❼✂✶è✧ç✙ï➉➊ì➈é✐è✧ä✥ø✠✡✙ÿ✂è✧ø➀ì➈é✫ì➈ë➏å➝✡✙ø➀å❛ê✬➊ì➈é✦✝
2Fs-s u OvE 2EE7X LEO AEF 1ii8 IP Linear --120- [deslant]Linear -8.4- Pairwise -76 K-NN Euclidean [deslant]K-NN Euclidean 2.4 40 PCA+quadratic 3.3 1000 RBF+linear [16x16]Tangent Distance 1 SVM poly 4 RS-SVMp的5 [dist]V-SVM poly 9 2828-300-10 [ds128x28-300-10 3.6 [deslant]20x20-300-10 16 2828-1000-10 [dis2828-1000-10 38 28x28-300-100-10 3.05 [dis28x28-300-100-10 28x28-500-150-10 2.95 [dis128x28-500-150-10 2.45 [16x16]LeNet-1 LeNet-4 1.1 LeNet-4/Local LeNet-4/K-NN LeNet-5 [dist]LeNet-5 [dist]Boosted LeNet-4 0.5 1.5 2.5 35 45 Rig.9.Error rate on the test set cRgfor various classigcation methods.deslantDindicates that the classiger was trained and tested on the deslanted version of the database.dist Dindicates that the training set was augmented with artigcially distorted eflamples.16fl16D indicates that the system used the 16f116 piflel images.The uncertainty in the quoted error rates is about 0.1 stant)indicates the class of the input character.On the However,the memory requirement and recognitiontime are regular data,the error rate is 11%.The network has 7i50 large:the complete p0,000 twenty by twenty pixel training free parameters.On the deslanted images,the test error images(about It Megabytes at one byte per pixel)must be rate is it%The network has t010 free parameters.The available at run time.Much more compact representations deficiencies of the linear classifier are well documented.1] could be devised with modest increase in error rate.On the and it is included here simply to form a basis of comparison regular test set the error rate was 5.0%.On the deslanted for more sophisticated classifiers.Various combinations of data,the error rate was It%with k8 B Naturally,a sigmoid units,linear units,gradient descent learning,and realistic Euclidean distance nearest-neighbor system would learning by directly solving linear systems gave similar re-operate on feature vectors rather than directly on the pix- sults. els,but since all of the other systems presented in this A simple improvement of the basic linear classifier was study operate directly on the pixels,this result is useful for tested,51].The idea is to train each unit of a single-layer a baseline comparison. network to separate each class from each other class.In our case this layer comprises t5 units labeled 0/1,0/1,..0/9, C.BXrincipal Component Analysis (CA)and olynomial 1/1..../9.Unit ivj is trained to produce +1 on patterns Classifier of class i,-1 on patterns of class j,and is not trained on Following 5,5t],a preprocessing stage was con- other patterns.The final score for class i is the sum of structed which computes the projection of the input pat- the outputs all the units labeled iu minus the sum of the tern on thet0 principal components of the set of training output of all the units labeled yvi,for all x and y.The vectors.To compute the principal components,the mean of error rate on the regular test set was 7.p%. each input component was first computed and subtracted C.I Baseline Nearest Neighbor Classifier from the training vectors.The covariance matrix of the re- sulting vectors was then computed and diagonalized using Another simple classifier is a K-nearest neighbor classi-Singular Value Decomposition.The0-dimensional feature fier with a Euclidean distance measure between input im-vector was used as the input of a second degree polynomial ages.This classifier has the advantage that no training classifier.This classifier can be seen as a linear classifier time,and no brain on the part of the designer,are required. with i1 inputs,preceded by a module that computes all
✂✁☎✄✝✆✟✞✠✄☛✡✌☞✎✍✟✏✒✑✓✏✂✏✂✏✎✔✖✕☛✄☎✗☛✏✙✘✛✚✙✏✂✁✢✜✤✣✥✣✧✦ ✜✰ K−NN Euclidean [deslant] K−NN Euclidean 40 PCA + quadratic 1000 RBF + linear SVM poly 4 RS−SVM poly 5 28x28−300−10 28x28−1000−10 28x28−300−100−10 28x28−500−150−10 LeNet−4 / Local LeNet−4 / K−NN LeNet−5 −−−− 12.0 −−−−> −−−− 8.4 −−−−> −−−− 7.6 −−−−> 5 2.4 3.3 3.6 1.1 1.1 1 0.8 4.7 3.6 1.6 4.5 3.8 3.05 2.5 2.95 2.45 1.7 1.1 1.1 1.1 0.95 0.8 0.7 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 [dist] V−SVM poly 9 [dist] 28x28−300−10 [dist] 28x28−1000−10 [dist] 28x28−300−100−10 [dist] 28x28−500−150−10 [16x16] LeNet−1 [dist] LeNet−5 [dist] Boosted LeNet−4 LeNet−4 [16x16] Tangent Distance [deslant] 20x20−300−10 Linear [deslant] Linear Pairwise ✁❼✿▲❍✪❦✦➁★❦✖❧✈✸✶✸✻✷✹✸❷✸❖✱❘✵✻✰✩✷✹❈➉✵✻✯✪✰✩✵✻✰❇✺✵➜✺✶✰r✵ ✁✂❳⑥✙✷✹✸❷▼✹✱✴✸✻✿▲✷✹✳✪✺❹❆❇❴➂✱✴✺✻✺✶✿✂♦❆❇✱❘✵✻✿▲✷✹❈➚❋✩✰r✵✻✯✪✷✒❉★✺❺❦✄✂❉✪✰❅✺✶❴➂✱✴❈❯✵✆☎✇✿▲❈✪❉✪✿▲❆❇✱❘✵✻✰❅✺➜✵✻✯♦✱❘✵➜✵✶✯✪✰✩❆❅❴➂✱✴✺✻✺✶✿✂♦✰❅✸❷✽✏✱✴✺❷✵✶✸✶✱✴✿▲❈✪✰❅❉➝✱✴❈✪❉➉✵✶✰❅✺✵✶✰❅❉➝✷✹❈ ✵✶✯✪✰➜❉✪✰❅✺✶❴➂✱✴❈❯✵✻✰❇❉✌▼❯✰❅✸✶✺✻✿▲✷✹❈✌✷✴⑥✥✵✻✯✪✰➜❉♦✱❘✵✶✱✴◗♦✱✴✺✶✰✹❦✝✂❉★✿▲✺✻✵✆☎❼✿▲❈✪❉✪✿▲❆❇✱❘✵✶✰❅✺❜✵✻✯♦✱❘✵❳✵✶✯✪✰➜✵✶✸✶✱✴✿▲❈✪✿▲❈✪❍✩✺✻✰❅✵❳✽✏✱✴✺❨✱✴✳★❍✹❋✩✰❇❈❯✵✻✰❇❉✌✽❀✿ ✵✻✯✌✱✴✸✻✵✻✿✂♦❆❅✿➂✱✴❴▲❴❙✞❉★✿▲✺✻✵✻✷✹✸✻✵✻✰❇❉✌✰✝☎★✱✴❋✩❃✪❴▲✰❅✺❺❦✞✂ ❬❇❽❃☎✑❬❇❽✟☎ ✿▲❈✪❉✪✿▲❆❇✱❘✵✶✰❅✺❀✵✶✯♦✱❘✵✏✵✻✯✪✰❹✺✻❙✒✺✻✵✻✰❅❋⑨✳✪✺✻✰❇❉✬✵✶✯★✰✛❬❇❽❃☎✑❬❇❽✔❃✪✿☎✒✰❅❴❩✿▲❋✛✱✴❍✹✰❅✺❺❦❼✮✏✯✪✰❷✳✪❈★❆❇✰❅✸✵❖✱✴✿▲❈❯✵●❙t✿▲❈✛✵✶✯★✰➜⑦✄✳✪✷✴✵✻✰❅❉t✰❅✸✶✸✶✷✹✸✏✸✶✱❘✵✶✰❅✺❀✿▲✺❳✱✴◗❄✷✹✳★✵❳❪★❦▲❬ ✁ ❦ ê✤è✥å➄é✐è✹➶✌ø➀é☎ù✂ø✁➊å✠è✥ï✖ê✒è✧ç✙ï➐❶û✓å➈ê✥ê✌ì➄ë❨è✥ç✙ï✶ø➀é✙æ✙ÿ✙è➝❿ç☎å➈ä✥å➎↔è✥ï➊ä❞ð➲ý♣é è✧ç✙ï ä✥ï✄✂➈ÿ☎û➀å➈ä✛ù☎å✠è✥å✗➌✂è✧ç☎ï✌ï➊ä✥ä✧ì❛ä➵ä❿å✠è✥ï✌ø➀ê➓➾ ✑✘✶✶ð➏ã✛ç✙ï✌é☎ï❶è④ó➵ì➈ä✥ô➽ç☎å➈ê ✔❆✺ ✙✻✘ ëíä✥ï➊ï➽æ☎å➈ä✥å➈î➻ï❶è✧ï✖ä✥ê✖ð➲ý♣é❺è✥ç✙ï✢ù✙ï✖ê✧û➀å➈é❛è✥ï✖ù➷ø➀î➓å❄✂➈ï❞ê✄➌❀è✧ç✙ï✶è✧ï❞ê④è✒ï✖ä✧ä✥ì➈ä ä❿å✠è✧ï➓ø✓ê✛✺☎ð ❝✝✶ ã✛ç✙ï➓é✙ï➊è④ó✇ì❛ä✧ô❖ç☎å➈ê✒❝✘❼➾✽✘✢ëíä✥ï➊ï➓æ☎å➈ä✥å➈î➻ï❶è✧ï✖ä✥ê✖ð➽ã✛ç✙ï ù✂ï✄➞✥❶ø➀ï➊é✗➊ø➑ï❞ê➉ì➈ë➏è✧ç☎ï✒û➀ø➑é✙ï❞å➄ät❶û✓å➈ê✥ê✤ø✙➞☎ï✖ä♣å➄ä✥ï➞ó➵ï➊û➀û❯ù✙ì☛➊ÿ✙î➻ï➊é✐è✧ï❞ù ✞✙➾✡✠ å➄é✎ù➞ø➑è❦ø✓ê❯ø➑é✥❶û➀ÿ☎ù✂ï✖ù❙ç✙ï✖ä✧ï✛ê✤ø➀î➻æ✙û✠✘➤è✧ì♣ëíì➈ä✥îòå✬✡☎å❛ê✤ø✓ê❯ì➄ë✥➊ì➈î➻æ☎å➈ä✧ø✓ê✤ì❛é ëíì➈ä➉î❭ì❛ä✧ï➞ê✧ì➈æ☎ç✙ø➀ê✤è✧ø✁➊å➄è✧ï❞ù➐➊û➀å❛ê✧ê✧ø✙➞☎ï➊ä❿ê➊ð➜✣➏å➄ä✥ø➑ì❛ÿ☎ê✛❶ì❛î➉✡✙ø➀é☎å➄è✧ø➀ì➈é☎ê✛ì➈ë ê✧ø✙✂❛î❭ì❛ø➀ù➲ÿ✙é✙ø➑è✥ê✒➌◆û➀ø➀é✙ï✖å➈ä♣ÿ✙é☎ø➩è❿ê✄➌✇✂➈ä❿å➈ù✙ø➑ï✖é❛è➳ù✂ï✖ê❘❶ï✖é❛è➳û➑ï❞å➄ä✥é✙ø➀é❼✂✗➌✟å➄é☎ù û➀ï✖å➄ä✥é✙ø➀é❼✂➝✡☛✘✶ù✂ø➀ä✧ï★↔è✧û✠✘✢ê✧ì➈û➀ú➇ø➑é✗✂❭û➀ø➀é✙ï✖å➈ä➔ê❇✘✂ê✤è✧ï➊î➓ê✔✂✐årú➈ï➤ê✤ø➀î➻ø➑û✓å➄ä❨ä✥ï❯✝ ê✧ÿ✙û➩è❿ê➊ð ✕ ê✤ø➀î➻æ✙û➑ï➓ø➀î➻æ✙ä✥ì✠ú➈ï➊î➻ï✖é❛è✌ì➈ë❫è✧ç☎ï➣✡☎å➈ê✧ø✁❙û➀ø➀é✙ï✖å➈ä➓❶û✓å➈ê✥ê✤ø✙➞☎ï➊ä➤ó✛å➈ê è✧ï❞ê④è✥ï✖ù✏✞✙✫✑ ✠♠ð✛ã✛ç✙ï➞ø✓ù✂ï✖å➻ø✓ê❨è✥ì➻è✧ä❿å➄ø➀é✫ï✖å➎❿ç✢ÿ✙é☎ø➩è➳ì➄ë➏å➓ê✤ø➀é❼✂❛û➑ï✄✝♠û✓å✪✘➈ï✖ä é✙ï➊è④ó✇ì❛ä✧ô♣è✧ì➤ê✤ï✖æ☎å➄ä❿å✠è✥ï➏ï✖å➎❿ç➉❶û✓å➈ê✥ê☛ëíä✥ì➈î ï❞å✑❿ç➞ì➈è✧ç✙ï✖ä❜➊û➀å❛ê✧ê✖ð✏➏➠é❙ì➈ÿ✙ä ➊å❛ê✤ï➻è✥ç✙ø✓ê✒û✓å✪✘➈ï✖ä➓❶ì➈î➻æ✙ä✥ø✓ê✤ï❞ê✹❝✵✙✿ÿ✙é☎ø➩è❿ê➞û✓å❄✡◆ï➊û➀ï✖ù ✘ ✄✦➾✑➌ ✘ ✄✫✑✦➌➀ð➑ð➀ð ✘ ✄✍❀✗➌ ➾ ✄✫✑✂ð➀ð➑ð➀ð ✺ ✄✍❀☎ð➓→➔é✙ø➑è✞✺✢☛✟✗✿ø✓ê➳è✧ä❿å➄ø➀é✙ï✖ù➲è✧ì✺æ✙ä✥ì✂ù✂ÿ✗❶ï✚✣➑➾❭ì➈é æ☎å✠è✧è✧ï➊ä✥é☎ê ì➄ë✔❶û✓å➈ê✥ê✭✺❘➌✏✝❘➾❭ì➈é æ☎å✠è✧è✧ï✖ä✧é☎ê➳ì➈ë✎❶û✓å➈ê✥ê ✗✥➌❀å➄é☎ù❺ø✓ê➤é✙ì➄è➤è✧ä❿å➄ø➀é✙ï❞ù❖ì❛é ì➄è✥ç✙ï➊ä➓æ✎å✠è✤è✥ï➊ä✥é☎ê✖ð ã✛ç✙ï➐➞☎é✎å➄û➔ê❘❶ì➈ä✥ï✢ëíì➈ä➣❶û✓å➈ê✥ê ✺➞ø✓ê❭è✧ç✙ï➲ê✧ÿ✙î❅ì➈ë è✧ç☎ï✌ì➈ÿ✂è✥æ✙ÿ✂è✥ê➉å➄û➀û✟è✧ç✙ï✌ÿ✙é☎ø➩è❿ê❨û➀å✑✡✎ï✖û➑ï❞ù ✺✢☛✦✼✿î➻ø➀é➇ÿ☎ê✛è✧ç☎ï➞ê✤ÿ✙î✴ì➄ë❯è✧ç✙ï ì➈ÿ✙è✧æ✙ÿ✂è➻ì➄ë➉å➈û➑û✇è✧ç☎ï✶ÿ✙é✙ø➑è✥ê❙û✓å❄✡◆ï➊û➀ï✖ù ❆❇☛✦✺❘➌❇ëíì➈ä➻å➄û➀û ✼❑å➄é✎ù ❆✟ð❺ã✛ç✙ï ï➊ä✥ä✥ì➈ä✛ä✥å➄è✧ï➤ì➈é✢è✧ç☎ï✌ä✧ï✒✂➈ÿ✙û✓å➄ä➵è✧ï❞ê④è➉ê✧ï❶è❨ó✛å➈ê ✔✂ð ✓✌✶➽ð ☞♣ð ✑➹✚✛å➈ê✧ï➊û➀ø➀é✙ï✌ñ➔ï✖å➈ä✧ï❞ê④è➔ñ➉ï➊ø✠✂➈ç☛✡✎ì❛ä✬☞✇û➀å❛ê✧ê✧ø➟➞✎ï➊ä ✕➔é✙ì➈è✧ç✙ï✖ä➤ê✧ø➀î❭æ☎û➑ï➝❶û✓å➈ê✥ê✤ø✙➞☎ï✖ä➉ø✓ê♣å➐☎✞✝♠é☎ï✖å➄ä✥ï✖ê✤è➉é✙ï✖ø✙✂❛ç☛✡✎ì❛ä④❶û✓å➈ê✥ê✧ø➟✝ ➞☎ï✖ä♣ó❨ø➑è✧ç❖å✜✭➏ÿ✗➊û➑ø✓ù✂ï❞å➄é➲ù✂ø✓ê④è❿å➄é✗➊ï✒î➻ï✖å❛ê✤ÿ☎ä✧ï➓✡◆ï❶è④ó➵ï➊ï✖é✫ø➑é☎æ✙ÿ✂è➤ø➑î➚✝ å❄✂❛ï✖ê✖ð❲ã✛ç☎ø➀ê➙❶û✓å➈ê✥ê✤ø✙➞☎ï➊ä➓ç✎å➈ê➻è✥ç✙ï❺å❛ù✂ú✠å➄é✐è✥å✑✂➈ï✿è✧ç☎å➄è✶é✙ì è✥ä✥å➈ø➑é☎ø➑é❼✂ è✧ø➀î➻ï✑➌➈å➈é☎ù✌é✙ì④✡✙ä❿å➄ø➀é➞ì➈é✌è✥ç✙ï➵æ☎å➄ä✧è❇ì➈ë✙è✧ç✙ï✛ù✂ï❞ê✤ø✠✂➈é✙ï✖ä✒➌✠å➈ä✧ï➏ä✥ï✒➍✐ÿ✙ø➀ä✥ï✖ù☛ð õ➔ì✠ó➵ï➊ú❛ï➊ä★➌❶è✥ç✙ï➏î➻ï➊î➻ì➈ä❘✘➔ä✥ï✒➍✐ÿ✙ø➀ä✧ï✖î➻ï➊é✐è❀å➄é☎ù➤ä✥ï✒➊ì✑✂❛é✙ø➩è✥ø➑ì❛é➉è✥ø➑î➻ï❫å➈ä✧ï û✓å➄ä❘✂➈ï✻✰❀è✧ç✙ït❶ì❛î➻æ✙û➑ï➊è✧ï ✓✻✘✗➌ ✘✗✘✻✘➳è④ó➵ï➊é✐è❅✘➑✡☛✘➻è④ó➵ï➊é✐è❅✘❭æ✙ø➟↔✂ï✖û✎è✥ä✥å➈ø➑é☎ø➑é❼✂ ø➀î➻å✑✂➈ï❞ê➃➪üå❄✡◆ì➈ÿ✂è ✑❝➤ö✫ï✒✂❛å❄✡☛✘✐è✧ï❞ê❇å➄è❯ì❛é✙ï✎✡❩✘✐è✥ï➵æ◆ï➊ä❦æ✙ø✙↔✂ï➊û●➶❇î✒ÿ☎ê✤è❷✡✎ï årú✠å➄ø➀û➀å✑✡✙û➀ï➔å✠è❫ä✥ÿ✙é➻è✧ø➀î➻ï➈ð❣ö✫ÿ✗❿ç➓î➻ì❛ä✧ï✛➊ì➈î➻æ☎å✑❶è❫ä✥ï➊æ✙ä✥ï✖ê✧ï➊é✐è✥å➄è✧ø➀ì➈é☎ê ❶ì❛ÿ✙û✓ù➓✡◆ï➵ù✂ï✖ú➇ø➀ê✧ï✖ù✌ó❨ø➑è✧ç❙î➻ì➇ù✙ï✖ê✤è❯ø➀é✗❶ä✥ï✖å❛ê✤ï✇ø➑é❙ï➊ä✥ä✧ì❛ä✝ä❿å✠è✧ï❛ð❦ý♣é✒è✧ç✙ï ä✥ï✄✂➈ÿ☎û➀å➈ä✇è✥ï✖ê✤è♣ê✧ï❶è❨è✧ç☎ï✌ï➊ä✥ä✧ì❛ä❨ä✥å➄è✧ï➤ó➵å❛ê ✙✂ð ✘✞✶✶ð➵ý♣é✿è✧ç☎ï✒ù✂ï✖ê✧û✓å➄é✐è✧ï❞ù ù✙å➄è✥å❼➌❦è✥ç✙ï✿ï➊ä✥ä✧ì❛ä➞ä✥å➄è✧ï✢ó➵å❛ê❄✑✙ð ❝✝✶➣➌❣ó❨ø➑è✧ç ✞◗✺ ❜☎ð➷ñ♣å✠è✥ÿ✙ä✥å➈û➑û✠✘✑➌❣å ä✥ï✖å➄û➀ø✓ê④è✥ø✠ ✭➏ÿ✥❶û➀ø➀ù✂ï❞å➄é➓ù✂ø✓ê④è❿å➄é✗➊ï❨é✙ï✖å➈ä✧ï❞ê④è❺✝♠é☎ï➊ø✠✂➈ç☛✡✎ì❛ä❣ê❇✘✂ê✤è✧ï➊î➶ó✇ì❛ÿ✙û✓ù ì➈æ◆ï➊ä❿å✠è✥ï♣ì❛é✶ëíï❞å✠è✥ÿ✙ä✧ï➳ú➈ï★↔è✥ì➈ä❿ê✇ä❿å✠è✥ç✙ï➊ä➵è✥ç☎å➄é✺ù✂ø➀ä✧ï★↔è✧û✠✘➓ì➈é✢è✧ç☎ï➳æ☎ø➟↔☛✝ ï➊û✓ê✒➌✬✡✙ÿ✂è✫ê✧ø➑é✗➊ï❺å➈û➑û➳ì➄ë➞è✥ç✙ï ì➄è✧ç☎ï➊ä✺ê❇✘✂ê✤è✧ï➊î➓ê✢æ✙ä✥ï✖ê✧ï➊é✐è✧ï❞ù❍ø➑é✲è✧ç✙ø✓ê ê✤è✧ÿ☎ù✦✘❙ì➈æ◆ï➊ä❿å✠è✥ï✛ù✙ø➑ä✥ï✒❶è✧û✠✘✒ì➈é❭è✥ç✙ï❨æ✙ø✙↔➇ï✖û➀ê✒➌➄è✥ç✙ø✓ê❣ä✧ï❞ê✤ÿ☎û➩è➏ø✓ê❣ÿ☎ê✧ï❶ëíÿ✙û☎ëíì➈ä å➚✡☎å➈ê✧ï➊û➀ø➀é✙ï✌❶ì➈î➻æ☎å➈ä✧ø✓ê✧ì➈é✝ð ☞♣ð ❜⑧✓❫ä✧ø➀é✗❶ø➀æ☎å➈û❼☞✇ì❛î➻æ✎ì❛é✙ï➊é✐è❷✕➔é☎å➈û✙✘✂ê✧ø➀ê✔➪✓✔☞✎✕④➶❯å➄é✎ù➑✓❦ì❛û✙✘➇é✙ì❛î❭ø✓å➄û ☞✇û✓å➈ê✥ê✤ø✙➞☎ï✖ä ✜✙ì➈û➀û➀ì✠ó❨ø➑é❼✂ ✞✙✬❜✬✠✶➌ ✞✙❝✫✠✶➌❺å▲æ✙ä✧ï✖æ✙ä✥ì☛➊ï✖ê✥ê✤ø➀é❼✂✭ê④è❿å❄✂➈ï ó✛å➈ê❭➊ì➈é✦✝ ê✤è✧ä✥ÿ✗↔è✥ï✖ù❺ó❨ç✙ø✠❿ç①➊ì➈î➻æ✙ÿ✂è✥ï✖ê➳è✧ç☎ï➓æ✙ä✧ì✫✓④ï✒❶è✧ø➀ì➈é❖ì➈ë❫è✥ç✙ï➓ø➑é☎æ✙ÿ✂è➞æ☎å➄è❇✝ è✧ï✖ä✧é✫ì❛é✺è✧ç✙ï ❝✗✘➻æ✙ä✥ø➀é✗❶ø➀æ☎å➄û❜❶ì❛î❭æ◆ì➈é☎ï➊é✐è✥ê➉ì➄ë❦è✧ç✙ï❭ê✤ï➊è♣ì➈ë❯è✥ä✥å➈ø➑é☎ø➑é❼✂ ú➈ï★↔è✥ì➈ä❿ê➊ð✝ã❇ì✬❶ì❛î➻æ✙ÿ✂è✧ï➏è✧ç✙ï✇æ✙ä✧ø➀é✗➊ø➑æ☎å➈û✑❶ì❛î➻æ✎ì❛é✙ï➊é✐è✥ê✒➌❞è✧ç✙ï❫î❭ï❞å➄é✌ì➈ë ï✖å➎❿ç➲ø➀é✙æ✙ÿ✂è✌❶ì❛î❭æ◆ì➈é☎ï➊é✐è➳ó✛å➈ê✬➞☎ä❿ê④èt❶ì❛î❭æ☎ÿ✂è✧ï❞ù❖å➄é✎ù➲ê✤ÿ✗✡✂è✧ä❿å✑❶è✧ï✖ù ëíä✥ì➈î➶è✧ç✙ï❨è✥ä✥å➈ø➑é☎ø➑é❼✂✌ú❛ï✒❶è✧ì➈ä❿ê✖ð❇ã✛ç✙ï✬➊ì✠ú✠å➄ä✥ø➀å➈é✗❶ï✛î➓å✠è✥ä✧ø✙↔❙ì➈ë◆è✥ç✙ï➔ä✥ï❯✝ ê✧ÿ✙û➩è✥ø➑é✗✂➓ú➈ï✒❶è✧ì❛ä✥ê➵ó✛å➈ê➵è✧ç☎ï➊é✆❶ì❛î❭æ☎ÿ✂è✧ï❞ù✿å➈é☎ù✿ù✂ø➀å✑✂➈ì❛é☎å➄û➀ø✙➽✖ï✖ù✶ÿ☎ê✤ø➀é❼✂ þ➇ø➀é❼✂➈ÿ☎û➀å➈ä❀✣➏å➄û➀ÿ✙ï✔✧➉ï★❶ì❛î❭æ◆ì❛ê✧ø➑è✧ø➀ì➈é✝ð❯ã✛ç✙ï ❝✘♦✝➠ù✂ø➑î➻ï✖é☎ê✤ø➀ì➈é✎å➄û✠ëíï❞å✠è✧ÿ☎ä✧ï ú➈ï★↔è✥ì➈ä❣ó✛å➈ê❦ÿ☎ê✧ï✖ù➻å➈ê❣è✧ç✙ï➔ø➀é✙æ✙ÿ✙è❫ì➄ë☛å✌ê✤ï★❶ì❛é☎ù➻ù✂ï✄✂❛ä✧ï✖ï➵æ◆ì➈û✠✘➇é✙ì➈î➻ø✓å➄û ❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✖ð✺ã✛ç✙ø✓ê➉➊û➀å❛ê✧ê✧ø➟➞✎ï➊ä➉✖å➄é①✡◆ï✶ê✤ï✖ï➊é å❛ê➞å✫û➀ø➑é✙ï❞å➄ä➉➊û➀å❛ê✧ê✧ø✙➞☎ï➊ä ó❨ø➑è✧ç✶✺ ✑✦➾❙ø➑é☎æ✙ÿ✂è✥ê✒➌☛æ✙ä✥ï✒➊ï✖ù✂ï❞ù↕✡☛✘➲å✶î➻ì✂ù✂ÿ✙û➀ï❭è✧ç☎å➄èt➊ì➈î➻æ✙ÿ✂è✥ï✖ê➤å➄û➀û
CXC.Ob CRE IEEE,AOVEy BEXFV R products of pairs of input variables.The error on the reg- onl0 marginallo improved error ratese 2.95%.Training ular test set was 3.3%. with distorted patterns improved the performance some- whate 2.5y%error for the 2sx2s-3yy-1yy-ly network,and C.4 Radial Basis Function Network 2.45%for the 2s x2-1yyy-15y-1y network. Following [55],an RBF network was constructed.The first laOer was composed of 1,yyy Gaussian RBF units with C.7 A Small Convolutional Networke LeNet-1 2sx2 inputs,and the second laOer was a simple 1yyy inputs Convolutional Networks are an attempt to solve the c ly outputs linear classifier.The RBF units were divided dilemma between small networks that cannot learn into ly groups of lyy.-ach group of units was trained the training set,and large networks that seem over- on all the training examples of one of the ly classes using parameterized.LeNet-1 was an earl0 embodiment of the the adaptive K-means algorithm.The second laDer weights Convolutional Network architecture which is included here were computed using a regularized pseudo-inverse method. for comparison purposes.The images were down-sampled The error rate on the regular test set was 3.P% to 1Px1P pixels and centered in the 2sx2s input la0er.Al- though about lyy,yyy multiplocadd steps are required to C.5 One-Hidden LaOer Full0 Connected MultilaOer Neural evaluate LeNet-1,its convolutional nature keeps the num- Network ber of free parameters to onl0 about 2Pyy.The LeNet- Another classifier that we tested was a full0 connected 1 architecture was developed using our own version of multi-laOer neural network with two la0ers of weights(one the US6S (US 6 ostal Service zip codes)database and its hidden laOer)trained with the version of back-propagation size was tuned to match the available data [35].LeNet-1 described in Appendix C.-rror on the regular test set was achieved 1.7%test error.The fact that a network with such 4.7%for a network with 3yy hidden units,and 4.5%for a a small number of parameters can attain such a good error network with lyyy hidden units.Using artificial distortions rate is an indication that the architecture is appropriate to generate more training data brought onlo marginal im- for the task. provemente 3.P%for 3yy hidden units,and 3.s for 1yyy hidden units.When deslanted images were used,the test C.s LeNet-4 error jumped down to 1.P%for a network with 3yy hidden -xperiments with LeNet-1 made it clear that a larger units. convolutional network was needed to make optimal use of It remains somewhat of a mOster0 that networks with the large size of the training set.LeNet-4 and later LeNet- such a large number of free parameters manage to achieve 5 were designed to address this problem.LeNet-4 is vero reasonablo low testing errors.We conjecture that the do-similar to LeNet-5,except for the details of the architec- namics of gradient descent learning in multilaOer nets has ture.It contains 4 first-level feature maps,followed bo a "self-regularization"effect.Because the origin of weight s subsampling maps connected in pairs to each first-laOer space is a saddle point that is attractive in almost evero feature maps,then 1P feature maps,followed bo 1P sub- direction,the weights invariablo shrink during the first sampling map,followed bo a fullo connected laDer with few epochs (recent theoretical analOsis seem to confirm 12y units,followed bo the output la0er (1y units).LeNet-4 this [5P).Small weights cause the sigmoids to operate contains about 2Py,yyy connections and has about 17,yyy in the quasi-linear region,making the network essentiallo free parameters.Test error was 1.1%.In a series of ex- equivalent to a low-capacito,single-la0er network.As the periments,we replaced the last laOer of LeNet-4 with a learning proceeds,the weights grow,which progressivel0 -uclidean Nearest Neighbor classifier,and with the "local increases the effective capacito of the network.This seems learning"method of Bottou and Vapnik [5s],in which a lo- to be an almost perfect,if fortuitous,implementation of cal linear classifier is retrained each time a new test pattern Vapniks "Structural Risk Minimization"principle P.A is shown.Neither of those methods improved the raw error better theoretical understanding of these phenomena,and rate,although the0 did improve the rejection performance. more empirical evidence,are definitel0 needed. C.9 Boosted LeNet-4 C.P Two-Hidden LaOer Fullo Connected MultilaOer Neural Following theoretical work bO R.Schapire [59],Drucker Network et al.[Py]developed the "boosting"method for combining To see the effect of the architecture,several two-hidden multiple classifiers.Three LeNet-4s are combinede the first laOer multilaOer neural networks were trained.Theoreti- one is trained the usual wa0.the second one is trained on cal results have shown that an0 function can be approxi- patterns that are filtered bO the first net so that the second mated bo a one-hidden laDer neural network [57].However, machine sees a mix of patterns,5y%of which the first net several authors have observed that two-hidden la0er archi- got right,and 5y%of which it got wrong.Finall0,the tectures sometimes Oield better performance in practical third net is trained on new patterns on which the first and situations.This phenomenon was also observed here.The the second nets disagree.During testing,the outputs of test error rate of a 2sx2s-3yy-lyy-1y network was 3.y5%, the three nets are simplo added.Because the error rate of a much better result than the one-hidden la0er network, LeNet-4 is vero low,it was necessar0 to use the artificiallO obtained using marginallo more weights and connections. distorted images (as with LeNet-5)in order to get enough Increasing the network size to 2x2s-1yyy-15y-1y Oielded samples to train the second and third nets.The test error
✂✁☎✄✝✆✟✞✠✄☛✡✌☞✎✍✟✏✒✑✓✏✂✏✂✏✎✔✖✕☛✄☎✗☛✏✙✘✛✚✙✏✂✁✢✜✤✣✥✣✧✦ ✜✢ æ✙ä✥ì✂ù✂ÿ✗↔è❿ê❨ì➄ë❦æ☎å➈ø➑ä❿ê✛ì➄ë❦ø➑é☎æ✙ÿ✂è➉ú✠å➄ä✥ø✓å❄✡✙û➀ï✖ê✖ð❫ã✛ç✙ï✌ï✖ä✧ä✥ì➈ä✛ì➈é✿è✥ç✙ï➞ä✥ï✄✂❄✝ ÿ✙û✓å➄ä✛è✥ï✖ê✤è➉ê✤ï➊è➔ó➵å❛ê❀❜☎ð ❜✌✶✶ð ☞♣ð ❝⑨✍➔å❛ù✂ø✓å➄û✏✚✛å➈ê✧ø➀ê✩✜☎ÿ✙é✗❶è✧ø➀ì➈é✫ñ➔ï➊è④ó✇ì❛ä✧ô ✜✙ì➈û➀û➀ì✠ó❨ø➑é❼✂ ✞✙✫✙ ✠✶➌❦å➄é✢✍✬✚✎✜❲é✙ï➊è④ó✇ì❛ä✧ô❺ó➵å❛ê➓❶ì❛é☎ê✤è✧ä✥ÿ✗↔è✥ï✖ù☛ð✫ã✛ç✙ï ➞☎ä❿ê④è➏û✓å✪✘➈ï✖ä❣ó➵å❛ê❷❶ì❛î❭æ◆ì❛ê✧ï✖ù❭ì➈ë❜➾➎➌ ✘✗✘✻✘➤â➤å➄ÿ☎ê✥ê✤ø✓å➄é➵✍✛✚✔✜❖ÿ✙é✙ø➑è✥ê➏ó❨ø➑è✧ç ✑❆✺❄↔✤✑❆✺➵ø➑é☎æ✙ÿ✂è✥ê✒➌➄å➄é✎ù➳è✧ç✙ï❫ê✧ï✒➊ì➈é☎ù➤û✓å✪✘➈ï✖ä☛ó➵å❛ê✝å➔ê✧ø➑î➻æ✙û➀ï✛➾✒✘✻✘✻✘✛ø➀é✙æ✙ÿ✙è✥ê ✄ ➾✽✘❙ì➈ÿ✂è✥æ✙ÿ✂è❿ê✛û➑ø➀é✙ï✖å➈ä✩❶û✓å➈ê✥ê✤ø✙➞☎ï➊ä❞ð❦ã✛ç✙ï✌✍✛✚✔✜ ÿ✙é✙ø➑è✥ê✛ó➵ï➊ä✥ï➤ù✂ø➑ú➇ø✓ù✂ï✖ù ø➀é❛è✥ì❭➾✽✘➒✂➈ä✥ì➈ÿ☎æ☎ê❭ì➄ë➝➾✽✘✻✘☎ð ✭❫å➎❿ç⑧✂❛ä✧ì❛ÿ✙æ❤ì➈ë➳ÿ✙é☎ø➩è❿ê➽ó✛å➈ê❭è✧ä❿å➄ø➀é✙ï❞ù ì➈é➲å➈û➑û❀è✧ç✙ï✒è✧ä❿å➄ø➀é✙ø➀é❼✂➓ï❯↔✙å➄î➻æ✙û➀ï✖ê➉ì➄ë❣ì➈é☎ï➞ì➄ë❣è✥ç✙ï➣➾✽✘ ❶û✓å➈ê✥ê✤ï❞ê❨ÿ☎ê✧ø➑é❼✂ è✧ç☎ï➔å➈ù✙å➈æ✂è✧ø➀ú➈ï✩☎✞✝⑥î❭ï❞å➄é☎ê❦å➈û✙✂❛ì➈ä✥ø➩è✥ç✙î✺ð❇ã✛ç✙ï➔ê✧ï✒➊ì➈é☎ù❙û➀å✪✘❛ï➊ä❦ó➵ï➊ø✠✂➈ç✐è✥ê ó➵ï➊ä✥ï✛❶ì➈î➻æ✙ÿ✙è✧ï✖ù➻ÿ✎ê✤ø➀é❼✂✒å➤ä✥ï✄✂❛ÿ✙û➀å➈ä✧ø✠➽➊ï❞ù✒æ✎ê✤ï✖ÿ☎ù✂ì❄✝⑥ø➀é✐ú❛ï➊ä❿ê✤ï✛î➻ï❶è✥ç✙ì✂ù☛ð ã✛ç✙ï✌ï✖ä✧ä✥ì➈ä➵ä✥å➄è✧ï➤ì➈é✿è✥ç✙ï✌ä✧ï✒✂➈ÿ✙û✓å➄ä✇è✧ï❞ê④è➉ê✧ï❶è➔ó✛å➈ê❙❜☎ð ✓✌✶ ☞♣ð ✙ ý♣é✙ï✄✝⑥õ➉ø➀ù☎ù✂ï➊é✘✗❇å✪✘➈ï✖ä➃✜✙ÿ☎û➑û✠✘ ☞✇ì➈é✙é✙ï★↔è✥ï✖ù✶ö➲ÿ✙û➩è✥ø➑û✓å✪✘➈ï✖ä✇ñ➉ï➊ÿ✙ä❿å➄û ñ➉ï❶è④ó➵ì➈ä✥ô ✕➔é✙ì➈è✧ç✙ï✖ä➉❶û✓å➈ê✥ê✤ø✙➞☎ï✖ä➤è✧ç☎å➄è✒ó✇ï➻è✥ï✖ê✤è✧ï❞ù➷ó➵å❛ê✌å✿ëíÿ✙û➑û✠✘ ❶ì➈é☎é✙ï✒❶è✧ï✖ù î✒ÿ☎û➩è✥ø➟✝⑥û➀å✪✘❛ï➊ä✛é✙ï✖ÿ✙ä✥å➈û✟é✙ï❶è④ó➵ì➈ä✥ô➓ó❨ø➩è✥ç✶è④ó➵ì➻û➀å✪✘❛ï➊ä❿ê❫ì➈ë❇ó➵ï➊ø✠✂➈ç✐è✥êt➪⑨ì➈é✙ï ç✙ø✓ù✙ù✂ï✖é✶û✓å✪✘➈ï➊ä✹➶❯è✥ä✥å➈ø➑é✙ï❞ù➓ó❨ø➑è✧ç➽è✥ç✙ï➳ú➈ï➊ä❿ê✧ø➑ì❛é➓ì➄ë❀✡☎å➎❿ô❩✝♠æ✙ä✥ì➈æ✎å❄✂❛å➄è✧ø➀ì➈é ù✂ï❞ê❺➊ä✧ø✠✡✎ï❞ù❭ø➑é➣✕➉æ✙æ✎ï✖é☎ù✂ø✙↔➙☞♣ð✝✭❫ä✧ä✥ì➈ä❦ì➈é❭è✥ç✙ï➔ä✥ï✄✂➈ÿ☎û➀å➈ä❦è✧ï✖ê✤è✇ê✧ï❶è➏ó✛å➈ê ❝☎ð ✔✖✶ ëíì➈ä➳å➓é✙ï➊è④ó✇ì❛ä✧ô✢ó❨ø➩è✥ç ❜✻✘✗✘➻ç✙ø➀ù☎ù✂ï➊é➲ÿ☎é✙ø➩è❿ê✄➌✟å➄é✎ù◆❝☎ð ✙✘✶ ëíì➈ä♣å é✙ï➊è④ó✇ì❛ä✧ô➳ó❨ø➩è✥ç➙➾✒✘✻✘✗✘❨ç✙ø✓ù✙ù✂ï➊é❙ÿ✙é✙ø➑è✥ê✖ð❜→♣ê✤ø➀é❼✂➳å➄ä✧è✧ø✙➞✥❶ø✓å➄û➇ù✂ø✓ê④è✥ì➈ä✧è✧ø➀ì➈é☎ê è✧ì ✂➈ï➊é☎ï➊ä❿å✠è✧ï➤î➻ì➈ä✥ï➤è✧ä❿å➄ø➀é✙ø➀é❼✂➽ù✙å➄è✥å➝✡✙ä✥ì➈ÿ❼✂❛ç✐è➔ì➈é✙û✠✘✶î➓å➄ä❘✂➈ø➀é☎å➈û☛ø➑î➚✝ æ✙ä✥ì✠ú➈ï✖î❭ï✖é✐è✽✰ ❜✙ð ✓✞✶ ëíì❛ä❊❜✗✘✻✘➓ç✙ø✓ù✙ù✂ï✖é❖ÿ✙é☎ø➩è❿ê✄➌✝å➈é☎ù ❜☎ð ✺✌✶ ëíì❛ä➑➾✒✘✻✘✗✘ ç✙ø✓ù✙ù✂ï✖é❺ÿ✙é☎ø➩è❿ê➊ð ✎ç☎ï➊é➷ù✂ï✖ê✧û➀å➈é✐è✧ï✖ù❖ø➑î➓å✑✂➈ï✖ê♣ó➵ï➊ä✥ï✒ÿ☎ê✧ï✖ù❢➌✟è✥ç✙ï❭è✧ï❞ê④è ï➊ä✥ä✥ì➈ä ✓④ÿ✙î➻æ✎ï❞ù✢ù✂ì✠ó❨é✶è✧ì➐➾➈ð ✓✞✶✹ëíì❛ä✛å❙é✙ï➊è④ó✇ì❛ä✧ô➻ó❨ø➑è✧ç◆❜✻✘✻✘✒ç✙ø✓ù✙ù✂ï➊é ÿ✙é✙ø➑è✥ê✖ð ➏⑥è➻ä✥ï➊î➓å➈ø➑é☎ê➻ê✧ì➈î➻ï➊ó❨ç✎å✠è➻ì➄ë♣å❺î➉✘✂ê✤è✧ï➊ä❘✘➷è✧ç✎å✠è➓é✙ï❶è④ó➵ì➈ä✥ô✂ê✒ó❨ø➑è✧ç ê✧ÿ✗❿ç✫å❙û✓å➄ä❘✂➈ï➳é➇ÿ✙î➉✡◆ï➊ä❨ì➈ë❇ëíä✥ï➊ï✌æ✎å➄ä❿å➄î➻ï❶è✥ï➊ä❿ê❫î➓å➄é✎å❄✂➈ï➳è✥ì➽å✑❿ç✙ø➀ï➊ú❛ï ä✥ï✖å➈ê✧ì➈é✎å❄✡✙û✠✘✶û➀ì✠ó è✧ï❞ê④è✥ø➑é❼✂✢ï➊ä✥ä✧ì❛ä✥ê✖ð ✎ï➚➊ì➈é✓④ï★↔è✧ÿ☎ä✧ï✒è✧ç☎å➄è➉è✧ç☎ï❭ù✦✘❩✝ é☎å➈î❭ø✁➊ê♣ì➄ë❹✂❛ä✥å❛ù✂ø➀ï➊é✐è➉ù✂ï✖ê❘❶ï✖é✐è♣û➀ï✖å➄ä✥é✙ø➀é❼✂➽ø➀é➲î✒ÿ☎û➩è✥ø➑û✓å✪✘➈ï✖ä➉é✙ï➊è✥ê♣ç☎å➈ê å✏☛✥ê✤ï✖û➩ë●✝⑥ä✧ï✒✂➈ÿ✙û✓å➄ä✥ø✠➽✖å✠è✥ø➑ì❛é✤✌➞ï✄➘◆ï★↔è❞ð✔✚➵ï★➊å➄ÿ✎ê✤ï➤è✥ç✙ï➞ì➈ä✥ø✠✂➈ø➀é✿ì➈ë❣ó✇ï✖ø✙✂❛ç❛è ê✧æ☎å✑➊ï➽ø➀ê❙å➲ê✥å➈ù✙ù✂û➀ï✶æ◆ì➈ø➀é✐è➞è✧ç✎å✠è❙ø✓ê✒å➄è✤è✥ä✥å➎↔è✧ø➀ú➈ï✶ø➑é❤å➈û➑î➻ì✐ê④è✒ï✖ú➈ï✖ä❺✘ ù✂ø➀ä✧ï★↔è✥ø➑ì❛é✏➌✌è✧ç☎ï ó✇ï✖ø✙✂❛ç✐è✥ê✫ø➀é➇úrå➈ä✧ø✓å❄✡☎û✙✘✲ê✧ç✙ä✥ø➑é☎ô❲ù✂ÿ☎ä✧ø➀é❼✂➘è✧ç☎ï①➞☎ä❿ê④è ëíï➊ó÷ï✖æ✎ì✦❿ç☎ê⑧➪íä✥ï✒❶ï✖é✐è✫è✧ç✙ï✖ì➈ä✥ï❶è✧ø✁➊å➈û✒å➄é✎å➄û✠✘➇ê✧ø✓ê➲ê✤ï✖ï➊î è✧ì❝➊ì➈é✦➞☎ä✥î è✧ç☎ø➀ê▲✞✙❆✓✠➭➶↔ð þ➇î➓å➄û➀û➤ó✇ï✖ø✙✂❛ç✐è✥ê ✖å➄ÿ☎ê✧ï➲è✥ç✙ï➷ê✤ø✠✂➈î➻ì➈ø✓ù✙ê✶è✧ì❑ì➈æ◆ï➊ä❿å✠è✧ï ø➀é❺è✥ç✙ï➵➍✐ÿ☎å➈ê✧ø➟✝⑥û➀ø➑é✙ï❞å➄ä➳ä✧ï✒✂➈ø➀ì➈é✏➌❀î➻å➈ô➇ø➑é❼✂✢è✧ç☎ï➓é✙ï❶è④ó➵ì➈ä✥ô✺ï✖ê✥ê✤ï✖é❛è✥ø➀å➈û➑û✠✘ ï✒➍✐ÿ✙ø➀ú✠å➄û➀ï➊é✐è➳è✧ì✫å✶û➀ì✠ó✔✝r➊å➄æ✎å✑❶ø➑è❅✘✑➌✟ê✧ø➑é❼✂❛û➑ï✄✝♠û✓å✪✘➈ï✖ä➉é✙ï➊è④ó✇ì❛ä✧ô✟ð➓✕♣ê➳è✧ç✙ï û➀ï✖å➄ä✥é✙ø➀é❼✂➷æ✙ä✥ì☛➊ï➊ï❞ù✙ê✄➌❫è✧ç✙ï❖ó✇ï✖ø✙✂❛ç❛è❿ê➝✂➈ä✥ì✠ó✌➌➏ó❨ç✙ø✁❿ç➘æ✙ä✧ì➎✂➈ä✥ï✖ê✥ê✤ø➀ú➈ï✖û✙✘ ø➀é✗❶ä✥ï✖å❛ê✤ï❞ê➏è✥ç✙ï➤ï❯➘✟ï✒↔è✥ø➑ú❛ï✞✖å➄æ☎å➎❶ø➑è❅✘➻ì➄ë❀è✥ç✙ï➳é✙ï❶è④ó➵ì➈ä✥ô✟ð❯ã✛ç✙ø✓ê❨ê✧ï➊ï➊î➓ê è✧ì➒✡◆ï✺å➈é å➈û➑î➻ì✐ê④è❭æ✎ï✖ä✤ëíï★↔è✒➌✇ø➩ë♣ëíì➈ä✧è✧ÿ✙ø➑è✧ì❛ÿ☎ê✒➌❫ø➀î❭æ☎û➑ï✖î❭ï✖é✐è✥å✠è✥ø➑ì❛é❤ì➈ë ✣➏å➄æ☎é✙ø➑ô☞❁ ê❖☛✥þ➇è✧ä✥ÿ✗↔è✥ÿ✙ä✥å➈û❹✍❨ø✓ê✤ô ö✫ø➑é☎ø➑î➻ø✠➽✖å✠è✥ø➑ì❛é✤✌✺æ✙ä✥ø➑é✥❶ø➀æ✙û➑ï❈✞✓✠⑥ð ✕ ✡◆ï❶è✤è✥ï➊ä➉è✧ç✙ï✖ì➈ä✥ï❶è✧ø✁➊å➈û✝ÿ✙é☎ù✙ï➊ä❿ê④è❿å➄é☎ù✂ø➀é❼✂➽ì➈ë❦è✧ç✙ï❞ê✤ï➞æ☎ç✙ï➊é✙ì❛î➻ï➊é☎å✗➌☎å➄é☎ù î➻ì➈ä✥ï➤ï➊î➻æ✙ø➀ä✧ø✁➊å➈û☛ï➊ú➇ø➀ù✙ï➊é✗➊ï✑➌✙å➄ä✥ï✌ù✂ï❯➞✎é✙ø➩è✥ï➊û✠✘✶é✙ï✖ï✖ù✂ï❞ù☛ð ☞♣ð ✓❑ã✛ó➵ì❄✝➠õ➔ø✓ù✙ù✂ï✖é❑✗❇å✪✘➈ï✖ä❨✜☎ÿ✙û➀û✙✘➵☞✇ì➈é✙é✙ï★↔è✥ï✖ù➻ö✫ÿ✙û➑è✧ø➀û➀å✪✘❛ï➊ä➏ñ➉ï➊ÿ✙ä❿å➄û ñ➉ï❶è④ó➵ì➈ä✥ô ã❀ì✿ê✤ï✖ï✌è✧ç✙ï❙ï❯➘✟ï✒❶è♣ì➄ë❣è✥ç✙ï❙å➈ä❘❿ç☎ø➩è✥ï✒↔è✥ÿ✙ä✥ï✑➌✎ê✧ï➊ú❛ï➊ä❿å➄û☛è④ó➵ì❄✝⑥ç✙ø✓ù✙ù✂ï✖é û✓å✪✘➈ï➊ä❭î✒ÿ☎û➩è✥ø➑û✓å✪✘➈ï✖ä❙é✙ï✖ÿ✙ä❿å➄û➵é✙ï❶è④ó➵ì➈ä✥ô✂ê✒ó➵ï➊ä✥ï✶è✥ä✥å➈ø➑é☎ï✖ù☛ð ã✛ç✙ï✖ì➈ä✥ï❶è✥ø➟✝ ➊å➈û➏ä✥ï✖ê✧ÿ✙û➩è❿ê➤ç☎årú➈ï➓ê✧ç✙ì✠ó❨é è✧ç☎å➄è✒å➄é☛✘✫ëíÿ✙é✥↔è✧ø➀ì➈é✢✖å➄é ✡◆ï➽å➄æ✙æ✙ä✥ì✪↔✂ø➟✝ î➓å✠è✥ï✖ù➑✡❩✘❙å♣ì❛é✙ï❯✝⑥ç✙ø✓ù✙ù✂ï✖é❙û✓å✪✘➈ï✖ä❯é☎ï➊ÿ✙ä❿å➄û➇é✙ï➊è④ó✇ì❛ä✧ô❵✞✙✗✔✆✠♠ð❯õ➔ì✠ó➵ï➊ú➈ï✖ä✒➌ ê✧ï➊ú➈ï✖ä✥å➈û◆å➈ÿ✂è✧ç☎ì➈ä❿ê❫ç☎årú❛ï➉ì➎✡☎ê✤ï✖ä✧ú❛ï✖ù➻è✥ç☎å✠è➵è④ó➵ì❄✝⑥ç✙ø➀ù☎ù✂ï➊é✢û➀å✪✘❛ï➊ä✛å➄ä✴❿ç✙ø➟✝ è✧ï★↔è✥ÿ✙ä✧ï❞ê➽ê✧ì➈î➻ï❶è✥ø➑î➻ï✖ê➣✘✐ø➀ï➊û✓ù⑨✡✎ï➊è✤è✥ï➊ä➽æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï✿ø➀é➘æ✙ä❿å✑↔è✥ø✠✖å➄û ê✧ø➩è✥ÿ☎å✠è✥ø➑ì❛é☎ê➊ð✇ã✛ç✙ø➀ê➔æ✙ç✙ï➊é☎ì➈î➻ï➊é✙ì❛é✢ó✛å➈ê❨å➈û➀ê✧ì➻ì✑✡☎ê✧ï➊ä✥ú➈ï❞ù✶ç☎ï➊ä✥ï➈ð➏ã✛ç✙ï è✧ï❞ê④è❙ï➊ä✥ä✧ì❛ä➳ä❿å✠è✥ï➻ì➄ë❨å❈✑❆✺♦↔✤✑✻✺♦✝✝❜✻✘✻✘❄✝❘➾✒✘✻✘❄✝❘➾✒✘✒é✙ï➊è④ó✇ì❛ä✧ô❖ó➵å❛ê✹❜✙ð ✘ ✙✘✶ ➌ å❺î✒ÿ✗❿ç⑧✡✎ï➊è✤è✧ï✖ä➓ä✧ï❞ê✤ÿ☎û➩è❭è✧ç☎å➈é❤è✥ç✙ï✺ì➈é✙ï✄✝♠ç☎ø➀ù✙ù✙ï➊é❤û✓å✪✘➈ï✖ä❙é☎ï❶è④ó➵ì➈ä✥ô✈➌ ì✑✡✙è✥å➄ø➀é✙ï❞ù❺ÿ✎ê✤ø➀é❼✂✫î➓å➄ä❘✂➈ø➀é☎å➄û➀û✠✘✫î❭ì❛ä✧ï➓ó➵ï➊ø✠✂➈ç✐è❿ê✌å➄é☎ù ❶ì❛é✙é✙ï★↔è✧ø➀ì➈é✎ê➊ð ➏➠é✗➊ä✧ï❞å➈ê✧ø➑é❼✂❖è✧ç✙ï✿é✙ï➊è④ó✇ì❛ä✧ô➷ê✤ø✠➽➊ï✢è✧ì✏✑❆✺❄↔✤✑❆✺♦✝✴➾✽✘✗✘✻✘❄✝❘➾✆✙❆✘✪✝✴➾✽✘➵✘➇ø➀ï➊û✓ù✂ï✖ù ì➈é☎û✙✘✻î➓å➈ä❺✂❛ø➑é☎å➈û➑û✠✘❍ø➀î❭æ☎ä✧ì✠ú❛ï✖ù✻ï✖ä✧ä✥ì➈ä✿ä❿å✠è✧ï❞ê✱✰☞✑✂ð ❀ ✙✞✶✶ð▼ã❇ä✥å➈ø➑é☎ø➑é❼✂ ó❨ø➑è✧ç➘ù✂ø➀ê✤è✧ì❛ä✤è✥ï✖ù❑æ☎å✠è✧è✧ï✖ä✧é☎ê❭ø➀î➻æ✙ä✧ì✠ú❛ï✖ù è✧ç✙ï✺æ◆ï➊ä✧ëíì➈ä✥î➻å➈é✗❶ï✺ê✧ì➈î➻ï❯✝ ó❨ç☎å➄è✽✰❊✑✙ð ✙✻✘✞✶✴ï➊ä✥ä✥ì➈ä➔ëíì❛ä➳è✧ç✙ï ✑❆✺❄↔✑✻✺♦✝✝❜✻✘✗✘♦✝✴➾✽✘✻✘❄✝❘➾✒✘➞é✙ï➊è④ó✇ì❛ä✧ô✇➌✟å➄é☎ù ✑✂ð ❝✵✙✞✶✹ëíì➈ä❨è✥ç✙ï ✑❆✺❄↔✤✑❆✺♦✝✴➾✽✘✗✘✻✘❄✝❘➾✆✙❆✘✪✝✴➾✱✘➳é✙ï❶è④ó➵ì➈ä✥ô✟ð ☞♣ð ✔✢✕òþ➇î➓å➈û➑û❜☞✇ì❛é➇ú➈ì➈û➀ÿ✂è✥ø➑ì❛é☎å➄û✟ñ➔ï➊è④ó✇ì❛ä✧ô☞✰ ✗✝ï❞ñ➔ï❶è❺✝❘➾ ☞✇ì➈é➇ú➈ì❛û➑ÿ✂è✥ø➑ì❛é☎å➄û➓ñ➉ï❶è④ó➵ì➈ä✥ô➇ê➷å➈ä✧ï➘å➄é å✠è✧è✧ï✖î❭æ✙è è✧ì ê✤ì❛û➑ú❛ï è✧ç✙ï ù✂ø➀û➑ï✖î➻î➻å ✡◆ï❶è④ó➵ï➊ï➊é ê✤î➓å➄û➀û✭é✙ï❶è④ó➵ì➈ä✥ô✂ê è✧ç✎å✠è ➊å➄é☎é✙ì➄è û➑ï❞å➄ä✥é è✧ç☎ï✹è✥ä✥å➈ø➑é✙ø➀é❼✂✴ê✧ï❶è★➌❤å➄é✎ù û➀å➈ä❺✂❛ïòé✙ï➊è④ó✇ì❛ä✧ô✂ê➘è✥ç☎å✠è ê✤ï✖ï➊î ì✠ú➈ï✖ä❇✝ æ☎å➈ä✥å➈î❭ï➊è✧ï✖ä✧ø✠➽➊ï❞ù☛ð✳✗❀ï✖ñ➉ï❶è❇✝✴➾➓ó✛å➈ê➞å➈é ï✖å➄ä✥û✠✘❖ï➊î➑✡✎ì✂ù✂ø➀î➻ï➊é✐è❙ì➈ë✛è✧ç✙ï ☞✇ì➈é➇ú❛ì➈û➀ÿ✂è✧ø➀ì➈é☎å➈û☎ñ➔ï➊è④ó✇ì❛ä✧ô❭å➄ä✴❿ç✙ø➑è✧ï✒❶è✧ÿ✙ä✥ï➔ó❨ç✙ø✁❿ç➽ø✓ê➏ø➀é✗❶û➀ÿ☎ù✂ï❞ù➽ç✙ï✖ä✧ï ëíì➈ät❶ì❛î➻æ☎å➄ä✥ø➀ê✧ì➈é✺æ✙ÿ☎ä✧æ◆ì❛ê✧ï✖ê✖ð♣ã✛ç✙ï❙ø➑î➓å✑✂➈ï✖ê➉ó✇ï✖ä✧ï❙ù✂ì✠ó❨é✦✝➠ê✧å➈î❭æ☎û➑ï❞ù è✧ì✆➾✒✓♦↔❢➾✽✓❙æ✙ø✙↔✂ï➊û✓ê❨å➄é☎ù✫❶ï✖é✐è✧ï➊ä✥ï✖ù✿ø➀é✿è✧ç✙ï ✑✻✺♦↔✤✑❆✺❙ø➑é☎æ✙ÿ✂è➉û✓å✪✘➈ï✖ä✖ð❷✕➔û✙✝ è✧ç☎ì➈ÿ❼✂❛ç➷å✑✡✎ì❛ÿ✂è➣➾✽✘✗✘❼➌ ✘✻✘✗✘✶î❙ÿ✙û➩è✥ø➑æ☎û✙✘ ✄✠å❛ù✙ù➷ê④è✥ï➊æ☎ê✒å➄ä✥ï➓ä✧ï★➍❛ÿ☎ø➑ä✥ï✖ù❖è✧ì ï➊ú✠å➈û➑ÿ☎å➄è✧ï✜✗❀ï✖ñ➔ï➊è❇✝✴➾✑➌✎ø➩è❿ê✛❶ì❛é✐ú❛ì➈û➀ÿ✂è✧ø➀ì➈é✎å➄û☛é☎å➄è✧ÿ✙ä✥ï✌ô➈ï✖ï➊æ☎ê✛è✥ç✙ï➞é➇ÿ✙î➚✝ ✡◆ï➊ä➽ì➈ë➳ëíä✥ï➊ï✫æ✎å➄ä❿å➄î➻ï❶è✥ï➊ä❿ê✒è✥ì ì➈é✙û✠✘❑å✑✡✎ì❛ÿ✂è✳✑✻✓✻✘✻✘☎ð✻ã✛ç✙ï❈✗✝ï❞ñ➔ï➊è❇✝ ➾➘å➄ä✴❿ç✙ø➩è✥ï✒❶è✧ÿ✙ä✥ï❑ó✛å➈ê ù✂ï✖ú➈ï➊û➀ì➈æ◆ï✖ù➶ÿ☎ê✧ø➑é✗✂✾ì❛ÿ✙ä➷ì✠ó❨é✹ú➈ï✖ä✥ê✧ø➀ì➈éòì➈ë è✧ç☎ï➝→➳þ✦✓✇þ⑨➪✻→♣þ➛✓❦ì❛ê✤è✥å➄û➏þ➇ï✖ä✧ú➇ø✁❶ï➚➽➊ø➀æ➒➊ì✂ù✂ï✖ê✴➶➳ù✙å✠è❿å❄✡☎å❛ê✤ï➻å➄é☎ù❺ø➑è✥ê ê✧ø✙➽✖ï❭ó✛å➈ê➳è✥ÿ✙é✙ï❞ù❖è✥ì✫î➓å✠è❘❿ç❺è✧ç☎ï✶årú✠å➄ø➀û➀å✑✡✙û➑ï➻ù✙å➄è✥å▲✞❜✵✙✆✠⑥ð❵✗✝ï❞ñ➔ï❶è❺✝❘➾ å✑❿ç☎ø➑ï✖ú➈ï✖ù➝➾❛ð✕✔✘✶➘è✥ï✖ê✤è❯ï✖ä✧ä✥ì➈ä❞ð✝ã✛ç✙ï❫ë⑨å✑↔è❇è✧ç☎å➄è❦å➔é✙ï❶è④ó➵ì➈ä✥ô♣ó❨ø➑è✧ç❙ê✤ÿ✗❿ç å➞ê✧î➓å➄û➀û☎é➇ÿ✙î➉✡◆ï➊ä✇ì➄ë☛æ☎å➈ä✥å➈î❭ï➊è✧ï✖ä✥ê❹✖å➄é➽å➄è✤è✥å➈ø➑é➽ê✧ÿ✗❿ç➽å✌✂❛ì➇ì➇ù➻ï✖ä✧ä✥ì➈ä ä❿å✠è✧ï✿ø✓ê❭å➈é❤ø➀é☎ù✂ø✁➊å➄è✧ø➀ì➈é è✥ç☎å✠è➻è✥ç✙ï✿å➈ä❘❿ç☎ø➩è✥ï✒↔è✥ÿ✙ä✥ï✢ø✓ê➻å➄æ✙æ✙ä✥ì➈æ☎ä✧ø✓å✠è✥ï ëíì➈ä✛è✥ç✙ï➤è✥å❛ê✤ô✟ð ☞♣ð ✺◗✗❀ï✖ñ➔ï➊è❇✝✿❝ ✭❷↔✂æ✎ï✖ä✧ø➀î➻ï➊é✐è✥ê➻ó❨ø➑è✧ç☞✗❀ï✖ñ➔ï➊è❇✝✴➾✺î➻å❛ù✂ï✺ø➩è ➊û➑ï❞å➄ä➻è✥ç☎å✠è✶å û✓å➄ä❘✂➈ï➊ä ❶ì❛é➇ú➈ì➈û➀ÿ✂è✥ø➑ì❛é☎å➄û✝é☎ï❶è④ó➵ì➈ä✥ô✶ó✛å➈ê➔é☎ï➊ï✖ù✙ï✖ù✺è✥ì✶î➓å➄ô❛ï➞ì➈æ✂è✥ø➑î➓å➈û❀ÿ☎ê✧ï✒ì➈ë è✧ç☎ï➉û✓å➄ä❘✂➈ï➉ê✧ø✙➽✖ï➉ì➈ë☛è✧ç✙ï➉è✧ä❿å➄ø➀é✙ø➑é✗✂✒ê✤ï➊è✖ð ✗✝ï❞ñ➔ï➊è❇✝✿❝❙å➄é✎ù➓û➀å➄è✧ï✖ä✄✗✝ï❞ñ➔ï➊è❇✝ ✙✶ó✇ï✖ä✧ï➻ù✂ï✖ê✧ø✠✂➈é✙ï❞ù✫è✧ì✫å❛ù✙ù✂ä✥ï✖ê✥ê➔è✥ç✙ø➀ê➤æ☎ä✧ì➎✡✙û➑ï✖î✺ð ✗✝ï❞ñ➔ï➊è❇✝✿❝✢ø✓ê➳ú➈ï➊ä❘✘ ê✧ø➑î➻ø➀û➀å➈ä✌è✧ì❈✗❀ï✖ñ➔ï➊è❇✝ ✙✦➌❯ï❯↔❼❶ï✖æ✂è➞ëíì❛ä✌è✧ç✙ï✢ù✂ï❶è❿å➄ø➀û➀ê✌ì➈ë✇è✥ç✙ï✶å➈ä❘❿ç✙ø➑è✧ï★✹✝ è✧ÿ☎ä✧ï❛ð❫➏⑥è✆❶ì❛é❛è❿å➄ø➀é☎ê ❝✢➞☎ä❿ê④è❺✝♠û➀ï➊ú❛ï➊û➉ëíï✖å✠è✥ÿ✙ä✥ï❖î➓å➄æ☎ê✒➌➔ëíì❛û➑û➀ì✠ó➵ï✖ù ✡☛✘ ✺✿ê✤ÿ❼✡✎ê✧å➈î❭æ☎û➑ø➀é❼✂✿î➻å➈æ☎êt❶ì❛é✙é✙ï✒❶è✧ï❞ù➲ø➀é❖æ☎å➈ø➑ä❿ê♣è✧ì✿ï✖å✑❿ç↕➞☎ä✥ê✤è❇✝⑥û✓å✪✘➈ï➊ä ëíï✖å➄è✧ÿ✙ä✥ï✶î➓å➈æ☎ê✄➌❦è✥ç✙ï➊é ➾✽✓➲ëíï✖å✠è✥ÿ✙ä✥ï✶î➓å➄æ☎ê✒➌❦ëíì➈û➀û➀ì✠ó✇ï❞ù①✡☛✘⑨➾✒✓➲ê✧ÿ❼✡✦✝ ê✥å➄î➻æ✙û➀ø➑é❼✂❑î➓å➄æ✏➌♣ëíì➈û➀û➑ì✠ó➵ï✖ù ✡❩✘❍å❤ëíÿ✙û➀û✙✘ ➊ì➈é✙é✙ï★↔è✥ï✖ù✻û✓å✪✘➈ï✖ä✢ó❨ø➑è✧ç ➾ ✑✻✘➤ÿ✙é✙ø➑è✥ê✒➌➈ëíì❛û➑û➀ì✠ó➵ï✖ù➝✡☛✘➞è✥ç✙ï➉ì❛ÿ✂è✧æ✙ÿ✙è✇û✓å✪✘➈ï➊ä✛➪❇➾✽✘✌ÿ☎é✙ø➩è❿ê❘➶❶ð ✗✝ï❞ñ➔ï❶è❺✝❩❝ ❶ì❛é✐è✥å➄ø➀é☎ê➳å❄✡◆ì➈ÿ✂è✜✑❆✓✗✘❼➌ ✘✻✘✗✘➵❶ì➈é☎é✙ï✒❶è✧ø➀ì➈é☎ê➤å➄é✎ù➲ç☎å❛ê♣å✑✡✎ì❛ÿ✂è➚➾✍✔☛➌ ✘✻✘✗✘ ëíä✥ï➊ï✶æ✎å➄ä❿å➄î➻ï❶è✥ï➊ä❿ê➊ð ã❀ï❞ê④è➻ï➊ä✥ä✧ì❛ä✒ó✛å➈ê ➾➈ð✠➾✚✶✶ð ➏➠é❤å❺ê✧ï➊ä✥ø➀ï✖ê➞ì➈ë➉ï❯↔☛✝ æ◆ï➊ä✥ø➑î➻ï➊é✐è❿ê✄➌➔ó➵ï➲ä✥ï➊æ✙û✓å✑➊ï✖ù è✥ç✙ï❖û✓å➈ê✤è✢û✓å✪✘➈ï✖ä➽ì➄ë ✗❀ï✖ñ➔ï➊è❇✝✿❝ ó❨ø➑è✧ç✲å ✭➏ÿ✥❶û➀ø➀ù✂ï❞å➄é✫ñ➔ï❞å➄ä✥ï✖ê✤è➉ñ➔ï✖ø✙✂❛ç❩✡◆ì➈ä④❶û✓å➈ê✥ê✤ø✙➞☎ï➊ä★➌☎å➈é☎ù✺ó❨ø➩è✥ç✺è✧ç✙ï ☛✤û➀ì☛✖å➄û û➀ï✖å➄ä✥é✙ø➀é❼✂✌♣î➻ï❶è✥ç✙ì✂ù➻ì➄ë❢✚✇ì➈è✤è✥ì➈ÿ➓å➄é☎ù➚✣➏å➈æ✙é✙ø➀ô ✞✙❆✺✠❖➌➄ø➀é➻ó❨ç✙ø✠❿ç➓å➤û➀ì❄✝ ➊å➈û➇û➑ø➀é✙ï✖å➈ä❜➊û➀å❛ê✧ê✧ø✙➞☎ï➊ä❯ø➀ê❯ä✥ï❶è✥ä✥å➈ø➑é✙ï❞ù✌ï✖å➎❿ç✌è✧ø➀î➻ï❨å➉é✙ï✖ó è✧ï❞ê④è❦æ☎å➄è✤è✧ï✖ä✧é ø✓ê➏ê✤ç☎ì✠ó❨é✝ð❇ñ➉ï➊ø➑è✧ç✙ï✖ä➏ì➄ë◆è✧ç✙ì✐ê✤ï❨î➻ï➊è✧ç✙ì✂ù✙ê❣ø➀î➻æ✙ä✥ì✠ú➈ï✖ù➞è✧ç✙ï❨ä❿åró❤ï✖ä✧ä✥ì➈ä ä❿å✠è✧ï➎➌❛å➈û➩è✥ç✙ì➈ÿ❼✂❛ç❭è✧ç✙ï✒✘❭ù✂ø✓ù➻ø➀î❭æ☎ä✧ì✠ú❛ï✇è✥ç✙ï➉ä✥ï✓④ï★↔è✥ø➑ì❛é❭æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï➈ð ☞♣ð ❀⑧✚➵ì➇ì❛ê✤è✧ï❞ù ✗❀ï✖ñ➔ï➊è❇✝✿❝ ✜✙ì➈û➀û➀ì✠ó❨ø➑é❼✂➻è✥ç✙ï➊ì❛ä✧ï➊è✧ø✁➊å➈û❀ó➵ì➈ä✥ô ✡☛✘✆✍✌ð◆þ✦❿ç✎å➄æ✙ø➀ä✧ï ✞✙✻❀✠❖➌✥✧♣ä✧ÿ✗❿ô❛ï➊ä ï❶è➔å➈ûüð ✞✓✻✘✬✠☛ù✙ï➊ú➈ï✖û➑ì❛æ✎ï❞ù➽è✥ç✙ï◆☛❇✡◆ì➇ì❛ê✤è✧ø➀é❼✂✌❙î➻ï❶è✧ç☎ì➇ù✶ëíì➈ä✛➊ì➈î➑✡✙ø➑é☎ø➑é❼✂ î✒ÿ☎û➩è✥ø➑æ✙û➀ï✛❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê✖ð❇ã✛ç☎ä✧ï✖ï❅✗❀ï✖ñ➉ï❶è❇✝✿❝❛ê❣å➈ä✧ï✔❶ì❛î➉✡✙ø➀é✙ï❞ù ✰✝è✥ç✙ï✔➞☎ä❿ê④è ì➈é☎ï✌ø➀ê➔è✧ä❿å➄ø➀é✙ï✖ù✿è✥ç✙ï➞ÿ☎ê✧ÿ☎å➄û❇ó➵å✪✘❛ð➏è✥ç✙ï✒ê✧ï✒❶ì❛é☎ù✺ì➈é☎ï✌ø➀ê➔è✧ä❿å➄ø➀é✙ï✖ù✿ì➈é æ☎å➄è✤è✧ï✖ä✧é✎ê❯è✧ç☎å➄è❫å➄ä✥ï➃➞✎û➩è✥ï➊ä✥ï✖ù➚✡☛✘✌è✧ç✙ï✛➞☎ä✥ê✤è❣é✙ï❶è❫ê✧ì➳è✧ç☎å➄è➏è✧ç☎ï➔ê✤ï★❶ì➈é✎ù î➓å✑❿ç✙ø➀é✙ï✌ê✧ï➊ï✖ê❨å❭î➻ø✙↔✢ì➄ë❦æ☎å➄è✤è✧ï✖ä✧é✎ê✄➌✤✙✻✘✞✶✭ì➈ë❯ó❨ç✙ø✁❿ç✢è✧ç✙ït➞☎ä❿ê✤è➔é✙ï❶è ✂➈ì➈è✶ä✥ø✙✂❛ç❛è★➌➔å➄é☎ù ✙❆✘✞✶❅ì➄ë✌ó❨ç✙ø✁❿ç✻ø➑è➙✂❛ì➄è✶ó❨ä✥ì➈é❼✂✎ð②✜❯ø➑é☎å➈û➑û✠✘✑➌➔è✧ç✙ï è✧ç☎ø➑ä❿ù➽é✙ï➊è➵ø✓ê❫è✧ä❿å➄ø➀é✙ï✖ù➓ì❛é✶é✙ï✖ó❍æ☎å✠è✧è✧ï✖ä✧é☎ê✇ì➈é✢ó❨ç✙ø✁❿ç➓è✧ç✙ï④➞☎ä❿ê④è✛å➄é☎ù è✧ç☎ï✿ê✧ï✒➊ì➈é☎ù é☎ï❶è✥ê➻ù✂ø✓ê✥å❄✂➈ä✥ï➊ï❛ð➒✧♣ÿ✙ä✥ø➑é✗✂❖è✥ï✖ê✤è✧ø➀é❼✂✗➌❣è✥ç✙ï✿ì➈ÿ✂è✥æ✙ÿ✂è✥ê❭ì➈ë è✧ç☎ï➤è✧ç✙ä✥ï➊ï➤é✙ï➊è✥ê➔å➈ä✧ï➤ê✤ø➀î➻æ✙û✠✘✢å➈ù☎ù✂ï✖ù☛ð❹✚➵ï★➊å➄ÿ✎ê✤ï♣è✧ç✙ï✌ï✖ä✧ä✥ì➈ä➵ä❿å✠è✧ï➤ì➈ë ✗✝ï❞ñ➔ï➊è❇✝✿❝❙ø✓ê✛ú➈ï✖ä❺✘➻û➀ì✠ó✌➌✂ø➩è❨ó✛å➈ê➵é✙ï✒➊ï✖ê✥ê✧å➈ä❺✘❭è✥ì❙ÿ☎ê✧ï➳è✧ç✙ï✌å➈ä✤è✥ø➟➞✥➊ø➀å➈û➑û✠✘ ù✂ø✓ê④è✥ì➈ä✧è✧ï❞ù✢ø➀î➓å❄✂❛ï✖ê✌➪⑨å❛ê✛ó❨ø➩è✥ç❈✗✝ï❞ñ➔ï❶è❺✝✝✙➎➶➵ø➑é✺ì❛ä✥ù✂ï✖ä✛è✧ì➵✂❛ï❶è➔ï➊é☎ì➈ÿ❼✂❛ç ê✥å➄î➻æ✙û➀ï✖ê➵è✧ì➻è✥ä✥å➈ø➑é✢è✧ç☎ï➞ê✤ï★❶ì➈é✎ù✢å➈é☎ù✢è✧ç✙ø➀ä✥ù✿é✙ï➊è✥ê✖ð➏ã✛ç✙ï➤è✧ï❞ê④è➔ï✖ä✧ä✥ì➈ä
PROC.OF THE IEEE,NOVEMBER 1998 ic rate was y.7Y,the best of an0 of our classifiers.At first 【1Bx1 Tangent Dis似anc 19 glance,boosting appears to be three times more expensive SVM poly4 as a single net.In fact,when the first net produces a high confidence answer,the other nets are not called.The desiant)20x20-300-10 average computational cost is about 1.75 times that of a I1Bx1周LeNd-1 LeNet-4 single net. LeNet-4/Local LeNel-4/K-NN C.1y Tangent Distance Classifier (TDC) cist Boosted LeNe The Tangent Distance classifier (TDC)is a nearest- neighbor method where the distance function is made in- sensitive to small distortions and translations of the input Ag.10.。 image 1].If we consider an image as a point in a high dimensional pixel space (where the dimensionalito equals the number of pixels),then an evolving distortion of a char- acter traces out a curve in pixel space.Taken together, .24.000- all these distortions define a low-dimensional manifold in deslanK-N Euclidear 40 PCA+quadratic 3出 pixel space.For small distortions,in the vicinito of the 1000Re original image,this manifold can be approximated bo a 16x18 Tangent Distance -20000----3 SVM poly 4 14000-- plane,known as the tangent plane.An excellent measure RS-SVM poly 5 of wclosenessy for character images is the distance between [dist]V-SVM paly 9 .28.000- their tangent planes,where the set of distortions used to deslan2020-300-10 123 generate the planes includes translations,scaling,skewing, 28x28-1000-10 squeezing,rotation,and line thickness variations.A test 2828-300-100-10 2828-500-150-10 error rate of 1.1Y was achieved using 1PxIP pixel images. 6refiltering techniques using simple-uclidean distance at [18x1间LeNet-1 100 LeNel-4 multiple resolutions allowed to reduce the number of nec- LeNel-4/Local 20.000- .10000----3 essar0 Tangent Distance calculations. LeNet-4/K-NN LeNet-5 C.11 Support Vector Machine (SVM) 300 600 900 6olOnomial classifiers are well-studied methods for gen- erating complex decision surfaces.Unfortunatel0,the0 A·出f老 cogni- tion of a cingl c are impractical for high-dimensional problems,because the number of product terms is prohibitive.The Support Vec- tor technique is an extremel0 economical wa0 of represent- has reached y.s Y using a modified version of the V-SVM. ing complex surfaces in high-dimensional spaces,including Unfortunatel0,V-SVM is extremel0 expensivee about twice polOnomials and mano other tOpes of surfaces as much as regular SVM.To alleviate this problem,Burges A particularl0 interesting subset of decision surfaces is has proposed the Reduced Set Support Vector technique the ones that correspond to hoperplanes that are at a max- (RS-SVM),which attained 1.1Y on the regular test set 3], imum distance from the convex hulls of the two classes in with a computational cost of onlo P5y,yyy multipl0-adds the high-dimensional space of the product terms.Boser, per recognition,ie.onl0 about PyY more expensive than GuOon,and Vapnik realized that an0 polOnomial of E eNet-5. degree in this umaximum marginy set can be computed bo first computing the dot product of the input image with D.Dtcbssbn a subset of the training samples(called the tsupport vec- A summar0 of the performance of the classifiers is shown tors),elevating the result to the [-th power,and linearl0 in Figures 9 to 1.Figure 9 shows the raw error rate of the combining the numbers thereb0 obtained.Finding the sup- classifiers on the ly,yyy example test set.Boosted e eNet-4 port vectors and the coeh cients amounts to solving a high- performed best,achieving a score of y.7Y,closel0 followed dimensional quadratic minimization problem with linear bo EeNet-5 at y.s Y. inequalito constraints.For the sake of comparison,we in- Figure ly shows the number of patterns in the test set clude here the results obtained bo Burges and Schdlkopf that must be rerected to attain a y.5Y error for some of reported in 3].N ith a regular SVM,their error rate the methods.6atterns are repected when the value of cor- on the regular test set was 1.4Y.Cortes and Vapnik had responding output is smaller than a predefined threshold. reported an error rate of 1.1Y with SVM on the same In man0 applications,repection performance is more signif- data using a slightlo different technique.The computa-icant than raw error rate.The score used to decide upon tional cost of this technique is ver0 highe about 14 million the repection of a pattern was the difference between the multipl0-adds per recognition.Using Schdikopfss Virtual scores of the top two classes.Again,Boosted EeNet-4 has Support Vectors technique (V-SVM),1.yY error was at- the best performance.The enhanced versions of E eNet-4 tained.More recent10,Schelkopf(personal communication)did better than the originaleeNet-4,even though the raw
✂✁☎✄✝✆✟✞✠✄☛✡✌☞✎✍✟✏✒✑✓✏✂✏✂✏✎✔✖✕☛✄☎✗☛✏✙✘✛✚✙✏✂✁✢✜✤✣✥✣✧✦ ✜ ä❿å✠è✧ï➓ó✛å➈ê ✘✙ð ✔✖✶ ➌✝è✥ç✙ï➣✡◆ï✖ê✤è✒ì➄ë➔å➄é☛✘❺ì➄ë✛ì➈ÿ✙ä➉➊û➀å❛ê✧ê✧ø✙➞☎ï➊ä❿ê➊ð➐✕➵è✌➞☎ä❿ê④è ✂➈û✓å➄é✥❶ï✑➌☛✡◆ì➇ì❛ê✤è✧ø➀é❼✂❙å➈æ✙æ✎ï❞å➄ä❿ê➏è✥ì➑✡◆ï➳è✧ç✙ä✥ï➊ï♣è✧ø➀î➻ï✖ê➵î➻ì➈ä✥ï♣ï✄↔✂æ✎ï✖é☎ê✤ø➀ú➈ï å➈ê✺å ê✤ø➀é❼✂❛û➑ï❺é✙ï❶è❞ð ➏➠é✲ë⑨å➎↔è★➌♣ó❨ç✙ï✖é✻è✥ç✙ï➒➞☎ä✥ê✤è✢é☎ï❶è✫æ✙ä✥ì➇ù✙ÿ✗❶ï❞ê✿å ç✙ø✠✂➈ç✫❶ì❛é✦➞✎ù✂ï✖é✗❶ï✌å➄é✎ê✤ó➵ï➊ä★➌✐è✧ç✙ï✌ì➈è✧ç✙ï✖ä➵é☎ï❶è✥ê❨å➈ä✧ï➤é✙ì➈è✛➊å➄û➀û➀ï✖ù☛ð❣ã✛ç✙ï årú➈ï✖ä✥å✑✂➈ï➵❶ì❛î➻æ✙ÿ✂è✥å➄è✧ø➀ì➈é☎å➈û➜❶ì✐ê④è❙ø➀ê❭å❄✡◆ì➈ÿ✂è➙➾➈ð ✔✬✙✢è✧ø➀î➻ï✖ê✒è✧ç☎å➄è✒ì➄ë➉å ê✧ø➑é❼✂❛û➑ï➤é✙ï➊è✖ð ☞♣ð✠➾✽✘ ã❇å➈é❼✂➈ï✖é❛è✛✧♣ø➀ê✤è✥å➈é✗❶ï➉☞✇û✓å➈ê✥ê✤ø✙➞☎ï✖ä➓➪⑨ã✩✧✌☞✔➶ ã✛ç✙ï✲ã❯å➄é❼✂❛ï➊é✐è➹✧➉ø✓ê✤è✥å➄é✥❶ï❝➊û➀å❛ê✧ê✧ø➟➞✎ï➊ä ➪⑨ã✩✧✌☞✔➶ ø✓ê å é✙ï❞å➄ä✥ï✖ê✤è❇✝ é✙ï✖ø✙✂❛ç❩✡◆ì➈ä➤î➻ï❶è✥ç✙ì✂ù ó❨ç✙ï➊ä✥ï✒è✥ç✙ï➽ù✂ø✓ê④è❿å➄é✗➊ï❭ëíÿ✙é✗↔è✥ø➑ì❛é❺ø✓ê➤î➓å➈ù✂ï➻ø➀é✦✝ ê✧ï➊é☎ê✧ø➩è✥ø➑ú❛ï➤è✧ì➽ê✧î➓å➄û➀û✝ù✂ø➀ê✤è✧ì❛ä✤è✥ø➑ì❛é☎ê❨å➄é✎ù✶è✥ä✥å➈é☎ê✧û➀å➄è✧ø➀ì➈é☎ê✛ì➈ë❇è✥ç✙ï✌ø➑é☎æ✙ÿ✂è ø➀î➻å✑✂➈ï❈✞✓❼➾❪✠♠ð✫➏⑥ë❨ó✇ï➙❶ì➈é✎ê✤ø✓ù✂ï➊ä❙å➄é➷ø➑î➓å✑✂➈ï➽å➈ê❙å✺æ✎ì❛ø➑é✐è❭ø➑é å✺ç✙ø✠✂➈ç ù✂ø➀î➻ï➊é☎ê✧ø➑ì❛é☎å➄û✇æ✙ø✙↔➇ï✖û✇ê✧æ☎å✑➊ï➔➪íó❨ç☎ï➊ä✥ï➓è✧ç✙ï✢ù✂ø➑î➻ï✖é☎ê✤ø➀ì➈é✎å➄û➀ø➩è❅✘ ï✒➍✐ÿ☎å➈û➀ê è✧ç☎ï✇é➇ÿ✙î➑✡✎ï✖ä❇ì➈ë☎æ✙ø✙↔➇ï✖û➀ê✴➶✹➌rè✥ç✙ï➊é❙å➄é✒ï✖ú➈ì❛û➑ú➇ø➀é❼✂➉ù✂ø✓ê④è✥ì➈ä✧è✧ø➀ì➈é✌ì➈ë✎å✬❿ç☎å➈ä❇✝ å✑❶è✧ï✖ä❙è✥ä✥å➎❶ï❞ê➻ì➈ÿ✂è➽å➒❶ÿ✙ä✥ú➈ï✺ø➀é❑æ✙ø✙↔✂ï➊û➔ê✧æ☎å✑➊ï➈ð➘ã❇å➈ô➈ï✖é è✧ì➎✂➈ï➊è✧ç✙ï✖ä✒➌ å➄û➀û➏è✥ç✙ï✖ê✧ï✶ù✂ø✓ê✤è✧ì➈ä✧è✧ø➀ì➈é✎ê➞ù✂ï❯➞✎é✙ï✶å✫û➀ì✠ó✔✝⑥ù✙ø➑î➻ï➊é✎ê✤ø➀ì➈é☎å➈û❣î➓å➄é✙ø➑ëíì➈û✓ù➷ø➑é æ✙ø✙↔✂ï➊û➉ê✧æ☎å✑➊ï➈ð ✜✙ì❛ä➽ê✤î➓å➄û➀û➔ù✂ø✓ê④è✥ì➈ä✧è✧ø➀ì➈é☎ê✒➌❫ø➀é❑è✧ç✙ï✫ú➇ø✁❶ø➀é✙ø➩è❅✘❑ì➄ë♣è✧ç✙ï ì➈ä✥ø✠✂➈ø➀é☎å➄û➔ø➀î➓å❄✂❛ï✑➌➵è✧ç✙ø✓ê✢î➻å➈é✙ø➑ëíì➈û✓ù❭➊å➄é❭✡◆ï❺å➈æ✙æ✙ä✥ì✪↔✂ø➑î➓å✠è✥ï✖ù⑨✡☛✘ å æ✙û✓å➄é✙ï➎➌✟ô✐é☎ì✠ó❨é❖å➈ê➉è✧ç✙ï❭è❿å➄é❼✂❛ï➊é✐è♣æ✙û✓å➄é☎ï➈ð✌✕➔é❺ï❯↔❼❶ï✖û➑û➀ï➊é✐è♣î➻ï❞å➈ê✧ÿ✙ä✧ï ì➄ë ✌✑❶û➀ì❛ê✧ï➊é☎ï✖ê✥ê❴✌➳ëíì❛ä✎❿ç☎å➈ä✥å➎↔è✥ï➊ä➏ø➀î➓å❄✂➈ï❞ê❫ø✓ê❫è✧ç✙ï➤ù✂ø✓ê④è❿å➄é✗➊ï✬✡◆ï❶è④ó➵ï➊ï➊é è✧ç☎ï➊ø➀ä✌è✥å➈é❼✂➈ï✖é❛è➞æ☎û➀å➈é✙ï✖ê✒➌❇ó❨ç☎ï➊ä✥ï❙è✥ç✙ï✶ê✧ï❶è➞ì➈ë❨ù✙ø➀ê✤è✧ì❛ä✤è✥ø➑ì❛é☎ê➤ÿ☎ê✧ï✖ù❺è✧ì ✂➈ï✖é✙ï➊ä❿å✠è✥ï❨è✧ç☎ï♣æ✙û✓å➄é☎ï✖ê✇ø➑é✗➊û➑ÿ✎ù✂ï✖ê✇è✧ä❿å➄é✎ê✤û✓å✠è✥ø➑ì❛é☎ê✄➌✐ê❺✖å➄û➀ø➑é✗✂✗➌➇ê✤ô❛ï➊ó❨ø➀é❼✂✗➌ ê❘➍❛ÿ☎ï➊ï✄➽✖ø➑é✗✂✗➌❯ä✧ì➈è✥å✠è✥ø➑ì❛é✏➌❣å➄é☎ù➷û➀ø➀é✙ï➓è✧ç☎ø✠❿ô➇é✙ï❞ê✧ê✒úrå➈ä✧ø✓å✠è✥ø➑ì❛é☎ê✖ð✫✕ è✧ï❞ê④è ï➊ä✥ä✥ì➈ä➔ä❿å✠è✥ï✒ì➄ë✛➾❛ð✙➾✎✶✴ó➵å❛ê➉å✑❿ç☎ø➑ï✖ú➈ï✖ù✺ÿ✎ê✤ø➀é❼✂➔➾✒✓♦↔❢➾✽✓➻æ☎ø➟↔✂ï➊û❯ø➀î➻å✑✂➈ï❞ê➊ð ✓➏ä✥ï❯➞✎û➩è✥ï➊ä✥ø➑é❼✂➽è✧ï✒❿ç☎é✙ø✠➍✐ÿ✙ï❞ê➔ÿ☎ê✧ø➀é❼✂✢ê✧ø➑î➻æ✙û➀ï ✭❫ÿ✗❶û➀ø✓ù✂ï✖å➈é❖ù✂ø✓ê④è❿å➄é✗➊ï✒å✠è î✒ÿ☎û➩è✥ø➑æ✙û➀ï❭ä✧ï❞ê✤ì❛û➑ÿ✂è✥ø➑ì❛é☎ê➳å➄û➀û➑ì✠ó➵ï✖ù✿è✥ì✶ä✥ï✖ù✂ÿ✗➊ï✒è✥ç✙ï❭é✐ÿ☎î➉✡◆ï➊ä➳ì➄ë❫é✙ï★✹✝ ï✖ê✥ê✥å➄ä❘✘➽ã❇å➈é❼✂➈ï✖é❛è✛✧♣ø➀ê✤è✥å➈é✗❶ï➓✖å➄û✁❶ÿ✙û✓å✠è✥ø➑ì❛é☎ê✖ð ☞♣ð✠➾✑➾ þ➇ÿ✙æ✙æ◆ì➈ä✧è✛✣❣ï✒❶è✧ì➈ä➔ö❖å✑❿ç✙ø➀é✙ï➵➪üþ☛✣➳ö➔➶ ✓❦ì➈û✠✘✐é☎ì➈î➻ø➀å➈û❳➊û➀å❛ê✧ê✧ø➟➞✎ï➊ä❿ê➉å➄ä✥ï➞ó➵ï➊û➀û➟✝➠ê✤è✧ÿ☎ù✂ø➀ï✖ù➲î❭ï➊è✧ç✙ì✂ù✙ê♣ëíì❛ä④✂❛ï➊é✦✝ ï➊ä❿å✠è✥ø➑é✗✂ ❶ì❛î➻æ✙û➑ï✄↔✹ù✂ï✒➊ø➀ê✧ø➀ì➈é➶ê✧ÿ✙ä✧ë⑨å✑➊ï✖ê✖ð →➉é✂ëíì➈ä✧è✧ÿ☎é☎å✠è✥ï➊û✠✘✑➌❭è✧ç✙ï✒✘ å➄ä✥ï✛ø➑î➻æ✙ä❿å✑❶è✧ø✁➊å➈û❛ëíì❛ä❣ç✙ø✠✂➈ç✦✝➠ù✂ø➀î❭ï✖é☎ê✧ø➑ì❛é☎å➄û✂æ✙ä✥ì✑✡☎û➑ï✖î➻ê✒➌❄✡◆ï✒✖å➄ÿ☎ê✧ï➵è✧ç✙ï é➇ÿ✙î➉✡◆ï➊ä✛ì➈ë❀æ✙ä✥ì➇ù✙ÿ✗↔è➵è✧ï✖ä✧î➓ê➵ø➀ê➵æ✙ä✥ì➈ç☎ø✙✡✙ø➑è✧ø➀ú➈ï❛ð❣ã✛ç✙ï➤þ✂ÿ✙æ✙æ◆ì➈ä✧è✎✣❣ï★✹✝ è✧ì❛ä➏è✥ï✒❿ç✙é✙ø✁➍✐ÿ✙ï➉ø✓ê➵å➄é➽ï✄↔✐è✥ä✧ï✖î➻ï➊û✠✘❙ï★❶ì➈é☎ì➈î➻ø✠✖å➄û✙ó✛å✪✘❭ì➄ë✝ä✥ï➊æ✙ä✥ï✖ê✧ï➊é✐è❇✝ ø➀é❼✂➚➊ì➈î➻æ✙û➀ï❯↔➽ê✧ÿ✙ä✤ë⑨å➎❶ï❞ê➏ø➀é✶ç✙ø✠✂➈ç✦✝➠ù✂ø➀î➻ï➊é☎ê✧ø➑ì❛é☎å➄û✟ê✧æ☎å✑➊ï✖ê✒➌❛ø➀é✗❶û➀ÿ☎ù✂ø➀é❼✂ æ◆ì➈û✠✘✐é☎ì➈î➻ø➀å➈û➀ê❨å➈é☎ù✢î➻å➈é☛✘➽ì➄è✥ç✙ï➊ä✛è❅✘➇æ◆ï✖ê❨ì➈ë❯ê✧ÿ✙ä✤ë⑨å➎❶ï❞ê✹✞✓✬✠♠ð ✕ æ☎å➈ä✤è✥ø✠➊ÿ✙û➀å➈ä✧û✠✘❖ø➀é✐è✧ï✖ä✧ï❞ê④è✥ø➑é❼✂❺ê✤ÿ❼✡✎ê✤ï➊è❙ì➄ë➉ù✙ï✒❶ø✓ê✧ø➑ì❛é ê✤ÿ✙ä✧ë⑨å✑➊ï✖ê✌ø✓ê è✧ç☎ï❨ì➈é✙ï❞ê❯è✧ç☎å➄è➜❶ì➈ä✥ä✥ï✖ê✧æ✎ì❛é☎ù✌è✥ì➳ç☛✘➇æ◆ï➊ä✥æ✙û➀å➈é✙ï✖ê❯è✥ç☎å✠è❫å➈ä✧ï✛å✠è❫å➳î➓å♦↔☛✝ ø➀î✒ÿ✙î ù✂ø✓ê④è❿å➄é✗➊ï➞ëíä✥ì➈îPè✧ç✙ï➚❶ì❛é✐ú❛ï❯↔✿ç✐ÿ☎û➑û✓ê➉ì➈ë❣è✧ç✙ï✒è④ó✇ì➙❶û✓å➈ê✥ê✤ï❞ê❨ø➑é è✧ç☎ï✢ç✙ø✠✂➈ç❼✝⑥ù✂ø➀î➻ï➊é☎ê✧ø➀ì➈é☎å➈û✇ê✧æ☎å✑➊ï➽ì➄ë❨è✥ç✙ï✢æ☎ä✧ì✂ù✂ÿ✗❶è✒è✥ï➊ä✥î➻ê✖ð ✚✇ì✐ê✤ï✖ä✒➌ â➳ÿ❼✘❛ì➈é✏➌✛å➄é☎ù⑧✣➏å➄æ☎é✙ø➑ô ✞✓✵✑✆✠➔ä✥ï✖å➈û➑ø✠➽➊ï❞ù è✥ç☎å✠è➽å➄é☛✘ æ◆ì➈û✠✘➇é✙ì➈î➻ø✓å➄û❨ì➈ë ù✂ï✒✂➈ä✥ï➊ï ✞✺ø➀é✫è✥ç✙ø➀ê ☛✤î➓å❄↔➇ø➀î✒ÿ☎îPî➓å➄ä❘✂➈ø➀é✤✌➓ê✧ï❶èt➊å➄é➔✡✎ï➚❶ì➈î➻æ✙ÿ✙è✧ï✖ù ✡☛✘➓➞☎ä❿ê✤è➜❶ì➈î➻æ✙ÿ✙è✧ø➀é❼✂➳è✧ç✙ï➉ù✙ì➄è➏æ✙ä✥ì✂ù✂ÿ✗↔è➏ì➈ë☎è✥ç✙ï➔ø➀é✙æ✙ÿ✂è❫ø➀î➓å❄✂❛ï➵ó❨ø➑è✧ç å✶ê✧ÿ❼✡☎ê✧ï❶è➤ì➄ë❣è✥ç✙ï❙è✧ä❿å➄ø➀é✙ø➑é✗✂✶ê✧å➈î➻æ✙û➑ï❞ê➑➪✻➊å➈û➑û➀ï✖ù➲è✧ç✙ï ☛✧ê✧ÿ✙æ✙æ◆ì➈ä✧è♣ú➈ï★✹✝ è✧ì❛ä✥ê❃✌➎➶❯➌✙ï➊û➀ï➊ú✠å➄è✧ø➀é❼✂❭è✧ç✙ï➞ä✥ï✖ê✧ÿ✙û➑è❨è✧ì➻è✥ç✙ï ✞❩✝♠è✧ç✺æ◆ì✠ó✇ï✖ä✒➌✙å➈é☎ù✿û➑ø➀é✙ï✖å➈ä✧û✠✘ ❶ì❛î➉✡✙ø➀é✙ø➀é❼✂♣è✧ç✙ï➵é➇ÿ✙î➉✡◆ï➊ä❿ê❇è✧ç✙ï✖ä✧ï✒✡❩✘➤ì➎✡✂è✥å➈ø➑é✙ï❞ù☛ð❨✜❇ø➀é☎ù✂ø➀é❼✂♣è✧ç✙ï✛ê✧ÿ✙æ✦✝ æ◆ì➈ä✧è➏ú➈ï✒❶è✧ì❛ä✥ê❯å➄é☎ù❙è✥ç✙ï✛❶ì➇ï❈✯➣➊ø➑ï✖é❛è❿ê❣å➄î➻ì➈ÿ☎é❛è❿ê❯è✧ì✌ê✧ì➈û➀ú✐ø➀é❼✂✌å➳ç✙ø✠✂➈ç✦✝ ù✂ø➀î➻ï➊é☎ê✧ø➑ì❛é☎å➄ût➍✐ÿ☎å➈ù✙ä✥å➄è✧ø✁➲î➻ø➑é☎ø➑î➻ø✠➽✖å✠è✥ø➑ì❛é✲æ✙ä✥ì✑✡☎û➑ï✖î ó❨ø➩è✥ç✲û➀ø➑é✙ï❞å➄ä ø➀é✙ï✒➍✐ÿ☎å➈û➑ø➑è❅✘✆❶ì➈é✎ê④è✥ä✥å➈ø➑é✐è✥ê✖ð✛✜✙ì❛ä➉è✥ç✙ï❙ê✥å➄ô❛ï➞ì➄ë➜➊ì➈î➻æ☎å➈ä✧ø✓ê✤ì❛é✏➌☎ó➵ï➞ø➀é✦✝ ❶û➀ÿ☎ù✂ï✺ç✙ï✖ä✧ï✶è✧ç✙ï✺ä✥ï✖ê✧ÿ✙û➑è✥ê❙ì✑✡✂è❿å➄ø➀é✙ï✖ù➹✡❩✘✢✚➵ÿ✙ä❘✂➈ï❞ê❙å➈é☎ù❑þ❼❿ç✁ì➈û➀ô➈ì❛æ✂ë ä✥ï➊æ◆ì➈ä✧è✧ï✖ù✻ø➑é ✞✓ ❜✠⑥ð ✎ø➑è✧ç å❑ä✧ï✒✂➈ÿ✙û✓å➄ä✺þ☛✣♣ö①➌➵è✥ç✙ï➊ø➀ä✺ï➊ä✥ä✧ì❛ä✶ä❿å✠è✧ï ì➈é❖è✧ç✙ï➻ä✥ï✄✂❛ÿ✙û➀å➈ä➔è✥ï✖ê✤è✌ê✧ï❶è➤ó✛å➈ê➑➾➈ð ❝✌✶✶ð➚☞✇ì➈ä✧è✧ï❞ê♣å➈é☎ù↕✣➏å➈æ✙é✙ø➀ô✫ç☎å➈ù ä✥ï➊æ◆ì➈ä✧è✧ï✖ù❲å➄é ï✖ä✧ä✥ì➈ä✿ä✥å➄è✧ï➷ì➄ë✫➾➈ð✠➾✚✶ ó❨ø➩è✥ç þ✦✣♣ö ì➈é❲è✧ç☎ï ê✧å➈î❭ï ù✙å➄è✥å➷ÿ☎ê✧ø➑é✗✂❤å ê✤û➀ø✠✂➈ç✐è✧û✠✘ ù✙ø➟➘✟ï➊ä✥ï➊é✐è➓è✧ï★❿ç✙é✙ø✁➍✐ÿ✙ï➈ð ã✛ç✙ï➔➊ì➈î➻æ✙ÿ✂è❿å♦✝ è✧ø➀ì➈é✎å➄û✏❶ì✐ê④è❨ì➈ë❀è✧ç☎ø➀ê✛è✥ï✒❿ç✙é☎ø✠➍✐ÿ✙ï➤ø✓ê✛ú➈ï➊ä❘✘➓ç✙ø✠✂➈ç ✰➏å✑✡✎ì❛ÿ✂è✌➾❪❝❭î➻ø➑û➀û➑ø➀ì➈é î✒ÿ☎û➩è✥ø➑æ✙û✠✘❩✝⑥å❛ù✙ù✙ê❙æ✎ï✖ä✒ä✥ï✒➊ì✑✂➈é☎ø➩è✥ø➑ì❛é✝ð↕→➉ê✧ø➑é✗✂❺þ✦❿ç✁ì❛û➑ô❛ì➈æ✂ë✱❁ ê➓✣♣ø➑ä✧è✧ÿ✎å➄û þ➇ÿ✙æ☎æ✎ì❛ä✤è➵✣❣ï✒❶è✧ì❛ä✥ê❙è✧ï★❿ç✙é✙ø✁➍✐ÿ✙ï①➪➭✣✛✝④þ☛✣➳ö➔➶✹➌④➾➈ð ✘✞✶❂ï➊ä✥ä✥ì➈ä❭ó➵å❛ê❭å➄è❇✝ è✥å➈ø➑é☎ï✖ù☛ð❯ö✫ì➈ä✥ï❣ä✧ï★❶ï✖é❛è✥û✙✘➎➌❞þ✦❿ç✂ì➈û➀ô➈ì➈æ✙ë✇➪⑨æ✎ï✖ä✥ê✧ì➈é☎å➈û✑❶ì❛î➻î✒ÿ✙é✙ø✁➊å➄è✧ø➀ì➈é✥➶ 8.1 1.9 1.8 3.2 3.7 1.8 1.4 1.6 0.5 [deslant] K−NN Euclidean [16x16] Tangent Distance SVM poly 4 LeNet−4 LeNet−4 / Local LeNet−4 / K−NN [dist] Boosted LeNet−4 0 1 2 3 4 5 6 7 8 9 [deslant] 20x20−300−10 [16x16] LeNet−1 ✁❼✿▲❍✪❦✏❬❇❪★❦✤◆✏✰☎✄✻✰❅❆r✵✶✿▲✷✹❈ ■❼✰❅✸✻⑥✙✷✹✸✶❋✛✱✴❈✪❆❅✰✹♠✬❃✑✰❅✸✻❆❇✰❅❈❯✵❖✱✴❍✹✰✌✷✴⑥❳✵✶✰❅✺✵✛❃♦✱❘✵✻✵✻✰❅✸✶❈✪✺✩✵✻✯♦✱❘✵ ❋✎✳★✺✻✵❢◗❄✰❷✸✶✰☎✄ ✰❅❆❅✵✻✰❅❉✞✵✻✷✩✱✴❆❖✯✪✿▲✰r▼❯✰➜❪★❦ ✁✁ ✰❅✸✶✸✻✷✹✸❢⑥✠✷✹✸❢✺✻✷✹❋✩✰❨✷✴⑥❼✵✶✯✪✰❷✺❙★✺✵✶✰❅❋✩✺❇❦ 4 36 −−−− 24,000 −−−−> 39 794 −−−− 20,000 −−−−> −−−− 14,000 −−−−> 650 −−−− 28,000 −−−−> 123 795 267 469 100 260 −−−− 20,000 −−−−> −−−− 10,000 −−−−> 401 460 [deslant] K−NN Euclidean 1000 RBF [16x16] Tangent Distance SVM poly 4 RS−SVM poly 5 [dist] V−SVM poly 9 [deslant] 20x20−300−10 28x28−1000−10 28x28−300−100−10 28x28−500−150−10 [16x16] LeNet−1 LeNet−4 LeNet−4 / Local LeNet−4 / K−NN LeNet−5 Boosted LeNet−4 0 300 600 900 Linear Pairwise 40 PCA+quadratic ✁❼✿▲❍✪❦❩❬✹❬✹❦➙❤❳✳★❋✎◗❄✰❅✸❀✷✴⑥❼❋➃✳✪❴✵✻✿▲❃✪❴❙✒❑●✱✴❆❇❆❅✳✪❋➃✳✪❴➂✱❘✵✻✰❨✷✹❃❄✰❇✸✶✱❘✵✶✿▲✷✹❈★✺❳⑥✙✷✹✸✏✵✻✯✪✰❹✸✶✰❅❆❅✷✹❍✹❈✪✿ ❑ ✵✻✿▲✷✹❈✞✷✴⑥✗✱✎✺✻✿▲❈✪❍✹❴▲✰❨❆❖✯♦✱✴✸✶✱✴❆r✵✶✰❅✸❜✺✵❖✱✴✸✵✶✿▲❈★❍✎✽❀✿ ✵✶✯✞✱✎✺✶✿▲❵❅✰r❑●❈✪✷✹✸✻❋✛✱✴❴▲✿▲❵❇✰❅❉✛✿▲❋✛✱✴❍✹✰✹❦ ç☎å❛ê➔ä✥ï✖å✑❿ç☎ï✖ù ✘✙ð ✺✞✶ ÿ☎ê✧ø➑é❼✂✢å➓î➻ì➇ù✙ø➟➞☎ï❞ù✫ú➈ï✖ä✥ê✧ø➀ì➈é✺ì➄ë❣è✥ç✙ï➓✣✛✝④þ☛✣➳öð →➔é✙ëíì➈ä✧è✧ÿ✙é☎å➄è✧ï✖û✙✘➎➌♦✣✛✝④þ☛✣♣ö ø✓ê❇ï❯↔➇è✧ä✥ï➊î➻ï➊û✠✘➤ï❯↔✂æ◆ï➊é☎ê✧ø➑ú❛ï✻✰✝å✑✡✎ì❛ÿ✂è❀è④ó❨ø✁❶ï å➈ê➏î✒ÿ✗❿ç➓å➈ê❦ä✥ï✄✂❛ÿ✙û✓å➄ä❫þ☛✣➳öð❛ã❇ì➞å➄û➀û➑ï✖ú➇ø➀å➄è✧ï➵è✧ç✙ø✓ê➏æ☎ä✧ì➎✡✙û➑ï✖î✫➌✑✚➵ÿ✙ä❺✂❛ï✖ê ç☎å❛ê✒æ✙ä✥ì➈æ◆ì❛ê✧ï✖ù è✧ç✙ï✫✍❨ï✖ù✙ÿ✗❶ï❞ù❑þ➇ï➊è➽þ✂ÿ✙æ✙æ◆ì➈ä✧è➚✣❣ï★↔è✧ì❛ä✒è✥ï✒❿ç✙é☎ø✠➍✐ÿ✙ï ➪✍♣þ❩✝➠þ✦✣♣ö➔➶❯➌❞ó❨ç☎ø✠❿ç➞å➄è✤è✥å➈ø➑é☎ï✖ù➝➾➈ð✠➾✚✶✻ì➈é➤è✥ç✙ï❫ä✥ï✄✂❛ÿ✙û✓å➄ä☛è✥ï✖ê✤è❇ê✤ï➊è❀✞✓✫❜✬✠✶➌ ó❨ø➑è✧ç➘å➔❶ì❛î➻æ✙ÿ✂è✥å➄è✧ø➀ì➈é☎å➈û✩❶ì❛ê✤è❙ì➈ë➉ì➈é☎û✙✘✮✓✵✙❆✘❼➌ ✘✻✘✗✘✺î✒ÿ✙û➑è✧ø➀æ✙û✠✘❩✝⑥å❛ù✙ù✙ê æ◆ï➊ä➳ä✧ï★❶ì✑✂❛é✙ø➑è✧ø➀ì➈é✏➌✎øüð ï➈ð♣ì➈é✙û✠✘✫å❄✡◆ì➈ÿ✂è ✓✻✘✞✶ î➻ì➈ä✥ï➞ï❯↔✂æ◆ï➊é☎ê✧ø➀ú➈ï➞è✥ç☎å➄é ✗✝ï❞ñ➔ï➊è❇✝ ✙✂ð ✧✛ ✧✣✁➩✱❄☛✡☛➩✴➩▼✣ ➯♦➨ ✕❍ê✤ÿ✙î➻î➓å➄ä❘✘✌ì➄ë◆è✧ç☎ï➵æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï✛ì➄ë☎è✥ç✙ï✛❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê❯ø✓ê❣ê✤ç☎ì✠ó❨é ø➀é➣✜❇ø✠✂➈ÿ☎ä✧ï❞ê✑❀➳è✧ì➵➾✆✑✂ð❨✜❇ø✠✂➈ÿ☎ä✧ï ❀✌ê✧ç✙ì✠ó➔ê❦è✥ç✙ï➔ä❿åró❤ï➊ä✥ä✥ì➈ä❣ä❿å✠è✧ï❨ì➈ë✟è✧ç✙ï ❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê❦ì❛é➻è✧ç✙ï➉➾✽✘❼➌ ✘✻✘✗✘➉ï✄↔✙å➄î➻æ✙û➀ï❨è✧ï❞ê④è✇ê✧ï❶è❞ð❜✚➵ì➇ì❛ê✤è✧ï❞ù ✗✝ï❞ñ➔ï❶è❺✝❩❝ æ◆ï➊ä✧ëíì➈ä✥î❭ï❞ù➣✡◆ï✖ê✤è✒➌☎å➎❿ç✙ø➀ï➊ú➇ø➑é✗✂❙å➻ê❺➊ì➈ä✥ï➉ì➈ë✝✘✙ð ✔✖✶ ➌✦❶û➀ì❛ê✧ï➊û✠✘➻ëíì➈û➀û➀ì✠ó✇ï❞ù ✡☛✘✘✗✝ï❞ñ➔ï➊è❇✝ ✙❙å➄è ✘✙ð ✺✞✶✶ð ✜❇ø✠✂➈ÿ☎ä✧ï✫➾✽✘✿ê✧ç✙ì✠ó➔ê♣è✧ç✙ï➓é➇ÿ✙î➑✡✎ï✖ä✌ì➄ë✇æ✎å✠è✤è✥ï➊ä✥é☎ê➤ø➀é❺è✥ç✙ï❭è✧ï❞ê④è✒ê✧ï❶è è✧ç✎å✠è❙î❙ÿ☎ê✤è➑✡◆ï✶ä✥ï✓④ï✒❶è✧ï❞ù➷è✧ì å✠è✤è❿å➄ø➀é å✦✘☎ð ✙✞✶ ï✖ä✧ä✥ì➈ä✌ëíì❛ä❙ê✧ì➈î➻ï➽ì➈ë è✧ç☎ï➞î➻ï❶è✧ç☎ì➇ù☎ê➊ð✎✓❣å✠è✤è✥ï➊ä✥é☎ê❨å➄ä✥ï✌ä✥ï✓④ï✒❶è✧ï❞ù✢ó❨ç☎ï➊é✺è✧ç☎ï➞ú✠å➄û➀ÿ✙ï✌ì➄ë❷❶ì❛ä❇✝ ä✥ï✖ê✧æ✎ì❛é☎ù✂ø➀é❼✂✶ì❛ÿ✂è✧æ✙ÿ✙è➳ø✓ê➤ê✤î➓å➄û➀û➀ï➊ä♣è✧ç☎å➈é❺å✢æ✙ä✧ï❞ù✂ï❯➞✎é✙ï✖ù➲è✧ç✙ä✥ï✖ê✧ç✙ì➈û✓ù☛ð ➏➠é➓î➓å➄é☛✘❙å➈æ✙æ✙û➀ø✠✖å✠è✥ø➑ì❛é☎ê✄➌➈ä✧ï✰✓④ï✒↔è✥ø➑ì❛é➓æ✎ï✖ä✤ëíì❛ä✧î➓å➈é✗❶ï✛ø✓ê➏î➻ì➈ä✥ï❨ê✤ø✠✂➈é✙ø➑ë●✝ ø✁➊å➄é✐è➳è✧ç☎å➈é❖ä❿åró ï➊ä✥ä✧ì❛ä➉ä❿å✠è✥ï➈ð➞ã✛ç✙ï➻ê❺➊ì➈ä✥ï✒ÿ☎ê✧ï✖ù➲è✧ì✺ù✂ï✒➊ø➀ù✙ï❙ÿ✙æ◆ì➈é è✧ç☎ï✶ä✥ï✓④ï✒❶è✧ø➀ì➈é ì➈ë❨å➲æ☎å✠è✧è✧ï➊ä✥é ó✛å➈ê➤è✥ç✙ï✢ù✙ø➟➘✟ï➊ä✥ï➊é✗➊ï➣✡◆ï❶è④ó➵ï➊ï➊é è✧ç✙ï ê❘❶ì➈ä✥ï✖ê✛ì➈ë❇è✥ç✙ï✌è✥ì➈æ✺è④ó➵ì➣❶û✓å➈ê✥ê✤ï❞ê➊ð➜✕✬✂❛å➈ø➑é✏➌✈✚➵ì✐ì✐ê④è✥ï✖ù◆✗✝ï❞ñ➔ï➊è❇✝✿❝➓ç☎å➈ê è✧ç☎ï➣✡✎ï❞ê④è➞æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï➈ð✶ã✛ç☎ï➓ï➊é✙ç☎å➈é✗❶ï❞ù ú➈ï➊ä❿ê✧ø➑ì❛é☎ê➳ì➄ë ✗✝ï❞ñ➔ï❶è❺✝❩❝ ù✂ø✓ù➐✡✎ï➊è✤è✧ï✖ä❨è✧ç✎å➄é✿è✧ç✙ï✌ì❛ä✧ø✠✂➈ø➀é☎å➈û ✗❀ï✖ñ➉ï❶è❇✝✿❝✗➌☎ï➊ú➈ï✖é✶è✥ç✙ì➈ÿ✗✂➈ç✢è✧ç✙ï✌ä❿åró
CFO-ob CRE EE7AOLEy BEF FV 五 Linear ing the template images.Not surprisingl0,neural networks Pairwise require much less memor0 than memor0-based methods. [desant]K-NN Eucidear 24.000 The Overall performance depends on mano factors in- 40 PCA+quadratic 40 1000 RB cluding accurac0,running time,and memor0 requirements. [16x16]Tangent Distance 25,000- As computer technologo improves,larger-capacito recog- SVM poly 4 -14000----3 RS-SVM poly 5 nizers become feasible.varger recognizers in turn require lis到V-SMpo时5 -28,000----3 larger training sets.veNet-1 was appropriate to the avail- 【deslant2020-300-10 123 able technologo in 1989,ust as veNet-vis appropriate now. 2828-1000-10 In 1989 a recognizer as complex as veNet-y would have re- 28x28-300-100-10 28x28-500-150-10 quired several weeks training,and more data than was available,and was the efore not even considered.For quite [16x16]LeNet 1 a long time,veNet-1as considered the state of the art. LeNet 4 17 LeNet 4/Local -24.000- The local learning classifier,the optimal margin classifier, LeNet 4/K-NN 24,000 and the tangent distance classifier were developed to im- LeNet 5 160 Boosted LeNet 4 prove upon veNet-1 E and the0 succeeded at that.How- 00 00 ever,the0 in turn motivated a search for improved neural network architectures.This search was guided in part bo RmbP。 8用营设2oao estimates of the capacito of various learning machines,de- rived from measurements of the training and test error as a function of the number of training examples.e dis- covered that more capacito was needed.Through a series accuracies were identical. of experiments in architecture,combined with an anal0- Figure 11 shows the number of multipl0-accumulate op- sis of the characteristics of recognition errors,veNet-4 and erations necessar0 for the recognition of a single size- veNet-y were crafted. normalized image for each method.-xpectedlo,neural J e find that boosting gives a substantial improvement in networks are much less demanding than memor0-based accurac0,with a relativel0 modest penalto in memor0 and methods.Convolutional Neural Networks are particu- computing expense.Also,distortion models can be used larl0 well suited to hardware implementations because of to increase the effective size of a data set without actuallo their regular structure and their low memor0 requirements requiring to collect more data. for the weights.Single chip mixed analog-digital imple- The Support Vector Machine has excellent accurac0, mentations of veNet-ys predecessors have been shown to which is most remarkable,because unlike the other high operate at speeds in cess of 1NMM characters per sec- performance classifiers,it does not include a prvrVknowl- ond j42.However,the rapid progress of mainstream com- edge about the problem.In fact,this classifier would do puter technologo renders those exotic technologies quicklo ust as well if the image pixels were permuted with a fixed obsolete.Cost-effective implementations of memor0-based mapping and lost their pictorial structure.However,reach- techniques are more elusive,due to their enormous memor0 ing levels of performance comparable to the Convolutional requirements,and computational requirements. Neural Networks can onl0 be done at considerable expense Training time was also measured.K-nearest neighbors in memor0 and computational requirements.The reduced- and TDC have essentiallo zero training time.J hile the set SVM requirements are within a factor of two of the single-laOer net,the pairwise net,and 6 CAg quadratic net Convolutional Networks,and the error rate is vero close. could be trained in less than an hour,the multilaDer net Improvements of those results are expected,as the tech- training times were expectedlo much longer,but onl0 re- nique is relativel0 new. quired 1M to IM passes through the training set.This J hen plento of data is available,mano methods can at- amounts to I to 3 da0s of C6U to train veNet-y on a Sil- tain respectable accurac0.The neural-net methods run icon Graphics Origin I NMM server,using a single I MAMHz much faster and require much less space than memor0- R1 NMNMprocessor.It is important to note that while the based techniques.The neural nets advantage will become training time is somewhat relevant to the designer,it is of more striking as training databases ontinue to increase in little interest to the final user of the sOstem.Given the size. choice between an existing technique,and a new technique that brings marginal accurac0 improvements at the price EwInvar in e and Noe Res Wtan e of considerable training time,an0 final user would chose Convolutioal networks are parficular10 well suited for the latter. recognizing or re ecting shapes with widel0 varOing size, Figure 1 shows the memor0 requirements,and therefore position,and orientation,such as the ones topicallo pro the number of free parameters,of the various classifiers duced bo heuristic segmenters in real-world string recogni- measured in terms of the number of variables that need tion sOstems. to be stored.Most methods require onl0 about one bOte In an experiment like the one described above,the im- per variable for adequate performance.However,Nearest- portance of noise resistance and distortion invariance is Neighbor methods ma0 get bo with 4 bits per pixel for stor- not obvious.The situation in most real applications is
✂✁☎✄✝✆✟✞✠✄☛✡✌☞✎✍✟✏✒✑✓✏✂✏✂✏✎✔✖✕☛✄☎✗☛✏✙✘✛✚✙✏✂✁✢✜✤✣✥✣✧✦ ✜✄ 4 35 −−− 24,000 −−−> 40 794 −−− 25,000 −−−> −−−− 14,000 −−−−> 650 −−−− 28,000 −−−−> 123 795 267 469 3 17 −−− 24,000 −−−> −−− 24,000 −−−> 60 51 1000 RBF [16x16] Tangent Distance SVM poly 4 RS−SVM poly 5 [dist] V−SVM poly 5 [deslant] 20x20−300−10 28x28−1000−10 28x28−300−100−10 28x28−500−150−10 [16x16] LeNet 1 LeNet 4 LeNet 4 / Local LeNet 4 / K−NN LeNet 5 Boosted LeNet 4 0 300 600 900 Linear Pairwise 40 PCA+quadratic [deslant] K−NN Euclidean ✁❼✿▲❍✪❦♦❬❇❾★❦➝❸t✰❅❋✩✷✹✸❙✎✸✻✰❇⑦✄✳✪✿▲✸✻✰❇❋✩✰❅❈❯✵✶✺❇❚❯❋✩✰❇✱✴✺✶✳✪✸✻✰❇❉✎✿▲❈✩❈✒✳✪❋➃◗❄✰❇✸✥✷✴⑥❩▼✴✱✴✸✶✿➂✱✴◗✪❴▲✰❅✺❺❚❯⑥✙✷✹✸ ✰❇✱✴❆❖✯✛✷✴⑥❩✵✻✯✪✰❜❋✩✰r✵✻✯✪✷✄❉✪✺❺❦✥❸t✷✹✺✻✵✈✷✴⑥❩✵✻✯✪✰❜❋✩✰r✵✻✯✪✷✄❉✪✺✇✷✹❈✪❴❙✩✸✻✰❅⑦✒✳✪✿▲✸✻✰❳✷✹❈★✰❜◗❯❙✒✵✻✰ ❃❄✰❅✸✏▼✴✱✴✸✶✿➂✱✴◗✪❴▲✰❨⑥✙✷✹✸❀✱✴❉✪✰❅⑦✒✳✪✱❘✵✶✰➜❃❄✰❇✸⑥✠✷✹✸✻❋✛✱✴❈✪❆❅✰✹❦ å✑✒❶ÿ✙ä❿å✑➊ø➑ï❞ê➵ó✇ï✖ä✧ï➤ø✓ù✂ï➊é✐è✧ø✁➊å➈ûüð ✜❇ø✠✂➈ÿ☎ä✧ï➝➾✑➾➤ê✧ç✙ì✠ó➔ê➏è✥ç✙ï✌é➇ÿ✙î➉✡◆ï➊ä❨ì➈ë❇î❙ÿ✙û➩è✥ø➑æ☎û✙✘❩✝➠å✑✄➊ÿ✙î✒ÿ☎û➀å➄è✧ï➳ì➈æ✦✝ ï➊ä❿å✠è✥ø➑ì❛é☎ê é☎ï✒❶ï❞ê✧ê✥å➄ä❘✘òëíì❛ä❤è✥ç✙ï✲ä✥ï✒➊ì✑✂❛é✙ø➩è✥ø➑ì❛é ì➄ë❖åòê✧ø➀é❼✂➈û➀ï✾ê✧ø✙➽✖ï❯✝ é✙ì❛ä✧î➓å➄û➀ø✠➽➊ï✖ù✾ø➀î➻å✑✂➈ï ëíì➈ä✫ï❞å✑❿ç❲î➻ï❶è✧ç☎ì➇ù✝ð ✭❷↔✂æ◆ï✒↔è✥ï✖ù✂û✠✘✑➌✌é☎ï➊ÿ✙ä❿å➄û é✙ï➊è④ó✇ì❛ä✧ô✂ê❖å➈ä✧ï❑î✒ÿ✗❿ç✹û➑ï❞ê✧ê➷ù✂ï✖î➻å➈é☎ù✂ø➀é❼✂✻è✥ç☎å➄é➶î➻ï➊î➻ì➈ä❘✘❩✝✶✡☎å❛ê✤ï❞ù î➻ï❶è✥ç✙ì✂ù✙ê➊ð❂☞✇ì➈é➇ú➈ì❛û➑ÿ✂è✥ø➑ì❛é☎å➄û✢ñ➔ï➊ÿ☎ä✥å➈û✢ñ➔ï➊è④ó✇ì❛ä✧ô✂ê å➈ä✧ï✻æ✎å➄ä✧è✧ø✁❶ÿ✦✝ û✓å➄ä✥û✙✘❺ó✇ï✖û➑û➵ê✤ÿ✙ø➑è✧ï❞ù è✧ì➲ç✎å➄ä❿ù✂ó➵å➈ä✧ï➻ø➀î➻æ✙û➑ï✖î➻ï➊é✐è✥å➄è✧ø➀ì➈é☎ê➉✡✎ï★➊å➄ÿ✎ê✤ï➓ì➈ë è✧ç☎ï➊ø➀ä✇ä✥ï✄✂❛ÿ✙û➀å➈ä❫ê✤è✧ä✥ÿ✗↔è✥ÿ✙ä✧ï♣å➄é☎ù❭è✥ç✙ï➊ø➀ä✇û➀ì✠ó➘î➻ï➊î➻ì➈ä❘✘✒ä✥ï✒➍✐ÿ✙ø➀ä✧ï✖î➻ï➊é✐è✥ê ëíì➈ä➽è✧ç✙ï➲ó➵ï➊ø✠✂➈ç✐è❿ê➊ð þ➇ø➑é✗✂➈û➀ï➔❿ç✙ø➀æ î➻ø✙↔✂ï✖ù❍å➄é✎å➄û➀ì✑✂❄✝➠ù✂ø✠✂➈ø➑è✥å➈û➵ø➀î➻æ✙û➀ï❯✝ î➻ï➊é✐è✥å➄è✧ø➀ì➈é☎ê➤ì➈ë❀✗❀ï✖ñ➔ï➊è❇✝ ✙ ❁ ê➤æ✙ä✥ï✖ù✂ï★❶ï✖ê✥ê✧ì➈ä❿ê➉ç☎årú❛ï➑✡◆ï➊ï✖é ê✤ç✙ì✠ó❨é❖è✧ì ì➈æ◆ï➊ä❿å✠è✥ï å✠è❖ê✧æ✎ï✖ï✖ù✙ê❖ø➑é ï✄↔❼❶ï✖ê✥ê✫ì➈ë➐➾✒✘✻✘✻✘ ❿ç☎å➈ä✥å➎↔è✥ï➊ä❿ê✢æ◆ï➊ä ê✤ï★✹✝ ì➈é✎ù ✞✓✬❝✫✠♠ð❣õ➔ì✠ó➵ï➊ú❛ï➊ä★➌✐è✧ç✙ï➤ä❿å➄æ✙ø✓ù✶æ☎ä✧ì➎✂➈ä✥ï✖ê✥ê❫ì➈ë❇î➓å➄ø➀é☎ê✤è✧ä✥ï✖å➄î ❶ì➈î➚✝ æ✙ÿ✂è✥ï➊ä❨è✥ï✒❿ç✙é☎ì➈û➀ì✑✂✑✘✶ä✧ï✖é☎ù✂ï➊ä❿ê➵è✧ç☎ì❛ê✧ï✌ï❯↔✂ì➄è✥ø✠♣è✧ï★❿ç✙é✙ì❛û➑ì➎✂➈ø➀ï✖ê✩➍✐ÿ✙ø✁❿ô✐û✠✘ ì✑✡✎ê✤ì❛û➑ï➊è✧ï➈ð✔☞✇ì❛ê✤è❇✝⑥ï❯➘✟ï✒↔è✥ø➑ú❛ï➤ø➑î➻æ✙û➀ï➊î➻ï➊é✐è❿å✠è✧ø➀ì➈é✎ê❨ì➄ë❦î➻ï➊î➻ì➈ä❘✘❩✝✶✡☎å❛ê✤ï❞ù è✧ï★❿ç✙é✙ø✁➍✐ÿ✙ï✖ê❯å➄ä✥ï❫î➻ì➈ä✥ï❫ï✖û➑ÿ✎ê✤ø➀ú➈ï➎➌rù✙ÿ✙ï❫è✥ì➔è✥ç✙ï➊ø➀ä❦ï➊é✙ì❛ä✧î➻ì➈ÿ✎ê✝î➻ï➊î➻ì➈ä❘✘ ä✥ï✒➍✐ÿ✙ø➀ä✧ï✖î❭ï✖é✐è✥ê✒➌✙å➄é☎ù➐❶ì❛î❭æ☎ÿ✂è✥å➄è✧ø➀ì➈é☎å➈û☛ä✧ï★➍❛ÿ☎ø➑ä✥ï➊î➻ï➊é✐è❿ê➊ð ã❀ä❿å➄ø➀é✙ø➀é❼✂✺è✧ø➀î➻ï✶ó✛å➈ê➞å➈û➀ê✧ì✫î➻ï✖å➈ê✧ÿ✙ä✥ï✖ù☛ð✆☎✞✝⑥é✙ï✖å➈ä✧ï❞ê④è➞é☎ï➊ø✠✂➈ç☛✡✎ì❛ä✥ê å➄é✎ù❤ã✩✧✌☞✭ç☎årú➈ï✢ï✖ê✥ê✤ï✖é✐è✧ø✓å➄û➀û✙✘ ➽➊ï✖ä✧ì❖è✧ä❿å➄ø➀é✙ø➀é❼✂➲è✥ø➑î➻ï➈ð ✎ç☎ø➑û➀ï✿è✧ç✙ï ê✧ø➑é❼✂❛û➑ï✄✝♠û✓å✪✘➈ï✖ä❫é✙ï➊è✒➌➇è✥ç✙ï♣æ✎å➄ø➀ä✧ó❨ø✓ê✤ï♣é✙ï❶è★➌✙å➄é☎ù➙✓✎☞✎✕✣✞➍❛ÿ✎å➈ù✂ä❿å✠è✥ø✠➉é✙ï❶è ❶ì❛ÿ✙û✓ù ✡✎ï➓è✥ä✥å➈ø➑é☎ï✖ù➷ø➀é û➑ï❞ê✧ê➞è✧ç☎å➈é å➄é ç✙ì❛ÿ✙ä✒➌❇è✧ç☎ï➽î✒ÿ✙û➑è✧ø➀û✓å✪✘➈ï➊ä✒é✙ï❶è è✧ä❿å➄ø➀é✙ø➀é❼✂✺è✧ø➀î➻ï✖ê✌ó➵ï➊ä✥ï➓ï❯↔✂æ◆ï✒❶è✧ï✖ù✙û✙✘❺î✒ÿ✗❿ç û➀ì➈é❼✂❛ï➊ä★➌✏✡✙ÿ✂è❙ì➈é✙û✠✘❖ä✥ï❯✝ ➍✐ÿ✙ø➀ä✧ï❞ù ➾✽✘ è✥ì ✑❆✘✻æ☎å➈ê✥ê✤ï❞ê✫è✥ç✙ä✧ì❛ÿ❼✂➈ç è✥ç✙ï è✥ä✥å➈ø➑é☎ø➑é❼✂✾ê✤ï➊è✖ð✯ã✛ç✙ø✓ê å➄î➻ì❛ÿ✙é✐è✥ê❨è✥ì ✑❙è✧ì ❜➽ù✙å✪✘✂ê❨ì➄ë➃☞✎✓➃→ è✧ì➓è✥ä✥å➈ø➑é❈✗❀ï✖ñ➉ï❶è❇✝ ✙➻ì❛é✺å✢þ➇ø➑û✙✝ ø✁❶ì➈é❤â➳ä❿å➄æ☎ç✙ø✠✖ê❙ý♣ä✥ø✙✂❛ø➑é ✑❆✘✗✘✻✘✺ê✧ï➊ä✥ú➈ï➊ä★➌❀ÿ✎ê✤ø➀é❼✂➲å➲ê✧ø➑é✗✂➈û➀ï✘✑❆✘✻✘❛ö➲õ✛➽ ✍✌➾✒✘✻✘✻✘✗✘✶æ☎ä✧ì✦❶ï❞ê✧ê✧ì➈ä❞ð➚➏⑥è➞ø➀ê✌ø➀î➻æ✎ì❛ä✤è❿å➄é✐è➤è✧ì✫é✙ì➈è✧ï➻è✥ç☎å✠è➞ó❨ç✙ø➀û➀ï➻è✧ç✙ï è✧ä❿å➄ø➀é✙ø➀é❼✂❭è✧ø➀î➻ï✌ø➀ê➔ê✧ì➈î➻ï✖ó❨ç☎å✠è❨ä✥ï➊û➀ï➊ú✠å➄é✐è✛è✧ì❭è✧ç☎ï➞ù✂ï✖ê✧ø✙✂❛é✙ï➊ä★➌✂ø➑è➔ø➀ê✛ì➈ë û➀ø➩è✧è✧û➀ï✫ø➑é✐è✧ï✖ä✧ï❞ê④è❭è✧ì è✥ç✙ï➐➞☎é☎å➄û❨ÿ✎ê✤ï✖ä❭ì➈ë➉è✥ç✙ï✫ê❇✘✂ê✤è✧ï➊î✺ð➘â➳ø➑ú❛ï➊é❑è✧ç✙ï ❿ç✙ì❛ø✠➊ï✛✡✎ï➊è④ó✇ï✖ï➊é✶å➈é➽ï❯↔✂ø✓ê④è✥ø➑é✗✂✌è✧ï★❿ç✙é✙ø✁➍❛ÿ☎ï✑➌✂å➄é☎ù✶å✌é✙ï➊ó➘è✥ï✒❿ç✙é☎ø✠➍✐ÿ✙ï è✧ç✎å✠è➉✡✙ä✥ø➀é❼✂❛ê✌î➓å➈ä❺✂❛ø➑é☎å➈û➏å✑✒❶ÿ✙ä❿å✑✄✘➲ø➑î➻æ✙ä✥ì✠ú➈ï✖î➻ï➊é✐è✥ê✌å➄è➞è✧ç✙ï✶æ✙ä✧ø✁❶ï ì➄ë✞❶ì➈é✎ê✤ø✓ù✂ï➊ä❿å❄✡☎û➑ï✶è✧ä❿å➄ø➀é✙ø➑é✗✂➲è✧ø➀î➻ï✑➌➵å➄é☛✘➒➞☎é☎å➄û✛ÿ☎ê✧ï➊ä➻ó➵ì➈ÿ✙û✓ù✢❿ç✙ì✐ê✤ï è✧ç☎ï✌û➀å➄è✤è✧ï✖ä✖ð ✜❇ø✠✂➈ÿ☎ä✧ï✞➾ ✑➉ê✤ç✙ì✠ó➔ê❀è✧ç☎ï✇î➻ï➊î➻ì❛ä❺✘➤ä✥ï✒➍✐ÿ✙ø➀ä✧ï✖î❭ï✖é✐è✥ê✒➌rå➈é☎ù✌è✥ç✙ï➊ä✥ï❶ëíì❛ä✧ï è✧ç☎ï❖é➇ÿ✙î➉✡◆ï➊ä✿ì➄ë✌ëíä✥ï➊ï❖æ☎å➄ä❿å➄î➻ï❶è✥ï➊ä❿ê✄➌✛ì➈ë✌è✧ç✙ï❺úrå➈ä✧ø➀ì➈ÿ✎ê ❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê î➻ï✖å❛ê✤ÿ✙ä✥ï✖ù❑ø➀é❑è✥ï➊ä✥î➓ê➻ì➄ë➳è✧ç✙ï✫é➇ÿ✙î➑✡✎ï✖ä➓ì➄ë➤úrå➈ä✧ø✓å❄✡☎û➑ï❞ê✒è✥ç☎å✠è➽é☎ï➊ï✖ù è✧ì✆✡◆ï✶ê✤è✧ì❛ä✧ï❞ù☛ð✶ö✫ì✐ê④è➞î➻ï❶è✥ç✙ì✂ù✙ê➞ä✥ï✒➍✐ÿ✙ø➀ä✥ï❭ì❛é✙û✠✘❺å❄✡◆ì➈ÿ✙è➞ì➈é✙ï➵✡☛✘✐è✧ï æ◆ï➊ä➉ú✠å➈ä✧ø✓å❄✡✙û➀ï➳ëíì➈ä➉å❛ù✂ï✒➍✐ÿ☎å➄è✧ï✌æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï➈ð❫õ➉ì✠ó✇ï✖ú➈ï➊ä★➌✙ñ➔ï❞å➄ä✥ï✖ê✤è❇✝ ñ➔ï✖ø✙✂❛ç☛✡✎ì❛ä❇î➻ï❶è✥ç✙ì✂ù✙ê❯î➓å✪✘✌✂➈ï❶è❜✡☛✘✌ó❨ø➩è✥ç ❝✞✡✙ø➩è❿ê❯æ✎ï✖ä❯æ☎ø➟↔✂ï➊û✐ëíì❛ä❯ê✤è✧ì❛ä❇✝ ø➀é❼✂➤è✧ç✙ï➔è✧ï➊î➻æ✙û✓å✠è✥ï➔ø➀î➻å✑✂➈ï❞ê➊ð❯ñ➔ì➄è➵ê✤ÿ✙ä✥æ✙ä✥ø➀ê✧ø➀é❼✂➈û✠✘✑➌➄é☎ï➊ÿ✙ä❿å➄û✙é☎ï❶è④ó➵ì➈ä✥ô➇ê ä✥ï✒➍✐ÿ✙ø➀ä✧ï➤î✒ÿ✥❿ç✢û➀ï✖ê✥ê✛î➻ï➊î➻ì➈ä❘✘➓è✧ç✎å➄é✺î➻ï➊î➻ì➈ä❘✘❩✝✶✡☎å❛ê✤ï❞ù➓î➻ï❶è✥ç✙ì✂ù✙ê➊ð ã✛ç✙ï❖ý♣ú➈ï➊ä❿å➄û➀û➵æ✎ï✖ä✤ëíì❛ä✧î➓å➄é✥❶ï✢ù✙ï➊æ◆ï➊é☎ù✙ê➻ì❛é❑î➓å➈é❩✘➷ë⑨å➎↔è✥ì➈ä❿ê✒ø➀é✦✝ ❶û➀ÿ☎ù✂ø➀é❼✂✌å✑✒❶ÿ✙ä❿å✑✄✘✑➌✠ä✥ÿ✙é✙é✙ø➀é❼✂♣è✧ø➀î➻ï✑➌❛å➈é☎ù❙î❭ï✖î➻ì➈ä❘✘✌ä✧ï★➍❛ÿ☎ø➑ä✥ï➊î➻ï➊é✐è❿ê➊ð ✕➉ê ❶ì➈î➻æ✙ÿ✙è✧ï➊ä➻è✥ï✒❿ç✙é☎ì➈û➀ì✑✂✑✘❤ø➑î➻æ✙ä✥ì✠ú➈ï❞ê✄➌✇û➀å➈ä❺✂❛ï➊ä❺✝❖✖å➄æ☎å➎❶ø➑è❅✘❺ä✥ï✒➊ì✑✂❄✝ é✙ø✠➽➊ï✖ä✥ê④✡✎ï★❶ì❛î❭ï❙ëíï✖å❛ê✤ø✠✡✙û➀ï➈ð✜✗❇å➄ä❘✂➈ï➊ä♣ä✧ï★❶ì➎✂➈é✙ø✠➽➊ï✖ä✥ê➉ø➑é❖è✧ÿ✙ä✥é❖ä✥ï✒➍✐ÿ✙ø➀ä✧ï û✓å➄ä❘✂➈ï➊ä➵è✥ä✥å➈ø➑é✙ø➀é❼✂➽ê✧ï❶è❿ê➊ð ✗❀ï✖ñ➉ï❶è❇✝✴➾✌ó✛å➈ê❨å➄æ☎æ✙ä✧ì❛æ✙ä✥ø➀å➄è✧ï➳è✧ì➻è✥ç✙ï➞årú✠å➄ø➀û➟✝ å❄✡☎û➑ï❣è✥ï✒❿ç✙é☎ì➈û➀ì✑✂✑✘♣ø➑é➵➾✒❀✻✺✗❀❼➌ ✓④ÿ✎ê④è❦å➈ê ✗❀ï✖ñ➔ï➊è❇✝ ✙✛ø✓ê❀å➄æ☎æ✙ä✧ì❛æ✙ä✥ø➀å➄è✧ï❣é✙ì✠ó✌ð ➏➠é↕➾✒❀✻✺✻❀✒å➞ä✥ï✒❶ì➎✂➈é✙ø✠➽➊ï✖ä❫å➈ê✎❶ì❛î➻æ✙û➑ï✄↔➽å➈ê ✗❀ï✖ñ➉ï❶è❇✝ ✙✌ó➵ì➈ÿ☎û➀ù➓ç✎årú➈ï➔ä✥ï❯✝ ➍✐ÿ✙ø➀ä✧ï❞ù✻ê✤ï✖ú➈ï✖ä✥å➈û➔ó✇ï✖ï➊ô✂ê✽❁❫è✥ä✥å➈ø➑é✙ø➀é❼✂✥➌➔å➄é☎ù✲î❭ì❛ä✧ï❺ù✙å✠è❿å è✧ç✎å➄é✻ó✛å➈ê årú✠å➄ø➀û➀å✑✡✙û➀ï✑➌➈å➈é☎ù➻ó➵å❛ê❯è✧ç✙ï✖ä✧ï➊ëíì➈ä✥ï❨é✙ì➄è✇ï➊ú➈ï✖é➵❶ì➈é✎ê✤ø✓ù✂ï➊ä✥ï✖ù✝ð❜✜☎ì➈ä➜➍✐ÿ✙ø➑è✧ï å✺û➀ì➈é❼✂✿è✧ø➀î➻ï✑➌ ✗✝ï❞ñ➔ï❶è❺✝❘➾➻ó✛å➈ê✌➊ì➈é☎ê✧ø➀ù✙ï➊ä✥ï✖ù❖è✧ç✙ï✶ê✤è✥å➄è✧ï➻ì➄ë➵è✧ç☎ï✶å➄ä✧è✖ð ã✛ç✙ï❙û➑ì✦➊å➈û❀û➀ï✖å➈ä✧é✙ø➀é❼✂➣➊û➀å❛ê✧ê✧ø➟➞✎ï➊ä★➌✙è✧ç☎ï✒ì➈æ✙è✧ø➀î➻å➈û❀î➓å➄ä❘✂➈ø➀é✆❶û✓å➈ê✥ê✤ø✙➞☎ï✖ä✒➌ å➄é✎ù è✧ç✙ï➓è❿å➄é❼✂❛ï➊é✐è❙ù✂ø✓ê✤è✥å➄é✥❶ï ➊û➀å❛ê✧ê✧ø➟➞✎ï➊ä✌ó➵ï➊ä✥ï✶ù✂ï➊ú❛ï➊û➀ì➈æ◆ï✖ù❺è✧ì❖ø➑î➚✝ æ✙ä✥ì✠ú➈ï❭ÿ✙æ◆ì➈é✏✗✝ï❞ñ➔ï❶è❺✝❘➾✁❖å➈é☎ù❺è✧ç✙ï✒✘❺ê✧ÿ✗✄➊ï➊ï✖ù✙ï✖ù➷å✠è✌è✥ç☎å✠è❞ð✶õ➔ì✠ó✔✝ ï➊ú❛ï➊ä★➌✙è✥ç✙ï✄✘✺ø➀é✫è✧ÿ✙ä✥é➲î➻ì➄è✥ø➑ú✠å➄è✧ï✖ù❖å➽ê✤ï❞å➄ä✴❿ç✿ëíì➈ä➉ø➀î➻æ✙ä✥ì✠ú➈ï✖ù✺é☎ï➊ÿ✙ä❿å➄û é✙ï➊è④ó✇ì❛ä✧ô✫å➄ä✴❿ç✙ø➑è✧ï★↔è✧ÿ☎ä✧ï❞ê➊ð➞ã✛ç✙ø✓ê✌ê✧ï✖å➄ä✴❿ç➲ó✛å➈ê④✂➈ÿ✙ø✓ù✂ï❞ù❖ø➀é❖æ☎å➈ä✤èt✡☛✘ ï✖ê✤è✧ø➀î➓å✠è✥ï✖ê➵ì➄ë❇è✧ç✙ï✌➊å➈æ☎å✑➊ø➩è❅✘➓ì➈ë❇ú✠å➄ä✥ø➀ì➈ÿ☎ê✇û➑ï❞å➄ä✥é✙ø➀é❼✂❙î➓å➎❿ç✙ø➑é☎ï✖ê✒➌➇ù✙ï❯✝ ä✥ø➑ú❛ï✖ù✿ëíä✥ì➈î▼î❭ï❞å➈ê✧ÿ✙ä✥ï➊î➻ï➊é✐è✥ê➔ì➄ë❣è✧ç☎ï➞è✧ä❿å➄ø➀é✙ø➀é❼✂✶å➈é☎ù✿è✧ï❞ê④è➳ï➊ä✥ä✧ì❛ä➉å➈ê å❺ëíÿ✙é✗❶è✧ø➀ì➈é➘ì➄ë♣è✧ç✙ï✫é➇ÿ✙î➑✡✎ï✖ä➓ì➄ë➉è✥ä✥å➈ø➑é☎ø➑é❼✂➷ï✄↔✂å➈î➻æ✙û➑ï❞ê➊ð ✎ï✫ù✂ø➀ê❇✝ ❶ì✠ú❛ï➊ä✥ï✖ù✢è✧ç✎å✠è➤î❭ì❛ä✧ï➑➊å➄æ✎å✑❶ø➑è❅✘✿ó➵å❛ê➔é✙ï✖ï✖ù✂ï❞ù☛ð➳ã✛ç☎ä✧ì❛ÿ❼✂➈ç➲å✢ê✤ï✖ä✧ø➀ï✖ê ì➄ë➤ï❯↔✂æ◆ï➊ä✥ø➑î➻ï✖é❛è❿ê➓ø➀é➘å➈ä❘❿ç☎ø➩è✥ï✒↔è✥ÿ✙ä✥ï✑➌➃➊ì➈î➑✡✙ø➑é☎ï✖ù ó❨ø➩è✥ç✻å➄é➘å➄é☎å➈û✙✘❩✝ ê✧ø➀ê✇ì➄ë☛è✥ç✙ï✞❿ç✎å➄ä❿å✑↔è✥ï➊ä✥ø➀ê✤è✧ø✁➊ê➏ì➄ë✝ä✥ï✒❶ì➎✂➈é✙ø➑è✧ø➀ì➈é➓ï✖ä✧ä✥ì➈ä❿ê✒➌✫✗✝ï❞ñ➔ï❶è❺✝❩❝❭å➄é☎ù ✗✝ï❞ñ➔ï➊è❇✝ ✙✒ó➵ï➊ä✥ï✌❶ä❿å✠ë➺è✥ï✖ù☛ð ✎ï➃➞☎é☎ù➤è✥ç☎å✠è❨✡◆ì✐ì✐ê④è✥ø➑é✗✂✛✂➈ø➀ú➈ï❞ê❇å➔ê✧ÿ❼✡☎ê✤è✥å➈é✐è✧ø✓å➄û✐ø➑î➻æ✙ä✥ì✠ú➈ï✖î❭ï✖é✐è❀ø➑é å✑✒❶ÿ✙ä❿å✑✄✘✑➌✐ó❨ø➩è✥ç✫å✒ä✥ï➊û✓å✠è✥ø➑ú❛ï➊û✠✘➓î❭ì✂ù✂ï❞ê④è❨æ◆ï➊é☎å➈û➩è❅✘✶ø➑é✿î➻ï➊î➻ì➈ä❘✘➽å➄é☎ù ❶ì❛î➻æ✙ÿ✂è✧ø➀é❼✂❺ï❯↔✂æ✎ï✖é☎ê✧ï➈ð↕✕➔û✓ê✧ì✗➌❣ù✂ø✓ê④è✥ì➈ä✧è✧ø➀ì➈é î➻ì✂ù✂ï➊û✓ê➑✖å➄é①✡◆ï✶ÿ☎ê✧ï✖ù è✧ì➻ø➀é✗❶ä✥ï✖å❛ê✤ï♣è✧ç☎ï✌ï❯➘✟ï✒↔è✥ø➑ú❛ï✌ê✤ø✠➽➊ï➤ì➄ë❦å➻ù✙å✠è❿å❭ê✧ï❶è❨ó❨ø➑è✧ç✙ì❛ÿ✂è➉å✑❶è✧ÿ☎å➈û➑û✠✘ ä✥ï✒➍✐ÿ✙ø➀ä✧ø➀é❼✂❙è✧ì➣❶ì❛û➑û➀ï✒❶è❨î❭ì❛ä✧ï✌ù✙å➄è✥å☎ð ã✛ç✙ï✲þ➇ÿ✙æ✙æ◆ì➈ä✧è✢✣❣ï✒❶è✧ì❛ä ö➲å✑❿ç✙ø➀é✙ï❍ç☎å❛ê➷ï❯↔❼❶ï✖û➑û➀ï➊é✐è❤å➎✄❶ÿ☎ä✥å➎❯✘✑➌ ó❨ç✙ø✁❿ç❤ø✓ê❭î❭ì✐ê④è➻ä✥ï➊î➓å➄ä✥ô✠å❄✡✙û➀ï✑➌❷✡◆ï✒✖å➄ÿ☎ê✧ï✶ÿ✙é☎û➑ø➀ô➈ï✢è✧ç✙ï✺ì➈è✧ç✙ï✖ä❙ç✙ø✠✂➈ç æ◆ï➊ä✧ëíì➈ä✥î➻å➈é✗❶ït❶û✓å➈ê✥ê✧ø➟➞☎ï✖ä✥ê✒➌➇ø➩è➉ù✙ì✐ï❞ê✛é✙ì➄è➔ø➀é✗➊û➑ÿ☎ù✙ï ➢✞➤✈➡▼✣ ➯❄➡▼✣✟ô✐é☎ì✠ó❨û➟✝ ï✖ù❼✂➈ï✶å✑✡✎ì❛ÿ✂è➞è✧ç☎ï➽æ✙ä✥ì✑✡✙û➀ï➊î✺ð✫➏➠é➷ë⑨å✑❶è✒➌❯è✧ç✙ø✓ê➉➊û➀å❛ê✧ê✧ø✙➞☎ï➊ä➞ó➵ì➈ÿ☎û➀ù➷ù✂ì ✓④ÿ☎ê✤è➔å➈ê➵ó✇ï✖û➑û☛ø➑ë❀è✥ç✙ï➤ø➑î➓å✑✂➈ï➳æ✙ø➟↔✂ï✖û➀ê➵ó➵ï➊ä✥ï♣æ◆ï➊ä✥î✒ÿ✂è✥ï✖ù✢ó❨ø➩è✥ç✿å➉➞❼↔✂ï✖ù î➓å➄æ✙æ☎ø➑é❼✂➤å➄é✎ù➞û➀ì❛ê✤è❇è✥ç✙ï➊ø➀ä❦æ✙ø✠❶è✧ì❛ä✧ø✓å➄û✂ê✤è✧ä✥ÿ✗↔è✥ÿ✙ä✧ï❛ð❇õ➉ì✠ó✇ï✖ú➈ï➊ä★➌rä✧ï❞å✑❿ç✦✝ ø➀é❼✂❙û➀ï➊ú❛ï➊û✓ê❫ì➈ë❇æ◆ï➊ä✧ëíì➈ä✥î➓å➄é✗➊ï④❶ì❛î➻æ☎å➄ä❿å❄✡✙û➀ï❨è✥ì✒è✥ç✙ï➉☞✇ì➈é➇ú➈ì❛û➑ÿ✂è✥ø➑ì❛é☎å➄û ñ➔ï✖ÿ✙ä❿å➄û✎ñ➉ï❶è④ó➵ì➈ä✥ô✂ê➜➊å➈é➓ì➈é✙û✠✘➚✡✎ï➤ù✂ì➈é☎ï➉å✠è✔❶ì➈é✎ê✤ø✓ù✂ï➊ä❿å❄✡☎û➑ï❨ï✄↔➇æ◆ï➊é✎ê✤ï ø➀é➓î❭ï✖î➻ì➈ä❘✘❙å➄é✎ù➚➊ì➈î➻æ✙ÿ✂è❿å✠è✥ø➑ì❛é☎å➄û✙ä✥ï✒➍✐ÿ✙ø➀ä✥ï➊î➻ï➊é✐è✥ê✖ð❇ã✛ç☎ï➔ä✧ï❞ù✂ÿ✗➊ï✖ù☛✝ ê✧ï❶è✿þ☛✣➳ö ä✧ï★➍✐ÿ✙ø➑ä✥ï➊î➻ï✖é❛è❿ê➽å➄ä✥ï✺ó❨ø➩è✥ç✙ø➀é✲å ë⑨å✑❶è✧ì➈ä✶ì➄ë➤è④ó✇ì ì➈ë➤è✧ç✙ï ☞✇ì➈é➇ú❛ì➈û➀ÿ✂è✧ø➀ì➈é☎å➈û➏ñ➔ï➊è④ó✇ì❛ä✧ô✂ê✒➌❇å➄é✎ù è✧ç✙ï✢ï➊ä✥ä✧ì❛ä✌ä✥å➄è✧ï➓ø✓ê➞ú➈ï✖ä❺✘➒❶û➀ì❛ê✧ï➈ð ➏➠î➻æ✙ä✥ì✠ú➈ï➊î➻ï✖é❛è❿ê➞ì➈ë➔è✧ç✙ì✐ê✤ï✿ä✥ï✖ê✧ÿ✙û➩è❿ê❙å➈ä✧ï✢ï❯↔✂æ◆ï✒↔è✥ï✖ù❢➌➵å➈ê✒è✧ç✙ï✢è✧ï★❿ç✦✝ é✙ø✁➍✐ÿ✙ï✌ø➀ê✛ä✥ï➊û✓å✠è✥ø➑ú❛ï➊û✠✘➓é✙ï➊ó✌ð ✎ç✙ï➊é✿æ✙û➀ï➊é✐è❅✘➓ì➄ë❦ù✙å✠è❿å✒ø✓ê✛årúrå➈ø➑û✓å❄✡☎û➑ï➎➌✐î➓å➈é❩✘➓î➻ï➊è✧ç✙ì✂ù✙ê✔➊å➈é✿å➄è❇✝ è✥å➈ø➑é❲ä✥ï✖ê✧æ✎ï★↔è✥å✑✡✙û➀ï å➎✄❶ÿ☎ä✥å➎❯✘➈ðPã✛ç✙ï➷é✙ï➊ÿ✙ä❿å➄û✙✝⑥é✙ï❶è✺î➻ï❶è✧ç☎ì➇ù☎ê✺ä✧ÿ✙é î✒ÿ✥❿ç✻ë⑨å➈ê✤è✧ï✖ä✫å➄é☎ù✲ä✧ï★➍❛ÿ☎ø➑ä✥ï❺î❙ÿ✗❿ç✲û➀ï✖ê✥ê✺ê✤æ✎å✑❶ï❺è✧ç✎å➄é✲î➻ï✖î❭ì❛ä❺✘❩✝ ✡☎å❛ê✤ï❞ù➽è✥ï✒❿ç✙é✙ø✁➍✐ÿ✙ï✖ê✖ð➏ã✛ç✙ï✌é✙ï✖ÿ✙ä❿å➄û☛é✙ï➊è✥ê✽❁☎å➈ù✂ú✠å➈é❛è❿å❄✂❛ï♣ó❨ø➀û➑û❀✡✎ï★❶ì➈î➻ï î➻ì➈ä✥ï➳ê✤è✧ä✥ø➀ô✐ø➀é❼✂❭å➈ê✇è✧ä❿å➄ø➀é✙ø➀é❼✂❭ù☎å✠è✥å✑✡☎å➈ê✧ï✖ê✎❶ì❛é✐è✧ø➀é✐ÿ☎ï➔è✧ì➻ø➀é✗❶ä✥ï✖å❛ê✤ï♣ø➑é ê✧ø✙➽✖ï➈ð ✔✛ ✪✹➨☎✍✪➢❄➡▼✣➢♦➨❄✴➳➝➢♦➨ ✪➉➸➑➯ ✣✁➩✄➳ ✁✬➳❯➩✱✣✁➩✹➺r➢♦➨☞❄❘➳ ☞✇ì➈é➇ú➈ì❛û➑ÿ✂è✥ø➑ì❛é☎å➄û✇é✙ï❶è④ó➵ì➈ä✥ô✂ê➞å➄ä✥ï✶æ✎å➄ä✧è✧ø✁❶ÿ✙û✓å➄ä✥û✙✘ ó➵ï➊û➀û➵ê✧ÿ✙ø➑è✧ï✖ù ëíì➈ä ä✥ï✒❶ì➎✂➈é✙ø✠➽➊ø➀é❼✂ ì➈ä➽ä✥ï✓④ï★↔è✥ø➑é❼✂ ê✧ç☎å➈æ✎ï❞ê➻ó❨ø➩è✥ç❍ó❨ø➀ù✂ï✖û✙✘❑ú✠å➄ä❘✘✐ø➀é❼✂ ê✧ø✙➽✖ï✑➌ æ◆ì❛ê✧ø➩è✥ø➑ì❛é✏➌➏å➄é☎ù ì➈ä✥ø➑ï✖é❛è❿å✠è✥ø➑ì❛é✏➌❯ê✧ÿ✗❿ç❤å❛ê✌è✥ç✙ï✶ì❛é✙ï✖ê✒è❅✘➇æ✙ø✠✖å➄û➀û✙✘ æ✙ä✥ì❄✝ ù✂ÿ✗➊ï✖ù ✡❩✘➓ç☎ï➊ÿ✙ä✥ø➀ê✤è✧ø✁➉ê✧ï✄✂❛î➻ï➊é✐è✧ï✖ä✥ê✇ø➑é✢ä✧ï❞å➄û✙✝♠ó➵ì➈ä✥û✓ù➓ê④è✥ä✧ø➀é❼✂✒ä✥ï✒➊ì✑✂❛é✙ø➟✝ è✧ø➀ì➈é✫ê❺✘✂ê④è✥ï➊î➓ê➊ð ➏➠é å➈é❺ï✄↔➇æ◆ï➊ä✥ø➀î❭ï✖é✐è➞û➑ø➀ô➈ï❭è✧ç☎ï❭ì❛é✙ï➓ù✂ï✖ê❘❶ä✥ø✠✡✎ï❞ù❺å✑✡✎ì✠ú❛ï✑➌✟è✧ç☎ï➓ø➑î➚✝ æ◆ì➈ä✧è✥å➄é✥❶ï ì➄ë❙é☎ì➈ø✓ê✤ï ä✧ï❞ê✤ø✓ê✤è✥å➄é✥❶ï❺å➈é☎ù❲ù✂ø➀ê✤è✧ì❛ä✤è✥ø➑ì❛é✲ø➀é➇úrå➈ä✧ø✓å➄é✥❶ï❖ø✓ê é✙ì➈è✿ì➎✡✐ú➇ø➀ì➈ÿ☎ê✖ð ã✛ç✙ï➷ê✤ø➑è✧ÿ✎å✠è✧ø➀ì➈é✾ø➑é❲î➻ì❛ê✤è✿ä✥ï✖å➈û➳å➈æ✙æ✙û➀ø✠✖å✠è✥ø➑ì❛é☎ê✶ø✓ê