Biometric Security QUTLINE ● Introduction Lecture 8 Face Recognition System Face Detection Location Traditional Uni-Modal Face normalization Face Recognition Feature Extraction Recognition Face Recognition Application Face Recognition Problems Introduction(1) Face 口 Current state k Face is the most common biometrics. Using the whole ce for automatic identification is a complex task ecause its appearance is constantly changing Introduction k One effective approach may employ rule-based logic and a neural network for the image classification process. The first face system is introduced in 1992. 口 Feature Set Size of to mouth, middle of cheek, size of mo radius vectors and feature points Introduction (2) Why Face Recognition? Introduction (3) F 日 FR analyzes facial 口Non- intrusive More nature, do not restrict user movement- socially more acceptable aIt requir This is how human beings are recognizing each other LEss expensive to setup considerable interest. Hardware is getting cheaper ole many legacy uses/database of face images to construct ne lH-image with or without consent of the 日 Fight terrorism °9ada90% asing need after the september 11 events/ Spot terrorists in to 50% only !) Require automated face detection system on suspect in sensitive Face Recognition Vendor Tests(FRVT) areas,e.g.airport, military facility ●httpl/www.frvt.org
1 Biometrics Research Centre (UGC/CRC) Lecture 6 - 1 Traditional Traditional Uni-Modal Face Recognition Face Recognition Biometric Security Biometric Security Lecture 8 Lecture 8 Biometrics Research Centre (UGC/CRC) Lecture 8 - 2 OUTLINE OUTLINE • Introduction • Face Recognition System • Face Detection & Location • Face Normalization • Feature Extraction & Recognition • Face Recognition Application • Face Recognition Problems Biometrics Research Centre (UGC/CRC) Lecture 8 - 3 Introduction Biometrics Research Centre (UGC/CRC) Lecture 8 - 4 Current State É Face is the most common biometrics. Using the whole face for automatic identification is a complex task because its appearance is constantly changing. É One effective approach may employ rule-based logic and a neural network for the image classification process. The first face system is introduced in 1992. Feature Set Facial geometry - Size of eye, distance from eye to mouth, middle of mouth to chin, side of eye to cheek, size of mouth, radius vectors and feature points Face Introduction (1) Introduction (1) Biometrics Research Centre (UGC/CRC) Lecture 8 - 5 Introduction (2) Introduction (2) Why Face Recognition? Why Face Recognition? Non-intrusive z More nature, do not restrict user movement - Socially more acceptable z This is how human beings are recognizing each other Less expensive to setup z Hardware is getting cheaper z Available many legacy uses/database of face images z Easy to construct new facial-image with or without consent of the people Fight terrorism z Increasing need after the September 11 events/ Spot terrorists in public z Require automated face detection system on suspect in sensitive areas, e.g. airport, military facility Biometrics Research Centre (UGC/CRC) Lecture 8 - 6 FR analyzes facial characteristics. It requires a digital (web) camera (of low quality is enough). This technique has attracted considerable interest. Uses distinctive features of the human face to verify or identify individuals Introduction (3) Introduction (3) Accuracy: the best performance had a 90% verification rate at a FAR of 1%. (However, when the face is captured at outdoor, for the same 1% FAR, the verification rate is dropped to 50% only!) Face Recognition Vendor Tests (FRVT) zhttp://www.frvt.org/
Introduction (4) Introduction (5) OFace Recognition is the identification or Basic Notions verification of a person solely from the facial OFacial recognition analyzes the characteristics of a persons face images input through a digital video camera 日 Source of face images OIt measures the overall facial structure, including distances ● Still Image between eyes, nose, mouth, and jaw edges uThese measurements are retained in a database and used o Color or black and white as a comparison when a user stands before the camera OThis biometric has been widely, and perhaps wildly, touted facial thermogram as a fantastic system for recognizing potential threats (whether terrorist, scam artist, or known criminal) but so far 3D techniques has been unproven in high-level usage Structured light Introduction(6) Introduction(7) How it Works Current Situation aUser faces the camera, standing about two feet from it. DThe system will locate the users face and perform matches against the claimed identity or the facial O After 911, US deployed FRS in airports to prevent terrorism, OIt is possible that the user may need to move and and used to capture suspect in public area ttempt the verification based on his facial position OBut, some US government departments have already O The system usually comes to a decision in less than 5 abandoned the use of FRS since it has found high"false positives rate"and"false negative rate DTo prevent a fake face or mold from faking out the s is easily to be abused by the oper system, many systems now require the user to smile d Revenue: from $34. 4m(2002)to $429 1m(2007) blink, or otherwise move in a way that is human before a Market Share: -10%of entire biometrics market(2007) FR System(1): Overview Get Reference Face Face Recognition System 2 品 Match Flowchart of face recognition system Face images or:Face image sequences
2 Biometrics Research Centre (UGC/CRC) Lecture 8 - 7 Introduction (4) Introduction (4) Face Recognition is the identification or verification of a person solely from the facial appearance Source of face images: zStill Image z Video z Color or Black and White z Non-visible wavelengths facial thermogram z 3D techniques - Stereo - Structured light Biometrics Research Centre (UGC/CRC) Lecture 8 - 8 Introduction (5) Introduction (5) Basic Notions Basic Notions Facial recognition analyzes the characteristics of a person's face images input through a digital video camera. It measures the overall facial structure, including distances between eyes, nose, mouth, and jaw edges. These measurements are retained in a database and used as a comparison when a user stands before the camera. This biometric has been widely, and perhaps wildly, touted as a fantastic system for recognizing potential threats (whether terrorist, scam artist, or known criminal) but so far has been unproven in high-level usage. Biometrics Research Centre (UGC/CRC) Lecture 8 - 9 Introduction (6) Introduction (6) How it Works How it Works User faces the camera, standing about two feet from it. The system will locate the user's face and perform matches against the claimed identity or the facial database. It is possible that the user may need to move and reattempt the verification based on his facial position. The system usually comes to a decision in less than 5 seconds. To prevent a fake face or mold from faking out the system, many systems now require the user to smile, blink, or otherwise move in a way that is human before verifying Biometrics Research Centre (UGC/CRC) Lecture 8 - 10 Facial Recognition System (FRS) usually be used in combination with general surveillance system for security control After 911, US deployed FRS in airports to prevent terrorism, and used to capture suspect in public area But, some US government departments have already abandoned the use of FRS since it has found high "false positives rate" and "false negative rate“ The FRS is easily to be abused by the operator Revenue: from $34.4m (2002) to $429.1m (2007) Market Share: ~10% of entire biometrics market (2007) Introduction (7) Introduction (7) Current Situation Biometrics Research Centre (UGC/CRC) Lecture 8 - 11 Face Recognition System Biometrics Research Centre (UGC/CRC) Lecture 8 - 12 1-to-1 Authentication Face detection and location Feature extraction & Face recognition Face images or image sequences Name Flowchart of face recognition system FR System (1): Overview
FR System(3): General Steps FR System(2): Two stages 曰 Face Detection In General Different Approaches Face detection and location O Locate face in a given image o Motion detecting and head tracking 1. Detect whether the input images or image sequences a Separate it from the scene D"Face Space distance 2. If they do include faces, figure out the position of the faces absEr Features extraction and Face recognition 1. Look for face features which distinguish individuals a Face Normalization -Adjustmen Rotation 2. Judge whether the people in image is the given person or in the database 口 Face Identification D Fe Tures extracton and Face recognition Face Detection Location(1): Two Kinds of methods CS-pase 2. Neural networks method (classification into face non- Face Detection Location ce classes) Knowle 3. Distribution ruler of gray-value-based (e.g. gray values of eyes 4. Contour ruler 5. Color, Movement Symmetry Information 7. Symmetry Information Face Detection Location(2): Face Detection Location(2) Subspace Method Subspace Method (cont featutes dspace of face hich shows common faces, which is a good representation of face 口 This can be done B品6 e-based normalized fi O Each face image is considered as a higher dimensional a Calculate the covariance matrix of the specimen images a Find out the eigenvalues(A t z,,A) and a Face images can be represented by fewer base vectors, the"eigenfaces
3 Biometrics Research Centre (UGC/CRC) Lecture 8 - 13 FR System (2): Two stages Face detection and location 1. Detect whether the input images or image sequences include faces 2. If they do include faces, figure out the position of the faces 3. Segment each face from background Features extraction and Face recognition 1. Look for face features which distinguish individuals 2. Judge whether the people in image is the given person or in the database Biometrics Research Centre (UGC/CRC) Lecture 8 - 14 FR System (3): FR System (3): General Steps General Steps Different Approaches Motion detecting and head tracking “Face Space” distance In General Locate face in a given image Separate it from the scene Face Detection Face Normalization - Adjustment – Expression – Rotation – Lighting – Scale – Head tilt – Eye location Face Identification z Features extraction and Face recognition Biometrics Research Centre (UGC/CRC) Lecture 8 - 15 Face Detection & Location Biometrics Research Centre (UGC/CRC) Lecture 8 - 16 Face Detection & Location (1): Face Detection & Location (1): Two Kinds of Methods Two Kinds of Methods Statistics-based method 1. Subspace method 2. Neural networks method (classification into face & nonface classes) Knowledge-based method 3. Distribution ruler of gray-value-based (e.g. gray values of eyes’ area) 4. Contour ruler 5. Color, Movement & Symmetry Information 6. Movement Information 7. Symmetry Information Biometrics Research Centre (UGC/CRC) Lecture 8 - 17 Face Detection & Location (2): Face Detection & Location (2): Subspace Method Find the subspace of face images which shows common features of faces, which is a good representation of face This can be done by using Karhunen-Loeve transformation, which is an image-gray-value-based method, and the image gray values have to be normalized first. Each face image is considered as a higher dimensional vector Calculate the covariance matrix of the specimen images Find out the eigenvalues (λ1,λ2,…,λd) and corresponding eigenvectors (ϕ1, ϕ 2,..., ϕ d) of the covariance matrix Face images can be represented by fewer base vectors, the “eigenfaces” Biometrics Research Centre (UGC/CRC) Lecture 8 - 18 Face Detection & Location (2): Face Detection & Location (2): Subspace Method (cont.) Subspace Method (cont.)
Face Detection Location(2): Face Detection Location (2) Subspace Method(cont Subspace Method (cont 「回司 「冂 The number of principal components Face Detection Location ( 3): Face Detection Location(4) Neural Network method Distribution Ruler of Gray-Value Based a Two-class classification problem face class and non- face class O Detect faces using the nearly universal distribution rulers O Need to train the neural network with face and non-face of gray values of faces under normal light condition 口 Mosaic method u Problem: many kinds of non-face images which are not O Divides the image areas into image blocks of 4x4 collected O IF face area-> satisfy some distribution rulers of gray a Slow-lots of specimens or input node values d Further divides these areas into image blocks of Face Detection Location(5): Face Detection Location(6): Contour Ruler Color Information O Contour is an important feature of fat O Face contour is modeled as ellipse O The skin colors are usually different to background color O Two straight lines(cheek) and two arcs of ellipse u the face colors in the same race is similar O Use snake techniques to get the face contour O the pixels in face areas are clustered in a small area
4 Biometrics Research Centre (UGC/CRC) Lecture 8 - 19 Face Detection & Location (2): Face Detection & Location (2): Subspace Method (cont.) Subspace Method (cont.) Biometrics Research Centre (UGC/CRC) Lecture 8 - 20 Detection rate The number of principal components Face Detection & Location (2): Face Detection & Location (2): Subspace Method (cont.) Subspace Method (cont.) Biometrics Research Centre (UGC/CRC) Lecture 8 - 21 Face Detection & Location (3): Face Detection & Location (3): Neural Network Method Two-class classification problem: face class and nonface class Need to train the neural network with face and non-face image specimens Problem: many kinds of non-face images which are not collected Slow - lots of specimens or input nodes Biometrics Research Centre (UGC/CRC) Lecture 8 - 22 Face Detection & Location (4): Face Detection & Location (4): Distribution Ruler of Gray-ValueBased Detect faces using the nearly universal distribution rulers of gray values of faces under normal light condition Mosaic method Divides the image areas into image blocks of 4x4 IF face area -> satisfy some distribution rulers of gray values Further divides these areas into image blocks of 8x8 and repeat the process. Biometrics Research Centre (UGC/CRC) Lecture 8 - 23 Detect and extract face contour with edge detection algorithms Contour is an important feature of face Face contour is modeled as ellipse Two straight lines (cheek) and two arcs of ellipse Use snake techniques to get the face contour Face Detection & Location (5): Face Detection & Location (5): Contour Ruler Contour Ruler Biometrics Research Centre (UGC/CRC) Lecture 8 - 24 Face Detection & Location (6): Face Detection & Location (6): Color Information Detect faces with the use of color information of face, as usually color of faces are different from that of background color in an image The skin colors are usually different to background color the face colors in the same race is similar the pixels in face areas are clustered in a small area
Face Detection Location(7) Face Detection Location(8): Movement Information Symmetry Information d Sequence of images showing people moving relative to the background are used as input of the system(e. g a Face is symmetrical in general video surveillance system) a Symmetrical objects in a face can be us O Movement information can be used to segment the face from the background 国 小小小 taya Face Normalization 1. Image is rotated to align the eyes(eye coordinates must 2. The image is scaled to make the distance between the eyes constant. The image is also cropped to a smaller Face Normalization size that is nearly just the fac 3. A mask is applied that zeros out pixels not in an oval that contains the typical face. The oval is generated of gray values for the non-masked pixels mean zero and standard deviation one Feature Extraction& Recognition (1 O Principal Component Analysis(PCA)ie Eigenfaces O Local Feature Analysis O Linear Discriminant Analysis Feature Extraction Recognition a Probabilistic Principal Component Analysis(PPCA) u Geometry-feature- based method(e.g position between eyes, nose, mouth& chin) 口 Deformation models D Automatic Face Processing(AFP) O Neural networks method 5
5 Biometrics Research Centre (UGC/CRC) Lecture 8 - 25 Face Detection & Location (7): Face Detection & Location (7): Movement Information Sequence of images showing people moving relative to the background are used as input of the system (e.g. video surveillance system) Movement information can be used to segment the face from the background Biometrics Research Centre (UGC/CRC) Lecture 8 - 26 Face Detection & Location (8): Face Detection & Location (8): Symmetry Information Face is symmetrical in general Symmetrical objects in a face can be used Biometrics Research Centre (UGC/CRC) Lecture 8 - 27 Face Normalization Biometrics Research Centre (UGC/CRC) Lecture 8 - 28 Face Normalization 1. Image is rotated to align the eyes (eye coordinates must be known). 2. The image is scaled to make the distance between the eyes constant. The image is also cropped to a smaller size that is nearly just the face. 3. A mask is applied that zeros out pixels not in an oval that contains the typical face. The oval is generated analytically. 4. Histogram equalization is used to smooth the distribution of gray values for the non-masked pixels. 5. The image is normalized so the non-masked pixels have mean zero and standard deviation one. Biometrics Research Centre (UGC/CRC) Lecture 8 - 29 Feature Extraction & Recognition Biometrics Research Centre (UGC/CRC) Lecture 8 - 30 Feature Extraction & Recognition(1) Feature Extraction & Recognition(1) Principal Component Analysis (PCA) i.e. Eigenfaces Local Feature Analysis Linear Discriminant Analysis Probabilistic Principal Component Analysis (PPCA) Geometry-feature-based method (e.g. position between eyes, nose, mouth & chin) Deformation models Automatic Face Processing (AFP) Neural networks method