Unlock with Your Heart:Heartbeat-based Authentication on Commercial Mobile Phones LEI WANG,State Key Laboratory for Novel Software Technology,Nanjing University,China KANG HUANG,State Key Laboratory for Novel Software Technology,Nanjing University,China KE SUN,State Key Laboratory for Novel Software Technology,Nanjing University,China WEI WANG,State Key Laboratory for Novel Software Technology,Nanjing University,China CHEN TIAN,State Key Laboratory for Novel Software Technology,Nanjing University,China LEI XIE,State Key Laboratory for Novel Software Technology,Nanjing University,China QING GU,State Key Laboratory for Novel Software Technology,Nanjing University,China In this paper,we propose to use the vibration of the chest in response to the heartbeat as a biometric feature to authenticate the user on mobile devices.We use the built-in accelerometer to capture the heartbeat signals on commercial mobile phones. The user only needs to press the phone on his/her chest,and the system can identify the user within a few heartbeats.To reliably extract heartbeat features,we design a two-step alignment scheme that can handle the natural variability in human heart rates.We further use an adaptive template selection scheme to authenticate the user under different body postures and body states.Based on heartbeat signals collected on twenty users,the experimental results show that our method can achieve an authentication accuracy of 96.49%and the heartbeat features are stable over a period of three months. CCS Concepts:.Security and privacy-Biometrics; Additional Key Words and Phrases:Biometrics-based Authentication,Mobile System ACM Reference Format: Lei Wang,Kang Huang.Ke Sun,Wei Wang.Chen Tian,Lei Xie,and Qing Gu.2018.Unlock with Your Heart:Heartbeat- based Authentication on Commercial Mobile Phones.Proc.ACM Interact.Mob.Wearable Ubiquitous Technol.2,3,Article 140 (September 2018),22 pages.https://doiorg/10.1145/3264950 1 INTRODUCTION Biometric features,including fingerprints and faces,have been used as metrics for user authentication on commercial mobile devices.Biometrics-based user authentication systems identify the user based on "who you are",instead of "what you know"(passwords)or "what you have"(tokens)[47].Since users often forget to carry their physical tokens and passwords are susceptible to leakage [5,54],biometrics-based authentication systems Authors'addresses:Lei Wang,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing,Jiangsu,China,wangl@smail nju.edu.cn:Kang Huang.State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.Jiangsu,China,hkwany520@ gmail.com;Ke Sun,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.Jiangsu,China,kesun@smail.nju.edu.cn Wei Wang.State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.Jiangsu,China,ww@njuedu.cn;Chen Tian, State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.Jiangsu,China,tianchen@nju.edu.cn;Lei Xie,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing,Jiangsu,China,Ixie@nju.edu.cn;Qing Gu,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.Jiangsu,China,guq@nju.edu.cn Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for components of this work owned by others than ACM must be honored.Abstracting with credit is permitted.To copy otherwise,or republish,to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.Request permissions from permissions@acm.org. 2018 Association for Computing Machinery. 2474-9567/2018/9-ART140$15.00 https:/doi.org/10.1145/3264950 Proc.ACM Interact.Mob.Wearable Ubiquitous Technol.,Vol.2,No.3,Article 140.Publication date:September 2018. 140
140 Unlock with Your Heart: Heartbeat-based Authentication on Commercial Mobile Phones LEI WANG, State Key Laboratory for Novel Software Technology, Nanjing University, China KANG HUANG, State Key Laboratory for Novel Software Technology, Nanjing University, China KE SUN, State Key Laboratory for Novel Software Technology, Nanjing University, China WEI WANG, State Key Laboratory for Novel Software Technology, Nanjing University, China CHEN TIAN, State Key Laboratory for Novel Software Technology, Nanjing University, China LEI XIE, State Key Laboratory for Novel Software Technology, Nanjing University, China QING GU, State Key Laboratory for Novel Software Technology, Nanjing University, China In this paper, we propose to use the vibration of the chest in response to the heartbeat as a biometric feature to authenticate the user on mobile devices. We use the built-in accelerometer to capture the heartbeat signals on commercial mobile phones. The user only needs to press the phone on his/her chest, and the system can identify the user within a few heartbeats. To reliably extract heartbeat features, we design a two-step alignment scheme that can handle the natural variability in human heart rates. We further use an adaptive template selection scheme to authenticate the user under different body postures and body states. Based on heartbeat signals collected on twenty users, the experimental results show that our method can achieve an authentication accuracy of 96.49% and the heartbeat features are stable over a period of three months. CCS Concepts: • Security and privacy → Biometrics; Additional Key Words and Phrases: Biometrics-based Authentication, Mobile System ACM Reference Format: Lei Wang, Kang Huang, Ke Sun, Wei Wang, Chen Tian, Lei Xie, and Qing Gu. 2018. Unlock with Your Heart: Heartbeatbased Authentication on Commercial Mobile Phones. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 140 (September 2018), 22 pages. https://doi.org/10.1145/3264950 1 INTRODUCTION Biometric features, including fingerprints and faces, have been used as metrics for user authentication on commercial mobile devices. Biometrics-based user authentication systems identify the user based on “who you are”, instead of “what you know” (passwords) or “what you have” (tokens) [47]. Since users often forget to carry their physical tokens and passwords are susceptible to leakage [5, 54], biometrics-based authentication systems Authors’ addresses: Lei Wang, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, wangl@smail. nju.edu.cn; Kang Huang, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, hkwany520@ gmail.com; Ke Sun, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, kesun@smail.nju.edu.cn; Wei Wang, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, ww@nju.edu.cn; Chen Tian, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, tianchen@nju.edu.cn; Lei Xie, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, lxie@nju.edu.cn; Qing Gu, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China, guq@nju.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2018 Association for Computing Machinery. 2474-9567/2018/9-ART140 $15.00 https://doi.org/10.1145/3264950 Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, No. 3, Article 140. Publication date: September 2018
140:2·L.Wang et al. provide a convenient and secure way to unlock private mobile devices,i.e.,devices that often have a singular user, including smartphones and smartwatches.However,most biometric features,such as fingerprints,faces,and voices,are vulnerable to spoofing and replaying attacks [4,13,17,43].For example,with the widely available 3D-reconstruction and 3D-printing technologies,it is easy to bypass face recognition systems with 3D masks [17]. Therefore,we need to find a new biometric feature that is easily accessible on mobile devices and yet difficult to be reproduced by attackers. The vibration of the chest in response to the heartbeat,which is called seismocardiogram(SCG)[26],can be used as a biometric feature for user authentication.Firstly,the heartbeat pattern depends on the biological features and geometric structure of the heart,which is unique for each person.Secondly,SCG provides strong protection against spoofing attacks.To access the SCG,the adversaries have to attach a device to the chest of the user,which is considerably harder than taking photos of the user's face or recording the voices of the user. While there are contactless radar systems that can measure the heartbeat from a distance [39,66],there is still no evidence that these signals are reliable enough for reconstructing the details of heartbeat dynamics.Furthermore, compared to replaying the heartbeat sound,it is harder for the adversaries to reproduce the small vibrations caused by heartbeats.Thirdly,the heartbeat pattern is closely linked to the"liveness"and the emotion of the user.By detecting the abnormality of the heartbeat pattern,the system can potentially reject the user when he/she is under threat.While SCG can serve as the biometric feature for user authentication,traditional SCG measurement schemes require specially designed devices and need to attach the device via chest bands [26].This makes traditional SCG approaches not applicable to authentication on commercial mobile devices. In this paper,we propose to use the built-in accelerometer to capture the heartbeat vibration and perform user authentication on commercial mobile devices.To unlock the device,the user only needs to press the device on his/her chest to collect heartbeat signals,and the system can identify the user within a few heartbeats,as shown in Figure 1(a).Our design is based on the observation that the detailed vibration patterns within one heartbeat cycle can serve as a unique identity for a person,and such patterns can be reliably captured by the accelerometers of commercial mobile phones.Using SCG collected from twenty volunteers,we find that different people have different heartbeat patterns even if their heart rates are similar.Moreover,these patterns are robust when the user slightly changes the position where the heartbeat is measured or the angle of the mobile phone.Therefore,this authentication scheme can be easily used in daily life.Heartbeat patterns can serve as the main authentication scheme for mobile devices,or as a supplementary authentication scheme in multi-factor authentication solutions For example,a two-factor authentication system may ask the user to press the phone on his/her chest and put one finger on the fingerprint scanner at the same time.In this way,the system checks both the fingerprint and the heartbeat pattern to improve the security level of the authentication process. When building heartbeat-based authentication system,we need to address the following technical challenges. First,human heartbeat patterns contain intrinsic Heart Rate Variability(HRV)[42].Even for a healthy person, the standard deviation of the time between two normal heartbeats(SDNN)could be larger than one hundred milliseconds(one-tenth of the heartbeat cycle).This is because heartbeats are susceptible to variations in the inputs from the parasympathetic nervous system(PSNS)caused by multiple human factors,e.g.,respiration.The variability in heartbeat duration leads to challenges in dividing and aligning the heartbeat signals.To address this challenge,we propose a two-step segmentation and alignment scheme that can precisely align the key timing features of the heartbeat even if the durations of the heartbeats are slightly different.Second,extracting reliable features from heartbeat signals with different durations is challenging.On one hand,the heartbeat signals from different persons contain similar peak-and-valley sequences with slightly different amplitudes and time intervals. On the other hand,directly using the raw heartbeat signal and matching in the time domain often wrongly reject the authorized user due to the variation in the duration of a heartbeat cycle.To address this challenge,we propose to use wavelet transform to extract features from heartbeat signals.Our experimental results show that features extracted by wavelet transform outperform both the Dynamic Time Warping(DTW)and time domain matching Proc.ACM Interact.Mob.Wearable Ubiquitous Technol..Vol 2.No.3.Article 140.Publication date:September 2018
140:2 • L. Wang et al. provide a convenient and secure way to unlock private mobile devices, i.e., devices that often have a singular user, including smartphones and smartwatches. However, most biometric features, such as fingerprints, faces, and voices, are vulnerable to spoofing and replaying attacks [4, 13, 17, 43]. For example, with the widely available 3D-reconstruction and 3D-printing technologies, it is easy to bypass face recognition systems with 3D masks [17]. Therefore, we need to find a new biometric feature that is easily accessible on mobile devices and yet difficult to be reproduced by attackers. The vibration of the chest in response to the heartbeat, which is called seismocardiogram (SCG) [26], can be used as a biometric feature for user authentication. Firstly, the heartbeat pattern depends on the biological features and geometric structure of the heart, which is unique for each person. Secondly, SCG provides strong protection against spoofing attacks. To access the SCG, the adversaries have to attach a device to the chest of the user, which is considerably harder than taking photos of the user’s face or recording the voices of the user. While there are contactless radar systems that can measure the heartbeat from a distance [39, 66], there is still no evidence that these signals are reliable enough for reconstructing the details of heartbeat dynamics. Furthermore, compared to replaying the heartbeat sound, it is harder for the adversaries to reproduce the small vibrations caused by heartbeats. Thirdly, the heartbeat pattern is closely linked to the “liveness” and the emotion of the user. By detecting the abnormality of the heartbeat pattern, the system can potentially reject the user when he/she is under threat. While SCG can serve as the biometric feature for user authentication, traditional SCG measurement schemes require specially designed devices and need to attach the device via chest bands [26]. This makes traditional SCG approaches not applicable to authentication on commercial mobile devices. In this paper, we propose to use the built-in accelerometer to capture the heartbeat vibration and perform user authentication on commercial mobile devices. To unlock the device, the user only needs to press the device on his/her chest to collect heartbeat signals, and the system can identify the user within a few heartbeats, as shown in Figure 1(a). Our design is based on the observation that the detailed vibration patterns within one heartbeat cycle can serve as a unique identity for a person, and such patterns can be reliably captured by the accelerometers of commercial mobile phones. Using SCG collected from twenty volunteers, we find that different people have different heartbeat patterns even if their heart rates are similar. Moreover, these patterns are robust when the user slightly changes the position where the heartbeat is measured or the angle of the mobile phone. Therefore, this authentication scheme can be easily used in daily life. Heartbeat patterns can serve as the main authentication scheme for mobile devices, or as a supplementary authentication scheme in multi-factor authentication solutions. For example, a two-factor authentication system may ask the user to press the phone on his/her chest and put one finger on the fingerprint scanner at the same time. In this way, the system checks both the fingerprint and the heartbeat pattern to improve the security level of the authentication process. When building heartbeat-based authentication system, we need to address the following technical challenges. First, human heartbeat patterns contain intrinsic Heart Rate Variability (HRV) [42]. Even for a healthy person, the standard deviation of the time between two normal heartbeats (SDNN) could be larger than one hundred milliseconds (one-tenth of the heartbeat cycle). This is because heartbeats are susceptible to variations in the inputs from the parasympathetic nervous system (PSNS) caused by multiple human factors, e.g., respiration. The variability in heartbeat duration leads to challenges in dividing and aligning the heartbeat signals. To address this challenge, we propose a two-step segmentation and alignment scheme that can precisely align the key timing features of the heartbeat even if the durations of the heartbeats are slightly different. Second, extracting reliable features from heartbeat signals with different durations is challenging. On one hand, the heartbeat signals from different persons contain similar peak-and-valley sequences with slightly different amplitudes and time intervals. On the other hand, directly using the raw heartbeat signal and matching in the time domain often wrongly reject the authorized user due to the variation in the duration of a heartbeat cycle. To address this challenge, we propose to use wavelet transform to extract features from heartbeat signals. Our experimental results show that features extracted by wavelet transform outperform both the Dynamic Time Warping (DTW) and time domain matching Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, No. 3, Article 140. Publication date: September 2018
Unlock with Your Heart:Heartbeat-based Authentication on Commercial Mobile Phones.140:3 Unlock With Your Heart Yes! (a)Capturing the SCG signals using the mo- (b)Interface of our Android APp bile phone Fig.1.Heartbeat-based authentication scenario schemes.Third,human heartbeat patterns change under various conditions.For example,the heartbeat patterns captured after exercising are different to the pattern when the same user is in the resting state,even if these patterns are normalized in the time domain so that their heartbeat cycles are stretched to the same duration.To address this challenge,we propose a heartbeat pattern selection scheme that chooses the best heartbeat patterns for authentication based on the scenario information,which indicates the status of the user(e.g.,whether the user is in the exercising or the resting state)and the body posture(e.g.whether the user is standing/sitting.lying down or leaning on the sofa). We have implemented our heartbeat-based authentication scheme on the Android platform.We collected more than 110,000 heartbeat samples from 35 volunteers.The implemented system achieves an Equal Error Rate(EER) of 3.51%for user authentication when using just five heartbeat cycles.Our experimental results also show that the system is robust against different ways of putting the mobile phone and different body postures. In summary,we have made the following contributions in this paper: To our best knowledge,we are the first to perform heartbeat-based user authentication using the built-in accelerometer on commercial mobile phones. We propose a set of novel signal processing schemes designed for heartbeat-based user authentication, including template-based heartbeat alignment,wavelet-based feature extraction,and dynamic heartbeat pattern selection We implement our authentication system on commercial smartphones and verify our design using heartbeat signals collected from twenty users. 2 RELATED WORK Existing work on heartbeat measurement and authentication can be divided into three categories:special equipment based heartbeat measurement,commodity device based heartbeat measurement,and biometrics-based authentication. Special Equipment based Heartbeat Measurement:Existing systems use specialized equipment to collect heartbeat signals,including electrocardiography(ECG),ballistocardiogram(BCG),seismocardiogram(SCG)and RF cardiac signals.ECG signal has been used for heart rate estimation [46,58]and disease diagnosis [33,37] for a long time.While ECG provides accurate heartbeat measurements,ECG systems have to attach electrodes Proc.ACM Interact.Mob.Wearable Ubiquitous Technol.,Vol.2.No.3,Article 140.Publication date:September 2018
Unlock with Your Heart: Heartbeat-based Authentication on Commercial Mobile Phones • 140:3 Z - a xis X-axis Y-axis (a) Capturing the SCG signals using the mobile phone (b) Interface of our Android APP Fig. 1. Heartbeat-based authentication scenario schemes. Third, human heartbeat patterns change under various conditions. For example, the heartbeat patterns captured after exercising are different to the pattern when the same user is in the resting state, even if these patterns are normalized in the time domain so that their heartbeat cycles are stretched to the same duration. To address this challenge, we propose a heartbeat pattern selection scheme that chooses the best heartbeat patterns for authentication based on the scenario information, which indicates the status of the user (e.g., whether the user is in the exercising or the resting state) and the body posture (e.g., whether the user is standing/sitting, lying down or leaning on the sofa). We have implemented our heartbeat-based authentication scheme on the Android platform. We collected more than 110, 000 heartbeat samples from 35 volunteers. The implemented system achieves an Equal Error Rate (EER) of 3.51% for user authentication when using just five heartbeat cycles. Our experimental results also show that the system is robust against different ways of putting the mobile phone and different body postures. In summary, we have made the following contributions in this paper: • To our best knowledge, we are the first to perform heartbeat-based user authentication using the built-in accelerometer on commercial mobile phones. • We propose a set of novel signal processing schemes designed for heartbeat-based user authentication, including template-based heartbeat alignment, wavelet-based feature extraction, and dynamic heartbeat pattern selection. • We implement our authentication system on commercial smartphones and verify our design using heartbeat signals collected from twenty users. 2 RELATED WORK Existing work on heartbeat measurement and authentication can be divided into three categories: special equipment based heartbeat measurement, commodity device based heartbeat measurement, and biometrics-based authentication. Special Equipment based Heartbeat Measurement: Existing systems use specialized equipment to collect heartbeat signals, including electrocardiography (ECG), ballistocardiogram (BCG), seismocardiogram (SCG) and RF cardiac signals. ECG signal has been used for heart rate estimation [46, 58] and disease diagnosis [33, 37] for a long time. While ECG provides accurate heartbeat measurements, ECG systems have to attach electrodes Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, No. 3, Article 140. Publication date: September 2018
140:4·L.Wang et al. to the skin of the user,which is inconvenient for daily use.BCG measures the micro recoil movements of the body caused by the blood traveling along the vascular tree [8,20,41].Such micro-movements can be captured by highly sensitive geophone mounted on the bed that the user is sleeping on [29,30].SCG measures the local vibration of the chest caused by the heartbeat and it has been used for heart rate estimation [9,36,52,56].SCG can also be used for assessments of the time interval of different mechanical events occurring during the systolic and diastolic phase [14-16].However,most SCG systems require specifically designed chest belt to attach the sensor to the chest of the user [14].Recently,RF-based systems provide a non-intrusive and contactless way for heartbeat measurement.Adib et al.[1]use Frequency Modulated Continuous Wave(FMCW)to monitor the heart rates with a median accuracy of 99%.Yang et al.[64]propose a system that uses 60GHz millimeter wave (mmWave)for heartbeat monitoring.However,most of these systems use expensive special hardware and only provide coarse heart rate estimations that are not applicable for user authentication. Commodity Device based Heartbeat Measurement:Low-cost commodity devices,including Wi-Fi devices and smartphones,can also be used for heartbeat monitoring.With the Channel State Information(CSI)captured from commercial WI-Fi devices,it is possible to estimate the heart rate by either the amplitude of CSI [40]or the phase of CSI [63].Furthermore,Zhao et al.[66]show that CSI provides enough details in heartbeat cycles so that it can be used for recognizing the emotional state of the user.Oian et al.[51]leverage inaudible acoustic signals emitted by commodity mobile phones to monitor the heart rates.However,these Wi-Fi and acoustic signal based measurements are sensitive to environmental changes,including the angle and the distance of the device to the target user. There are systems that use the built-in accelerometers or gyroscopes in commodity mobile phone to capture the SCG signals [35,44,59].Most of these systems only provide coarse measurements,such as heart rates or Heart Rate Variability(HRV)[35,44].In a recent system deployed on smartphones,Wang et al.[59]detect the detailed fiducial point of the SCG signals with the aid of photoplethysmogram(PPG)to measure the blood pressure of the user.In comparison,our system solely relies on the SCG signals captured by the built-in accelerometer to extract detailed heart movement pattern without help from other sensors. Biometrics based Authentication:Biometrics-based authentication uses features,such as fingerprint [53,55], face [18,21],voice [7,19,31,49],breath [11],iris [57],and heartbeat [12,24],to authenticate the user.Among these features,the heartbeat pattern is a relatively new and hard-to-spoof biometric feature for authentication. Choudhary and Manikandan [12]propose a heartbeat extraction framework for authentication based on ECG signals.BreathLive [24]uses a heartbeat sound based authentication system,which relies on the inherent correlation between chest motion and sounds caused by deep respiration to protect the user from replay attacks Auth'n'Scan [23]uses physiological information,including heart rates,HRV,and respiration rates,derived from PPG to authenticate the user.Cardiac Scan [39]uses a remote,high-resolution heartbeat monitoring system based on DC-coupled continuous-wave radar to achieve continuous user authentication.However,most of these heartbeat-based authentication systems use specially designed equipment and cannot be easily applied to current commodity mobile devices. 3 SYSTEM OVERVIEW 3.1 Authentication Model and System Components Our heartbeat-based authentication system aims at identifying the owner of the mobile device.We assume that the mobile device only has one owner.However,our system can be extended to identify multiple users on the same device by updating our training and recognition process. The first step of our system is the training process as shown in Figure 2.During the training process,the user needs to press the mobile device on his/her chest,more specifically,put the bottom of the phone perpendicularly on the lower portion of the sternum,to collect training heartbeat samples,as shown in Figure 1(a).The training Proc.ACM Interact.Mob.Wearable Ubiquitous Technol.,Vol.2,No.3,Article 140.Publication date:September 2018
140:4 • L. Wang et al. to the skin of the user, which is inconvenient for daily use. BCG measures the micro recoil movements of the body caused by the blood traveling along the vascular tree [8, 20, 41]. Such micro-movements can be captured by highly sensitive geophone mounted on the bed that the user is sleeping on [29, 30]. SCG measures the local vibration of the chest caused by the heartbeat and it has been used for heart rate estimation [9, 36, 52, 56]. SCG can also be used for assessments of the time interval of different mechanical events occurring during the systolic and diastolic phase [14–16]. However, most SCG systems require specifically designed chest belt to attach the sensor to the chest of the user [14]. Recently, RF-based systems provide a non-intrusive and contactless way for heartbeat measurement. Adib et al. [1] use Frequency Modulated Continuous Wave (FMCW) to monitor the heart rates with a median accuracy of 99%. Yang et al. [64] propose a system that uses 60GHz millimeter wave (mmWave) for heartbeat monitoring. However, most of these systems use expensive special hardware and only provide coarse heart rate estimations that are not applicable for user authentication. Commodity Device based Heartbeat Measurement: Low-cost commodity devices, including Wi-Fi devices and smartphones, can also be used for heartbeat monitoring. With the Channel State Information (CSI) captured from commercial WI-Fi devices, it is possible to estimate the heart rate by either the amplitude of CSI [40] or the phase of CSI [63]. Furthermore, Zhao et al. [66] show that CSI provides enough details in heartbeat cycles so that it can be used for recognizing the emotional state of the user. Qian et al. [51] leverage inaudible acoustic signals emitted by commodity mobile phones to monitor the heart rates. However, these Wi-Fi and acoustic signal based measurements are sensitive to environmental changes, including the angle and the distance of the device to the target user. There are systems that use the built-in accelerometers or gyroscopes in commodity mobile phone to capture the SCG signals [35, 44, 59]. Most of these systems only provide coarse measurements, such as heart rates or Heart Rate Variability (HRV) [35, 44]. In a recent system deployed on smartphones, Wang et al. [59] detect the detailed fiducial point of the SCG signals with the aid of photoplethysmogram (PPG) to measure the blood pressure of the user. In comparison, our system solely relies on the SCG signals captured by the built-in accelerometer to extract detailed heart movement pattern without help from other sensors. Biometrics based Authentication: Biometrics-based authentication uses features, such as fingerprint [53, 55], face [18, 21], voice [7, 19, 31, 49], breath [11], iris [57], and heartbeat [12, 24], to authenticate the user. Among these features, the heartbeat pattern is a relatively new and hard-to-spoof biometric feature for authentication. Choudhary and Manikandan [12] propose a heartbeat extraction framework for authentication based on ECG signals. BreathLive [24] uses a heartbeat sound based authentication system, which relies on the inherent correlation between chest motion and sounds caused by deep respiration to protect the user from replay attacks. Auth’n’Scan [23] uses physiological information, including heart rates, HRV, and respiration rates, derived from PPG to authenticate the user. Cardiac Scan [39] uses a remote, high-resolution heartbeat monitoring system based on DC-coupled continuous-wave radar to achieve continuous user authentication. However, most of these heartbeat-based authentication systems use specially designed equipment and cannot be easily applied to current commodity mobile devices. 3 SYSTEM OVERVIEW 3.1 Authentication Model and System Components Our heartbeat-based authentication system aims at identifying the owner of the mobile device. We assume that the mobile device only has one owner. However, our system can be extended to identify multiple users on the same device by updating our training and recognition process. The first step of our system is the training process as shown in Figure 2. During the training process, the user needs to press the mobile device on his/her chest, more specifically, put the bottom of the phone perpendicularly on the lower portion of the sternum, to collect training heartbeat samples, as shown in Figure 1(a). The training Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, No. 3, Article 140. Publication date: September 2018
Unlock with Your Heart:Heartbeat-based Authentication on Commercial Mobile Phones.140:5 AuthenticatingHeartbeat Feature User Extract Scenario Infomation Segmentation Alignment Extraction Authentication Heartbeat Heart Rate Fine Template SVM Model Collection Estimation Generate Feature SVM Model Fine Alianmen Extractior Training Template Generation Fig.2.Authentication System Components process normally takes less than two minutes(for collecting 60 heartbeats).Users may be instructed to change the position or the angle of the device during the training process to introduce more variations in the training samples When collecting the training samples,our system records the built-in accelerometer readings at a sampling rate of 100~250 Hz(depending on the hardware support of the device).With the readings of the accelerometer,we first extract the heart rates and the body posture of the user.With this information,the collected training samples can be classified into one of the predefined scenarios,e.g,the heart rates are in the range of 50~80 Beats per Minute(BPM)and the user is sitting on a chair.The training samples are then used for generating heartbeat patterns for that given scenario.Each heartbeat pattern includes one heartbeat template for signal alignment and one Support Vector Machine(SVM)model for identifying the owner of the device.The SVM model is a two-class classifier that is trained using the training heartbeats from the owner(as the positive samples)and the benchmark heartbeats from a global heartbeat database(as the negative samples).The SVM model can give the likelihood whether an unknown heartbeat signal belongs to the owner or not. After the training process,our system uses the heartbeat patterns to perform user authentication.Similar to the training process,the authentication process first collects the heartbeat signals and then extracts the scenario information from the readings of the accelerometer.The scenario information is used for selecting one set of the heartbeat patterns,including both the template for signal alignment and the SVM model for authentication.If there is a matching heartbeat pattern in the database,the system first uses the template to segment the continuous SCG signals into individual heartbeat cycles and align the key features of each cycle.The system then extracts features using wavelet transform and applies the SVM model to classify the heartbeats.If there is no heartbeat pattern for the identified scenario,the system fallbacks to another authentication scheme,such as asking the user to input a PIN.If the user is authenticated through the PIN,the buffered heartbeat signals are used for generating the new heartbeat pattern(both the alignment template and the SVM model)for the identified scenario. The key components of our system are described in the following sections: Heartbeat Segmentation and Alignment(Section 4):In the heartbeat segmentation component,we use a two-step segmentation algorithm to divide the continuous acceleration signals into individual heartbeat cycles. The first step is coarse heart rate estimation,which uses a coarse template to estimate the heart rates from the accelerometer readings.The estimated heart rates are used for selecting the heartbeat pattern which contains the template for fine-grained heartbeat alignment.In the second step of heartbeat segmentation,we use the fine template to perform a cross-correlation on the continuous heartbeat signals.By this way,we can precisely align the key features of each heartbeat cycle in the time domain. Feature Extraction(Section 5):After the segmentation step,our system performs data preprocessing,e.g., normalizing the amplitude of the heartbeat signals,before the feature extraction step.Then,we use Discrete Wavelet Transform(DWT)to extract features from the heartbeat.Each heartbeat cycle is decomposed into multiple levels of wavelet coefficients,and we choose the wavelet coefficients that are most closely related to the heartbeat patterns.This way,we reduce noises that come from different sources,including the respiration movements,small limb movements,and small variations in accelerometer readings. Proc.ACM Interact.Mob.Wearable Ubiquitous Technol..Vol.2.No.3.Article 140.Publication date:September 2018
Unlock with Your Heart: Heartbeat-based Authentication on Commercial Mobile Phones • 140:5 Heartbeat Collection Training Extract Scenario Infomation Heart Rate Estimation Posture Estimation Scenario Selection Authenticating Heartbeat Segmentation & Alignment Generate Fine Alignment Template Feature Extraction User Authentication Feature Extraction SVM Model Generation Fine Template SVM Model Fig. 2. Authentication System Components process normally takes less than two minutes (for collecting 60 heartbeats). Users may be instructed to change the position or the angle of the device during the training process to introduce more variations in the training samples. When collecting the training samples, our system records the built-in accelerometer readings at a sampling rate of 100∼250 Hz (depending on the hardware support of the device). With the readings of the accelerometer, we first extract the heart rates and the body posture of the user. With this information, the collected training samples can be classified into one of the predefined scenarios, e.g., the heart rates are in the range of 50 ∼ 80 Beats per Minute (BPM) and the user is sitting on a chair. The training samples are then used for generating heartbeat patterns for that given scenario. Each heartbeat pattern includes one heartbeat template for signal alignment and one Support Vector Machine (SVM) model for identifying the owner of the device. The SVM model is a two-class classifier that is trained using the training heartbeats from the owner (as the positive samples) and the benchmark heartbeats from a global heartbeat database (as the negative samples). The SVM model can give the likelihood whether an unknown heartbeat signal belongs to the owner or not. After the training process, our system uses the heartbeat patterns to perform user authentication. Similar to the training process, the authentication process first collects the heartbeat signals and then extracts the scenario information from the readings of the accelerometer. The scenario information is used for selecting one set of the heartbeat patterns, including both the template for signal alignment and the SVM model for authentication. If there is a matching heartbeat pattern in the database, the system first uses the template to segment the continuous SCG signals into individual heartbeat cycles and align the key features of each cycle. The system then extracts features using wavelet transform and applies the SVM model to classify the heartbeats. If there is no heartbeat pattern for the identified scenario, the system fallbacks to another authentication scheme, such as asking the user to input a PIN. If the user is authenticated through the PIN, the buffered heartbeat signals are used for generating the new heartbeat pattern (both the alignment template and the SVM model) for the identified scenario. The key components of our system are described in the following sections: Heartbeat Segmentation and Alignment (Section 4): In the heartbeat segmentation component, we use a two-step segmentation algorithm to divide the continuous acceleration signals into individual heartbeat cycles. The first step is coarse heart rate estimation, which uses a coarse template to estimate the heart rates from the accelerometer readings. The estimated heart rates are used for selecting the heartbeat pattern which contains the template for fine-grained heartbeat alignment. In the second step of heartbeat segmentation, we use the fine template to perform a cross-correlation on the continuous heartbeat signals. By this way, we can precisely align the key features of each heartbeat cycle in the time domain. Feature Extraction (Section 5): After the segmentation step, our system performs data preprocessing, e.g., normalizing the amplitude of the heartbeat signals, before the feature extraction step. Then, we use Discrete Wavelet Transform (DWT) to extract features from the heartbeat. Each heartbeat cycle is decomposed into multiple levels of wavelet coefficients, and we choose the wavelet coefficients that are most closely related to the heartbeat patterns. This way, we reduce noises that come from different sources, including the respiration movements, small limb movements, and small variations in accelerometer readings. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, No. 3, Article 140. Publication date: September 2018