计算机科学与技术（参考文献）QGesture - Quantifying Gesture Distance and Direction with WiFi Signals

39 QGesture:Quantifying Gesture Distance and Direction with WiFi Signals NAN YU,State Key Laboratory for Novel Software Technology,Nanjing University,China WEI WANG,State Key Laboratory for Novel Software Technology,Nanjing University,China ALEX X.LIU,Deptartment of Computer Science of Engineering.Michigan State University,USA LINGTAO KONG,State Key Laboratory for Novel Software Technology,Nanjing University,China Many HCI applications,such as volume adjustment in a gaming system,require quantitative gesture measurement for metrics such as movement distance and direction.In this paper,we propose QGesture,a gesture recognition system that uses CSI values provided by COTS WiFi devices to measure the movement distance and direction of human hands.To achieve high accuracy in measurements,we first use phase correction algorithm to remove the phase noise in CSI measurements.We then propose a robust estimation algorithm,called LEVD,to estimate and remove the impact of environmental dynamics. To separate gesture movements from daily activities,we design simple gestures with unique characteristics as preambles to determine the start of the gesture.Our experimental results show that QGesture achieves an average accuracy of 3 cm in the measurement of movement distance and more than 95%accuracy in the movement direction detection in the one dimensional case.Furthermore,it achieves an average absolute direction error of 15 degrees and an average accuracy of 3.7 cm in the measurement of movement distance in the two-dimensional case. CCS Concepts:Human-centered computing-Ubiquitous and mobile computing systems and tools; Additional Key Words and Phrases:Gesture Recognition,WiFi Signals,Wireless Sensing ACM Reference Format: Nan Yu,Wei Wang.Alex X.Liu,and Lingtao Kong.2018.QGesture:Quantifying Gesture Distance and Direction with WiFi Signals.Proc.ACM Hum.-Comput.Interact.1,4,Article 39(March 2018),22 pages.https://doi.org/0000001. 0000001 1 INTRODUCTION Recently a number of interesting WiFi-based gesture recognition schemes have been proposed [1,8,19,24,29]. As human bodies are mostly made of water,they reflect WiFi signals and introduce distortions in the received signal when they move.Different gestures cause different types of distortions in WiFi signals.Thus,by analyzing the changes in WiFi signals,we can recognize the corresponding gesture.WiFi-based gesture recognition has many advantages over traditional approaches that use cameras [4]or wearable sensors [10,28,36].For exam- ple,WiFi-based gesture recognition requires neither lighting nor carrying any devices.It also provides better coverage as WiFi signals can penetrate through walls. Authors'addresses:Nan Yu,State Key Laboratory for Novel Software Technology,Nanjing University,State Key Laboratory for Novel Software Technology,Nanjing.Jiangsu,China;Wei Wang.State Key Laboratory for Novel Software Technology,Nanjing University,State Key Laboratory for Novel Software Technology,Nanjing.Jiangsu,China;Alex X.Liu,Deptartment of Computer Science of Engineering. Michigan State University,Computer Science and Engineering.USA:Lingtao Kong.State Key Laboratory for Novel Software Technology, Nanjing University,State Key Laboratory for Novel Software Technology,Nanjing.Jiangsu,China. ACM acknowledges that this contribution was authored or co-authored by an employee,contractor,or affiliate of the United States govern ment.As such,the United States government retains a nonexclusive,royalty-free right to publish or reproduce this article,or to allow others to do so,for government purposes only. 2018 Association for Computing Machinery. 2573-0142/2018/3-ART39$15.00 https:/doi.org/0000001.0000001 Proceedings of the ACM on Human-Computer Interaction,Vol.1,No.4,Article 39.Publication date:March 2018

39 QGesture: Quantifying Gesture Distance and Direction with WiFi Signals NAN YU, State Key Laboratory for Novel Software Technology, Nanjing University, China WEI WANG, State Key Laboratory for Novel Software Technology, Nanjing University, China ALEX X. LIU, Deptartment of Computer Science of Engineering, Michigan State University, USA LINGTAO KONG, State Key Laboratory for Novel Software Technology, Nanjing University, China Many HCI applications, such as volume adjustment in a gaming system, require quantitative gesture measurement for metrics such as movement distance and direction. In this paper, we propose QGesture, a gesture recognition system that uses CSI values provided by COTS WiFi devices to measure the movement distance and direction of human hands. To achieve high accuracy in measurements, we first use phase correction algorithm to remove the phase noise in CSI measurements. We then propose a robust estimation algorithm, called LEVD, to estimate and remove the impact of environmental dynamics. To separate gesture movements from daily activities, we design simple gestures with unique characteristics as preambles to determine the start of the gesture. Our experimental results show that QGesture achieves an average accuracy of 3 cm in the measurement of movement distance and more than 95% accuracy in the movement direction detection in the one￾dimensional case. Furthermore, it achieves an average absolute direction error of 15 degrees and an average accuracy of 3.7 cm in the measurement of movement distance in the two-dimensional case. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; Additional Key Words and Phrases: Gesture Recognition, WiFi Signals, Wireless Sensing ACM Reference Format: Nan Yu, Wei Wang, Alex X. Liu, and Lingtao Kong. 2018. QGesture: Quantifying Gesture Distance and Direction with WiFi Signals. Proc. ACM Hum.-Comput. Interact. 1, 4, Article 39 (March 2018), 22 pages. https://doi.org/0000001. 0000001 1 INTRODUCTION Recently a number of interesting WiFi-based gesture recognition schemes have been proposed [1, 8, 19, 24, 29]. As human bodies are mostly made of water, they reflect WiFi signals and introduce distortions in the received signal when they move. Different gestures cause different types of distortions in WiFi signals. Thus, by analyzing the changes in WiFi signals, we can recognize the corresponding gesture. WiFi-based gesture recognition has many advantages over traditional approaches that use cameras [4] or wearable sensors [10, 28, 36]. For exam￾ple, WiFi-based gesture recognition requires neither lighting nor carrying any devices. It also provides better coverage as WiFi signals can penetrate through walls. Authors’ addresses: Nan Yu, State Key Laboratory for Novel Software Technology, Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing, Jiangsu, China; Wei Wang, State Key Laboratory for Novel Software Technology, Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing, Jiangsu, China; Alex X. Liu, Deptartment of Computer Science of Engineering, Michigan State University, Computer Science and Engineering, USA; Lingtao Kong, State Key Laboratory for Novel Software Technology, Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing, Jiangsu, China. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor, or affiliate of the United States govern￾ment. As such, the United States government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for government purposes only. © 2018 Association for Computing Machinery. 2573-0142/2018/3-ART39 $15.00 https://doi.org/0000001.0000001 Proceedings of the ACM on Human-Computer Interaction, Vol. 1, No. 4, Article 39. Publication date: March 2018

39:2·N.Yu et al One of the most important applications of WiFi-based gesture recognition is to interact with smart home devices.Existing home appliances use physical interfaces,such as knobs and levers,to provide quantitative inputs,including volume adjustment for TVs and brightness adjustment for lights.These physical inputs allow the user to fine-tune the input value based on immediate feedback.It is difficult to emulate these physical inputs using popular voice-based interactions provided by Amazon Echo or Google Home.However,WiFi-based gesture control can enable such fine-grained quantitative control.For example,the user can push his hand forward to increase the volume of the TV set,where the magnitude of volume increase is proportional to the distance of pushing.To enable this,we need not only to recognize different predefined gestures,but also to quantify gesture movement distance in a granularity of a few centimeters so that the system can adjust the volume according to the distance that the user pushes his hand,while providing audio feedback on the current volume setting along the pushing process.In this way,the user can quantitatively adjust the volume to the desired value using a single action rather than repeating the gesture to increase or decrease the volume by a small amount at each time. The task of using Radio Frequency(RF)signal obtained from commercial hardware to measure the ges- ture movement distance and direction is difficult.Prior systems that use WiFi measurements from commer- cial devices often require whole-body movements,such as walking,to track movement speeds and directions [3,24,33,37].With the coarse measurement from commercial WiFi devices,existing schemes cannot trace hand/finger movements,which introduces weaker WiFi signal distortions than whole-body movements,in a fine-granularity.(delete?)They can only recognize hand/finger movements by matching them to predefined ges- ture patterns [1,18].Using customized ASIC chips based on the 60 GHz radar technology,Google's recent Soli system can quantify micro gestures so that those gestures can serve as human input for small wearable devices (such as smart watches)whose touch screens are too small for human to conveniently input [19].However,due to the fast decay of 60 GHz signal in the air,60 GHz system requires the gesture to be performed within tens of centimeters [31].The limited operational range makes them unsuitable to serve as remote control interfaces for home appliances. In this paper,we propose QGesture,a Quantifying Gesture distance and direction system,which uses Commercial- Off-The-Shelf(COTS)WiFi devices to measure the movement distance and direction of human hands.Figure 1 shows the basic system structure of OGesture.When the user pushes towards the target device,the device collects Channel State Information(CSI),which is perturbed by the WiFi signal reflected by the moving hand The signal reflected by the hand appears as a dynamic vector component in the CSI values,which causes the complex-valued CSI measurement to rotate.The distance of movement can be calculated by the phase change of complex-valued CSI measurement and the direction of movement can be determined by the rotation direction. Therefore,the user can push forward to increase the volume while pulling away to reduce the volume and the amount of increase/decrease is determined by the movement distance.As the perturbation of WiFi signals can be captured at a long distance,QGesture can work at a distance as far as 2 meters.QGesture is the first step to- wards quantitative remote control for home appliances.It shows the feasibility of fine-grained distance/direction measurement of hand movement over a few meters using COTS WiFi devices.Note that currently only a limited modules of WiFi network cards can provide CSI measurements [12]and CSI is not available on smartphones. We envision that more commercial WiFi devices would open their CSI information so that our approach can be deployed on smartphones in the near future. There are four key challenges that need to be addressed in designing QGesture. Reconstruct the phase of CSI measurements:The phase of CSI measurements is important for determining the movement directions [29].However,due to hardware imperfections in COTS WiFi Network Interface Cards (NICs),there are Carrier Frequency Offsets(CFO)and Sampling Frequency Offsets(SFO)between the transmitter and the receiver [17,34].Both the CFO and SFO introduce high variations in the phase of CSI and these variations are sensitive to temperature and hardware conditions.Therefore,it is difficult to predict and remove such phase Proceedings of the ACM on Human-Computer Interaction,Vol.1,No.4,Article 39.Publication date:March 2018

39:2 • N. Yu et al. One of the most important applications of WiFi-based gesture recognition is to interact with smart home devices. Existing home appliances use physical interfaces, such as knobs and levers, to provide quantitative inputs, including volume adjustment for TVs and brightness adjustment for lights. These physical inputs allow the user to fine-tune the input value based on immediate feedback. It is difficult to emulate these physical inputs using popular voice-based interactions provided by Amazon Echo or Google Home. However, WiFi-based gesture control can enable such fine-grained quantitative control. For example, the user can push his hand forward to increase the volume of the TV set, where the magnitude of volume increase is proportional to the distance of pushing. To enable this, we need not only to recognize different predefined gestures, but also to quantify gesture movement distance in a granularity of a few centimeters so that the system can adjust the volume according to the distance that the user pushes his hand, while providing audio feedback on the current volume setting along the pushing process. In this way, the user can quantitatively adjust the volume to the desired value using a single action rather than repeating the gesture to increase or decrease the volume by a small amount at each time. The task of using Radio Frequency (RF) signal obtained from commercial hardware to measure the ges￾ture movement distance and direction is difficult. Prior systems that use WiFi measurements from commer￾cial devices often require whole-body movements, such as walking, to track movement speeds and directions [3, 24, 33, 37]. With the coarse measurement from commercial WiFi devices, existing schemes cannot trace hand/finger movements, which introduces weaker WiFi signal distortions than whole-body movements, in a fine-granularity.(delete?) They can only recognize hand/finger movements by matching them to predefined ges￾ture patterns [1, 18]. Using customized ASIC chips based on the 60 GHz radar technology, Google’s recent Soli system can quantify micro gestures so that those gestures can serve as human input for small wearable devices (such as smart watches) whose touch screens are too small for human to conveniently input [19]. However, due to the fast decay of 60 GHz signal in the air, 60 GHz system requires the gesture to be performed within tens of centimeters [31]. The limited operational range makes them unsuitable to serve as remote control interfaces for home appliances. In this paper, we propose QGesture, a Quantifying Gesture distance and direction system, which uses Commercial￾Off-The-Shelf (COTS) WiFi devices to measure the movement distance and direction of human hands. Figure 1 shows the basic system structure of QGesture. When the user pushes towards the target device, the device collects Channel State Information (CSI), which is perturbed by the WiFi signal reflected by the moving hand. The signal reflected by the hand appears as a dynamic vector component in the CSI values, which causes the complex-valued CSI measurement to rotate. The distance of movement can be calculated by the phase change of complex-valued CSI measurement and the direction of movement can be determined by the rotation direction. Therefore, the user can push forward to increase the volume while pulling away to reduce the volume and the amount of increase/decrease is determined by the movement distance. As the perturbation of WiFi signals can be captured at a long distance, QGesture can work at a distance as far as 2 meters. QGesture is the first step to￾wards quantitative remote control for home appliances. It shows the feasibility of fine-grained distance/direction measurement of hand movement over a few meters using COTS WiFi devices. Note that currently only a limited modules of WiFi network cards can provide CSI measurements [12] and CSI is not available on smartphones. We envision that more commercial WiFi devices would open their CSI information so that our approach can be deployed on smartphones in the near future. There are four key challenges that need to be addressed in designing QGesture. • Reconstruct the phase of CSI measurements: The phase of CSI measurements is important for determining the movement directions [29]. However, due to hardware imperfections in COTS WiFi Network Interface Cards (NICs), there are Carrier Frequency Offsets (CFO) and Sampling Frequency Offsets (SFO) between the transmitter and the receiver [17, 34]. Both the CFO and SFO introduce high variations in the phase of CSI and these variations are sensitive to temperature and hardware conditions. Therefore, it is difficult to predict and remove such phase Proceedings of the ACM on Human-Computer Interaction, Vol. 1, No. 4, Article 39. Publication date: March 2018

QGesture:Quantifying Gesture Distance and Direction with WiFi Signals.39:3 variations without disturbing the small phase changes caused by hand movements.To address this challenge,we carefully analyze the phase offsets in different antenna pairs and design our phase correction algorithm so that phase changes of hand movements can be preserved.Hence,we can determine the movement direction with an accuracy of more than 95% Separate the channel state changes caused by the moving hands from the mixture of changes caused by other body parts:This is particularly important for a gesture recognition system to operate over a long distance, i.e.,several meters,because such system captures both the gesture movements and the environmental dynamics. When the user performs the gesture,their torso and arms also move at the same time,which significantly perturb the measurements of the wireless channel.To address this challenge,we analyze the CSI signals and find the typical signal frequencies generated by gestures,which are different from those generated by movements of other body parts.We then design a robust estimation algorithm,called LEVD,to remove the impact of environmental dynamics. Separate gesture movements from daily activities:Daily activities,such as walking and sitting down,also distort the wireless channel state information.To ensure that QGesture only responses to the channel distortion caused by specific gestures,we design simple gestures with unique characteristics as preambles to determine the start of the gesture.Our experimental results show that QGesture can efficiently recognize the preamble with an accuracy of 92.5%and a low False Positive Rate(FPR)of 3.2%. Accommodate arbitrary pushing angles:The phase changes of CSI measurements are determined by the changes in path length,which depends on the movement angle and the position of the hand with respect to the sender and receiver.When the hand moves along the line connecting the sender and receiver,the path length changes by two times of the movement distance.However,when the movement is in other directions,we may get smaller path length change for the same movement distance.To allow pushing along arbitrary angle,we need to perform the 2D tracking of the hand.To address this challenge,we propose to use multiple receivers to track path length changes of different paths at the same time.By doing this,we can triangulate the position of the hand and measure both the pushing angle and the movement distance.Our experimental results show that we can measure the movement angle with an accuracy of 15 degrees and movement distance with an accuracy of 3.7 cm. We implemented QGesture using COTS WiFi routers and laptops.Our experimental results show that QGes- ture can measure the gesture movement distance with an accuracy of 3 cm within a distance of 1 meters in normal indoor environments.OGesture can also reliably detect the hand movement direction with an accuracy of more than 95%in the one-dimensional case.Furthermore,it achieves an average absolute direction error of 15 degrees and an average accuracy of 3.7 cm in the measurement of movement distance in the two-dimensional case. 2 RELATED WORK We classify existing related gesture systems into two groups:RF-based recognition/tracking and non-RF-based recognition/tracking.Considering the way of collecting RF signal,we further classify RF-based into two cate- gories:RF-based recognition/tracking using COTS hardware and RF-based recognition/tracking using special- ized devices. RF-based Recognition/Tracking Using COTS Hardware:Most COTS hardware based on recognition and tracking systems uses the Received Signal Strength Indicator(RSSI)or CSI obtained from WiFi NICs to capture gesture signals [1,5,7,13,18,22,25,26].The WiKey scheme proposed to use CSI dynamics to recognize micro human activities such as keystrokes [6].The WiFinger scheme used CSI to recognize a set of eight gestures with an accuracy of 93%[26].The WiGest scheme used three wireless links to recognize a special set of gestures, where user hands blocked the signal and thus introduced significant RSSI changes,and achieved a recognition Proceedings of the ACM on Human-Computer Interaction,Vol.1,No.4,Article 39.Publication date:March 2018

QGesture: Quantifying Gesture Distance and Direction with WiFi Signals • 39:3 variations without disturbing the small phase changes caused by hand movements. To address this challenge, we carefully analyze the phase offsets in different antenna pairs and design our phase correction algorithm so that phase changes of hand movements can be preserved. Hence, we can determine the movement direction with an accuracy of more than 95%. • Separate the channel state changes caused by the moving hands from the mixture of changes caused by other body parts: This is particularly important for a gesture recognition system to operate over a long distance, i.e., several meters, because such system captures both the gesture movements and the environmental dynamics. When the user performs the gesture, their torso and arms also move at the same time, which significantly perturb the measurements of the wireless channel. To address this challenge, we analyze the CSI signals and find the typical signal frequencies generated by gestures, which are different from those generated by movements of other body parts. We then design a robust estimation algorithm, called LEVD, to remove the impact of environmental dynamics. • Separate gesture movements from daily activities: Daily activities, such as walking and sitting down, also distort the wireless channel state information. To ensure that QGesture only responses to the channel distortion caused by specific gestures, we design simple gestures with unique characteristics as preambles to determine the start of the gesture. Our experimental results show that QGesture can efficiently recognize the preamble with an accuracy of 92.5% and a low False Positive Rate (FPR) of 3.2%. • Accommodate arbitrary pushing angles: The phase changes of CSI measurements are determined by the changes in path length, which depends on the movement angle and the position of the hand with respect to the sender and receiver. When the hand moves along the line connecting the sender and receiver, the path length changes by two times of the movement distance. However, when the movement is in other directions, we may get smaller path length change for the same movement distance. To allow pushing along arbitrary angle, we need to perform the 2D tracking of the hand. To address this challenge, we propose to use multiple receivers to track path length changes of different paths at the same time. By doing this, we can triangulate the position of the hand and measure both the pushing angle and the movement distance. Our experimental results show that we can measure the movement angle with an accuracy of 15 degrees and movement distance with an accuracy of 3.7 cm. We implemented QGesture using COTS WiFi routers and laptops. Our experimental results show that QGes￾ture can measure the gesture movement distance with an accuracy of 3 cm within a distance of 1 meters in normal indoor environments. QGesture can also reliably detect the hand movement direction with an accuracy of more than 95% in the one-dimensional case. Furthermore, it achieves an average absolute direction error of 15 degrees and an average accuracy of 3.7 cm in the measurement of movement distance in the two-dimensional case. 2 RELATED WORK We classify existing related gesture systems into two groups: RF-based recognition/tracking and non-RF-based recognition/tracking. Considering the way of collecting RF signal, we further classify RF-based into two cate￾gories: RF-based recognition/tracking using COTS hardware and RF-based recognition/tracking using special￾ized devices. RF-based Recognition/Tracking Using COTS Hardware: Most COTS hardware based on recognition and tracking systems uses the Received Signal Strength Indicator (RSSI) or CSI obtained from WiFi NICs to capture gesture signals [1, 5, 7, 13, 18, 22, 25, 26]. The WiKey scheme proposed to use CSI dynamics to recognize micro human activities such as keystrokes [6]. The WiFinger scheme used CSI to recognize a set of eight gestures with an accuracy of 93% [26]. The WiGest scheme used three wireless links to recognize a special set of gestures, where user hands blocked the signal and thus introduced significant RSSI changes, and achieved a recognition Proceedings of the ACM on Human-Computer Interaction, Vol. 1, No. 4, Article 39. Publication date: March 2018

39:4·N.Yu et al CSI values Target device WiFi signal Volume increase WiFi router Fig.1.QGesture system overview. accuracy of 96%[1].However,most of these systems only recognized a predefined set of gestures without consid- ering movement distance/direction measurements.WiDir used WiFi CSI to estimate the whole-body movement direction,such as walking,with an error of 10 degrees [33].For small hand movements,WiDraw used the Angle- Of-Arrival(AOA)measurement to achieve a tracking accuracy of 5 cm [25].However,the AOA-based approach also had a limited working range of fewer than 2 feet,so that it cannot be used as remote controls in HCI applications.QGesture is inspired by previous WiFi CSI processing technologies,including the noise removal algorithm,basic phase correction algorithm,and preamble gesture design.QGesture advances the state-of-art design by capturing the small phase variations caused by hand movements at a long distance.In addition to WiFi-based schemes,existing schemes also use COTS RFID readers and tags to track gestures [9,28].However, these systems require users to wear RFIDs or operate close to the RFID array,which makes them inconvenient to use. RF-based Recognition/Tracking Using Specialized Devices:RF signals can also be captured by specialized devices such as software radio systems.Software radio systems,such as USRP or WARP,have access to the fine- grained baseband signal so that they can provide the capability of quantifying hand/finger movement distance and speed [2,3,14,31,35].WiSee used USRP software radio to identify and classify nine whole-body gestures with an accuracy of 94%[24].WiTrack used specially designed Frequency-Modulated Continuous-Wave(FMCW) radar with a high bandwidth of 1.79 GHz to track human movements behind the wall with a resolution of about 11 cm to 20 cm [2,3].WiDeo used the WARP hardware to enable a tracking accuracy of 7 cm for multiple objects [14].AllSee used a specially designed analog circuit to extract the envelopes of the received signals and recognize gestures within a short distance of 2.5 feet[15].While these system provided valuable insights on the dynamics of the wireless signal,tracking with the coarse-grained CSI measurements requires a different set of signal processing algorithms. Non-RF-based Recognition/Tracking:Gesture recognition can be enabled by non-RF based technologies, including computer vision,wearable devices,and sound waves.Computer vision based gesture recognition uses cameras and infrared sensors to reconstruct the depth information from videos.The distance measurement accuracy for computer vision based solutions could be a few millimeters when the target is within one meter [32],but the depth accuracy degrades to a few centimeters for an operational range of 5 meters [16].The key limitation of computer vision based solutions is that the accuracy is highly dependent on the viewing angle and lighting conditions.Moreover,users may also have privacy concerns for video-camera-based solutions.Sound waves can be used to measure moving distances [23,38,39]or moving speeds [11].When the user is holding the Proceedings of the ACM on Human-Computer Interaction,Vol.1,No.4,Article 39.Publication date:March 2018

39:4 • N. Yu et al. Target device WiFi signal WiFi router Volume increase CSI values Fig. 1. QGesture system overview. accuracy of 96% [1]. However, most of these systems only recognized a predefined set of gestures without consid￾ering movement distance/direction measurements. WiDir used WiFi CSI to estimate the whole-body movement direction, such as walking, with an error of 10 degrees [33]. For small hand movements, WiDraw used the Angle￾Of-Arrival (AOA) measurement to achieve a tracking accuracy of 5 cm [25]. However, the AOA-based approach also had a limited working range of fewer than 2 feet, so that it cannot be used as remote controls in HCI applications. QGesture is inspired by previous WiFi CSI processing technologies, including the noise removal algorithm, basic phase correction algorithm, and preamble gesture design. QGesture advances the state-of-art design by capturing the small phase variations caused by hand movements at a long distance. In addition to WiFi-based schemes, existing schemes also use COTS RFID readers and tags to track gestures [9, 28]. However, these systems require users to wear RFIDs or operate close to the RFID array, which makes them inconvenient to use. RF-based Recognition/Tracking Using Specialized Devices: RF signals can also be captured by specialized devices such as software radio systems. Software radio systems, such as USRP or WARP, have access to the fine￾grained baseband signal so that they can provide the capability of quantifying hand/finger movement distance and speed [2, 3, 14, 31, 35]. WiSee used USRP software radio to identify and classify nine whole-body gestures with an accuracy of 94% [24]. WiTrack used specially designed Frequency-Modulated Continuous-Wave (FMCW) radar with a high bandwidth of 1.79 GHz to track human movements behind the wall with a resolution of about 11 cm to 20 cm [2, 3]. WiDeo used the WARP hardware to enable a tracking accuracy of 7 cm for multiple objects [14]. AllSee used a specially designed analog circuit to extract the envelopes of the received signals and recognize gestures within a short distance of 2.5 feet [15]. While these system provided valuable insights on the dynamics of the wireless signal, tracking with the coarse-grained CSI measurements requires a different set of signal processing algorithms. Non-RF-based Recognition/Tracking: Gesture recognition can be enabled by non-RF based technologies, including computer vision, wearable devices, and sound waves. Computer vision based gesture recognition uses cameras and infrared sensors to reconstruct the depth information from videos. The distance measurement accuracy for computer vision based solutions could be a few millimeters when the target is within one meter [32], but the depth accuracy degrades to a few centimeters for an operational range of 5 meters [16]. The key limitation of computer vision based solutions is that the accuracy is highly dependent on the viewing angle and lighting conditions. Moreover, users may also have privacy concerns for video-camera-based solutions. Sound waves can be used to measure moving distances [23, 38, 39] or moving speeds [11]. When the user is holding the Proceedings of the ACM on Human-Computer Interaction, Vol. 1, No. 4, Article 39. Publication date: March 2018

QGesture:Quantifying Gesture Distance and Direction with WiFi Signals.39:5 device,sound wave based solutions can provide distance measurement accuracy of a few centimeters [23,38]. Due to the weakness of sound energy reflected by hand,device-free gesture recognition solutions mostly use the Doppler effect,which only provides low-resolution speed measurements that cannot be used for fine-grained control over a long distance [11].Recent fine-grained tracking solution only works for a short distance of 50 cm [21,30].QGesture uses the similar phase based distances measurement algorithm as LLAP [30].However, our long-range WiFi gesture tracking system needs to handle the phase noises and interferences from nearby movements,which can be ignored in short-range sound-based systems. 3 SYSTEM MODEL In this section,we first present the theoretical model that quantifies the gesture movement distance and direction.We then discuss the noise sources that make CSI measurements from COTS devices deviate from theoretical models.Finally,we present methods to remove the CFO and SFO in CSI measurements so that we can measure the movement distance and direction using theoretical models. 3.1 Modeling Phase-Distance Relationship The magnitude and phase changes in CSI are closely related to the distance and direction of gesture move- ments.For simplicity,we first consider signals traveling through only two paths,i.e.,the Line-Of-Sight(LOS) path(path A)and the hand-reflected path(path B),between a pair of transmitter/receiver as shown in Figure 2. In theory,the resulting Channel Frequency Response(CFR)H(f,t)in CSI measured at time t can be represented as[29,34]: H(f.t)=aa(f.t)e(f,t)e() (1) where jis the imaginary unit with j2=-1,f is the frequency of the WiFi signal,aa(f,t)and aB(f,t)represent the magnitude attenuation and the initial phase in path a and B.As the path length of path A and B are different, their propagation delay tA(t)and rg(t)are also different as we have the relationship ra(t)=la(t)/c,where lA(t) is the length of path A and c is the speed of light. The CFR H(f,t)contains two components:one static component for path A and one dynamic component for path B,as shown in Figure 3.Furthermore,the magnitude of the static component of different pairs of antenna of different subcarriers is different as a result of different propagation delay and different carrier frequencies as showed in our Section 5.5.Note that the CFR H(f,t)is a complex value,where the real and imaginary part are called the In-phase(I)part and Quadrature(Q)part,respectively.Therefore,when we plot CFR in the complex plane,the CFR value at each time instance will be a vector and the end of the vector draws an I/O trace as time evolves.In case that the hand pushes towards the transmitter/receiver,the I/Q trace for a single subcarrier is an arc as shown in Figure 3.This is because when the hand moves,the vector for path A,which is a(f,t)ejf) is not changed as both the transmitter and the receiver remain static.The vector for path A is a static component. However,the vector for path B,which is ag(f,t)e(),significantly changes when the path length of lg(t) changes.When lg(t)reduces,the attenuation ag(f,t)only changes slowly and the phase (t)=-2mfrg(t)= -2flB(t)/c increases significantly.For WiFi signals at 5 GHz,the radio wavelength A=c/f is equal to 6 cm. Therefore,the phase for the vector corresponding to path B,which is o(t)=-2mlg(t)/A,will increase by 2 when lB(t)is reduced by the radio wavelength of 6 cm.By measuring the phase change Aog of the dynamic component,we can get the movement distance d as: △pB入 d=- 2aπ (2) Proceedings of the ACM on Human-Computer Interaction,Vol.1,No.4,Article 39.Publication date:March 2018

QGesture: Quantifying Gesture Distance and Direction with WiFi Signals • 39:5 device, sound wave based solutions can provide distance measurement accuracy of a few centimeters [23, 38]. Due to the weakness of sound energy reflected by hand, device-free gesture recognition solutions mostly use the Doppler effect, which only provides low-resolution speed measurements that cannot be used for fine-grained control over a long distance [11]. Recent fine-grained tracking solution only works for a short distance of 50 cm [21, 30]. QGesture uses the similar phase based distances measurement algorithm as LLAP [30]. However, our long-range WiFi gesture tracking system needs to handle the phase noises and interferences from nearby movements, which can be ignored in short-range sound-based systems. 3 SYSTEM MODEL In this section, we first present the theoretical model that quantifies the gesture movement distance and direction. We then discuss the noise sources that make CSI measurements from COTS devices deviate from theoretical models. Finally, we present methods to remove the CFO and SFO in CSI measurements so that we can measure the movement distance and direction using theoretical models. 3.1 Modeling Phase-Distance Relationship The magnitude and phase changes in CSI are closely related to the distance and direction of gesture move￾ments. For simplicity, we first consider signals traveling through only two paths, i.e., the Line-Of-Sight (LOS) path (path A) and the hand-reflected path (path B), between a pair of transmitter/receiver as shown in Figure 2. In theory, the resulting Channel Frequency Response (CFR) H(f ,t) in CSI measured at time t can be represented as [29, 34]: H(f ,t) = aA(f ,t)e −j2π f τA (t) + aB (f ,t)e −j2π f τB (t) , (1) where j is the imaginary unit with j 2 = −1, f is the frequency of the WiFi signal, aA (f ,t) and aB (f ,t) represent the magnitude attenuation and the initial phase in path A and B. As the path length of path A and B are different, their propagation delay τA(t) and τB (t) are also different as we have the relationship τA(t) = lA(t)/c, where lA(t) is the length of path A and c is the speed of light. The CFR H(f ,t) contains two components: one static component for path A and one dynamic component for path B, as shown in Figure 3. Furthermore, the magnitude of the static component of different pairs of antenna of different subcarriers is different as a result of different propagation delay and different carrier frequencies as showed in our Section 5.5. Note that the CFR H(f ,t) is a complex value, where the real and imaginary part are called the In-phase (I) part and Quadrature (Q) part, respectively. Therefore, when we plot CFR in the complex plane, the CFR value at each time instance will be a vector and the end of the vector draws an I/Q trace as time evolves. In case that the hand pushes towards the transmitter/receiver, the I/Q trace for a single subcarrier is an arc as shown in Figure 3. This is because when the hand moves, the vector for path A, which is aA(f ,t)e −j2π f τA (t) , is not changed as both the transmitter and the receiver remain static. The vector for path A is a static component. However, the vector for path B, which is aB (f ,t)e −j2π f τB (t) , significantly changes when the path length of lB (t) changes. When lB (t) reduces, the attenuation aB (f ,t) only changes slowly and the phase φB (t) = −2π f τB (t) = −2π f lB (t)/c increases significantly. For WiFi signals at 5 GHz, the radio wavelength λ = c/f is equal to 6 cm. Therefore, the phase for the vector corresponding to path B, which is φB (t) = −2πlB (t)/λ, will increase by 2π when lB (t) is reduced by the radio wavelength of 6 cm. By measuring the phase change ∆φB of the dynamic component, we can get the movement distance d as: d = − ∆φBλ 2aπ , (2) Proceedings of the ACM on Human-Computer Interaction, Vol. 1, No. 4, Article 39. Publication date: March 2018