Motivation System Design Experiments Coodusic 和京大盛 Basic Idea Ultrasound High sampling rate(48 kHz) Sensitive to the depth direction Only 1D information Mono-camera Low frame per second(30 fps) Accurate 2D information o Use ultrasound based sensing,along with one COTS mono-camera, to enable 3D tracking of user fingers with high frame rate. ·口0,42，平·2刀00 Ke Sun,et al Mobisys'18 June 3.2018 Depth Aware Finger Tapping on Virtual Display 6/22
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Motivation System Design Experiments Conclusion Basic Idea Time (millisecond) 0 50 100 150 200 250 300 350 400 450 500 I/Q (normalized) -300 -200 -100 0 100 200 300 400 I Q Ultrasound High sampling rate (48 kHz) Sensitive to the depth direction Only 1D information Mono-camera Low frame per second (30 fps) Accurate 2D information Use ultrasound based sensing, along with one COTS mono-camera, to enable 3D tracking of user fingers with high frame rate. Ke Sun, et al. Mobisys’18 June 3, 2018 Depth Aware Finger Tapping on Virtual Display 6 / 22
Motivation System Design Experimonts Coodusio 动京大盘 System Architecture Fingertip Localization Hand Fingertips Recelve frame detection detection Camera Ultrasound Signal Phase Extraction 0 Recelve Sound signal Sound signal phase change Microphone sound signal measurement 勿 Emit 18~22kHz Speaker CW sound signal Keystroke Localization Tapping Detection and Tapping Depth Measurement Confirm the tapping action Keystroke localization based on finite state machine based on the depth Tapping Depth measurement Measurement Detect the start of the finger tapping action Ke Sun,et al Mobisys'18 June 3.2018 Depth Aware Finger Tapping on Virtual Display 7/22
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Motivation System Design Experiments Conclusion System Architecture Microphone Speaker Emit 18~22kHz CW sound signal Receive sound signal Sound signal down conversion Sound signal phase change measurement Detect the start of the finger tapping action Ultrasound Signal Phase Extraction Camera Receive frame Hand detection Fingertips detection Fingertip Localization Tapping Detection and Tapping Depth Measurement Confirm the tapping action based on finite state machine Tapping Depth Measurement Keystroke Localization Keystroke localization based on the depth measurement Ke Sun, et al. Mobisys’18 June 3, 2018 Depth Aware Finger Tapping on Virtual Display 7 / 22