Tracking Human Motions in Photographing: A Context-Aware Energy-Saving Scheme for Smart Phones YAFENG YIN,LEI XIE,YUANYUAN FAN,and SANGLU LU,State Key Laboratory for Novel Software Technology,Nanjing University Due to the portability of smart phones,more and more people tend to take photos with smart phones.How- ever,energy-saving continues to be a thorny problem,since photographing is a rather power hungry function. To extend the battery life of phones while taking photos,we propose a context-aware energy-saving scheme called"SenSave."SenSave senses the user's activities during photographing and adopts suitable energy-saving strategies accordingly.SenSave works based on the observation that a lot of energy during photographing is wasted in preparations before shooting.By leveraging the low power-consuming embedded sensors,such as accelerometer and gyroscope,we can recognize the user's activities and reduce unnecessary energy con- sumption.Besides,by maintaining an activity state machine,SenSave can determine the user's activity pro- gressively and improve the recognition accuracy.Experiment results show that SenSave can recognize the user's activities with an average accuracy of 95.5%and reduce the energy consumption during photographing by 30.0%,when compared to the approach by frequently turning ON/OFF the camera or screen.Additionally, we enhance"SenSave"by introducing an extended Markov chain to predict the next activity state and adopt the energy-saving strategy in advance.Then,we can reduce the energy consumption during photographing 2 by36.1%. CCS Concepts:Human-centered computing-Ubiquitous and mobile computing design and eval- uation methods;Empirical studies in ubiquitous and mobile computing; Additional Key Words and Phrases:Activity sensing.energy saving.smart phones,photographing ACM Reference format: Yafeng Yin,Lei Xie,Yuanyuan Fan,and Sanglu Lu.2017.Tracking Human Motions in Photographing:A Context-Aware Energy-Saving Scheme for Smart Phones.ACM Trans.Sen.Netw.13,4,Article 29(September 2017),37 pages. https:/doi.org/.10.1145/3085578 A preliminary version of this work was presented in Proceedings of the 12th IEEE International Conference on Mobile Ad Hoc and Sensor Systems (IEEE MASS 2015)(Fan et al.2015). This work is supported in part by National Natural Science Foundation of China under Grant Nos.61472185,61373129, 61321491,61502224;JiangSu Natural Science Foundation under Grant No.BK20151390:Fundamental Research Funds for the Central Universities under Grant No.020214380035;2016 Program A for outstanding PhD candidate of Nanjing University under Grant No.201601A008.This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization. Authors'addresses:Y.Yin,L Xie(corresponding author),Y.Fan,and S.Lu,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China;emails:yyf@dislab.nju.edu.cn,lxie@nju.edu.cn,fyymonica@ dislab.nju.edu.cn,sanglu@nju.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted.To copy otherwise,or republish,to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.Request permissions from permissions@acm.org. ©2017ACM1550-4859/2017/09-ART29$15.00 https:/∥doi.org/10.1145/3085578 ACM Transactions on Sensor Networks,Vol.13,No.4,Article 29.Publication date:September 2017
29 Tracking Human Motions in Photographing: A Context-Aware Energy-Saving Scheme for Smart Phones YAFENG YIN, LEI XIE, YUANYUAN FAN, and SANGLU LU, State Key Laboratory for Novel Software Technology, Nanjing University Due to the portability of smart phones, more and more people tend to take photos with smart phones. However, energy-saving continues to be a thorny problem, since photographing is a rather power hungry function. To extend the battery life of phones while taking photos, we propose a context-aware energy-saving scheme called “SenSave.” SenSave senses the user’s activities during photographing and adopts suitable energy-saving strategies accordingly. SenSave works based on the observation that a lot of energy during photographing is wasted in preparations before shooting. By leveraging the low power-consuming embedded sensors, such as accelerometer and gyroscope, we can recognize the user’s activities and reduce unnecessary energy consumption. Besides, by maintaining an activity state machine, SenSave can determine the user’s activity progressively and improve the recognition accuracy. Experiment results show that SenSave can recognize the user’s activities with an average accuracy of 95.5% and reduce the energy consumption during photographing by 30.0%, when compared to the approach by frequently turning ON/OFF the camera or screen. Additionally, we enhance “SenSave” by introducing an extended Markov chain to predict the next activity state and adopt the energy-saving strategy in advance. Then, we can reduce the energy consumption during photographing by 36.1%. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing design and evaluation methods; Empirical studies in ubiquitous and mobile computing; Additional Key Words and Phrases: Activity sensing, energy saving, smart phones, photographing ACM Reference format: Yafeng Yin, Lei Xie, Yuanyuan Fan, and Sanglu Lu. 2017. Tracking Human Motions in Photographing: A Context-Aware Energy-Saving Scheme for Smart Phones. ACM Trans. Sen. Netw. 13, 4, Article 29 (September 2017), 37 pages. https://doi.org/10.1145/3085578 A preliminary version of this work was presented in Proceedings of the 12th IEEE International Conference on Mobile Ad Hoc and Sensor Systems (IEEE MASS 2015) (Fan et al. 2015). This work is supported in part by National Natural Science Foundation of China under Grant Nos. 61472185, 61373129, 61321491, 61502224; JiangSu Natural Science Foundation under Grant No. BK20151390; Fundamental Research Funds for the Central Universities under Grant No. 020214380035; 2016 Program A for outstanding PhD candidate of Nanjing University under Grant No. 201601A008. This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization. Authors’ addresses: Y. Yin, L. Xie (corresponding author), Y. Fan, and S. Lu, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China; emails: yyf@dislab.nju.edu.cn, lxie@nju.edu.cn, fyymonica@ dislab.nju.edu.cn, sanglu@nju.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2017 ACM 1550-4859/2017/09-ART29 $15.00 https://doi.org/10.1145/3085578 ACM Transactions on Sensor Networks, Vol. 13, No. 4, Article 29. Publication date: September 2017
29:2 Y.Yin et al. 1 INTRODUCTION 1.1 Motivation Nowadays,smart phones are widely used in our daily lives.Due to the portability of smart phones, more and more people tend to take photos with their smart phones,for example,taking photos at a tourist attraction.However,energy-saving continues to be an upsetting problem for smart camera phones,since photographing is a very power-hungry function.For example,according to KS Mobile's(KS Mobile Inc.2014)report in 2014,the application Camera360 Ultimate is listed in first place of the top 10 battery-draining applications for Android.Therefore,the huge energy consumption becomes a non-negligible pain point for the users of smart camera phones.Conse- quently,it is essential to reduce the unnecessary energy consumption during photographing to extend the battery life of smart camera phones. 1.2 Limitations of Prior Art Prior work on energy saving of smart phones can be classified into the following three parts:energy consumption of hardware(Fan et al.2007;Bellosa et al.2003;Rajan et al.2006; Balasubramanian et al.2009),power consumption models,and energy-saving schemes for specific applications.For hardware,Chen et al.(2013a)analyze the power consumption of AMOLED displays in multimedia applications and reveal that camera recording incurs high power cost. LiKamWa et al.(2013)report the experimental and analytical characterization of CMOS image sensors and reveal two energy-proportional mechanisms for energy saving.For models,Dong and Zhong(2011)propose Sesame,with which a mobile system constructs an energy model of itself without any external assistance.Xu et al.(2013)propose a new way called V-edge to generate power models based on battery voltage dynamics.For specific applications,Han et al.(2013)study the energy cost made by human-screen interaction,such as scrolling on the screen.Dietrich and Chakraborty(2013)detect the game's current state and decrease the processor's voltage and fre- quency whenever possible to save energy.Hu et al.(2013)propose a Mobility-Assisted User Contact detection algorithm(MAUC),which utilizes the accelerometer of the phone to detect user move- ments for energy-saving.The Bluetooth scans only when user movements have a high possibility to cause contact changes.LiKamWa et al.(2013)improve the energy efficiency of image sensors based on hardware modifications.There are fewer energy-saving schemes for photographing. Being different from these prior work,we aim to propose an energy-saving scheme for pho- tographing.We aim to recognize the user's activity and reduce unnecessary energy cost when the user is not taking photos.The scheme does not need hardware modifications and user interaction, to guarantee a good user experience. 1.3 Proposed Approach A straight solution to reduce energy cost is to turn off the camera or screen while not taking photos.However,frequently turning ON/OFF the camera or screen is very annoying and leads to a bad user experience.Besides,frequently turning on the camera or screen will incur high energy consumption.Take the Samsung Galaxy Nexus smart phone as an example,the energy consumption of the pair of operations,that is,turning off the camera and screen and then turning on the screen and camera,can keep the camera working on preview mode for about 7s. To propose an efficient energy-saving scheme,we conduct extensive observations.We find that during photographing.a fairly large proportion of energy is wasted in preparations before shoot- ing.For example,the user usually first turns on the camera.Then,he/she will probably adjust the phone time and again,to find a good camera view.Finally,when the camera focuses on the target, the user will press the button to shoot.Between two consecutive shots,the camera works with ACM Transactions on Sensor Networks,Vol 13.No.4,Article 29.Publication date:September 2017
29:2 Y. Yin et al. 1 INTRODUCTION 1.1 Motivation Nowadays, smart phones are widely used in our daily lives. Due to the portability of smart phones, more and more people tend to take photos with their smart phones, for example, taking photos at a tourist attraction. However, energy-saving continues to be an upsetting problem for smart camera phones, since photographing is a very power-hungry function. For example, according to KS Mobile’s (KS Mobile Inc. 2014) report in 2014, the application Camera360 Ultimate is listed in first place of the top 10 battery-draining applications for Android. Therefore, the huge energy consumption becomes a non-negligible pain point for the users of smart camera phones. Consequently, it is essential to reduce the unnecessary energy consumption during photographing to extend the battery life of smart camera phones. 1.2 Limitations of Prior Art Prior work on energy saving of smart phones can be classified into the following three parts: energy consumption of hardware (Fan et al. 2007; Bellosa et al. 2003; Rajan et al. 2006; Balasubramanian et al. 2009), power consumption models, and energy-saving schemes for specific applications. For hardware, Chen et al. (2013a) analyze the power consumption of AMOLED displays in multimedia applications and reveal that camera recording incurs high power cost. LiKamWa et al. (2013) report the experimental and analytical characterization of CMOS image sensors and reveal two energy-proportional mechanisms for energy saving. For models, Dong and Zhong (2011) propose Sesame, with which a mobile system constructs an energy model of itself without any external assistance. Xu et al. (2013) propose a new way called V-edge to generate power models based on battery voltage dynamics. For specific applications, Han et al. (2013) study the energy cost made by human-screen interaction, such as scrolling on the screen. Dietrich and Chakraborty (2013) detect the game’s current state and decrease the processor’s voltage and frequency whenever possible to save energy. Hu et al. (2013) propose a Mobility-Assisted User Contact detection algorithm (MAUC), which utilizes the accelerometer of the phone to detect user movements for energy-saving. The Bluetooth scans only when user movements have a high possibility to cause contact changes. LiKamWa et al. (2013) improve the energy efficiency of image sensors based on hardware modifications. There are fewer energy-saving schemes for photographing. Being different from these prior work, we aim to propose an energy-saving scheme for photographing. We aim to recognize the user’s activity and reduce unnecessary energy cost when the user is not taking photos. The scheme does not need hardware modifications and user interaction, to guarantee a good user experience. 1.3 Proposed Approach A straight solution to reduce energy cost is to turn off the camera or screen while not taking photos. However, frequently turning ON/OFF the camera or screen is very annoying and leads to a bad user experience. Besides, frequently turning on the camera or screen will incur high energy consumption. Take the Samsung Galaxy Nexus smart phone as an example, the energy consumption of the pair of operations, that is, turning off the camera and screen and then turning on the screen and camera, can keep the camera working on preview mode for about 7s. To propose an efficient energy-saving scheme, we conduct extensive observations. We find that during photographing, a fairly large proportion of energy is wasted in preparations before shooting. For example, the user usually first turns on the camera. Then, he/she will probably adjust the phone time and again, to find a good camera view. Finally, when the camera focuses on the target, the user will press the button to shoot. Between two consecutive shots, the camera works with ACM Transactions on Sensor Networks, Vol. 13, No. 4, Article 29. Publication date: September 2017
Tracking Human Motions in Photographing 29:3 the same settings(e.g.,the same preview size),which result in comparable energy consumption to that of shooting photos.If we can recognize the user's activity and detect the duration between two consecutive shots,then we can decrease the screen brightness,preview size,or the preview frame rate to reduce energy cost. In this article,by leveraging activity sensing,we propose a context-aware energy-saving scheme for smart camera phones.Our idea works based on the observation that most smart phones are equipped with low power-consuming sensors,such as the accelerometer and gyroscope.We can leverage these tiny sensors to recognize the user's activities,such that the corresponding energy- saving strategies (e.g,decreasing the screen brightness,decreasing the frame rate,etc.)can be applied.To reduce the error of activity recognition,we maintain an activity state machine to de- termine the activity state progressively.In addition,we also introduce an extended Markov chain to predict the next activity state,to adopt a suitable energy-saving strategy in advance to fur- ther reduce energy cost.Without user interaction,we can reduce the energy consumption during photographing while guaranteeing a good user experience. 1.4 Technique Challenges and Solutions There are some challenges in activity sensing and designing the energy-saving scheme for taking photos with smart phones. -Activity sensing:The first challenge is how to use the sensor data for activity recognition. To address this challenge,we propose a three-level architecture,which classifies the activi- ties into three levels:body level,arm level,and wrist level.For the sensor data of a potential activity,we first utilize the variance and periodicity of the sensor data to classify the activity into one of the three levels.For activities in the same level,we combine data from differ- ent sensors to distinguish one from another based on the features of activities.To reduce the error of activity recognition,we maintain an activity state machine and determine the user's activity state progressively. -Energy-saving scheme design:The second challenge is how to design an appropriate energy-saving scheme with the recognized activities during photographing.To address this challenge,we propose a context-aware energy-saving scheme SenSave,which adopts suit- able energy-saving strategies based on the user's activities.In body level,SenSave focuses on turning ON/OFF sensors,camera,and screen.In arm level,SenSave focuses on adjust- ing the screen brightness,starting or stopping the camera preview.In wrist level,SenSave focuses on adjusting the preview size,the preview frame rate of the camera.In each level, we will adjust the parameters in an energy-saving strategy for the specific activity. -Trade-off between activity sensing and energy saving:The third challenge is how to make an appropriate trade-off between the accuracy of activity sensing and energy con- sumption.Obviously,more types of sensor data and larger sampling rates contribute to higher accuracy of activity sensing,while resulting in more energy consumption.To ad- dress this challenge,we only leverage the low power-consuming sensors like accelerom- eter and gyroscope for activity recognition.When guaranteeing the recognition accuracy, we choose the sampling rates of sensors as small as possible.For further energy saving,we introduce an extended Markov chain to predict the next activity state and adopt the suitable energy-saving strategy in advance. 1.5 Key Contributions We make the following contributions in this article(a preliminary version of this work appeared in Fan et al.(2015)). ACM Transactions on Sensor Networks,Vol.13,No.4,Article 29.Publication date:September 2017
Tracking Human Motions in Photographing 29:3 the same settings (e.g., the same preview size), which result in comparable energy consumption to that of shooting photos. If we can recognize the user’s activity and detect the duration between two consecutive shots, then we can decrease the screen brightness, preview size, or the preview frame rate to reduce energy cost. In this article, by leveraging activity sensing, we propose a context-aware energy-saving scheme for smart camera phones. Our idea works based on the observation that most smart phones are equipped with low power-consuming sensors, such as the accelerometer and gyroscope. We can leverage these tiny sensors to recognize the user’s activities, such that the corresponding energysaving strategies (e.g., decreasing the screen brightness, decreasing the frame rate, etc.) can be applied. To reduce the error of activity recognition, we maintain an activity state machine to determine the activity state progressively. In addition, we also introduce an extended Markov chain to predict the next activity state, to adopt a suitable energy-saving strategy in advance to further reduce energy cost. Without user interaction, we can reduce the energy consumption during photographing while guaranteeing a good user experience. 1.4 Technique Challenges and Solutions There are some challenges in activity sensing and designing the energy-saving scheme for taking photos with smart phones. —Activity sensing: The first challenge is how to use the sensor data for activity recognition. To address this challenge, we propose a three-level architecture, which classifies the activities into three levels: body level, arm level, and wrist level. For the sensor data of a potential activity, we first utilize the variance and periodicity of the sensor data to classify the activity into one of the three levels. For activities in the same level, we combine data from different sensors to distinguish one from another based on the features of activities. To reduce the error of activity recognition, we maintain an activity state machine and determine the user’s activity state progressively. —Energy-saving scheme design: The second challenge is how to design an appropriate energy-saving scheme with the recognized activities during photographing. To address this challenge, we propose a context-aware energy-saving scheme SenSave, which adopts suitable energy-saving strategies based on the user’s activities. In body level, SenSave focuses on turning ON/OFF sensors, camera, and screen. In arm level, SenSave focuses on adjusting the screen brightness, starting or stopping the camera preview. In wrist level, SenSave focuses on adjusting the preview size, the preview frame rate of the camera. In each level, we will adjust the parameters in an energy-saving strategy for the specific activity. —Trade-off between activity sensing and energy saving: The third challenge is how to make an appropriate trade-off between the accuracy of activity sensing and energy consumption. Obviously, more types of sensor data and larger sampling rates contribute to higher accuracy of activity sensing, while resulting in more energy consumption. To address this challenge, we only leverage the low power-consuming sensors like accelerometer and gyroscope for activity recognition. When guaranteeing the recognition accuracy, we choose the sampling rates of sensors as small as possible. For further energy saving, we introduce an extended Markov chain to predict the next activity state and adopt the suitable energy-saving strategy in advance. 1.5 Key Contributions We make the following contributions in this article (a preliminary version of this work appeared in Fan et al. (2015)). ACM Transactions on Sensor Networks, Vol. 13, No. 4, Article 29. Publication date: September 2017
29:4 Y.Yin et al. (a)Walking (b)Lifting up the (c)Rotating the phone(d)Fine-tuning and (e)Laying down arm shooting the arm Fig.1.Human activities during photographing. -First,we propose a context-aware energy-saving scheme for smart camera phones,by lever- aging the built-in sensors for activity sensing.Based on the activity recognition results,we can adopt corresponding energy-saving strategies. -Second,we build a three-level architecture for activity sensing,including body level,arm level,and wrist level.We use the low power-consuming sensors like accelerometer and gyroscope to extract representative features to distinguish one activity from another.By maintaining an activity state machine,we can determine the user's activity progressively and reduce the error of activity recognition. -Third,we design an efficient energy-saving scheme,which can adaptively choose a suitable energy-saving strategy without user interaction,according to the activity state.Besides, we also introduce an extended Markov chain to predict the next activity state,to adopt a suitable energy-saving strategy in advance for further energy saving. -Fourth,we have implemented a system prototype in android-powered smart camera phones. The experiment results show that our solution is able to recognize the user's activities with an average accuracy of 95.5%.Besides,we can reduce the energy consumption during pho- tographing by 30.0%,when compared to the approach by frequently turning ON/OFF the camera or screen.By introducing the extended Markov chain,we can reduce the energy consumption during photographing by 36.1%. 2 OBSERVATIONS ON PHOTOGRAPHING 2.1 Human Activities Related to Photographing During photographing,the users tend to have similar activities,as shown in Figure 1.Before or after the user takes photos,he/she may stay motionless or keep moving,for example,walking. jogging,and so on.While taking photos,the user usually lifts up the arm,rotates the phone, makes fine-tuning,shoots a picture,then lays down the arm.We categorize all the activities into three levels.(1)Body level:motionlessness,body movement.(2)Arm level:lifting up the arm,laying down the arm.(3)Wrist level:rotating the phone,making fine-tuning,shooting a picture.If the user wants to take multiple photos,then he/she may keep the camera working on the preview state, to take the next photo conveniently.However,it is rather energy consuming to keep the camera working.Therefore,many users tend to turn off the camera between two consecutive shots,unless he/she needs to take multiple photos in a short time. 2.2 Energy Consumption Related to Photographing According to Figure 1,before shooting a photo,there will be a preparation time,during which the user needs to move his/her locations,adjusts the position of the phone or makes fine-tuning,to find a good camera view.Obviously,if the user keeps the camera working with large preview size, it will incur much energy consumption,because the preparation time cannot be ignored.Besides, ACM Transactions on Sensor Networks,Vol 13.No.4,Article 29.Publication date:September 2017
29:4 Y. Yin et al. Fig. 1. Human activities during photographing. —First, we propose a context-aware energy-saving scheme for smart camera phones, by leveraging the built-in sensors for activity sensing. Based on the activity recognition results, we can adopt corresponding energy-saving strategies. —Second, we build a three-level architecture for activity sensing, including body level, arm level, and wrist level. We use the low power-consuming sensors like accelerometer and gyroscope to extract representative features to distinguish one activity from another. By maintaining an activity state machine, we can determine the user’s activity progressively and reduce the error of activity recognition. —Third, we design an efficient energy-saving scheme, which can adaptively choose a suitable energy-saving strategy without user interaction, according to the activity state. Besides, we also introduce an extended Markov chain to predict the next activity state, to adopt a suitable energy-saving strategy in advance for further energy saving. —Fourth, we have implemented a system prototype in android-powered smart camera phones. The experiment results show that our solution is able to recognize the user’s activities with an average accuracy of 95.5%. Besides, we can reduce the energy consumption during photographing by 30.0%, when compared to the approach by frequently turning ON/OFF the camera or screen. By introducing the extended Markov chain, we can reduce the energy consumption during photographing by 36.1%. 2 OBSERVATIONS ON PHOTOGRAPHING 2.1 Human Activities Related to Photographing During photographing, the users tend to have similar activities, as shown in Figure 1. Before or after the user takes photos, he/she may stay motionless or keep moving, for example, walking, jogging, and so on. While taking photos, the user usually lifts up the arm, rotates the phone, makes fine-tuning, shoots a picture, then lays down the arm. We categorize all the activities into three levels. (1) Body level: motionlessness, body movement. (2) Arm level: lifting up the arm, laying down the arm. (3) Wrist level: rotating the phone, making fine-tuning, shooting a picture. If the user wants to take multiple photos, then he/she may keep the camera working on the preview state, to take the next photo conveniently. However, it is rather energy consuming to keep the camera working. Therefore, many users tend to turn off the camera between two consecutive shots, unless he/she needs to take multiple photos in a short time. 2.2 Energy Consumption Related to Photographing According to Figure 1, before shooting a photo, there will be a preparation time, during which the user needs to move his/her locations, adjusts the position of the phone or makes fine-tuning, to find a good camera view. Obviously, if the user keeps the camera working with large preview size, it will incur much energy consumption, because the preparation time cannot be ignored. Besides, ACM Transactions on Sensor Networks, Vol. 13, No. 4, Article 29. Publication date: September 2017
Tracking Human Motions in Photographing 29:5 3200 280 多 70 240 E2000 1600 10 1200 10 60 400 P1 P2 P3 U1 U2 U3 U4 U5 U6 U7 U8 U9 U10 P1 P2 P3 (a)Energy consumption of three (b)Probability of turning ON/OFF(c)Energy consumption of built-in components camera or screen of 10 users sensors Fig.2.In(a)and(c):P1,Samsung Galaxy Nexus;P2,Samsung Galaxy S5;P3,Samsung Galaxy Note4. frequently turning on/off the camera can also incur extra energy consumption.To optimize the energy-consuming parts and propose an efficient energy-saving scheme,we first use Monsoon power monitor(Monsoon Solutions Inc.2015)to measure the energy cost in photographing. 2.2.1 Energy Consumption in Preparation for Photographing.Power consumption in preparation time of photographing is large.We observe the power consumption in the following three android- based phones,that is,Samsung Galaxy Nexus,Samsung Galaxy S5,and Samsung Galaxy Note4. In Figure 2(a),we show the power of the phone for three components,that is,"Base,""Display," and "Camera."Here,"Base"means the power when the screen is turned off and the phone stays in the idle state,that is,no app runs except for the system program."Display"represents the added power(ie.,compared with"Base")by keeping the screen on."Camera"represents the added power (i.e.,compared with"Display"and "Base")by keeping the camera working in preview mode with default settings.During the experiment,we measure the energy consumption in an office with the same light conditions,while each phone adjusts its screen brightness in an automatic way. According to Figure 2(a),the power of keeping the screen on is dozens of times larger than that of "Base,"while the power of keeping the camera working is several times of that of"Display." Therefore,when keeping the camera working in preview mode,much energy will be wasted in preparation for photographing.However,if we can recognize the user's activities and detect the preparation time accurately,we can decrease the screen brightness,preview size,preview frame rate,and so on,to reduce the unnecessary energy cost. 2.2.2 Energy Consumption of Turning ON/OFF the Camera.Frequently turning ON/OFF the cam- era and screen is annoying and wasting energy.Usually,when the user needs to take multiple photos in a period of time,he/she tends to turn ON/OFF the camera or screen frequently for energy sav- ing,instead of keeping the camera working all the time.To know how users adopt the common energy-saving schemes,that is,turning ON/OFF the camera or screen,we invite 10 users to take photos freely in our campus for 20min.For each user,when he/she takes photos,there will be an observer,who will record the user's behavior,that is,the number of times for taking photos,turn- ing ON/OFF camera,turning ON/OFF screen,turning ON/OFF camera and screen.Here,"turning ON/OFF"means a pair of operations,that is,turn off and turn on.For example,turning off the camera and turning it on again means one time for turning ON/OFF camera. In regard to the three energy-saving schemes,"Turn ON/OFF Camera"means the user turns off the camera after photographing and turning on the camera again for retaking photos.In the process,the user does not turn on/off the screen."Turn ON/OFF Screen"means the user turns off the screen after photographing and turning on the screen again for retaking photos.Considering the privacy issues,users usually need to unlock the screen(e.g.,enter password),when they turn ACM Transactions on Sensor Networks,Vol.13,No.4,Article 29.Publication date:September 2017
Tracking Human Motions in Photographing 29:5 Fig. 2. In (a) and (c): P1, Samsung Galaxy Nexus; P2, Samsung Galaxy S5; P3, Samsung Galaxy Note4. frequently turning on/off the camera can also incur extra energy consumption. To optimize the energy-consuming parts and propose an efficient energy-saving scheme, we first use Monsoon power monitor (Monsoon Solutions Inc. 2015) to measure the energy cost in photographing. 2.2.1 Energy Consumption in Preparation for Photographing. Power consumption in preparation time of photographing is large. We observe the power consumption in the following three androidbased phones, that is, Samsung Galaxy Nexus, Samsung Galaxy S5, and Samsung Galaxy Note4. In Figure 2(a), we show the power of the phone for three components, that is, “Base,” “Display,” and “Camera.” Here, “Base” means the power when the screen is turned off and the phone stays in the idle state, that is, no app runs except for the system program. “Display” represents the added power (i.e., compared with “Base”) by keeping the screen on. “Camera” represents the added power (i.e., compared with “Display” and “Base”) by keeping the camera working in preview mode with default settings. During the experiment, we measure the energy consumption in an office with the same light conditions, while each phone adjusts its screen brightness in an automatic way. According to Figure 2(a), the power of keeping the screen on is dozens of times larger than that of “Base,” while the power of keeping the camera working is several times of that of “Display.” Therefore, when keeping the camera working in preview mode, much energy will be wasted in preparation for photographing. However, if we can recognize the user’s activities and detect the preparation time accurately, we can decrease the screen brightness, preview size, preview frame rate, and so on, to reduce the unnecessary energy cost. 2.2.2 Energy Consumption of Turning ON/OFF the Camera. Frequently turning ON/OFF the camera and screen is annoying and wasting energy. Usually, when the user needs to take multiple photos in a period of time, he/she tends to turn ON/OFF the camera or screen frequently for energy saving, instead of keeping the camera working all the time. To know how users adopt the common energy-saving schemes, that is, turning ON/OFF the camera or screen, we invite 10 users to take photos freely in our campus for 20min. For each user, when he/she takes photos, there will be an observer, who will record the user’s behavior, that is, the number of times for taking photos, turning ON/OFF camera, turning ON/OFF screen, turning ON/OFF camera and screen. Here, “turning ON/OFF” means a pair of operations, that is, turn off and turn on. For example, turning off the camera and turning it on again means one time for turning ON/OFF camera. In regard to the three energy-saving schemes, “Turn ON/OFF Camera” means the user turns off the camera after photographing and turning on the camera again for retaking photos. In the process, the user does not turn on/off the screen. “Turn ON/OFF Screen” means the user turns off the screen after photographing and turning on the screen again for retaking photos. Considering the privacy issues, users usually need to unlock the screen (e.g., enter password), when they turn ACM Transactions on Sensor Networks, Vol. 13, No. 4, Article 29. Publication date: September 2017