IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 RF-3DScan:RFID-based 3D Reconstruction on Tagged Packages Yanling Bu,Student Member,IEEE,Lei Xie,Member,IEEE,Yinyin Gong,Jia Liu,Member,IEEE, Bingbing He,Jiannong Cao,Fellow,IEEE,Baoliu Ye,Member,IEEE,and Sanglu Lu,Member,IEEE Abstract-Currently,the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods,which are restricted to the line-of-sight or the battery life.Therefore,we propose a passive RFID-based solution,called RF-3DScan,to perform 3D reconstruction on tagged packages,including the package orientation and the package stacking.The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning.Through extracting phase differences to build angle profiles for each tag,RF-3DScan derives their relative positions,further determines the package orientation and the coarse-grained package stacking.By simply performing the 2D scanning,RF-3DScan can provide the fine-grained package stacking determination.We implement a prototype system of RF-3DScan and evaluate its performance in real settings.Our experiment results show that RF-3DScan can achieve about 92.5%identification accuracy of the bottom face,and average error about 4.08 of the rotation angle. For the package stacking,1D scanning can achieve the comparable performance in comparison with 2D scanning. Index Terms-RFID,3D reconstruction,package orientation,package stacking. INTRODUCTION URRENTLY,in the logistic industry,traditional applica- tions like the warehouse management and the logistic transportation,are emerging with brand new requirements. For instance,considering the safety and space utilization Package <right Tag issues,packages are required to be placed based on specified Stacking left Tagged Package regulations.Specifically,with regard to a single package,if it contains orientation-sensitive objects,i.e.,chemical reagents, precision instruments,it is protected from getting rollover or upside down.While with regard to multiple packages,to Antenna ensure the package safety during the transportation process, Package unaligned Orientation rollover Linear Mobile Scanning they are required to be precisely arranged in order,i.e., Linear Track heavy packages are placed on the bottom and light ones are on the top.To satisfy the above requirements,3D reconstruc- Fig.1.3D reconstruction on tagged packages via linear mobile scanning tion has been introduced to handle these issues for monitor- ing the package placement.Generally,3D reconstruction is Previous 3D reconstruction solutions mainly utilize com- a process of capturing the shape and appearance of a single puter vision or sensor-based methods.Computer vision- or multiple real objects.Fig.1 shows the principle of 3D based solutions capture the appearance of objects with cam- reconstruction on packaged objects:1)Package orientation eras,and then build 3D profiles of objects [1,2].They can of a single object,which means determining the relative reconstruct objects in a vivid way.However,they suffer from orientation of each object,i.e.,pinpointing the bottom/top the line-of-sight constraint,easily leading to blind angles face and estimating angles of vertical sides of the object in when capturing images.Sensor-based approaches attach the specified coordinate system.2)Package stacking of mul- inertial sensors onto items so as to monitor the orientation tiple objects,which means determining the relative stacking variation of targets [3,4].However,the main disadvantages situation of multiple packages,i.e.,figuring out the up- of them are the high hardware cost and the limited battery down,front-back or left-right relationships among objects. life of sensors.Thankfully,the promising RFID technology has brought great chances for the 3D reconstruction on packaged objects in the logistic industry.Nowadays,passive Yanling Bu,Lei Xie,Yinyin Gong,Jia Liu,Bingbing He,Baoliu Ye, and Sanglu Lu are with the State Key Laboratory for Novel Software RFID tags have been broadly used to label packages with Technology,Nanjing University,China. detailed logistics information.Compared to the above two E-mail: yanling@smail.nju.edu.cn, Ixie@nju.edu.cn, yy- approaches,the passive RFID tag is battery-free and cheap. gong@dislab.nju.edu.cn,jialiu@nju.edu.cn, hebb@dislab.nju.edu.cn, yebl@nju.edu.cn,sanglu@nju.edu.cn Also,RFID technology uses the backscatter communication, Jiannong Cao is with the Department of Computing,The Hong Kong so it has no requirement of the line of sight or the light Polytechnic University,Hong Kong,China. condition.Most importantly,in order to scan and identify E-mail:csjcao@comp.polyu.edu.hk. packages,RFID systems have been already widely deployed Lei Xie and Baoliu Ye are the co-corresponding authors. in the sites for most logistic applications in our daily life
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1 RF-3DScan: RFID-based 3D Reconstruction on Tagged Packages Yanling Bu, Student Member, IEEE, Lei Xie, Member, IEEE, Yinyin Gong, Jia Liu, Member, IEEE, Bingbing He, Jiannong Cao, Fellow, IEEE, Baoliu Ye, Member, IEEE, and Sanglu Lu, Member, IEEE Abstract—Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08◦ of the rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning. Index Terms—RFID, 3D reconstruction, package orientation, package stacking. ✦ 1 INTRODUCTION C URRENTLY, in the logistic industry, traditional applications like the warehouse management and the logistic transportation, are emerging with brand new requirements. For instance, considering the safety and space utilization issues, packages are required to be placed based on specified regulations. Specifically, with regard to a single package, if it contains orientation-sensitive objects, i.e., chemical reagents, precision instruments, it is protected from getting rollover or upside down. While with regard to multiple packages, to ensure the package safety during the transportation process, they are required to be precisely arranged in order, i.e., heavy packages are placed on the bottom and light ones are on the top. To satisfy the above requirements, 3D reconstruction has been introduced to handle these issues for monitoring the package placement. Generally, 3D reconstruction is a process of capturing the shape and appearance of a single or multiple real objects. Fig. 1 shows the principle of 3D reconstruction on packaged objects: 1) Package orientation of a single object, which means determining the relative orientation of each object, i.e., pinpointing the bottom/top face and estimating angles of vertical sides of the object in the specified coordinate system. 2) Package stacking of multiple objects, which means determining the relative stacking situation of multiple packages, i.e., figuring out the updown, front-back or left-right relationships among objects. • Yanling Bu, Lei Xie, Yinyin Gong, Jia Liu, Bingbing He, Baoliu Ye, and Sanglu Lu are with the State Key Laboratory for Novel Software Technology, Nanjing University, China. E-mail: yanling@smail.nju.edu.cn, lxie@nju.edu.cn, yygong@dislab.nju.edu.cn, jialiu@nju.edu.cn, hebb@dislab.nju.edu.cn, yebl@nju.edu.cn, sanglu@nju.edu.cn. • Jiannong Cao is with the Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China. E-mail: csjcao@comp.polyu.edu.hk. • Lei Xie and Baoliu Ye are the co-corresponding authors. Tag Package Orientation Package Stacking Linear Mobile Scanning Antenna Linear Track Tagged Package above below left right rollover unaligned Fig. 1. 3D reconstruction on tagged packages via linear mobile scanning Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods. Computer visionbased solutions capture the appearance of objects with cameras, and then build 3D profiles of objects [1, 2]. They can reconstruct objects in a vivid way. However, they suffer from the line-of-sight constraint, easily leading to blind angles when capturing images. Sensor-based approaches attach inertial sensors onto items so as to monitor the orientation variation of targets [3, 4]. However, the main disadvantages of them are the high hardware cost and the limited battery life of sensors. Thankfully, the promising RFID technology has brought great chances for the 3D reconstruction on packaged objects in the logistic industry. Nowadays, passive RFID tags have been broadly used to label packages with detailed logistics information. Compared to the above two approaches, the passive RFID tag is battery-free and cheap. Also, RFID technology uses the backscatter communication, so it has no requirement of the line of sight or the light condition. Most importantly, in order to scan and identify packages, RFID systems have been already widely deployed in the sites for most logistic applications in our daily life
IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 Therefore,in this paper,we propose a passive RFID- This paper presents the first study of using RFID to based 3D reconstruction approach,called RF-3DScan.As perform the 3D reconstruction on tagged packages.We shown in Fig.1,RF-3DScan aims at performing the 3D make three contributions.First,for the 3D reconstruction reconstruction on packaged objects attached with passive on packages,we attach a set of passive RFID tags onto RFID tags,including the package orientation and the pack- packages,and respectively handle issues of the package age stacking.The basic idea is that by attaching multiple orientation and the package stacking through angle profiles tags onto the surface of packages,we are capable of obtain- of tags.We build an angle-profile-based model to depict ing the orientation of each single package and the stacking the relationship between RF-signals of tags and the orien- status of multiple packages based on the backscattered RF- tation/stacking status of packages.Second,we propose a signals from these tags.RF-3DScan works as follows.We mobile scanning approach to perform the 3D reconstruction attach a set of passive RFID tags onto the package surface, of tagged packages via RFID.Generally,with the 1D mobile and leverage one mobile RFID antenna to move along the scanning,we can determine the package orientation and straight line to continuously scan the tagged packages.With coarse-grained package stacking;while with the 2D mobile the mobile scanning,we collect RF-signals from tags when scanning,we can determine the fine-grained package stack- the antenna is at different positions.Then,we extract phase ing.Third,We implement a prototype system of RF-3DScan differences of tags at different time points,and build angle to evaluate its performance.Our experiment results in real profiles for each tag to depict the geometry angle variation settings show that RF-3DScan can achieve about 92.5% between antenna-tag pairs during the moving process.Re- identification accuracy of the bottom face,and average error ferring to the angle profiles of tags,we can derive their about 4.08 of the rotation angle.The 1D scanning is much relative positions,and further determine the package place- easier to perform than the 2D scanning,while achieving the ment status,including the package orientation for each single comparable performance in terms of the package stacking. package and the package stacking for multiple packages. To realize the 3D reconstruction via RFID systems,there 2 RELATED WORK are three key challenges.The first challenge is that the 2.1 Computer Vision and Sensor-based Approach uncertain tag direction is easy to create dead zones of RFID communication.How to optimize the layout of multiple Computer-vision-based solutions mainly leverage the depth tags for avoiding dead zones and achieving the robust 3D camera to perform 3D reconstruction of multiple objects reconstruction is a key problem.To tackle this challenge,we [1,2].To avoid the blind angles in 3D reconstruction for deploy tags along three mutual orthogonal orientations,so specified objects,usually multiple depth cameras are de- that there are always some tags that can be collected by the ployed at different positions to perform multi-view recon- reader easily,which guarantees the high sampling rate and struction for their 3D models [2],or a moving depth camera reliable 3D reconstruction.The second challenge is that the is used to build the 3D models in a mobile approach [1].In existing work can only derive the 2D relative localization a word,these approaches suffer from the line-of-sight(LOS) of tag objects via once mobile scanning.How to locate the constraint in 3D perception,and they are vulnerable to the package and determine the package placement in the 3D limitation of the light intensity.Sensor-based solutions [3,4] space is still under-investigated.To tackle this challenge,we mainly attach the battery-powered sensors(such as inertial build an angle-profile model and combine this model with sensors or GPS modules)to the surface of the objects,and the priori knowledge of tag layout to sense the package continuously monitor the 3D placement of specified objects placement in the 3D space.Through once mobile linear scan- so as to track the orientation variation [3],or the stacking ning,we can extract angle profiles from phase differences to situation among multiple objects.However,they suffer from obtain position indicators and further determine the pack- the high hardware cost of sensors,as well as the limited age orientation with the known tag layout.By performing battery life of the sensor. one more scanning along the direction orthogonal to the previous one,we can combine the twice position indicators 2.2 RFID-based Approach to accurately estimate the package stacking.Although the Orientation tracking:By attaching RFID tags onto the spec- 2D scanning is a fine-grained solution for the package stack- ified object,it is possible to track the orientation variation ing,it requires the extra mobile scanning,so we propose of the object according to the variation of the corresponding a coarse-grained solution by the 1D scanning.With the RF-signals [5-10].Tagball [5]is proposed as a 3D human- known tag layout,we can localize the package via only computer interaction system,where multiple passive tags once scanning to determine the package stacking.The third are attached to a controlling ball,such that the motions challenge is that the RF-signal is likely to be distorted due of the ball from users can be detected from the phase to the tag orientation or the multi-path effect.How to select changes of multiple tags.Tagyro [6]attaches an array of effective data to ensure the performance is to be studied.To passive RFID tags as orientation sensors on the objects, tackle this challenge,we leverage phase differences to derive by transforming the runtime phase offsets between tags angle profiles,so as to eliminate the phase variation caused into the orientation angle.Compared with our RF-3DScan by the changing tag's orientation relative to the antenna system,these approaches track the orientation variation of during the mobile scanning.Furthermore,at the start or end the dynamically moving objects,whereas our approach aims of the scanning,as the antenna is relatively far from the tag, to determine the orientation of statically placed packages. the RF-signal is more seriously distorted,so we propose an Localization:RFID localization generally falls into two adaptive algorithm to automatically filter outliers and keep categories:absolute localization [11-17]and relative local- remaining data for the later estimation. ization [18-25].By attaching multiple tags and pinpointing
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 2 Therefore, in this paper, we propose a passive RFIDbased 3D reconstruction approach, called RF-3DScan. As shown in Fig. 1, RF-3DScan aims at performing the 3D reconstruction on packaged objects attached with passive RFID tags, including the package orientation and the package stacking. The basic idea is that by attaching multiple tags onto the surface of packages, we are capable of obtaining the orientation of each single package and the stacking status of multiple packages based on the backscattered RFsignals from these tags. RF-3DScan works as follows. We attach a set of passive RFID tags onto the package surface, and leverage one mobile RFID antenna to move along the straight line to continuously scan the tagged packages. With the mobile scanning, we collect RF-signals from tags when the antenna is at different positions. Then, we extract phase differences of tags at different time points, and build angle profiles for each tag to depict the geometry angle variation between antenna-tag pairs during the moving process. Referring to the angle profiles of tags, we can derive their relative positions, and further determine the package placement status, including the package orientation for each single package and the package stacking for multiple packages. To realize the 3D reconstruction via RFID systems, there are three key challenges. The first challenge is that the uncertain tag direction is easy to create dead zones of RFID communication. How to optimize the layout of multiple tags for avoiding dead zones and achieving the robust 3D reconstruction is a key problem. To tackle this challenge, we deploy tags along three mutual orthogonal orientations, so that there are always some tags that can be collected by the reader easily, which guarantees the high sampling rate and reliable 3D reconstruction. The second challenge is that the existing work can only derive the 2D relative localization of tag objects via once mobile scanning. How to locate the package and determine the package placement in the 3D space is still under-investigated. To tackle this challenge, we build an angle-profile model and combine this model with the priori knowledge of tag layout to sense the package placement in the 3D space. Through once mobile linear scanning, we can extract angle profiles from phase differences to obtain position indicators and further determine the package orientation with the known tag layout. By performing one more scanning along the direction orthogonal to the previous one, we can combine the twice position indicators to accurately estimate the package stacking. Although the 2D scanning is a fine-grained solution for the package stacking, it requires the extra mobile scanning, so we propose a coarse-grained solution by the 1D scanning. With the known tag layout, we can localize the package via only once scanning to determine the package stacking. The third challenge is that the RF-signal is likely to be distorted due to the tag orientation or the multi-path effect. How to select effective data to ensure the performance is to be studied. To tackle this challenge, we leverage phase differences to derive angle profiles, so as to eliminate the phase variation caused by the changing tag’s orientation relative to the antenna during the mobile scanning. Furthermore, at the start or end of the scanning, as the antenna is relatively far from the tag, the RF-signal is more seriously distorted, so we propose an adaptive algorithm to automatically filter outliers and keep remaining data for the later estimation. This paper presents the first study of using RFID to perform the 3D reconstruction on tagged packages. We make three contributions. First, for the 3D reconstruction on packages, we attach a set of passive RFID tags onto packages, and respectively handle issues of the package orientation and the package stacking through angle profiles of tags. We build an angle-profile-based model to depict the relationship between RF-signals of tags and the orientation/stacking status of packages. Second, we propose a mobile scanning approach to perform the 3D reconstruction of tagged packages via RFID. Generally, with the 1D mobile scanning, we can determine the package orientation and coarse-grained package stacking; while with the 2D mobile scanning, we can determine the fine-grained package stacking. Third, We implement a prototype system of RF-3DScan to evaluate its performance. Our experiment results in real settings show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08◦ of the rotation angle. The 1D scanning is much easier to perform than the 2D scanning, while achieving the comparable performance in terms of the package stacking. 2 RELATED WORK 2.1 Computer Vision and Sensor-based Approach Computer-vision-based solutions mainly leverage the depth camera to perform 3D reconstruction of multiple objects [1, 2]. To avoid the blind angles in 3D reconstruction for specified objects, usually multiple depth cameras are deployed at different positions to perform multi-view reconstruction for their 3D models [2], or a moving depth camera is used to build the 3D models in a mobile approach [1]. In a word, these approaches suffer from the line-of-sight (LOS) constraint in 3D perception, and they are vulnerable to the limitation of the light intensity. Sensor-based solutions [3, 4] mainly attach the battery-powered sensors (such as inertial sensors or GPS modules) to the surface of the objects, and continuously monitor the 3D placement of specified objects, so as to track the orientation variation [3], or the stacking situation among multiple objects. However, they suffer from the high hardware cost of sensors, as well as the limited battery life of the sensor. 2.2 RFID-based Approach Orientation tracking: By attaching RFID tags onto the specified object, it is possible to track the orientation variation of the object according to the variation of the corresponding RF-signals [5–10]. Tagball [5] is proposed as a 3D humancomputer interaction system, where multiple passive tags are attached to a controlling ball, such that the motions of the ball from users can be detected from the phase changes of multiple tags. Tagyro [6] attaches an array of passive RFID tags as orientation sensors on the objects, by transforming the runtime phase offsets between tags into the orientation angle. Compared with our RF-3DScan system, these approaches track the orientation variation of the dynamically moving objects, whereas our approach aims to determine the orientation of statically placed packages. Localization: RFID localization generally falls into two categories: absolute localization [11–17] and relative localization [18–25]. By attaching multiple tags and pinpointing
IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 each tag's 3D coordinates,the absolute localization can be 45 tailored to our problem for 3D reconstruction.However,this approach suffers from the complicated system deployment or the high computational complexity.For example,the state-of-the-art absolute localization schemes Tagoram [13] and RFind [15]are able to achieve the cm-level localization accuracy,however,they require either high computation overhead or dedicated device calibration,which are unfit 60120180240300360 Rotation (deg.) for estimating many packages concurrently.In addition,the localization work focuses on pinpointing only a single tag, (a)Rotate along the Z axis (b)Phase change during rotation how to estimate the package placement with the tag array Fig.2.Measured phase of a single tag rotating along the Z axis is still under-investigated.Rather than the absolute localiza- tion of a single tag,our approach utilizes the tag array to provide the localization result,which can achieve the com- parable performance without the above limits.Moreover, the relative localization investigates the relative locations of objects as opposed to absolute coordinates.STPP [19]is the first work to tackle the 2D relative localization.It investi- △d≈ △ds gates the spatial-temporal dynamics with phase profiles.A d x cosa d x cosa Moving V-zone (comprised of phase sequences)based solution is direction proposed to determine the relative localization of tagged A1 d P2 objects in the 2D plane.However,STPP cannot always get Fig.3.Angle-of-Arrival in static Fig.4.Angle-of-Arrival in mobile a V-zone in practice,especially when the sampling rate of scanning scanning per tag is low (due to many tags)or there are some dead zones of the RFID communication.Unlike STPP,regardless Fig.2 plots the phase change when a tag rotates along the of the V-zone,our work takes full advantages of all phase Z axis.It shows that the phase varies continuously over the measurements for localization and 3D reconstruction. rotation.Next,we discuss how to use the angle-of-arrival approach to overcome above three challenges,and benefit ANGLE-PROFILE-BASED MODELING our system design in the sequel. In this section,we first discuss the limitations of directly using phase values,and introduce how to use the phase dif- 3.2 Angle Profile ference to model the angle profile for the 3D reconstruction. Angle-of-Arrival (AoA)is one of the most popular RF-based localization measurements using phase difference.The basic idea of our approach is that by moving the antenna to scan 3.1 Limitations of Phase-based Measurement the tags,we extract phase differences from the specified tags The RF phase is a widely used attribute of the wireless at different time points,then we derive the geometry angles signal that reflects the phase offset between the received between the tag and the mobile antenna when the antenna electromagnetic wave and the emitted one,ranging from is at different positions,which is called angle profile. 0 to 2m.Due to the ultra-high working frequency in RFIDs and fine-grained measurement resolution of phase values by 3.2.1 Angle in Static Scanning COTS readers,the phase is very sensitive to the distance be- As shown in Fig.3,a tag is set at T,A and A2 are two tween the antenna and the tag,which gives us the potential antennas separated by d,M is the middle point of A1A2.V chance to achieve the accurate 3D reconstruction.Suppose is the projected point of T on the antenna pair line A1A2, s is the distance between the antenna and the tag.Since the the perpendicular distance is h.The included angle between backscatter communication of RFID is round-trip,the signal line TM and line MV is the AoA for tag T at position M, totally traverses a distance of 2s in each communication. denoted as a.Let dr.A and dr.A2 represent the distances Besides the distance,some hardware characteristics will also between T and the antennas,the antennas collect the phases distort the phase value.Hence,the phase6 reported by the as 0A:and 0A2,respectively.0A1,0A2 E [0,2). reader can be expressed as: The phase difference is related to the distance difference 2π from the tag to the antennas.When h>d,the relationship ×2s+7 mod2π between the phase difference (A0=0A:-0A2+n,0n means the phase offset caused by the hardware characteristics of A where A is the wavelength,n represents the phase offset and A2)and the distance difference (Ad dr.A:-dr.A2 caused by the hardware characteristics. dcosa)can be approximated as: Although the phase accurately reflects the distance,we face three challenges before putting it into use:1)The distort 2dcosaA0 (1) 入 2π +n, factor n caused by the physical hardware is unknown;2)The phase value repeats periodically,it is not feasible to use it wherencan be any integer in【-头-尝，装-L,its directly;3)In addition to s and n,our extensive experiments range isd.Whend<,the value range ofn is smaller than show that the tag orientation influences the phase value 6. 1,which means n has a unique value,so a is deterministic
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 3 each tag’s 3D coordinates, the absolute localization can be tailored to our problem for 3D reconstruction. However, this approach suffers from the complicated system deployment or the high computational complexity. For example, the state-of-the-art absolute localization schemes Tagoram [13] and RFind [15] are able to achieve the cm-level localization accuracy, however, they require either high computation overhead or dedicated device calibration, which are unfit for estimating many packages concurrently. In addition, the localization work focuses on pinpointing only a single tag, how to estimate the package placement with the tag array is still under-investigated. Rather than the absolute localization of a single tag, our approach utilizes the tag array to provide the localization result, which can achieve the comparable performance without the above limits. Moreover, the relative localization investigates the relative locations of objects as opposed to absolute coordinates. STPP [19] is the first work to tackle the 2D relative localization. It investigates the spatial-temporal dynamics with phase profiles. A V-zone (comprised of phase sequences) based solution is proposed to determine the relative localization of tagged objects in the 2D plane. However, STPP cannot always get a V-zone in practice, especially when the sampling rate of per tag is low (due to many tags) or there are some dead zones of the RFID communication. Unlike STPP, regardless of the V-zone, our work takes full advantages of all phase measurements for localization and 3D reconstruction. 3 ANGLE-PROFILE-BASED MODELING In this section, we first discuss the limitations of directly using phase values, and introduce how to use the phase difference to model the angle profile for the 3D reconstruction. 3.1 Limitations of Phase-based Measurement The RF phase is a widely used attribute of the wireless signal that reflects the phase offset between the received electromagnetic wave and the emitted one, ranging from 0 to 2π. Due to the ultra-high working frequency in RFIDs and fine-grained measurement resolution of phase values by COTS readers, the phase is very sensitive to the distance between the antenna and the tag, which gives us the potential chance to achieve the accurate 3D reconstruction. Suppose s is the distance between the antenna and the tag. Since the backscatter communication of RFID is round-trip, the signal totally traverses a distance of 2s in each communication. Besides the distance, some hardware characteristics will also distort the phase value. Hence, the phase θ reported by the reader can be expressed as: θ = 2π λ × 2s + η mod 2π, where λ is the wavelength, η represents the phase offset caused by the hardware characteristics. Although the phase accurately reflects the distance, we face three challenges before putting it into use: 1) The distort factor η caused by the physical hardware is unknown; 2) The phase value repeats periodically, it is not feasible to use it directly; 3) In addition to s and η, our extensive experiments show that the tag orientation influences the phase value θ. ͵Ͳι Ͳι Ͳ ι ͻͲι Ͳ ι ͵Ͳι Ͳι ͻͲι (a) Rotate along the Z axis 0 60 120 180 240 300 360 Rotation (deg.) 1 1.5 2 2.5 3 3.5 4 4.5 Phase (rad.) (b) Phase change during rotation Fig. 2. Measured phase of a single tag rotating along the Z axis 1 2 -. 3 . × cos2 )" )& 4 5 6 . Fig. 3. Angle-of-Arrival in static scanning ) 1 2 -. 3 . × cos2 " & 4 5 . 6789:; direction Fig. 4. Angle-of-Arrival in mobile scanning Fig. 2 plots the phase change when a tag rotates along the Z axis. It shows that the phase varies continuously over the rotation. Next, we discuss how to use the angle-of-arrival approach to overcome above three challenges, and benefit our system design in the sequel. 3.2 Angle Profile Angle-of-Arrival (AoA) is one of the most popular RF-based localization measurements using phase difference. The basic idea of our approach is that by moving the antenna to scan the tags, we extract phase differences from the specified tags at different time points, then we derive the geometry angles between the tag and the mobile antenna when the antenna is at different positions, which is called angle profile. 3.2.1 Angle in Static Scanning As shown in Fig. 3, a tag is set at T, A1 and A2 are two antennas separated by d, M is the middle point of A1A2. V is the projected point of T on the antenna pair line A1A2, the perpendicular distance is h. The included angle between line TM and line MV is the AoA for tag T at position M, denoted as α. Let dT ,A1 and dT ,A2 represent the distances between T and the antennas, the antennas collect the phases as θA1 and θA2 , respectively. θA1 , θA2 ∈ [0, 2π). The phase difference is related to the distance difference from the tag to the antennas. When h d, the relationship between the phase difference (∆θ = θA1−θA2+θη, θη means the phase offset caused by the hardware characteristics of A1 and A2) and the distance difference (∆d = dT ,A1 − dT ,A2 ' d cos α) can be approximated as: 2d cos α λ = ∆θ 2π + n, (1) where n can be any integer in − 2d λ − ∆θ 2π , 2d λ − ∆θ 2π , its range is 4d λ . When d < λ 4 , the value range of n is smaller than 1, which means n has a unique value, so α is deterministic.
IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 Perpendicular cota distance T2 Equal angle point T 0 T Perpendkcular point Ty Moving a Moving distance OEqual angle Perpendicular A distance Fig.6.Model of the angle profile point point Fig.5.Metrics of the angle profile 3.4 Model of Angle Profile To depict the angle-profile-based measurement metrics in 3.2.2 Angle in Mobile Scanning mathematics,we build a linear model to derive the metrics As for multiple antennas,the phase offsets related to their from the angle profile automatically.Considering Fig.4,the own hardware characteristics are different,so it is hard to angle-of-arrival can be expressed as: determine 0n.Hence,we prefer a mobile antenna to multiple static antennas,in which case can be canceled. cot a= V-A (2) h For a mobile antenna,the angle-of-arrival is a little different.Without the loss of generality,we redefine the AoA where cot means the cotangent function,h is the perpen- in a mobile case,as shown in Fig.4.Similarly,T is the tag dicular distance between the tag and the antenna moving position and V is its projected point on the antenna moving trace.yA and yv represent the coordinates of point A and line,its perpendicular distance is h.Let the mobile antenna V along the antenna moving direction.Assume there is an be at position A,then the included angle of line TA and the antenna starting point S,the distance from S to V is lo,the antenna moving direction is just the angle-of-arrival (a)for antenna moved distance is l.Thus,(lo-l)represents the the tag when the antenna is at position A. distance from the antenna to the perpendicular point(same To estimate the angle at position A,we only need the as (yv -yA)),the angle can be rewritten as: phases collected at the two nearby positions(P and P2), 1 centered on the antenna(P1A AP2).Thus,the phase cota=L+b,k=-6s场 h (3) difference at position P and P2 can be used to estimate where the slope k is related to the minus reciprocal of h,the a with Eq.(1).By combining the angles at different antenna intercept b depends on the ratio of lo and h. positions,we can derive an angle profile for a specified tag. Taking the tags in Fig.5,the transformed angle expres- sion based on Eg.(3)should look like the lines shown 3.3 Metrics of Angle Profile in Fig.6.As l increases continuously during the moving process,a increases as well.When the antenna reaches the Suppose there are two tags and one antenna in the same perpendicular point,a is equal to /2,so cot a=0.The line plane (Fig.5).The antenna moves linearly from O to A, of T reaches 0 earlier than T2.Thus,the order of such zero so it passes through Ti first,followed by T2.When the points are corresponding to the tags'perpendicular points, antenna passes through the tag(corresponding to point V and the spacing between two zero points just reflects the in Fig.4),the angle-of-arrival (a)of that tag reaches /2, separation of tags'perpendicular points.In addition,the naming this point as the perpendicular point.Similarly,we call intersection of the two lines represents the position where the distance from the tag to the perpendicular point perpen- the tags are projected on the same line with the antenna, dicular distance,the direction perpendicular to the antenna corresponding to the equal angle point.Specifically,the moving direction as perpendicular direction.As T is on the smaller h is,the larger is,and the sharper the line is. left along the antenna moving direction,its perpendicular As the h of Ti is smaller than T2,the of Ti is larger,so point shows earlier than T2.Hence,the perpendicular point the line of n decreases faster than T2. is the key metric for the tags'relative positions along the For a certain tag,its angle profile records its angles moving direction. at different positions,as {(cot ai,li)},i=1,2,...,n,n Besides the perpendicular point,there is the other special represents the amount of samples.As described in Section point:equal angle point.The equal angle point is where the 3.2.2,it is easy to obtain cot a;at a specific position with the antenna and the two tags are in the same line,so Ti and separation distance and the phase difference of two nearby T2 share the same angle.Before equal point,the angle of positions,here comes a new question,how to determine T1 is smaller than the angle of T2.On the contrary,the the location of the antenna during the moving process? angle of Ti changes to be bigger than that of T2 after the Actually,we only care about the relative position of the equal angle point.No matter for Ti or T2,its angle increases antenna along the linear scanning direction when it collects continuously during the antenna moving process,so it is data,we need not the absolute position of the antenna in obvious that the angle of Ti changes faster than that of the 3D space,but we require to know the relative moving T2.Such phenomenon is due to the smaller perpendicular distance along the scanning direction so as to determine the distance of T1.Thus,according to the angle change rate, value of li in the angle profile.Note that,it is the relative we can determine the tags'relative positions along the positions among tags that matters,so we can randomly set perpendicular direction. a position along the linear scanning direction as the starting
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 4 ܶଵ ߙ ߙ ߙ ᇱ ᇱᇱ ଵߙ ଶߙ ଵߙ Ԣ ଶߙ Ԣ ݄ଶ ݄ଵ ܶଶ ܶଵ ܣ ܱ ܶଶ ܶଵ ߙ Ͳ Fig. 5. Metrics of the angle profile 3.2.2 Angle in Mobile Scanning As for multiple antennas, the phase offsets related to their own hardware characteristics are different, so it is hard to determine θη. Hence, we prefer a mobile antenna to multiple static antennas, in which case θη can be canceled. For a mobile antenna, the angle-of-arrival is a little different. Without the loss of generality, we redefine the AoA in a mobile case, as shown in Fig. 4. Similarly, T is the tag position and V is its projected point on the antenna moving line, its perpendicular distance is h. Let the mobile antenna be at position A, then the included angle of line T A and the antenna moving direction is just the angle-of-arrival (α) for the tag when the antenna is at position A. To estimate the angle at position A, we only need the phases collected at the two nearby positions (P1 and P2), centered on the antenna (P1A = AP2). Thus, the phase difference at position P1 and P2 can be used to estimate α with Eq. (1). By combining the angles at different antenna positions, we can derive an angle profile for a specified tag. 3.3 Metrics of Angle Profile Suppose there are two tags and one antenna in the same plane (Fig. 5). The antenna moves linearly from O to A, so it passes through T1 first, followed by T2. When the antenna passes through the tag (corresponding to point V in Fig. 4), the angle-of-arrival (α) of that tag reaches π/2, naming this point as the perpendicular point. Similarly, we call the distance from the tag to the perpendicular point perpendicular distance, the direction perpendicular to the antenna moving direction as perpendicular direction. As T1 is on the left along the antenna moving direction, its perpendicular point shows earlier than T2. Hence, the perpendicular point is the key metric for the tags’ relative positions along the moving direction. Besides the perpendicular point, there is the other special point: equal angle point. The equal angle point is where the antenna and the two tags are in the same line, so T1 and T2 share the same angle. Before equal point, the angle of T1 is smaller than the angle of T2. On the contrary, the angle of T1 changes to be bigger than that of T2 after the equal angle point. No matter for T1 or T2, its angle increases continuously during the antenna moving process, so it is obvious that the angle of T1 changes faster than that of T2. Such phenomenon is due to the smaller perpendicular distance of T1. Thus, according to the angle change rate, we can determine the tags’ relative positions along the perpendicular direction. ܶଵ ߙ ߙ ߙ ᇱ ᇱᇱ ଵߙ ଶߙ ଵߙ Ԣ ଶߙ Ԣ ݄ଶ ݄ଵ ܶଶ ܶଵ ܣ ܱ ܶଶ ܶଵ ߙ Ͳ Fig. 6. Model of the angle profile 3.4 Model of Angle Profile To depict the angle-profile-based measurement metrics in mathematics, we build a linear model to derive the metrics from the angle profile automatically. Considering Fig. 4, the angle-of-arrival can be expressed as: cot α = yV − yA h , (2) where cot means the cotangent function, h is the perpendicular distance between the tag and the antenna moving trace. yA and yV represent the coordinates of point A and V along the antenna moving direction. Assume there is an antenna starting point S, the distance from S to V is l0, the antenna moved distance is l. Thus, (l0 − l) represents the distance from the antenna to the perpendicular point (same as (yV − yA)), the angle can be rewritten as: cot α = kl + b, k = − 1 h , b = l0 h , (3) where the slope k is related to the minus reciprocal of h, the intercept b depends on the ratio of l0 and h. Taking the tags in Fig. 5, the transformed angle expression based on Eq. (3) should look like the lines shown in Fig. 6. As l increases continuously during the moving process, α increases as well. When the antenna reaches the perpendicular point, α is equal to π/2, so cot α = 0. The line of T1 reaches 0 earlier than T2. Thus, the order of such zero points are corresponding to the tags’ perpendicular points, and the spacing between two zero points just reflects the separation of tags’ perpendicular points. In addition, the intersection of the two lines represents the position where the tags are projected on the same line with the antenna, corresponding to the equal angle point. Specifically, the smaller h is, the larger kkk is, and the sharper the line is. As the h of T1 is smaller than T2, the kkk of T1 is larger, so the line of T1 decreases faster than T2. For a certain tag, its angle profile records its angles at different positions, as {(cot αi , li)}, i = 1, 2, ..., n, n represents the amount of samples. As described in Section 3.2.2, it is easy to obtain cot αi at a specific position with the separation distance and the phase difference of two nearby positions, here comes a new question, how to determine the location of the antenna during the moving process? Actually, we only care about the relative position of the antenna along the linear scanning direction when it collects data, we need not the absolute position of the antenna in the 3D space, but we require to know the relative moving distance along the scanning direction so as to determine the value of li in the angle profile. Note that, it is the relative positions among tags that matters, so we can randomly set a position along the linear scanning direction as the starting
EEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 point.Then based on the moving distance from the starting Moving point,we get the value of li at any time ti.After extracting direction h2 the angle profile,referring to Eq.(3),we are able to estimate T2 the two unknown parameters h and lo through the linear fitting method.Specifically,h depends on the perpendicular h distance between the tag and the antenna moving trace. lo is related to the projected position where the antenna passes through the tag,the larger lo is,the later that line reaches 0,and the tag is more ahead along the antenna moving direction.Thus,by leveraging these properties,we Fig.7. Localization based on the tag array can determine tags'relative positions with the following two principles: Preprocess Determine Determine 1) The value of reflects the perpendicular distance package orientation package stacking Angle computation h from the tag to the antenna moving trace:the RF (single) (multiple) larger,the smaller the perpendicular distance. Signals Angle smoothing 2) The value of lo determines the projected position ·Bottom/top face Linear fitting Relative stacking of the corresponding tag along the antenna moving ·Rotation angle situation direction.The difference of lo between two tags indi- Fig.8.Architecture of RF-3DScan cates their interval in the antenna moving direction. Product Code (EPC)of each attached tag.Based on the 3.5 Localization based on Angle Profile with Tag Array tag's EPC from RF-signals,it is easy to obtain the extensive corresponding package information.Specially,the tagged With angle profiles of each tag,we obtain the perpendicular packages ought to be produced by machines in the real distance and projected position along the scanning direction logistic industry,that is,the priori knowledge is determined of each tag.The perpendicular distance is related to the position of the tag,including the height difference and the by the designer,so it is easy to obtain such priori knowledge without extra labors for acquisition.Aiming at using as depth of the tag from the antenna.Although it is hard to fewer tags as possible to depict the package uniquely,accu- decompose the perpendicular distance to get the height rately and conveniently,the tag deployment should obey the difference and depth of each tag,we can localize each tag two design rules in Section 4.3.1.Meanwhile,we make the by taking these tags as a whole,which has the known tag following assumptions:1)The antenna moves at a constant separation distances along each axes in a certain coordinate speed;2)Each package is a standard cube,and they are fully system.Specifically,taking a tag array with two tags for on the ground or parallel to the ground (on the ground is a example,if the separation distances of two tags along the special case of parallel to the ground). orthogonal directions of the antenna scanning direction are Fig.8 illustrates the architecture of RF-3DScan.RF- known,the positions of the two tags can be obtained based 3DScan takes RF-signals from the tags as input,then out- on their perpendicular distances.As shown in Fig.7,the puts 3D profiles for multiple packages.The whole system antenna moves along the Y axis,the tag separation distances consists of three components:1)Preprocess:With RF-signals of Ti and T2 along the X and Z axes are△rand△z, from the tags,RF-3DScan builds angle profiles by using and the perpendicular distances of Ti and T2 are hi and the phase differences at different time points for each tag, h2,respectively.Assume the depth and height difference and extracts position indicators from angle profiles for the from Ti to the antenna moving trace are d and d:,the relative localization among tags by linear fitting.2)Deter- perpendicular distances can be represented as: mine package orientation for a single package:By comparing h好=d+d, the relative positions of tags on a specified package,RF- (4) 3DScan can determine which side of the package is on the h=(d+△x)2+(d2-△z)2, ground,and then evaluates the angle of the vertical sides in where hi and h2 are extracted from angle profiles of each a specified coordinate system.3)Determine package stacking tag,Ax and Az are known according to the tag array layout for multiple packages:After deriving the orientation of a single and the package orientation,so (dr,d=)can be computed package,the centers of the packages are also determined using Eq.(4).Meanwhile,as the projected point of the tag along the scanning direction.By performing a 2D mobile along the scanning direction is extracted from the angle pro- scanning,RF-3DScan combines the results from the two file,its position along the Y axis is obtained,so the relative orthogonal scanning,so the accurate relative positions of position of the tag in the 3D space is totally determined. these packages in the 3D space can be determined.Note that,based on the tag array localization,using only 1D scanning can also achieve the comparable performance. SYSTEM DESIGN 4.1 System Overview 4.2 Data Preprocess RF-3DScan is a 3D reconstruction system for tagged pack-With raw RF-signals,we need to process the collected data ages via RFID.In RF-3DScan,the information of each pack- and build angle profiles of tags.The preprocessing can be age is priori and stored in the database individually,includ-divided into three steps:angle computation,angle smooth- ing the package size,the position and unique Electronic ing and linear fitting
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 5 point. Then based on the moving distance from the starting point, we get the value of li at any time ti . After extracting the angle profile, referring to Eq. (3), we are able to estimate the two unknown parameters h and l0 through the linear fitting method. Specifically, h depends on the perpendicular distance between the tag and the antenna moving trace. l0 is related to the projected position where the antenna passes through the tag, the larger l0 is, the later that line reaches 0, and the tag is more ahead along the antenna moving direction. Thus, by leveraging these properties, we can determine tags’ relative positions with the following two principles: 1) The value of kkk reflects the perpendicular distance h from the tag to the antenna moving trace: the larger kkk, the smaller the perpendicular distance. 2) The value of l0 determines the projected position of the corresponding tag along the antenna moving direction. The difference of l0 between two tags indicates their interval in the antenna moving direction. 3.5 Localization based on Angle Profile with Tag Array With angle profiles of each tag, we obtain the perpendicular distance and projected position along the scanning direction of each tag. The perpendicular distance is related to the position of the tag, including the height difference and the depth of the tag from the antenna. Although it is hard to decompose the perpendicular distance to get the height difference and depth of each tag, we can localize each tag by taking these tags as a whole, which has the known tag separation distances along each axes in a certain coordinate system. Specifically, taking a tag array with two tags for example, if the separation distances of two tags along the orthogonal directions of the antenna scanning direction are known, the positions of the two tags can be obtained based on their perpendicular distances. As shown in Fig. 7, the antenna moves along the Y axis, the tag separation distances of T1 and T2 along the X and Z axes are ∆x and ∆z, and the perpendicular distances of T1 and T2 are h1 and h2, respectively. Assume the depth and height difference from T1 to the antenna moving trace are dx and dz, the perpendicular distances can be represented as: ( h 2 1 = d 2 x + d 2 z , h 2 2 = (dx + ∆x) 2 + (dz − ∆z) 2 , (4) where h1 and h2 are extracted from angle profiles of each tag, ∆x and ∆z are known according to the tag array layout and the package orientation, so (dx, dz) can be computed using Eq. (4). Meanwhile, as the projected point of the tag along the scanning direction is extracted from the angle pro- file, its position along the Y axis is obtained, so the relative position of the tag in the 3D space is totally determined. 4 SYSTEM DESIGN 4.1 System Overview RF-3DScan is a 3D reconstruction system for tagged packages via RFID. In RF-3DScan, the information of each package is priori and stored in the database individually, including the package size, the position and unique Electronic Moving direction Y X Z 01 02 ℎ2 ∆4 ℎ1 67 68 ∆9 Fig. 7. Localization based on the tag array ݄ ݈ Preprocess Angle computation Linear fitting Angle smoothing RF Signals Determine package orientation (single) Determine package stacking (multiple) • Bottom / top face • Rotation angle Relative stacking situation Fig. 8. Architecture of RF-3DScan Product Code (EPC) of each attached tag. Based on the tag’s EPC from RF-signals, it is easy to obtain the extensive corresponding package information. Specially, the tagged packages ought to be produced by machines in the real logistic industry, that is, the priori knowledge is determined by the designer, so it is easy to obtain such priori knowledge without extra labors for acquisition. Aiming at using as fewer tags as possible to depict the package uniquely, accurately and conveniently, the tag deployment should obey the two design rules in Section 4.3.1. Meanwhile, we make the following assumptions: 1) The antenna moves at a constant speed; 2) Each package is a standard cube, and they are fully on the ground or parallel to the ground (on the ground is a special case of parallel to the ground). Fig. 8 illustrates the architecture of RF-3DScan. RF- 3DScan takes RF-signals from the tags as input, then outputs 3D profiles for multiple packages. The whole system consists of three components: 1) Preprocess: With RF-signals from the tags, RF-3DScan builds angle profiles by using the phase differences at different time points for each tag, and extracts position indicators from angle profiles for the relative localization among tags by linear fitting. 2) Determine package orientation for a single package: By comparing the relative positions of tags on a specified package, RF- 3DScan can determine which side of the package is on the ground, and then evaluates the angle of the vertical sides in a specified coordinate system. 3) Determine package stacking for multiple packages: After deriving the orientation of a single package, the centers of the packages are also determined along the scanning direction. By performing a 2D mobile scanning, RF-3DScan combines the results from the two orthogonal scanning, so the accurate relative positions of these packages in the 3D space can be determined. Note that, based on the tag array localization, using only 1D scanning can also achieve the comparable performance. 4.2 Data Preprocess With raw RF-signals, we need to process the collected data and build angle profiles of tags. The preprocessing can be divided into three steps: angle computation, angle smoothing and linear fitting