3-Dimensional Reconstruction on Tagged Packages via RFID Systems Yanling Bu,Lei Xie,Jia Liu,Bingbing He,Yinyin Gong and Sanglu Lu State Key Laboratory for Novel Software Technology,Nanjing University,China yanlingbu@foxmail.com,Ixie@nju.edu.cn.jialiu.cs@gmail.com, [hebb,yygong}@dislab.nju.edu.cn,sanglu@nju.edu.cn Abstract-Nowadays,3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications.Existing 3D Package reconstruction methods are mainly orientation Package based on computer vision or sensor-based approaches,which are stacking limited by the line-of-sight or battery life constraint.In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID,by attaching multiple reference tags -Antenna onto the surface of the packages.The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space,the antenna obtains the RF-signals of the Moving direction reference tags attached on the packages.By extracting the phase differences to build the angle profile for each tag,RF-3DScan can compare the angle profiles of the different reference tags and Fig.1.3D reconstruction on tagged packages via mobile scanning derive their relative positions,then further determine the package Existing 3D reconstruction methods are mainly based on orientation and stacking for 3D reconstruction.We implement computer vision or sensor-based approaches.Computer vision- RF-3DScan and evaluate its performance in real settings.The experiment results show that the average identification accuracy based approaches leverage the cameras to capture the appear- of the bottom face is about 92.5%,and the average estimation ance and build the 3D profiles of objects [1.21.They are able error of the rotation angle is about 4.08. to reconstruct the shape of objects in a vivid approach.The main disadvantage is the line-of-sight constraint in capturing I.INTRODUCTION images,which leads to blind angles in 3D reconstruction Nowadays,the traditional logistic industry-related appli- with only one fixed camera.Sensor-based approaches usually cations,such as warehouse management and logistic trans- leverage inertial sensors attached to the items to detect the portation,are emerging with brand new requirements.For orientation variation of the specified items [3,4].However, example,during the process of warehouse management or they suffer from high hardware cost of the sensors,as well logistic transportation,the packages are usually required to be as the limited battery life for the sensors.Fortunately,the placed according to some specified regulations.In particular,rising use of RFID technology in the logistic industry has in regard to a single package,if it contains orientation-sensitive brought brand new opportunities to 3D reconstruction on goods,such as chemical reagents or precision instruments, packaged objects.In current logistic industry,RFID tags have then it is prohibited from being rollover or upside down; been widely used to label the packages with exact logistics in regard to multiple packages,they are also required to be information.In comparison to the above two approaches,the precisely arranged in some specified order,e.g.,heavy objects passive RFID tag is battery-free and very cheap,and the are placed on the bottom,whereas light objects are placed backscatter-based communication from RFID is not limited on the top,to ensure safety in the transportation.To deal by the line of sight requirement.Most importantly,for most with the above requirements,the technology of 3-dimensional logistic applications,the RFID systems are already deployed (3D)reconstruction has been introduced to tackle these issues in the sites to scan and identify the tagged packages. in monitoring the package placement.3D reconstruction is a Therefore,in this paper,we propose RF-3DScan,which process of capturing the shape and appearance of a single or aims to perform 3D reconstruction on packaged objects via multiple real objects.In principle,there are two key aspects the RFID systems (in Fig.1).Our idea is based on the to realize 3D reconstruction for packaged objects:1)Package observation that by attaching multiple tags onto the surface orientation for a single object,it refers to determining the of the packages,we are able to derive the 3D orientation of relative orientation for each package,i.e..figuring out the each single package and the 3D stacking situation of multiple bottom/top face,as well as the angles of the other vertical sides packages according to the backscattered RF-signals from these for the specified object in the specified coordinate system.2) reference tags.Our approach of RF-3DScan is as follows:We Package stacking for multiple objects,it refers to determining attach a set of reference tags on the surface of the packages, the relative stacking situation for multiple packages,i.e.,then we utilize a single RFID antenna to continuously scan performing the relative localization of multiple objects. the tagged packages,while the antenna is moving along
3-Dimensional Reconstruction on Tagged Packages via RFID Systems Yanling Bu, Lei Xie, Jia Liu, Bingbing He, Yinyin Gong and Sanglu Lu State Key Laboratory for Novel Software Technology, Nanjing University, China yanlingbu@foxmail.com, lxie@nju.edu.cn, jialiu.cs@gmail.com, {hebb, yygong}@dislab.nju.edu.cn, sanglu@nju.edu.cn Abstract—Nowadays, 3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications. Existing 3D reconstruction methods are mainly based on computer vision or sensor-based approaches, which are limited by the line-of-sight or battery life constraint. In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID, by attaching multiple reference tags onto the surface of the packages. The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space, the antenna obtains the RF-signals of the reference tags attached on the packages. By extracting the phase differences to build the angle profile for each tag, RF-3DScan can compare the angle profiles of the different reference tags and derive their relative positions, then further determine the package orientation and stacking for 3D reconstruction. We implement RF-3DScan and evaluate its performance in real settings. The experiment results show that the average identification accuracy of the bottom face is about 92.5%, and the average estimation error of the rotation angle is about 4.08◦. I. INTRODUCTION Nowadays, the traditional logistic industry-related applications, such as warehouse management and logistic transportation, are emerging with brand new requirements. For example, during the process of warehouse management or logistic transportation, the packages are usually required to be placed according to some specified regulations. In particular, in regard to a single package, if it contains orientation-sensitive goods, such as chemical reagents or precision instruments, then it is prohibited from being rollover or upside down; in regard to multiple packages, they are also required to be precisely arranged in some specified order, e.g., heavy objects are placed on the bottom, whereas light objects are placed on the top, to ensure safety in the transportation. To deal with the above requirements, the technology of 3-dimensional (3D) reconstruction has been introduced to tackle these issues in monitoring the package placement. 3D reconstruction is a process of capturing the shape and appearance of a single or multiple real objects. In principle, there are two key aspects to realize 3D reconstruction for packaged objects: 1) Package orientation for a single object, it refers to determining the relative orientation for each package, i.e., figuring out the bottom/top face, as well as the angles of the other vertical sides for the specified object in the specified coordinate system. 2) Package stacking for multiple objects, it refers to determining the relative stacking situation for multiple packages, i.e., performing the relative localization of multiple objects. 7DJ $QWHQQD 3DFNDJH RULHQWDWLRQ 3DFNDJH VWDFNLQJ Fig. 1. 3D reconstruction on tagged packages via mobile scanning Existing 3D reconstruction methods are mainly based on computer vision or sensor-based approaches. Computer visionbased approaches leverage the cameras to capture the appearance and build the 3D profiles of objects [1, 2]. They are able to reconstruct the shape of objects in a vivid approach. The main disadvantage is the line-of-sight constraint in capturing images, which leads to blind angles in 3D reconstruction with only one fixed camera. Sensor-based approaches usually leverage inertial sensors attached to the items to detect the orientation variation of the specified items [3, 4]. However, they suffer from high hardware cost of the sensors, as well as the limited battery life for the sensors. Fortunately, the rising use of RFID technology in the logistic industry has brought brand new opportunities to 3D reconstruction on packaged objects. In current logistic industry, RFID tags have been widely used to label the packages with exact logistics information. In comparison to the above two approaches, the passive RFID tag is battery-free and very cheap, and the backscatter-based communication from RFID is not limited by the line of sight requirement. Most importantly, for most logistic applications, the RFID systems are already deployed in the sites to scan and identify the tagged packages. Therefore, in this paper, we propose RF-3DScan, which aims to perform 3D reconstruction on packaged objects via the RFID systems (in Fig. 1). Our idea is based on the observation that by attaching multiple tags onto the surface of the packages, we are able to derive the 3D orientation of each single package and the 3D stacking situation of multiple packages according to the backscattered RF-signals from these reference tags. Our approach of RF-3DScan is as follows: We attach a set of reference tags on the surface of the packages, then we utilize a single RFID antenna to continuously scan the tagged packages, while the antenna is moving along
straight lines within a constrained 2-dimensional space.As the II.RELATED WORK antenna is moving,by extracting the phase differences from A.Computer Vision and Sensor-based Approach the specified tags at different time points,we build the angle Computer-vision-based solutions mainly leverage the depth profiles to depict the geometry angles between the antenna-tag camera to perform 3D reconstruction of multiple objects pairs.By comparing the angle profiles of different reference [1,2].To avoid the blind angles in 3D reconstruction for the tags,we are able to derive the relative positions of these tags specified objects.usually multiple depth cameras are deployed on the specified package,and further figure out the package at different positions to perform multi-view reconstruction for orientation and package stacking for multiple packages. their 3D models [1],or a moving depth camera is used to There are three key challenges to realize 3D reconstruction build the 3D models in a mobile approach [2].In a word, via RFID systems.The first challenge is to determine the these approaches suffer from the line-of-sight (LOS)constraint package orientation according to the RF-signals from the reference tags attached to a specified package.To tackle this in 3D perception,and they are vulnerable to the limitation of the light intensity.Sensor-based solutions [3,4]mainly challenge,we extract angle profiles from the phases of the attach the battery-powered sensors (such as inertial sensors or RF-signals,then we build an angle-profile-based model to GPS modules)to the surface of the objects,and continuously transform the RF-signals into the indicators for the relative monitor the 3D placement of the specified objects,so as to localization among the reference tags.Thus,after performing track the orientation variation [3],or the stacking situation 1-dimensional mobile scanning along a straight line,we can among multiple objects. determine the relative positions of the reference tag pairs on the package,and use this information to further derive the B.RFID-based Approach package orientation.The second challenge is to determine Orientation tracking:By attaching multiple RFID tags onto the stacking situation among multiple packages,according to the specified object,it is possible to track the orientation the RF-signals from the reference tags attached to multiple variation of the object according to the variation of the packages.To tackle this challenge,we further perform a 2- corresponding RF-signals [5,6].Tagball [5]is proposed as dimensional mobile scanning to scan the packages along the a 3D human-computer interaction system,where multiple orthogonal direction of the previous scanning direction,such passive tags are attached to a controlling ball,such that the that the relative 3D positions of the reference tags from motions of the ball rotation from users can be detected from different packages can be determined.In this way,we can the phase changes of multiple tags.Tagyro [6]attaches an estimate the centers of packages according to the reference array of passive RFID tags as orientation sensors on the tags,and derive the relative locations of different packages in objects,by transforming the runtime phase offsets between the 3D space.The third challenge is to select effective refer- tags into the orientation angle.Compared with our RF-3DScan ence tag pairs for accurately deriving the package orientation system,these approaches track the orientation variation of the and stacking situation.To tackle this challenge,we filter out dynamically moving objects,whereas our approach aims to those reference tags with unstable phases.which are located determine the orientation of statically placed packages. outside the field of major antenna beam during scanning,by Localization:RFID localization generally falls into two cat- referring to the received signal strength(RSS).Further,as our egories:absolute localization [7-10]and relative localization empirical study shows that the absolute phase of the RF-signal [11-15].By attaching multiple tags and pinpointing each tag's varies with different orientations of the reference tag,thus we 3D coordinates,the absolute localization can be tailored to our measure the phase differences to extract the angle profiles from problem for 3D reconstruction.However,this approach suffers the reference tags during the mobile scanning. from complicated system deployment and collaboration.For To the best of our knowledge,this paper presents the example,the state-of-the-art absolute localization schemes first study of using RFID for 3D reconstruction on tagged PinIt [7]and Tagoram [10]are able to achieve cm-level packages.We make three contributions as follows.1)For 3D localization accuracy,however,they either need to deploy reconstruction on the packages,we attach multiple reference many reference tags or require sophisticated calibration of RFID tags onto the packages,and respectively tackle the issues multiple readers.Rather than absolute localization,recent of package orientation and package stacking,by leveraging RFID researches start to focus on the relative localization the angle profiles extracted from the RF-signals.We build an of multiple objects without any pre-deployment of reference angle-profile-based model to depict the relationship between nodes.Relative localization investigates the relative locations the RF-signals from the reference tags and the package ori- of a set of objects as oppose to their absolute coordinates entation/stacking.2)We propose a mobile scanning solution STPP [13]is the first work to tackle 2D relative localization.It to realize the 3D reconstruction of tagged packages.We are investigates the spatial-temporal dynamics in the phase profiles able to determine the package orientation via /-dimensional However,this approach leverages large-range scanning to mobile scanning,and further determine the package stacking detect the Vzone from the phase sequences,as it requires the via 2-dimensional mobile scanning.3)We have implemented a antenna to cross the perpendicular point during the scanning prototype system to evaluate the performance,the experiment to collect enough phases.Compared with STPP,our approach results in real settings show that RF-3DScan achieves about performs 3D relative localization by leveraging the angle 92.5%bottom face accuracy and about 4.08 angle error. profiles from rather small-range scanning
straight lines within a constrained 2-dimensional space. As the antenna is moving, by extracting the phase differences from the specified tags at different time points, we build the angle profiles to depict the geometry angles between the antenna-tag pairs. By comparing the angle profiles of different reference tags, we are able to derive the relative positions of these tags on the specified package, and further figure out the package orientation and package stacking for multiple packages. There are three key challenges to realize 3D reconstruction via RFID systems. The first challenge is to determine the package orientation according to the RF-signals from the reference tags attached to a specified package. To tackle this challenge, we extract angle profiles from the phases of the RF-signals, then we build an angle-profile-based model to transform the RF-signals into the indicators for the relative localization among the reference tags. Thus, after performing 1-dimensional mobile scanning along a straight line, we can determine the relative positions of the reference tag pairs on the package, and use this information to further derive the package orientation. The second challenge is to determine the stacking situation among multiple packages, according to the RF-signals from the reference tags attached to multiple packages. To tackle this challenge, we further perform a 2- dimensional mobile scanning to scan the packages along the orthogonal direction of the previous scanning direction, such that the relative 3D positions of the reference tags from different packages can be determined. In this way, we can estimate the centers of packages according to the reference tags, and derive the relative locations of different packages in the 3D space. The third challenge is to select effective reference tag pairs for accurately deriving the package orientation and stacking situation. To tackle this challenge, we filter out those reference tags with unstable phases, which are located outside the field of major antenna beam during scanning, by referring to the received signal strength (RSS). Further, as our empirical study shows that the absolute phase of the RF-signal varies with different orientations of the reference tag, thus we measure the phase differences to extract the angle profiles from the reference tags during the mobile scanning. To the best of our knowledge, this paper presents the first study of using RFID for 3D reconstruction on tagged packages. We make three contributions as follows. 1) For 3D reconstruction on the packages, we attach multiple reference RFID tags onto the packages, and respectively tackle the issues of package orientation and package stacking, by leveraging the angle profiles extracted from the RF-signals. We build an angle-profile-based model to depict the relationship between the RF-signals from the reference tags and the package orientation/stacking. 2) We propose a mobile scanning solution to realize the 3D reconstruction of tagged packages. We are able to determine the package orientation via 1-dimensional mobile scanning, and further determine the package stacking via 2-dimensional mobile scanning. 3) We have implemented a prototype system to evaluate the performance, the experiment results in real settings show that RF-3DScan achieves about 92.5% bottom face accuracy and about 4.08◦ angle error. II. RELATED WORK A. Computer Vision and Sensor-based Approach Computer-vision-based solutions mainly leverage the depth camera to perform 3D reconstruction of multiple objects [1, 2]. To avoid the blind angles in 3D reconstruction for the specified objects, usually multiple depth cameras are deployed at different positions to perform multi-view reconstruction for their 3D models [1], or a moving depth camera is used to build the 3D models in a mobile approach [2]. In a word, these approaches suffer from the line-of-sight (LOS) constraint in 3D perception, and they are vulnerable to the limitation of the light intensity. Sensor-based solutions [3, 4] mainly attach the battery-powered sensors (such as inertial sensors or GPS modules) to the surface of the objects, and continuously monitor the 3D placement of the specified objects, so as to track the orientation variation [3], or the stacking situation among multiple objects. B. RFID-based Approach Orientation tracking: By attaching multiple RFID tags onto the specified object, it is possible to track the orientation variation of the object according to the variation of the corresponding RF-signals [5, 6]. Tagball [5] is proposed as a 3D human-computer interaction system, where multiple passive tags are attached to a controlling ball, such that the motions of the ball rotation from users can be detected from the phase changes of multiple tags. Tagyro [6] attaches an array of passive RFID tags as orientation sensors on the objects, by transforming the runtime phase offsets between tags into the orientation angle. Compared with our RF-3DScan system, these approaches track the orientation variation of the dynamically moving objects, whereas our approach aims to determine the orientation of statically placed packages. Localization: RFID localization generally falls into two categories: absolute localization [7–10] and relative localization [11–15]. By attaching multiple tags and pinpointing each tag’s 3D coordinates, the absolute localization can be tailored to our problem for 3D reconstruction. However, this approach suffers from complicated system deployment and collaboration. For example, the state-of-the-art absolute localization schemes PinIt [7] and Tagoram [10] are able to achieve cm-level localization accuracy, however, they either need to deploy many reference tags or require sophisticated calibration of multiple readers. Rather than absolute localization, recent RFID researches start to focus on the relative localization of multiple objects without any pre-deployment of reference nodes. Relative localization investigates the relative locations of a set of objects as oppose to their absolute coordinates. STPP [13] is the first work to tackle 2D relative localization. It investigates the spatial-temporal dynamics in the phase profiles However, this approach leverages large-range scanning to detect the V-zone from the phase sequences, as it requires the antenna to cross the perpendicular point during the scanning to collect enough phases. Compared with STPP, our approach performs 3D relative localization by leveraging the angle profiles from rather small-range scanning
△d △d d x cosa d x cosa 0 60 120180240300360 Moving Rotation(deg.）】 direction (a)rotate along the Z axis (b)phase change during rotation A Pi d Fig.2.Measured phase of a single tag rotating along the Z axis Fig.3.AoA in static scanning Fig.4.AoA in mobile scanning III.ANGLE-PROFILE-BASED MODELING The phase difference is related to the distance difference A.Limitations of Phase-based Measurement from the tag to the antennas.When h>d,the relationship between the phase difference (A0 =0A:-0A2+n.n means The RF phase is a widely used attribute of the wireless the phase offset caused by the hardware characteristics of A signal,ranging from 0 to 2.Due to the ultra-high working and A2)and the distance difference (Ad dr.A-dr.A2 frequency (indicating short wave-length)in RFIDs and fine- grained measure resolution of phase value by COTS readers, d cos a)can be approximated as: the phase is very sensitive to the tag-antenna distance,which 2 d cos a△6 2π +n (2) gives us the potential chance to achieve accurate 3D recon- 入 struction.Suppose dis is the distance between the antenna wherencan be any integer in[-共-2，共-]，its range and the tag.Since the backscatter communication of RFID is 4.When d<the value range of n is smalier than 1, is round-trip,the signal totally traverses a distance of 2dis which means n has a unique value,so a is deterministic. in each communication.Besides the distance,some hardware 2)Angle in Mobile Scanning:With respect to multiple characteristics will also distort the phase value.Hence,the antennas,the phase offsets related to their own hardware phase 6 reported by the reader can be expressed as: characteristics are different,so it is hard to determine n Hence,we prefer a mobile antenna to multiple static antennas, 2元 ×2dis+n mod 2 (1) in which case the can be canceled. For a mobile antenna,the angle-of-arrival is a little different. where A is the wavelength,n represents the phase offset caused Without the loss of generality,we redefine the AoA in a mobile by the hardware characteristics.Although the phase accurately case,as shown in Fig.4.Similarly,T is the tag position reflects the distance,we face three challenges before putting and V is its projected point on the antenna moving line, into use:1)The distort factor n is unknown;2)The phase value its perpendicular distance is h.Let the mobile antenna be repeats periodically,it is not feasible to use it directly;3)In at position A,then the included angle of line TA and the addition to dis and n,our extensive experiments show that the antenna moving direction is just the angle-of-arrival (a)for tag orientation influences the phase value 0.Fig.2 plots the the tag when the antenna is at position A. phase change as a tag rotates along the Z axis,as the phase To estimate the angle at position A,we only need the phases varies continuously over the rotation.Next,we discuss how collected at the two nearby positions (P and P2),centered to use the angle-of-arrival approach to overcome above three on the antenna (PA =AP2).Thus,the phase difference at challenges,and benefit our system design in the sequel. position P and P2 can be used to estimate a with Eg.2.By combining the angles at different antenna positions,we can B.Angle Profile derive an angle profile for a specified tag. Angle-of-Arrival (AoA)is one of the most popular RF- C.Metrics of Angle Profile based localization measurements using phase difference.The Suppose there are two tags and one antenna in the same basic idea of our approach is that by moving the antenna plane(Fig.5).The antenna moves linearly from O to A,so to scan the tags,we extract the phase differences from the it passes through T first,followed by T2.When the antenna specified tags at different time points,then we derive the passes through the tag (corresponding to point V in Fig.4), geometry angles between the tag-antenna pairs at different the angle-of-arrival (o)of that tag reaches /2,naming this positions,which is called angle profile. point as the perpendicular point.Similarly,we call the distance 1)Angle in Static Scanning:As shown in Fig.3,a tag is from the tag to the perpendicular point perpendicular distance, set at T,A1 and A2 are two antennas separated by d,M is the direction perpendicular to the antenna moving direction as the middle point of A1A2.V is the projected point of T on perpendicular direction.As Ti is on the left along the antenna the tag pair line A1A2,the perpendicular distance is h.The moving direction,its perpendicular point shows earlier than included angle between line TM and line MV is the AoA T2.Hence,the perpendicular point is the key metric for the for tag T,denoted as a.Let dr.A,and dr.A represent the tags'relative positions along the moving direction. distances between T and the antennas,the antennas collect the Besides the perpendicular point,there is the other special phases as 6A and 042 respectively.641,6A2 E0,27). point:equal angle point.The equal angle point is where the
(a) rotate along the Z axis 5RWDWLRQ GHJ 3KDVH UDG (b) phase change during rotation Fig. 2. Measured phase of a single tag rotating along the Z axis ܶ ߙ ο݀ ൎ ߙ ൈ݀ ଶܣ ଵܣ ݄ ܸ ܯ ݀ Fig. 3. AoA in static scanning ܣ ܶ ߙ ο݀ ൎ ߙ ൈ݀ ܲଵ ܲଶ ݄ ܸ ݀ ݃݊݅ݒܯ Fig. 4. AoA in mobile scanning III. ANGLE-PROFILE-BASED MODELING A. Limitations of Phase-based Measurement The RF phase is a widely used attribute of the wireless signal , ranging from 0 to 2π. Due to the ultra-high working frequency (indicating short wave-length) in RFIDs and finegrained measure resolution of phase value by COTS readers, the phase is very sensitive to the tag-antenna distance, which gives us the potential chance to achieve accurate 3D reconstruction. Suppose dis is the distance between the antenna and the tag. Since the backscatter communication of RFID is round-trip, the signal totally traverses a distance of 2dis in each communication. Besides the distance, some hardware characteristics will also distort the phase value. Hence, the phase θ reported by the reader can be expressed as: θ = 2π λ × 2dis + η mod 2π (1) where λ is the wavelength, η represents the phase offset caused by the hardware characteristics. Although the phase accurately reflects the distance, we face three challenges before putting into use: 1) The distort factor η is unknown; 2) The phase value repeats periodically, it is not feasible to use it directly; 3) In addition to dis and η, our extensive experiments show that the tag orientation influences the phase value θ. Fig. 2 plots the phase change as a tag rotates along the Z axis, as the phase varies continuously over the rotation. Next, we discuss how to use the angle-of-arrival approach to overcome above three challenges, and benefit our system design in the sequel. B. Angle Profile Angle-of-Arrival (AoA) is one of the most popular RFbased localization measurements using phase difference. The basic idea of our approach is that by moving the antenna to scan the tags, we extract the phase differences from the specified tags at different time points, then we derive the geometry angles between the tag-antenna pairs at different positions, which is called angle profile. 1) Angle in Static Scanning: As shown in Fig. 3, a tag is set at T, A1 and A2 are two antennas separated by d, M is the middle point of A1A2. V is the projected point of T on the tag pair line A1A2, the perpendicular distance is h. The included angle between line TM and line MV is the AoA for tag T, denoted as α. Let dT ,A1 and dT ,A2 represent the distances between T and the antennas, the antennas collect the phases as θA1 and θA2 respectively. θA1 , θA2 ∈ [0, 2π). The phase difference is related to the distance difference from the tag to the antennas. When h d, the relationship between the phase difference (Δθ = θA1 −θA2 +θη, θη means the phase offset caused by the hardware characteristics of A1 and A2) and the distance difference (Δd = dT ,A1 − dT ,A2 d cos α) can be approximated as: 2d cos α λ = Δθ 2π + n (2) where n can be any integer in −2d λ − Δθ 2π , 2d λ − Δθ 2π , its range is 4d λ . When d < λ 4 , the value range of n is smaller than 1, which means n has a unique value, so α is deterministic. 2) Angle in Mobile Scanning: With respect to multiple antennas, the phase offsets related to their own hardware characteristics are different, so it is hard to determine θη. Hence, we prefer a mobile antenna to multiple static antennas, in which case the θη can be canceled. For a mobile antenna, the angle-of-arrival is a little different. Without the loss of generality, we redefine the AoA in a mobile case, as shown in Fig. 4. Similarly, T is the tag position and V is its projected point on the antenna moving line, its perpendicular distance is h. Let the mobile antenna be at position A, then the included angle of line T A and the antenna moving direction is just the angle-of-arrival (α) for the tag when the antenna is at position A. To estimate the angle at position A, we only need the phases collected at the two nearby positions (P1 and P2), centered on the antenna (P1A = AP2). Thus, the phase difference at position P1 and P2 can be used to estimate α with Eq. 2. By combining the angles at different antenna positions, we can derive an angle profile for a specified tag. C. Metrics of Angle Profile Suppose there are two tags and one antenna in the same plane (Fig. 5). The antenna moves linearly from O to A, so it passes through T1 first, followed by T2. When the antenna passes through the tag (corresponding to point V in Fig. 4), the angle-of-arrival (α) of that tag reaches π/2, naming this point as the perpendicular point. Similarly, we call the distance from the tag to the perpendicular point perpendicular distance, the direction perpendicular to the antenna moving direction as perpendicular direction. As T1 is on the left along the antenna moving direction, its perpendicular point shows earlier than T2. Hence, the perpendicular point is the key metric for the tags’ relative positions along the moving direction. Besides the perpendicular point, there is the other special point: equal angle point. The equal angle point is where the
Perpendicular distance Preprocess Determine Determine package orientation package stacking RF Angle compuation (single) (multiple) Signals Angle smoothing T ·Bottom/top face Relative stacking Lincar fitin Rotation angle situation 2 Moving Fig.7. Architecture of RF-3DScan 0 Equal angle Perpendicular A distance between two zero points just reflects the tags'perpendicular point point points separation.In addition,the intersection of the two lines Fig.5. Metrics for the angle profiles represents the position where the tags are projected on the cota same line with the antenna,corresponding to the equal angle point.Specially,the smaller h is,the larger is,and the Equal angle point sharper the line is.As the h of Ti is smaller than T2,thelkll T2 of Ti is larger,so the line of Ti decreases faster than T2. Perpendicular point For a certain tag,its angle profile records its angles at T Moving different positions.Note that,for analyzing the change of distance the angles,it is the separation between the positions and Fig.6.Model of the angle profiles the corresponding change of cot o that matter,so the moved antenna and the two tags are in the same line,so n and 72 distance l does not necessarily the actual moved distance of the share the same angle.Before equal point,the angle of Ti is antenna.That is,we have no constrict to the coordinates of the smaller than the angle of T2.On the contrary,the angle of T positions,as long as they refer to the same basis.For example, changes to be bigger than that of T2 after the equal angle point. let the antenna moves at a constant speed,and set a random No matter for T or T2,its angle increases continuously during time as the starting moving time.When the antenna collects the antenna moving process,so it is obvious that the angle of phase during the moving process,it records the corresponding time as well,then that position can be estimated with the Ti changes faster than that of T2.Such phenomenon is due to the smaller perpendicular distance of T1.Thus,according time interval and the constant moving speed.So,in Eq.4, to the angle change rate,we can determine the tags'relative the angle a and the moved distance I are known parameters, positions along the perpendicular direction. there're two remaining unknown parameters:h and lo,which can be estimated by linear fitting with multiple angles during D.Model of Angle Profile the moving process in the angle profile.Meanwhile,lo depends To depict the angle-profile-based measurement metrics in on when the antenna passes through the tag.For different mathematics,we build a linear model to derive the metrics tags,the antenna start point should be the same one (as they from the angle profile automatically.Considering Fig.4,the share the same starting time),so the larger lo is.the later that angle-of-arrival can be expressed as: line reaches 0,and the tag is more ahead along the antenna cot a=y-yA moving direction.Thus,by leveraging these properties,we can (3) h determine the tags'relative positions,as: where cot means the cotangent function,h is the perpendicular 1)The value of reflects the perpendicular distance from distance between the tag and the antenna moving trace.yA the tag to the antenna moving trace:the larger is, and yv represent the coordinates of point A and V along the the smaller the perpendicular distance is. antenna moving direction.Assume there is an antenna start 2)The value of lo determines the projected position of the point S,the distance from S to V is lo,the antenna moved corresponding tag along the antenna moving direction. distance be 1.Thus,(lo-l)represents the distance from the The difference of lo between two tags indicates their antenna to the perpendicular point (same as (yv-yA)),the interval in the antenna moving direction. angle can be rewritten as: IV.SYSTEM OVERVIEW cot a=kl +b,k = h6= lo (4) RF-3DScan is a 3D reconstruction system for tagged pack- h ages via RFID technology.For RF-3DScan,the geometry where the scope k is related to the minus reciprocal of h,the relationships of the tags attached on a single package is known intercept b depends on the ratio of lo and h. as priori and the tag deployment obeys two important rules. Taking the tags in Fig.5,the transformed angle expression Also,we make the following assumptions:1)The antenna using Eq.4 should look like the lines shown in Fig.6.As I moves at a constant speed;2)Each package is a standard cube. increases continuously during the moving process,o increases and they are fully on the ground or parallel to the ground (on as well.When the antenna reaches the perpendicular point, the ground is the special case of parallel to the ground). a is equal to /2,so cota =0.The line of T reaches Fig.7 illustrates the architecture of RF-3DScan.RF-3DScan 0 earlier than T2.Thus,the order of such zero points are takes the RF-signals from the tags as input,then outputs 3D corresponding to the tags perpendicular points,and the spacing profiles for multiple packages.The whole system consists
ଵߙ ଶߙ ଵԢߙ ଶԢߙ ݄ଶ ݄ଵ ܶଶ ܶଵ ܣ ܱ Fig. 5. Metrics for the angle profiles ܶଶ ܶଵ ߙ Ͳ Fig. 6. Model of the angle profiles antenna and the two tags are in the same line, so T1 and T2 share the same angle. Before equal point, the angle of T1 is smaller than the angle of T2. On the contrary, the angle of T1 changes to be bigger than that of T2 after the equal angle point. No matter for T1 or T2, its angle increases continuously during the antenna moving process, so it is obvious that the angle of T1 changes faster than that of T2. Such phenomenon is due to the smaller perpendicular distance of T1. Thus, according to the angle change rate, we can determine the tags’ relative positions along the perpendicular direction. D. Model of Angle Profile To depict the angle-profile-based measurement metrics in mathematics, we build a linear model to derive the metrics from the angle profile automatically. Considering Fig. 4, the angle-of-arrival can be expressed as: cot α = yV − yA h (3) where cot means the cotangent function, h is the perpendicular distance between the tag and the antenna moving trace. yA and yV represent the coordinates of point A and V along the antenna moving direction. Assume there is an antenna start point S, the distance from S to V is l0, the antenna moved distance be l. Thus, (l0 − l) represents the distance from the antenna to the perpendicular point (same as (yV − yA)), the angle can be rewritten as: cot α = kl + b, k = − 1 h, b = l0 h (4) where the scope k is related to the minus reciprocal of h, the intercept b depends on the ratio of l0 and h. Taking the tags in Fig. 5, the transformed angle expression using Eq. 4 should look like the lines shown in Fig. 6. As l increases continuously during the moving process, α increases as well. When the antenna reaches the perpendicular point, α is equal to π/2, so cot α = 0. The line of T1 reaches 0 earlier than T2. Thus, the order of such zero points are corresponding to the tags perpendicular points, and the spacing 3UHSURFHVV $QJOHFRPSXWDWLRQ /LQHDUILWWLQJ $QJOHVPRRWKLQJ 5) 6LJQDOV 'HWHUPLQH SDFNDJHRULHQWDWLRQ VLQJOH 'HWHUPLQH SDFNDJHVWDFNLQJ PXOWLSOH %RWWRPWRSIDFH 5RWDWLRQDQJOH 5HODWLYHVWDFNLQJ VLWXDWLRQ Fig. 7. Architecture of RF-3DScan between two zero points just reflects the tags’ perpendicular points separation. In addition, the intersection of the two lines represents the position where the tags are projected on the same line with the antenna, corresponding to the equal angle point. Specially, the smaller h is, the larger k is, and the sharper the line is. As the h of T1 is smaller than T2, the k of T1 is larger, so the line of T1 decreases faster than T2. For a certain tag, its angle profile records its angles at different positions. Note that, for analyzing the change of the angles, it is the separation between the positions and the corresponding change of cot α that matter, so the moved distance l does not necessarily the actual moved distance of the antenna. That is, we have no constrict to the coordinates of the positions, as long as they refer to the same basis. For example, let the antenna moves at a constant speed, and set a random time as the starting moving time. When the antenna collects phase during the moving process, it records the corresponding time as well, then that position can be estimated with the time interval and the constant moving speed. So, in Eq. 4, the angle α and the moved distance l are known parameters, there’re two remaining unknown parameters: h and l0, which can be estimated by linear fitting with multiple angles during the moving process in the angle profile. Meanwhile, l0 depends on when the antenna passes through the tag. For different tags, the antenna start point should be the same one (as they share the same starting time), so the larger l0 is, the later that line reaches 0, and the tag is more ahead along the antenna moving direction. Thus, by leveraging these properties, we can determine the tags’ relative positions, as: 1) The value of k reflects the perpendicular distance from the tag to the antenna moving trace: the larger k is, the smaller the perpendicular distance is. 2) The value of l0 determines the projected position of the corresponding tag along the antenna moving direction. The difference of l0 between two tags indicates their interval in the antenna moving direction. IV. SYSTEM OVERVIEW RF-3DScan is a 3D reconstruction system for tagged packages via RFID technology. For RF-3DScan, the geometry relationships of the tags attached on a single package is known as priori and the tag deployment obeys two important rules. Also, we make the following assumptions: 1) The antenna moves at a constant speed; 2) Each package is a standard cube, and they are fully on the ground or parallel to the ground (on the ground is the special case of parallel to the ground). Fig. 7 illustrates the architecture of RF-3DScan. RF-3DScan takes the RF-signals from the tags as input, then outputs 3D profiles for multiple packages. The whole system consists
of three components:1)Preprocess:with the RF-signals of VI.DETERMINE PACKAGE ORIENTATION the tags,RF-3DScan builds the angle profiles by using the FOR EACH SINGLE PACKAGE phase differences at different time points for each tag,and To reconstruct a single package,it makes the same sense extracts the indicators in the angle profiles for the relative to determine the package orientation.so we just need to localization among the tags by linear fitting.2)Determine identify the bottom face of this package,and estimate the package orientation for a single package:by comparing the relative rotation angle of the vertical sides against the antenna relative positions of the tags on a specified package,RF- plane in a specified coordinate system.The basic idea of our 3DScan can determine which side of the package is on the approach is that we attach a set of passive RFID tags on the ground,and then evaluates the angle of the vertical sides in package under special rules,then employ one antenna to do a specified coordinate system.3)Determine package stacking 1-dimensional mobile scanning to build the angle profiles for for multiple packages:after deriving the orientation of a single each tag.Next,we compare them to determine the relative package,the centers of the packages are also determined positions of the tags,thus we can further realize the 3D along the scanning direction.By performing a 2-dimensional reconstruction for a single package. mobile scanning,RF-3DScan combines the results from the two orthogonal scanning,so the relative positions of these A.Deploy Reference Tags packages in the 3D space can be determined. In order to determine the package orientation only by 1- dimensional mobile scanning,we need to deploy the tags in an V.DATA PREPROCESS efficient way.The design principle of the tag deployment is to With the raw RF-signals,we need to build angle profiles of use as fewer as tags to depict the package uniquely,accurately different tags first.The preprocessing can be divided into three and conveniently.So,we make two rules as follows: steps:angle computation,angle smoothing and linear fitting. Rule 1:The orientations of the tags should be along different A.Angle Computation orthogonal axes.As the package can be with any orientation As the antenna collects phases at different time points in the 3D space,we should pay attention to ensuring there are always enough effective tags reflecting the signals to the during its mobile scanning,we can extract the phase dif- ferences for a tag at different positions.Using these phase antenna.As the 3D space can be defined with three orthogonal differences,we compute the angle-of-arrivals with Eg.2.To axes,we can just let the tags deployed along these axes alone, get a deterministic angle,as mentioned above,the separation as shown in Fig.8(a).With this rule,tags along one direction of the positions among two phases should be within A/4. at most are in the blind direction,so other tags can get enough power to reflect their signals to the antenna effectively. B.Angle Smoothing Rule 2:The tags should be deployed along different orthog- Although using the phase difference from the two positions onal axes.As for identifying the bottom face of the package with the small separation can get a unique angle,the noise like it is the same to find which tags are along the vertical axis and multi-path effect would influence the measured phases,there what order these tags are.So,there should be at least three would exist large fluctuation in angles,so angle smoothing is tag pairs (four tags)along three orthogonal axes separately,as required.Usually,the phases collected by the antenna is not shown in Fig.8(b).Under this rule,whatever the orientation uniform,so is the angle distribution,thus it is not suitable of the package is,there is always one tag pair along the to use the common smooth algorithms,e.g.,low-pass filter. △9+y十 vertical axis,so we can transform the identification of the Taking the noise u into consideration:cosa =2d2 bottom face of the package into finding the vertical tag pair when d is very small,has much influence on cosa. and determining their orders along the vertical axis. While when d increases,such distortion effect decreases,but Combining the above two rules,Fig.8(c)illustrates a there exist redundant angles in the results,only one of them possible tag deployment satisfying these two rules.No matter is the true value.Hence,we can derive two sets of angles what orientation the package is,there are four tags at least to from two phase separations:a small one and a large one,then avoid the signal blind direction.Also,there is always one tag use the unique angles from the small separation to filter the pair along the Z axis.By determining the tags'order of this several angle candidates from the large separation,thus,we get tag pair,we can derive which side of the package is on the a relative accurate angle profile with less fluctuation [12].Note ground then. that,too large separation will bring too much environmental B.Determine Package Orientation change and break the restraint of the angle estimation method. Thus,we set the small separation around 5-8cm and the large To determine the package orientation,it demands to identify the bottom face of the package and the relative angle of the separation within 15cm empirically when the antenna is in front of the packages about 1m. vertical sides in a specified coordinate system.Considering our assumption that one side of the cube package must be C.Linear Fitting parallel to the ground,when we deploy the tags of a package With the smoothed angles at different positions from an like the solution described in Fig.8(c),we can identify which angle profile for a certain tag,we can use the linear model as tag pair is along the Z axis and what order the tag pair is Eq.4 to fit them,then derive the two important indicators(h instead.In this case,let the antenna do mobile scanning along and lo)of that tag for the later relative localization comparison. the X or Y axis only once,we can determine the orientation
of three components: 1) Preprocess: with the RF-signals of the tags, RF-3DScan builds the angle profiles by using the phase differences at different time points for each tag, and extracts the indicators in the angle profiles for the relative localization among the tags by linear fitting. 2) Determine package orientation for a single package: by comparing the relative positions of the tags on a specified package, RF- 3DScan can determine which side of the package is on the ground, and then evaluates the angle of the vertical sides in a specified coordinate system. 3) Determine package stacking for multiple packages: after deriving the orientation of a single package, the centers of the packages are also determined along the scanning direction. By performing a 2-dimensional mobile scanning, RF-3DScan combines the results from the two orthogonal scanning, so the relative positions of these packages in the 3D space can be determined. V. DATA PREPROCESS With the raw RF-signals, we need to build angle profiles of different tags first. The preprocessing can be divided into three steps: angle computation, angle smoothing and linear fitting. A. Angle Computation As the antenna collects phases at different time points during its mobile scanning, we can extract the phase differences for a tag at different positions. Using these phase differences, we compute the angle-of-arrivals with Eq. 2. To get a deterministic angle, as mentioned above, the separation of the positions among two phases should be within λ/4. B. Angle Smoothing Although using the phase difference from the two positions with the small separation can get a unique angle, the noise like multi-path effect would influence the measured phases, there would exist large fluctuation in angles, so angle smoothing is required. Usually, the phases collected by the antenna is not uniform, so is the angle distribution, thus it is not suitable to use the common smooth algorithms, e.g., low-pass filter. Taking the noise μ into consideration: cos α = λ 2d Δθ+μ 2π + nλ 2d , when d is very small, μ has much influence on cos α. While when d increases, such distortion effect decreases, but there exist redundant angles in the results, only one of them is the true value. Hence, we can derive two sets of angles from two phase separations: a small one and a large one, then use the unique angles from the small separation to filter the several angle candidates from the large separation, thus, we get a relative accurate angle profile with less fluctuation [12]. Note that, too large separation will bring too much environmental change and break the restraint of the angle estimation method. Thus, we set the small separation around 5-8cm and the large separation within 15cm empirically when the antenna is in front of the packages about 1m. C. Linear Fitting With the smoothed angles at different positions from an angle profile for a certain tag, we can use the linear model as Eq. 4 to fit them, then derive the two important indicators (h and l0) of that tag for the later relative localization comparison. VI. DETERMINE PACKAGE ORIENTATION FOR EACH SINGLE PACKAGE To reconstruct a single package, it makes the same sense to determine the package orientation, so we just need to identify the bottom face of this package, and estimate the relative rotation angle of the vertical sides against the antenna plane in a specified coordinate system. The basic idea of our approach is that we attach a set of passive RFID tags on the package under special rules, then employ one antenna to do 1-dimensional mobile scanning to build the angle profiles for each tag. Next, we compare them to determine the relative positions of the tags, thus we can further realize the 3D reconstruction for a single package. A. Deploy Reference Tags In order to determine the package orientation only by 1- dimensional mobile scanning, we need to deploy the tags in an efficient way. The design principle of the tag deployment is to use as fewer as tags to depict the package uniquely, accurately and conveniently. So, we make two rules as follows: Rule 1: The orientations of the tags should be along different orthogonal axes. As the package can be with any orientation in the 3D space, we should pay attention to ensuring there are always enough effective tags reflecting the signals to the antenna. As the 3D space can be defined with three orthogonal axes, we can just let the tags deployed along these axes alone, as shown in Fig. 8(a). With this rule, tags along one direction at most are in the blind direction, so other tags can get enough power to reflect their signals to the antenna effectively. Rule 2: The tags should be deployed along different orthogonal axes. As for identifying the bottom face of the package, it is the same to find which tags are along the vertical axis and what order these tags are. So, there should be at least three tag pairs (four tags) along three orthogonal axes separately, as shown in Fig. 8(b). Under this rule, whatever the orientation of the package is, there is always one tag pair along the vertical axis, so we can transform the identification of the bottom face of the package into finding the vertical tag pair and determining their orders along the vertical axis. Combining the above two rules, Fig. 8(c) illustrates a possible tag deployment satisfying these two rules. No matter what orientation the package is, there are four tags at least to avoid the signal blind direction. Also, there is always one tag pair along the Z axis. By determining the tags’ order of this tag pair, we can derive which side of the package is on the ground then. B. Determine Package Orientation To determine the package orientation, it demands to identify the bottom face of the package and the relative angle of the vertical sides in a specified coordinate system. Considering our assumption that one side of the cube package must be parallel to the ground, when we deploy the tags of a package like the solution described in Fig. 8(c), we can identify which tag pair is along the Z axis and what order the tag pair is instead. In this case, let the antenna do mobile scanning along the X or Y axis only once, we can determine the orientation