Connected Vehicle Safety Science, System, andFrameworkGuide Words:Connected vehicle; intelligent transportation system, driver assistance system,internet-of-thingsAbstract:In this paper, we propose a framework to develop an M2M-based (machine-to-machine)proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M inintelligenttransportationsystem(ITS):1)expansionof sensorcoverage,2)increaseoftimeallowedto react, and 3)mediation of bidding for right of way,to help driver avoiding potential trafficaccidents.Todevelopsuchasystem,wedivideitintothreemainparts:1)driverbehaviormodelingand prediction, which collects grand driving data to learn and predict the future behaviors of drivers; 2)M2M-based neighbor map building, which includes sensing, communication, and fusion technologiesto build a neighbor map, where neighbor map mentions the locations of all neighboring vehicles; 3)design of passive information visualization and proactive warning mechanism, which researches onhow to provide user-needed information and warning signals to drivers without interfering theirdrivingactivities.I.INTRODUCTIONThe most profound technologies are those that disappear. They weave themselves into the fabricof everyday life until they are indistinguishable from it, dubbed by Mark Weiser. Theinternet-of-things (IOT) is a realization of the ubiquitous computing vision, whereas (1) the bestcomputer is a quiet, invisible servant; (2) the computer should extend your unconscious; (3)technology informs but does not demand our attention. The usefulness of IOT will emerge whenproducts, applications, and services are connected and interacting with each other. Intelligenttransportation system (ITs), which has been extensively researched in the last decade, compliesadvanced mechanisms to provide innovative,proactive services relating to traffic management anddriving safety. For example, drivers'behaviors are limited to their line of sight. Connected vehiclescannot only share their sensory information, but also actively send out alerts to nearby vehicles indange. Forming an even larger vehicular network, comprising connected vehicles and infrastructures,make it possible to proactively perform load balancing across multiple routes. It is anticipated thattraffic accidents can be eliminated from one of the leading causes of death and the catastrophic onescanbeeffectivelyprevented.In this paper, we present the challenges arise from realizing intelligent transports, and provide
Connected Vehicle Safety Science, System, and Framework Guide Words：Connected vehicle; intelligent transportation system, driver assistance system, internet-of-things Abstract：In this paper, we propose a framework to develop an M2M-based (machine-to-machine) proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M in intelligent transportation system (ITS): 1) expansion of sensor coverage, 2) increase of time allowed to react, and 3) mediation of bidding for right of way, to help driver avoiding potential traffic accidents. To develop such a system, we divide it into three main parts: 1) driver behavior modeling and prediction, which collects grand driving data to learn and predict the future behaviors of drivers; 2) M2M-based neighbor map building, which includes sensing, communication, and fusion technologies to build a neighbor map, where neighbor map mentions the locations of all neighboring vehicles; 3) design of passive information visualization and proactive warning mechanism, which researches on how to provide user-needed information and warning signals to drivers without interfering their driving activities. I. INTRODUCTION The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it, dubbed by Mark Weiser. The internet-of-things (IOT) is a realization of the ubiquitous computing vision, whereas (1) the best computer is a quiet, invisible servant; (2) the computer should extend your unconscious; (3) technology informs but does not demand our attention. The usefulness of IOT will emerge when products, applications, and services are connected and interacting with each other. Intelligent transportation system (ITS), which has been extensively researched in the last decade, complies advanced mechanisms to provide innovative, proactive services relating to traffic management and driving safety. For example, drivers’ behaviors are limited to their line of sight. Connected vehicles cannot only share their sensory information, but also actively send out alerts to nearby vehicles in dange. Forming an even larger vehicular network, comprising connected vehicles and infrastructures, make it possible to proactively perform load balancing across multiple routes. It is anticipated that traffic accidents can be eliminated from one of the leading causes of death and the catastrophic ones can be effectively prevented. In this paper, we present the challenges arise from realizing intelligent transports, and provide
insights on resolution in thepresence of machine-to-machine (M2M) communications,includingvehicle-to-vehicle(V2V),vehicle-to-infrastructure(V21)and vehicle-to-cloud (V2C).Interms ofubiquitous computing, the internet-of-things in ITS (1) is a large-scale distributed computing server,(2) can extend human perception; (3) interacts with one another and, most importantly, with humanbeings to ensure against potential traffic violations and accidents.II.PROBLEMFORMULATIONTraffic violations do not necessarily lead to traffic collisions, if timely warnings can be sent outin accordance with the traffic situation.However, due to the line of sight, the perceptual capabilities ofany individuals are limited. In terms of ITS, things (or devices) work not just as individuals, but asmembers of a hierarchy.Thus, it is necessary to consider the problem of not just individual groups, butalsotheproblemof setsofgroupsasawholeX,Fig. 1. The hierarchy of the ITS problemAs shown in Fig. 1, lines are used to indicate roads. Analytics (A) optimizes decision making inthe cloud by aggregating every bit of information, whereas communication (C) and user experience (X)addressconnectivityandusability,respectively,inanadhocmanner,toensureagainstpotentialtrafficcollisions. Apart from the above problems, crowdsourcing also plays an essential role in developmentof analytics. Learning from crowdsourcing can serve as the key to making ITS reality.To develop such a proactive driver assistance system based on M2M communications, we divideit into three main parts and several technical components, as shown in Fig. 2. The three parts are 1)driver behavior modeling and prediction, 2) M2M-based neighbor map building, where neighbor mapmentions the locations of all neighboring vehicles, and 3) design of passive information visualizationand proactive warning mechanism
insights on resolution in the presence of machine-to-machine (M2M) communications, including vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) and vehicle-to-cloud (V2C). In terms of ubiquitous computing, the internet-of-things in ITS (1) is a large-scale distributed computing server; (2) can extend human perception; (3) interacts with one another and, most importantly, with human beings to ensure against potential traffic violations and accidents. II. PROBLEM FORMULATION Traffic violations do not necessarily lead to traffic collisions, if timely warnings can be sent out in accordance with the traffic situation. However, due to the line of sight, the perceptual capabilities of any individuals are limited. In terms of ITS, things (or devices) work not just as individuals, but as members of a hierarchy. Thus, it is necessary to consider the problem of not just individual groups, but also the problem of sets of groups as a whole. Fig. 1. The hierarchy of the ITS problem As shown in Fig. 1, lines are used to indicate roads. Analytics (A) optimizes decision making in the cloud by aggregating every bit of information, whereas communication (C) and user experience (X) address connectivity and usability, respectively, in an ad hoc manner, to ensure against potential traffic collisions. Apart from the above problems, crowdsourcing also plays an essential role in development of analytics. Learning from crowdsourcing can serve as the key to making ITS reality. To develop such a proactive driver assistance system based on M2M communications, we divide it into three main parts and several technical components, as shown in Fig. 2. The three parts are 1) driver behavior modeling and prediction, 2) M2M-based neighbor map building, where neighbor map mentions the locations of all neighboring vehicles, and 3) design of passive information visualization and proactive warning mechanism
1)MachineLearning3)UserExperienceDriverBehaviorData CollectionLearningUser-CentricSimulationDriver Behavior ModelAnticipatoryReasoningHCI DesignNeighbor MapNeighbor MapGPS,CameraPassive InformationBuildingLidar,etc.Proactive WarningDriver2)NeighborMapCommunicationFig. 2. System flowchart of the proactive driver assistance system.The rest of the paper is organized as follows. Next, the analytics and reasoning methodology isdescribed, prior to which the data collection process, as well as the data, is presented. In Sec. V, theconstraints and limitations in communication will be addressed. In Sec. VI, we explore the variety ofdesign challenges of the frontend exposed to end-users. Finally, we present the concluding remarksand future outlook in Sec. VIl.IILDATACOLLECTIONScooters are one of themost important transportation means in Taiwan.Out of 22millionregistered vehicles in Taiwan, scooters account for 67.2% of the vehicles - every 1.56 persons inTaiwan own a scooter. It is popular due to its higher fuel efficiency, lower sale price, and better abilitytomove through heavy traffic jams in theurban area, compared to a regular passenger car.Howeveralso due to its lower sale price, which results in less safety features incorporated in it, and its highermobility,which increases theprobability of a collision with other vehicles,scooters havecontributedto more than 80% of deaths in traffic accidents in Taiwan, causing more than 2,000 fatalities annuallyin the past decade.It is therefore crucial to develop a safety system that can helpto improve the safetyof the scooters on the road, while the solution needs to be able to be implemented within the costmargin of a regular scooter, which usually sells for approximately 2,000 U.S. dollars, about one tenthof that of a regular passenger car.Onepossiblesolutiontoutilizeamobiledevice,suchasasmartphone,to implementsomeofthese safety features. As the market penetration rate of smartphones grows to be over the 50% markglobally, they are owned by the majority of the drivers and thus, when the safety features areimplemented on the smartphone, it does not increase the cost of the vehicle. In addition, smartphoneshave many built-in sensors that can be usedtoobservethe driving behavior of the scooter drivers andthe surrounding vehicles; these sensors include gyroscopes, accelerometers, cameras, GPS, etc. If
Fig. 2. System flowchart of the proactive driver assistance system. The rest of the paper is organized as follows. Next, the analytics and reasoning methodology is described, prior to which the data collection process, as well as the data, is presented. In Sec. V, the constraints and limitations in communication will be addressed. In Sec. VI, we explore the variety of design challenges of the frontend exposed to end-users. Finally, we present the concluding remarks and future outlook in Sec. VII. III. DATA COLLECTION Scooters are one of the most important transportation means in Taiwan. Out of 22 million registered vehicles in Taiwan, scooters account for 67.2% of the vehicles - every 1.56 persons in Taiwan own a scooter. It is popular due to its higher fuel efficiency, lower sale price, and better ability to move through heavy traffic jams in the urban area, compared to a regular passenger car. However, also due to its lower sale price, which results in less safety features incorporated in it, and its higher mobility, which increases the probability of a collision with other vehicles, scooters have contributed to more than 80% of deaths in traffic accidents in Taiwan, causing more than 2,000 fatalities annually in the past decade. It is therefore crucial to develop a safety system that can help to improve the safety of the scooters on the road, while the solution needs to be able to be implemented within the cost margin of a regular scooter, which usually sells for approximately 2,000 U.S. dollars, about one tenth of that of a regular passenger car. One possible solution to utilize a mobile device, such as a smartphone, to implement some of these safety features. As the market penetration rate of smartphones grows to be over the 50% mark globally, they are owned by the majority of the drivers and thus, when the safety features are implemented on the smartphone, it does not increase the cost of the vehicle. In addition, smartphones have many built-in sensors that can be used to observe the driving behavior of the scooter drivers and the surrounding vehicles; these sensors include gyroscopes, accelerometers, cameras, GPS, etc. If
behavior models can be established and used to predict hazardous behaviors in advance with thecollected smartphone sensor data, then advance warning can be provided to the driver of that vehicleor,via someforms of communications,tothe driver of a neighboring vehicle.The behaviors of thescooters are significantly different from the behaviors of cars, due to its smaller dimensions and that ithas one more degree of freedom in its movement - the lean angle of its body, ie., roll angle. Althoughthere have been many efforts in collecting driving behavior data for cars, to the best of ourknowledge,thereisalmostnoeffortsincollectingextensivedrivingbehaviordataforscootersormotorcyclesTable I. Description of the Collected Sensor TypesrsopneVideo Carideo that is split into 10-min segments, The resolution of the video depends on t1020x10,640x480,320x240,ofollowing fou06x144.ThevideousesH.264/AdvancedVideoCoding(AVC)recorded audio is recorded as part of the video file, using the Adaptive Multi-RateMicrophonAD:CDvelocity,and bearine (vehicle)are )10 - 30 Hz, depending on the phone modelAccelerometerin m/s that is applied to the device on all three physicalz),includingthefon10 - 30 Hz, depending on the phone modelacceleationforceinm/sthat isappliedtothedeviceonallthreephysicalLineartheGyroscope10 - 30 Hz, depending on the phone modethe device'srate of rotationn rad/s around each of the three physical axes (x,y.sticfieldforall threephvsaxes(x,y,z)inpendineonthephreesofrotationthatthedevicemakesaroundallthreephvsicalaTo obtain the necessary data for developing various scooter driver behavior model, in June toSeptember2013,wehaveconductedalarge-scaledata collectionevent.in which100 scooterdriversare hired to collect sensor data during their daily use of scooters, using an app that we developed thatis executed on theirown Android smartphone.Before the event, wehave also distributed phonemountstoall participants,sothattheirsmartphonescanbeplacedonthehandlebarofthescootersandthe back camera of the smartphones can be used to capture video of the surrounding environment ofthe scooters.The app functions as a video event data recorder for the user, but in addition to recording thevideo and the audio, it also collects data from many sensors in the phone. Table I shows the type ofsensorsthat weused inthesmartphonefordata collection andtheirdescription.Notethat some of thelistedsensorsarevirtual sensors,whosedataiscalculatedwiththerawdatacollectedbyothersensors.Data collected by the smartphones is uploaded to a back-end server via cellular data connections orWi-Fi connections in real-time
behavior models can be established and used to predict hazardous behaviors in advance with the collected smartphone sensor data, then advance warning can be provided to the driver of that vehicle, or, via some forms of communications, to the driver of a neighboring vehicle. The behaviors of the scooters are significantly different from the behaviors of cars, due to its smaller dimensions and that it has one more degree of freedom in its movement - the lean angle of its body, i.e., roll angle. Although there have been many efforts in collecting driving behavior data for cars, to the best of our knowledge, there is almost no efforts in collecting extensive driving behavior data for scooters or motorcycles. Table I. Description of the Collected Sensor Types To obtain the necessary data for developing various scooter driver behavior model, in June to September 2013, we have conducted a large-scale data collection event, in which 100 scooter drivers are hired to collect sensor data during their daily use of scooters, using an app that we developed that is executed on their own Android smartphone. Before the event, we have also distributed phone mounts to all participants, so that their smartphones can be placed on the handlebar of the scooters and the back camera of the smartphones can be used to capture video of the surrounding environment of the scooters. The app functions as a video event data recorder for the user, but in addition to recording the video and the audio, it also collects data from many sensors in the phone. Table I shows the type of sensors that we used in the smartphone for data collection and their description. Note that some of the listed sensors are virtual sensors, whose data is calculated with the raw data collected by other sensors. Data collected by the smartphones is uploaded to a back-end server via cellular data connections or Wi-Fi connections in real-time
Fig. 3. The footprints of the participating scooter drivers over the 3-month event duration.Overthe3-monthperiod, a large amountofdatawas collected.Thefollowing summarizes somestatisticsofthecollecteddata:(1) 10,858 video files, with a total size of 473.8 GB, were collected. Most of the files are 10minutes in length.(2) In total, we collected 28,273 kilometers of driving behavior data. Out of the 100 participants.8 of them collectedmore than 1,000 kilometers of data,while 22 of them collected 100-1,000kilometersofdata(3) The majority of the participants operate the vehicle in the urban area of Taipei city, whilesome of them operate thevehicle in other parts of Taiwan.IV.ANALYTICSANDREASONINGMETHODOLOGYIn this section, we will present two main technical components for anticipatory reasoning: driverbehavior learning and neighbor map building.A.DriverBehavior LearningIn the past few years, researchers have spent lots of money and human efforts to study how toimprove the quality of driving and to avoid traffic accidents caused by improper drivingbehaviorwiththeaidfromcomputers.In2oo9,a studyreportedbytheAmericanAutomobileAssociation(AAA)Foundation for Traffic Safety shows that there are 56% of deadly crashes between 2003 and 2007involve one or more unsafe driving behaviors typically associated with aggressive driving. In thiswork, we want to analyze whether it is possible to predict dangerous events and to alert in advanceusing heterogeneous sensor data. Also we want to learn whether being able to recognize the drivingstyles of drivers can boost the above performance.In this project, we have collected the heterogeneous sensor data of 100 drivers, which bring some
Fig. 3. The footprints of the participating scooter drivers over the 3-month event duration. Over the 3-month period, a large amount of data was collected. The following summarizes some statistics of the collected data: (1) 10,858 video files, with a total size of 473.8 GB, were collected. Most of the files are 10 minutes in length. (2) In total, we collected 28,273 kilometers of driving behavior data. Out of the 100 participants, 8 of them collected more than 1,000 kilometers of data, while 22 of them collected 100 –1,000 kilometers of data. (3) The majority of the participants operate the vehicle in the urban area of Taipei city, while some of them operate the vehicle in other parts of Taiwan. IV. ANALYTICS AND REASONING METHODOLOGY In this section, we will present two main technical components for anticipatory reasoning: driver behavior learning and neighbor map building. A. Driver Behavior Learning In the past few years, researchers have spent lots of money and human efforts to study how to improve the quality of driving and to avoid traffic accidents caused by improper driving behavior with the aid from computers. In 2009, a study reported by the American Automobile Association (AAA) Foundation for Traffic Safety shows that there are 56% of deadly crashes between 2003 and 2007 involve one or more unsafe driving behaviors typically associated with aggressive driving. In this work, we want to analyze whether it is possible to predict dangerous events and to alert in advance using heterogeneous sensor data. Also we want to learn whether being able to recognize the driving styles of drivers can boost the above performance. In this project, we have collected the heterogeneous sensor data of 100 drivers, which bring some