Collision Avoidance System Field Operational Test
14 FIELD OPERATIONAL TEST (Task E)
The objectives of this task center on the preparations for and execution of the field operational test. In Phase I of this project, the objectives include:
The general approach has been to apply as much of UMTRI's prior methodology and learning from the ICC FOT as possible, adapting the field-testing techniques to the ACAS platform. The approach also involves advancing the state of practice with updated hardware and software for data acquisition and updated procedures to bring about an efficient and highly informative field test of ACAS. Noting that the ACAS system is much more complex than was the ICC system tested in 1997 and that much more of the system function is being developed in the course of the ACAS project, as opposed to simply testing a mostly pre-developed ICC system, UMTRI's approach has included significant engagement in technical discussions with the ACAS team. Thus, UMTRI staff have become increasingly knowledgeable on the makeup and operation of the entire system, while also giving critique and trial test results by way of feedback on system design to GM and Delphi team members.
From the viewpoint of an architecture for data acquisition, processing, and analysis, UMTRI has undertaken a top-down review of the approach, resulting in a major reconfiguration and upgrade of the data handling system. The intent is that the largest portion of this system plan will be confirmed through implementation of the data acquisition system and the associated database tools that are used for testing Stage 1 Pilot vehicles and the Prototype Phase Vehicle. Experience from these preliminary rounds of application will help in guiding a highly efficient and productive approach for the later field test of the ACAS fleet of vehicles.
Milestones and Deliverables through June 2000
The FOT Pilot Test Plan was delivered to NHTSA in January 2000.
The work accomplished will be summarized in three sections as follows:
The Data Acquisition System
Figure 14.1 shown below is the conceptual layout of the ACAS test vehicle, showing the points at which it is to be interfaced with the DAS unit. Work on DAS development has included a substantial effort to understand the complete ACAS system and the proper means for interfacing with it. While various ancillary signals from the ACAS vehicle are given an interface with the DAS to support power control functions, the primary datalink is via the CAN bus. The DAS package interfaces with the UMTRI lab facility both via logic control and data link mechanisms. The DAS also supports data collection outside of the domain of the CAN interface by means of a concern button, video cameras, and a microphone/speaker provision for driver comment.
Figure 14.1 Vehicle Interfaced With the DAS
The operation of the DAS is depicted in Figure 14.2, indicating that objective, quantitative data and audio/video data are handled separately. The primary control of the data collection process is vested in the computer that handles objective data, yielding time-stamped and stored samples of more than a hundred selected variables as well as transition files that identify when certain logical states have been satisfied. Among these states are those matching certain criteria that trigger the storage of audio and video files that have been temporarily buffered in a loop-&-store memory.
Figure 14.2 Objective, Quantitative Data and Audio/Video Data
The DAS communicates both between its two primary units and, upon return of the test car, directly with UMTRI's main archival computers via Ethernet connections. When a subject returns the vehicle to the UMTRI building, an Ethernet cable connection is patched to the DAS in order to recover the full data set. Throughout the test operation, cell modem connections to UMTRI are made at the end of each ignition-off cycle of the vehicle to download a trip summary that affords a means of monitoring progress in the field.
Figure 14.3 shows the basic hardware plan for the DAS. Two single-board computers support the respective "main" and "video" (including audio) storage modules. The link with GPS, aside from the recovery of GPS coordinates via the CAN bus, is to support the synchronism of DAS records with the Pulse-Per-Second (PPS) signal that allows GPS to serve as the master clock. The link to a box labeled "ignition, door locks, dome light" is a provision for accelerating the start-up of the DAS in order to minimize the time delay of DAS availability at the start of each new trip.
14.3 Basic Hardware Plan for the DAS
The Larger Architecture for a "Data System"
Figure 14.4 depicts the overall architecture of the data system, which supports research on FOT data. Included are elements that provide for the creation of test data via the DAS package, investigation of data based upon browsing and analyzing the contents of the database, appending analytical results as an augmentation of the test-derived database, high-level examination of reduced or aggregated data using data mining and visualization tools, and sharing data with others via a Web server. This overall architecture provides for an integrated multi-use of software that may be downloaded onto an individual DAS package in a vehicle as well as installed as part of the archival record of FOT results within the database.
The diagram shows that data are stored in each of three forms: namely as metadata, the database, and a data warehouse. Metadata serve to store objective attributes of the test design, test conditions, measured variables, real-time computations conducted on board the DAS package, etc., as well as corresponding attributes which define post-test analyses and simulations that were performed on test data drawn from the database. In general, metadata constitute instructions within the DAS package as well as a permanent bookkeeping record accompanying all measured and analyzed data elements.
The database represents one or more relational databases in which the FOT data and analytical derivations are stored. The most detailed study of the FOT results will involve exercise of the indicated "Explorer" and "Cruncher" tools, shown at the upper right, as specialized utilities for accessing and operating upon the relational database.
The data warehouse provides for convenient study by many researchers of results aggregated for high-level analysis. Significant response "FACTS" are defined for aggregating the data—for example, warning events, ACC interventions, driver-cited "miss" events, etc. Each FACT has "click-able" dimensions such as driver, road, time, cruise state, etc., and each dimension has attributes such as driver age, gender, driving style, velocity range, etc. A so-called "Pivot table", then, provides a query engine that is optimized for immediate analysis of the multiple dimensions of the FACT. The practical result is that the insight process is better stimulated by richer initial displays of data whose study can be better sustained, once the researcher gets on a good discovery track, because the pursuit process has been made so efficient.
14.4 Overall Architecture of the Data System
Testing the Opel Vectra Engineering Development Vehicle
Eight UMTRI staff members drove the Opel ACC vehicle over a 94-mile route during June of 2000. Each of these individuals was experienced in developing and testing ACC vehicles, some for as many as seven years and some for as little as one year. Prior to beginning the route, each person was given an orientation drive of approximately twenty minutes in order to learn system operation.
The route originated in Ann Arbor, proceeded along mostly freeway segments into Southfield, went south on freeways and a 24-mile segment of Telegraph Road to Flat Rock, and proceeded back to Ann Arbor on mostly freeways. Each person who drove the route filled out a 21-point questionnaire and the results were compiled and summarized for informing later stages of the ACAS activity.
The intent of this exercise was to gain first-hand experience that would guide the development of UMTRI and GM Institutional Review Board (IRB) and NHTSA Human Use Review Panel (HURP) applications required under the ACAS study. Additional collection of quantitative data from the Opel also facilitated the planning of UMTRI's data acquisition system, as well.
The Opel package afforded a brake-assisted ACC functionality and multi-target radar that was seen as reasonably approximating those which will support the ACC features of the ACAS system, albeit an earlier generation of the same having various differences in calibration of its ACC controller. Since the test drivers were encouraged to get as much ACC driving experience as possible along the route, they each took a special effort to maximize engagement while also maintaining reasonable safety margins. As a brief indicator of the scope of this pilot testing activity, Figure 14.5 presents the utilization results covering four cases defined as follows:
Both the actual and estimated (month-later) Opel utilization results are shown as bars whose width represents +/- one standard deviation about the reported mean value for eight drivers. The ICC FOT and CCC benchmark results are shown only as the average values obtained across the 108 test subjects in the earlier field test. Road types are distinguished by freeways, two kinds of surface streets, and the overall set of roads that were driven.
Figure 14.5 Cruise Utilization Level
Figure 14.6 shows that this rather motivated group of professionals drove with ACC engaged as much as possible on each leg of the route. Thus we see 80% to 90% utilizations of the Opel's ACC system across the entire route. While utilization on freeways is projected to remain within the 80 to 90% range in normal usage after a month's experience, the corresponding utilization values on surface streets would drop typically in half, or less. That is, UMTRI's drivers were pushing it during the 2-hr test drive in order to gain the operational experience, but they anticipate that a lesser utilization level—perhaps in the vicinity of 40%—might be adopted as normal behavior on surface streets, once a substantial level of familiarity prevails.
Against the ICC-FOT and CCC system utilization benchmarks, it is clear that the brake-assisted ACC system will be more heavily utilized. On surface streets, especially, the higher deceleration authority is expected to induce much higher utilizations than the two benchmark cases, presumably exposing users to the more complex conflicts that such environments present.
ACC utilization during the ACAS FOT is expected to lie above 60%, overall, whenever the vehicle is being operated above the minimum speed needed for ACC engagement.
Research findings fall into two categories: those associated with usage of a pilot version of the DAS package and those deriving from subjective evaluation of the Opel's ACC system.
Regarding Data Acquisition
A Data Acquisition System that included many but not all of the features and software architecture of the eventual FOT package was successfully constructed and operated on the Opel EDV. The system collected a large set of quantitative variables from the Opel's CAN bus and, among other things, provided UMTRI an early look at the multi-target radar data. A sample summary of such data is shown below in Figure 14.6, presenting a range vs. azimuth histogram that resulted from driving over the same 94-mile route as had been used in the subjective testing series. The figure shows the range/azimuth data for all targets flagged as stationary—and which thus resided outside of the proximate zone of the lane ahead of the vehicle. Thus, there are roadside objects and parked vehicles lying left and right of center at intermediate ranges and, at long range, objects that are assumed to be overhead bridges and signs.
Moreover, the early operation of an UMTRI data acquisition system on the Opel EDV served to confirm readiness for handling the later tasks of data collection in this project.
14.6 Range vs. Azimuth Histogram
Regarding the Opel ACC Evaluation
The Opel ACC system was found to be highly operable and was successfully driven in the Engaged Mode over approximately 90% of its mixed-route miles by eight different individuals. The 80%-and-above utilizations that were achieved with this ACC system on surface streets are acknowledged to be unusually high. Nevertheless, the intent was to explore and identify several of the challenging conflict types that primarily manifest themselves in this roadway environment. Utilization of such an ACC system on major surface streets in normal usage is anticipated to go up into the 40% range, with conflicts accompanying.
Additional conflicts were also observed on freeways due to a peculiar aspect of the Opel's control rule whereby braking was applied to resolve temporary headway incursions, even when no overtaking transient is present. At the same time, drivers observed conflict-mitigating aspects of their own behavior with ACC engaged, operating at longer headways and passing other vehicles less often with ACC engaged than they would have if driving manually.
Moreover, the Opel driving tests provide subjective indication that an ACC functionality of this kind calls for careful preparation of test subjects if HURP approval is to be ensured. It also confirms UMTRI's prior experience in the ICC-FOT, which showed that various conflicts would be encountered during ACC operation, some of which are not altogether unlike their occurrence in normal driving. Clearly, a central object of the FOT investigation will be to determine the ability of laypersons to resolve these conflicts and to elect ACC utilization levels and patterns of vigilance that serve to contain the risks.
Plans through June 2000
Shown on the next page is the task schedule covering the first phase of the ACAS FOT project. The schedule shows that all of UMTRI's Task E assignments, including the Pilot Test Plan, the preparation of a DAS package for testing engineering phase vehicles, and pilot testing by UMTRI professionals using the Opel EDV were all on schedule.
Plans through December 2000
Over the next six months, the following important subtasks and milestones will be completed:
UMTRI is also heavily engaged in advancing the DAS package for use on the Prototype Phase Vehicle during the next six months.
14.7 Task E Schedule
[TITLE PAGE] [TABLE OF CONTENTS]
[1 Executive Summary] [2 Introduction] [3 System Integration] [4 Forward Radar Sensor]
[5 Forward Vision Sensor] [6 Brake Control System] [7 Throttle Control System]
[8 Driver-Vehicle Interface] [9 Data Fusion] [10 Tracking & Identification] [11 CW Function]
[12 ACC Function] [13 Fleet Vehicle Build] [14 Field Operational Test]
[Appendix A] [Acronyms]