banner of Methodology for Determining Motorcycle Operator Crash Risk and Alcohol Impairment

4. Research Priorities

As part of the expert panel meeting, panelists were asked to assign priorities to each methodology discussed. A high priority would mean that panelists felt that the methodology was among the best candidates for future research funding because it provided a high likelihood of providing scientifically valid data. No attempt was made to reach a consensus on these priorities, and there was some disagreement as to the relative priorities of each methodology. Panelists used their own system for expressing their priorities, making it impossible to calculate “scores” for each methodology. However an examination of the comments given by panelists provides an excellent sense of how they viewed each methodology. For example, of the panelists expressing an opinion on the priority of the case control study, one described it as “highly recommended,” one described it as high priority if sufficient funding could be obtained to do it properly, one said this it was highest priority, one said it was second highest, and one rated it third highest. One panelist listed it as low priority out of a concern that it may be too difficult to get valid comparison data.

Project staff, using panel member comments and priorities, created a system for prioritizing the various research methodologies discussed within this project. Using the comments made by the attendees of the meeting, both in their listing of priorities and the comments made during the panel meeting, one of three levels of “scientific validity” (low, medium, and high) was assigned to each methodology. These levels represent how scientifically valid the results for each methodology would likely be, given the barriers to obtaining complete and accurate data for that methodology. The ratings of scientific validity contribute to the assigning of priorities for each methodology. It is important to note, however, that scientific validity was not the only contributing factor to assignment of priorities. The relative cost (low, medium, and high) and practicality of a methodology could affect its priority. Because cost categories were broad and costs were only estimates, it is possible that priorities would be revised if accurate cost figures were available for each methodology.

Part of the process of assigning priorities was to pair methodologies that only provide crash data with methodologies that only provide comparison data (e.g., Geo-General Comparison Data with Fatal Crash Records). Table 4 shows the methodologies arranged in a matrix. One dimension of the matrix shows cost categories (low, medium, and high), whereas the other shows levels of scientific validity (low, medium, and high) of the results expected from the methodologies. For the most part, the table serves as a good indication of which methodologies are the highest priority, i.e., those methodologies with the highest scientific validity within each cost category would be the best candidates for future research within that cost category. Exceptions to this rule of thumb occur when a methodology would either be so inexpensive as to be warranted, or so expensive as to be practically impossible. It should also be noted that the cost categories are somewhat broad. Of two methodologies in the same cost category, one may ultimately be sufficiently less expensive than another so as to increase its priority relative to the other. This might occur when the actual costs are estimated. It should also be noted that levels of scientific validity are set relative to other methodologies within the same cost category. For that reason, a methodology rated as “low” or “medium” within a given cost category (e.g., Geo-General + FARS) may be more scientifically valid than one rated “high” in another cost category (e.g., Fuel Station Survey with Injury Crash records).

Table 4. Cost by Validity Matrix



Scientific Validity





Contemporary Case Control



Fuel Station Survey with Fatal Crash Records
Fuel Station Survey with Injury Crash Records

Emergency Department

Geo-Specific + Fatal
Crash Records


Induced Exposure*


Geo-General + Fatal Crash Records

* Note that the position in the table for these two methodologies does not correspond with the priority ultimately given to them.