(Washington, DC)–A report from the Government Accountability Office (GAO) regarding the National Aviation Operations Monitoring Service (NAOMS) project was released today by senior Members of the Committee on Science and Technology. GAO found that NAOMS successfully demonstrated surveys of airline personnel could be used to identify emerging safety trends that might be precursors to aviation accidents. However, the Federal Aviation Administration (FAA), the target customer for the project’s results, rejected the premise that surveys air system professionals could provide useful information.
NAOMS was a research and development project conceived and designed in 1997 to use personnel surveys to provide broad, long-term measures on aviation safety trends. The goal of the project was to demonstrate that regular surveys of commercial pilots, general aviation pilots, ground and flight crew members and air traffic controllers could provide insight into how safety events were changing over time in the national airspace system. However, before NAOMS ended in 2004, only commercial pilots had been surveyed and further development of the tool was stopped due to changing priorities at NASA, budget cuts in the aeronautics program and hostility from FAA.
GAO found that the program was a success as a proof of concept with adherence to generally accepted survey principles and high rates of survey completion. However, GAO also found that the data collected in this program did not lend itself to ready analysis. Changes in sampling strategies, the pool of pilots surveyed, and problems with the possibility of multiple reporting of single events all contributed to making the NAOMS data extremely difficult to reliably analyze. The Committee had asked GAO to conduct a thorough analysis of the data, but released GAO from that request in light of the costs associated with overcoming methodological problems.
“This was a well-designed project that failed because it was executed without proper agency oversight and didn’t have the support it needed from its primary customer–the FAA,” said Science and Technology Committee Chairman Bart Gordon (D-TN). “I applaud the GAO report for the painstaking examination it made of the strengths and weaknesses of the NAOMS survey effort. I am disappointed that the data is not solid enough for careful analysis, but GAO has provided very valuable advice on how to restart a survey to get the most accurate results for policy makers to use in keeping the public safe.”
“GAO found that the Federal Aviation Administration was hostile to the NAOMS survey,” said Subcommittee on Investigations and Oversight Chairman Brad Miller (D-NC). “FAA still doesn’t see the value in asking pilots, crew members and air traffic controllers about their safety experiences as a way to identify problems. People who work on the front lines in the air safety system are exactly the people we should be listening to.”
“This is a perfect example of how good ideas and resources are wasted due to poor planning,” said Rep. Jerry Costello (D-IL), a senior Member of the Committee and Chairman of the Aviation Subcommittee on the Committee on Transportation and Infrastructure. “It should not take seven years to figure out that the program as constructed, for whatever reason, won’t work. This is particularly unacceptable given that the GAO found that NAOMS could offer valuable insights on aviation safety. NASA and the FAA must demonstrate a better ability to coordinate going forward, as they must collaborate on other important projects, such as NextGen.”
GAO found that a new survey would require “more coherent planning and sampling methods, a cost-benefit analysis, closer collaboration with potential customers, a detailed analysis plan, a reexamination of the sampling strategy, and a detailed project management plan to accommodate concerns inherent in any survey endeavor.”
For more information, please see the Committee’s website.
