The government in the US (DOT) creates airlines' on-time arrival rankings based only on the fraction of their flights that arrive 15 or more minutes late. This kind of threshold rule is common in other industries (restaurant hygiene grades, hospital report cards, green building standards, students test scores) generates incentives for firms to game the system by putting relatively more effort to lower delays in those flights that are expected to be around 15 min late so they fall below 15min. The way firms usually do it is by tying employees's rewards to improvements in the ranking.
From a new paper by Silke Forbes, Mara Lederman and Trevor Tombe:
"We find little evidence of gaming by airlines that have no incentive programs in place or by
airlines that have implemented incentive programs with targets that are unrealistically hard to achieve. On the other hand, we find strong evidence of “gaming” by airlines that have incentive programs with a target level of performance that can realistically be achieved."
What's more striking is that the technology used to report delays to the DOT differed among airlines: some used automatic system (computer reports to the DOT) and some had a manual system (employee fills delay form and sends it to the DOT). Yes, you are guessing right. The gaming was coming mainly from the airlines using manual systems. Look at the following 2 figures taken from the paper:
The spikes at 0 and 14 reflect that the gaming is coming from cheating more than increasing effort. And guess what, Southwest, the company that was promoting their awesome on-time arrivals rankings in the 90s and early 00s used manual reporting! There are many HBS case studies that show Southwest as a leading case on how internal organization and bundle of activities can be used to create competitive advantages. Competitors never succeeded at imitating SW's business model (e.g. TED). Maybe it was just one decision that made the whole difference: manual reporting!
Btw, Southwest now uses automatic reporting... And is not doing so well on the rankings.
