If the TSA’s system at the airport (let’s say for example) is 99% accurate, does this mean they are 99% accurate in catching terrorists? That would be my assumption, until I read an article our guest poster, Art Carden wrote on Forbes.com.
According to Art, when a NSA program (talking about a different governmental body now) is 99% accurate, this means it tends to find one false positive per 100 people. But since the sample size of terrorists is much smaller than the group of public at large, this skews the effectiveness in actually identifying terrorists. Art, the economist, does the math. I am just badly reiterating, so check out his article.
He concludes organizations like the NSA or the TSA “…would have to have a test that’s about as accurate as a DNA test. I’m willing to believe that’s possible, but possible certainly isn’t the same thing as likely.”
I tried to track down an accuracy study of the TSA scanning process. I was surprised to find a study showed that over time, scanners get better at spotting specific shapes. In this study, they were 88% accurate as opposed to the amateurs, who were 82% accurate.
That doesn’t really give me an overall picture of who accurate the process is though. What Art’s art-icle (see what I did there) did show me is I should always question a statistic I am given. A 99% accurate test does not mean it is 99% accurate at catching terrorists.