Monday, February 24, 2020

Crime metrics matter: two charts of the big mess we're in, even if we're not sure how big it is

[Update: Advancing Accurate and Objective Cybercrime Metrics, my article in the Journal of National Security Law & Policy is now available online.]

We are now about 50 years into the information age, so let me ask you: How secure is your personal information? If you're like most adults in America, the answer is probably: "not as secure as it used to be."

Chart linked to original report.
That is what the social scientists at Pew Research Center found last year when they carried out the survey behind the chart on the right (click to access the full report).

As you can see, 70% of folks said that they felt their personal information was less secure than it was five years ago; furthermore, they were more likely to think that way if they were 50 or older, more educated, or in a higher income bracket.

My take on these numbers is that they reflect the relentless increase cybercrime, or what my good friend, the gifted security researcher Cameron Camp, calls cyberbadness: the apparently never-ending litany of technology-enabled scams, frauds, thefts, losses, and disruptions that seem to be victimizing more and more people and organizations. Note that I used the word "seem" intentionally because some observers will point out that public perception of criminal activity is not always in sync with reality. At times that may be true, but before we can determine whether people are over- or under-reacting to cybercrime, we need to ask: what is the true scale and impact of cybercrime? And quite frankly, nobody has a good answer at the moment.

Why? Well, I've said it before, years ago, and again last year: "the importance of metrics to crime deterrence would appear to be both critical and obvious, but despite this there is a persistent cybercrime metrics gap." As far as I am concerned, that is a problem, one that I addressed at some length in a recent law journal article that is currently available online. The following quote may help to put the problem in perspective:

“[u]ntil there are accepted measures and benchmarks for the incidence and damage caused by computer-related crime, it will remain a guess whether we are spending enough resources to investigate or protect against such crimes… In short, metrics matter.”

Those words were spoken 16 years ago by an FBI agent, Edward J. Appel, someone who knew thing or two about metrics (his father, Charles A. Appel, founded the FBI's Technical Laboratory).

Unfortunately, casual use of Google gives the impression that we have an abundance of metrics of about cybercrime, with search results like "300+ Terrifying Cybercrime & Cybersecurity Statistics" and "110 Must-Know Cybersecurity Statistics for 2020." The problem is, the sources for such numbers are often suspect in terms of methodology and/or confirmation bias.

I addressed these issues in the Journal of National Security Law & Policy article mentioned above (Advancing Accurate and Objective Cybercrime Metrics (publication pending, but currently available online). And I had spoken at length about the problem at the 2015 Virus Bulletin security conference (you can find my paper, a video of my talk, and my slides here: Sizing Cybercrime: incidents and accidents, hints and allegations. The sad reality is that, when it comes to timely and objective official statistics about crimes committed in cyberspace, they are in short supply.

Even sadder is that fact that the metrics we do have, such the Internet Crimes Reports issued by the FBI and IC3, make for depressing reading, not to mention depressing charts like the one on the right. This documents the rise in total annual crime losses reported to the Internet Crime Complaint Center or IC3 from 2003 through 2019.

As you can see, the year-on-year increase has become quite acute. Yes, I know the chart is somewhat compressed to fit this page layout, but you would have to spread it quite wide to get rid of the "hockey stick" that is the last five years. I certainly wouldn't bet against it blowing through $4 billion in the next report.

And yes, there are issues with using the IC3 numbers as crime metrics. They are not collected as an exercise in crime metrics, but rather as part of just one avenue of attack against the crimes they represent. However, I have studied each annual report and am satisfied that collectively they provide solid evidence of a real world cybercrime impact trend that looks very much like the line shown here.

My law review article was one of several generated by a range of independent subject matter experts as part of the Third Way Cyber Enforcement Initiative. The initiative was an impressive multi-stage effort to coordinate inter-disciplinary input on efforts to tackle the cybercrime problem. When commissioned papers reached draft stage, authors attended a day-long, mid-summer workshop at New York University School of Law for live peer review. By October, this Third Way initiative had already produced results, including an excellent summary of the metrics issue in The Need for Better Metrics on Cybercrime, from Third Way Policy Advisor, Ishan Mehta.

As papers continue to appear, check the website of the Journal of National Security Law & Policy. For example, right now you can access this important contribution from Amy Jordan and Allison Peters on Countering the Cyber Enforcement Gap: Strengthening Global Capacity on Cybercrime, and this excellent review of the use of criminal charges as a response to nation-state hacking from Tim Maurer.

(Acknowledgement: I am deeply grateful to all who participated in this project, for their input, insight, enthusiasm, and support.)