Lies, damned lies, and statistics

Making sense of crime data.

‘‘It’s hard to argue with the data” is one of the most overused phrases wreaking havoc on honest debates — especially regarding crime. Politicians, academics, and pundits use crime data to argue whether certain public safety policies are working or failing. But with so many variables and policy agendas, it becomes difficult to determine whether crime is up or down, and if it’s down, does that necessarily equate to increased safety? 

The truth about crime data  

Data is often considered neutral and unassailable. While that’s technically true, many variables can call into question the veracity of the data, such as how it’s collected, by whom, for what purpose, who’s analyzing it, and even who is reporting it.  

Crime data is no different, and in recent years it has been used to make arguments about the impact various policies have had on crime and public safety.  

Don’t expect the media to offer clarification, as evidenced by internet search results about crime rates: 

Crime data requires analysis and interpretation for it to be of any real value and is often used to support arguments for or against something — such as policy proposals. When used this way, accusations of cherry-picking and manipulation are common.  

Data should be used in policy debates — both in primary arguments and counterarguments. It’s this adversarial approach, like our system of justice, which allows for a full examination of an issue.  

With this understanding, available crime data should be interpreted to challenge the prevailing narrative that “crime is down” and, correspondingly, that Minnesota’s public safety is improving in any meaningful way.  

Where does our crime data come from?  

There are two major forms of crime data collected in the United States. The first and most prominent is crimes reported to law enforcement as recorded by the Federal Bureau of Investigation (FBI) in its Uniform Crime Reporting (UCR) Program. The second is crime victimization as recorded by the U.S. Bureau of Justice Statistics (BJS) in its annual National Crime Victimization Survey (NCVS). Together, they comprise the bulk of U.S. crime data. 

UCR DATA  

The FBI began collecting UCR crime data voluntarily submitted by local law enforcement in 1930. The information collected in the UCR system represents our nation’s standard for crime data. It is UCR data that is most often referred to by politicians, law enforcement leaders, and the media when referencing national crime rates.  

The data is collected by individual police agencies throughout the country and reported to the FBI, historically through the Summary Reporting System (SRS), and more recently through the National Incident-Based Reporting System (NIBRS).  

The FBI publishes the annual data in its “Crime in the Nation” data summary document. In recent years the FBI has automated much of the data, making it available live on the FBI’s web-based Crime Data Explorer. The live data is continuously being added as law enforcement agencies submit data, so there is often a slight difference between it and the annual publication. 

NCVS DATA  

The BJS began collecting and reporting data on victimization in 1972. Each year it conducts a survey of criminal victimization by interviewing a nationally representative sample of approximately 240,000 persons in about 150,000 households. The annual NCVS report represents the primary source of information on national criminal victimization.  

There is no comparable state-level victimization data. 

MINNESOTA’S CRIME DATA  

In Minnesota, reported crimes are documented by hundreds of local law enforcement agencies and transferred to the Minnesota Bureau of Criminal Apprehension (BCA), which then reports the data to the FBI via NIBRS.  

Minnesota successfully transitioned to reporting via NIBRS in 2016, years before the 2021 transition was mandated. Local and county agencies reporting their crime data to the BCA exceeds 99 percent. The Minnesota crime data accessible in the FBI Crime Data Explorer is as complete and comparable as any crime data available, notwithstanding some limitations. The Minnesota data in the FBI Crime Data Explorer differs slightly from data published by the Minnesota BCA, likely due to a combination of the estimation process by the FBI and the fact the data is live and continuously updated.  

The limitations of crime data  

The most significant limitation of crime data is that most crimes are not reported to law enforcement. This is further compounded by the fact that the reporting rate varies each year as circumstances around crime change. According to the NCVS, less than a third of property crimes and only about half of violent crimes are reported. This means the data in the UCR is based on a major under-reporting of actual crimes committed and a rate of underreporting that can’t be reliably known — the best we have is the NCVS survey.  

Another limitation of crime data was the consequence of several transitions that have taken place in recent years.  

In 2021, the FBI attempted to transition away from the SRS to NIBRS. The transition was problematic because approximately 35 percent of the nation’s law enforcement agencies, including those in many of the nation’s largest cities, didn’t make the transition, leaving the UCR grossly underreported in 2021 and 2022, making comparison with those years suspect. 

This transition and the large percentage of agencies that failed to report are believed to have contributed to a recent controversy involving crime data. This fall, the FBI updated 2022 crime data after originally reporting crime reductions for 2022. Conservative outlets noted that the updates changed the data enough to erase many of the reported reductions in crime, and that the FBI did little to explain the discrepancy. While likely representing the addition of data received from some of the delinquent reporting, the controversy has, for some, diminished the integrity of our crime data.  

Other transitions that have occurred include retiring the historic Part 1 and Part 2 crimes designations and moving to Group A and Group B designations that include a more varied set of crimes. This transition has made accurate historical comparisons more difficult, but not impossible, especially when using the live Crime Data Explorer.  

Another limitation of crime data is in the final UCR data. It is actually an estimate and not the raw numbers submitted by law enforcement. This estimation process, while likely scientifically sound, adds to the uncertainty of the data — a fact the FBI acknowledges in its publications. 

One limitation associated with the NCVS data is that it is based on voluntary participation from a “representative sample” — 240,000 individuals out of over 375 million Americans. The survey is also not administered by the BJS, but rather by the U.S. Census Bureau. Finally, the NCVS has historically estimated victimization incidents but has not estimated victimization prevalence to determine the number of unique victims. 

Analysis and interpretation of crime data  

METHODOLOGY  

For consistency in comparison, the following analysis of crime data was made using UCR data culled from the FBI’s Crime Data Explorer and victimization data published in the annual NCVS.  

The analysis uses the two crime categories collected and reported in the UCR: Violent Crime (Murder, Aggravated Assault, Robbery, and Rape), and Property Crime (Larceny/Theft, Burglary, Motor Vehicle Theft, and Arson). The combination of these two categories has historically been called Part 1 crimes or the Crime Index.  

The analysis focuses on the years 2018 — a baseline year comfortably before COVID and the civil unrest in 2020, and 2023 — the most recent full year that complete crime data is available.  

WHAT THE CRIME DATA SHOWS  

A cursory look at crime data appears to support the argument that “crime is down” both nationally and in Minnesota. But a deeper analysis of the state data, especially in Minneapolis, offers compelling evidence that disputes the narrative that crime is decreasing or that public safety is improving.  

While the U.S. crime data shows slight decreases in the crime rates nationally between 2018 and 2023, those indicators are not consistently shown in Minnesota’s crime rates and are non-existent in Minneapolis’s crime rates — this despite frequent efforts by our politicians and many media outlets to suggest otherwise. 

The biggest takeaway from the data is the clear evidence of significant increases in our violent crime rate (murder, aggravated assault, rape, and robbery) both at a state level (up nearly 19 percent) and in Minneapolis (up 41 percent). These increases appear consistently in both the NCVS and the UCR data. This is a critical revelation because when people consider whether their public safety situation is improving, they consider the direction the violent crime rate is moving, not whether property crimes are rising or falling.  

The following figure illustrates how Minnesota’s violent crime rate is trending compared to the national violent crime rate. The difference is alarming. 

In an environment where murders, assaults, rapes, and robberies have increased dramatically, it’s difficult to believe that property crimes have somehow decreased over the same time. While some of the data suggests this is the case, a closer look at the data disputes that conclusion.  

The extrapolation of victimization data in the NCVS shows that total property crimes in Minnesota were down 2.3 percent between 2018-2023, while reported property crimes in the UCR show a decrease of just over 14 percent.  

The data reveals that the crime category of larceny (which includes crimes such as shoplifting and thefts from coin-operated machines) accounts for two-thirds of all Part 1 crimes reported each year. Therefore, when larceny reports drop, it can have a significant effect on the overall crime rate. That is exactly what happened in Minnesota between 2018-2023 when the number of larcenies reported to law enforcement fell by 13,713. The decrease in larcenies was more than the total decrease in all reported Part 1 crimes (just 11,243) over the same time — a reduction that skews the data and leads to misinterpretation of overall crime reduction — and certainly misinterpretation of whether our safety has improved.  

We know from the NCVS that three out of every four larcenies were not reported on a national level in 2023. Given the environment in Minnesota, still fresh off the rioting and anti-police movement that took hold following 2020, it’s probable that the reporting rate for larcenies and other property crimes was even lower than the national level. It is, therefore, irrational to believe the large drop in larcenies reported reflects an actual drop in larceny crimes or other crimes being committed.  

What is more likely is the public has become desensitized to crime and concluded that it isn’t worth their time, or the time of overburdened law enforcement or an underperforming court system, to report petty thefts. If true, this foretells a sad degradation of our state’s overall wellness — one that could be avoided by valuing and prioritizing law and order and a consequential criminal justice system.  

Furthermore, if larcenies are removed from the evaluation of Part 1 crimes due to unreliable reporting and misleading conclusions, the remaining Part 1 crime rate in Minnesota worsens by 14.8 percent from 2018-2023. 

THE MINNEAPOLIS EFFECT  

Minneapolis has an undisputed, oversized impact on Minnesota’s real — and perceived — well-being. Naturally, no evaluation of our crime situation can take place without understanding Minneapolis crime statistics:  

• Minneapolis accounts for a quarter of all Part 1 crimes, a third of all violent crimes, a third of all murders, and nearly half of all robberies in the entire state.  

• Between 2018-2023 in Minneapolis, robberies increased 21 percent, aggravated assaults increased 29 percent, murders increased 171 percent, and motor vehicle thefts increased 254 percent.  

• The increase in the number of motor vehicle thefts in Minneapolis from 2018-2023 (5,584) exceeded the total increase in statewide motor vehicle thefts over the same period (5,199). As crime in Minneapolis goes, so does crime in our state. That’s quite a legacy for a city representing seven percent of the state’s population. 

ASSESSMENT OF CRIME  

Perception, intuition, and lived experience are involved in the public’s assessment of crime in the U.S. Between 2018-2023 the public’s concern over crime in the U.S. increased from 48 percent to 63 percent, according to Gallup polling. This concern grew while our nation’s crime data suggested crime was decreasing. The Thinking Minnesota Poll taken in Dec. 2021 reflects a wide majority of Minnesotans (86 percent) being concerned with the level of crime in the state.  

CONCLUSION  

Some may continue to put their money on “neutral and unassailable” data showing a decrease in reported crime. However, upon reviewing the data and uncovering its limitations, more credence should be given to the public’s assessment that crime is not down.  

The public may not know the data, but they know what they see, hear, and feel about crime and their personal safety and they’re justified in their assessment. Crime reporting may be down, especially property crimes like larceny, but the crimes that matter, such as crimes of violence, are not down, and Minnesotans feel it in their everyday lives and communities — a fact not even the data disputes.