Why the economics of minimum wage increases are so tough to measure

Last week, several researchers at the Federal Reserve posted an analysis of the minimum wage in New York state. In the grand sweep of the research and debate done on the minimum wage, both the blog post and the dispute it kicked up were minor blips. But as a microcosm for that grander debate, it was actually rather instructive.

Basically, a trio of researchers at the Federal Reserve branch of New York spotted a chance for a natural experiment. Over the last few years, New York state has passed a series of minimum wage hikes that are gradually phasing in: a three-year hike for tipped workers, started in 2013, from $7.25 to $9.00 an hour; a 2015 law that raises fast food workers to $15 an hour by 2021; and a 2016 law that will raise the whole state to $15 an hour by 2022. Meanwhile, neighboring Pennsylvania remains at the federal minimum wage of $7.25 an hour. Since the effect of the minimum wage on jobs is hotly contested — critics contend it raises pay at the cost of less employment — why not look at the counties that lie on either side of the New York-Pennsylvania border, and see how they performed? If minimum wage hikes harm employment, that should show up in a comparison.

The researchers wound up looking at two sectors from 2010 to 2018 — retail, and leisure and hospitality — across 10 New York counties and 9 Pennsylvania counties. They found that earnings grew faster on the New York side. But since the minimum wage was going up, that's not surprising. What's more important is that, while retail is declining everywhere, both sets of counties declined at essentially the same rate. Meanwhile, leisure and hospitality employment grew at the same rate. Essentially, they found zero employment effect.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

It didn't take long for a conservative-leaning think tank, the Empire Center, to post a critique of the Fed's analysis.

The logic for focusing on counties on either side of the border is that, being right next to each other, they presumably share similar regional economies. The chances that you're comparing apples and oranges — that your analysis will be thrown off by random events and confounding variables — is minimized. "The Fed researchers sort of assumed, well, these are border areas so they're similar," E.J. McMahon, the founder and research director at Empire Center, told The Week. "But they're not actually similar, and they're in different regional economies."

Specifically, McMahon pointed to two casinos that opened in Sullivan and Tioga counties on the New York side. He argued that casinos are an unusual enough business model — state-licensed gambling — that including the jobs they create in an analysis of the minimum wage obscures more than it reveals. He also pointed to the Fed's inclusion of Orange county on the New York side as problematic: It's an exurb of New York City, commuters into the city live there, and it's integrated with the greater metro area's public transit. The other counties in the study are more rural, and one widely agreed upon point in the minimum wage debate is that the policy's effects, whatever they are, are going to be different for low-cost-of-living rural areas than for expensive urban ones. McMahon also argued that the New York side is just bigger in population and more economically dynamic across the board.

McMahon reran the numbers from 2013 to 2018, but removed Sullivan, Tioga, and Orange counties. He found that leisure and hospitality jobs on the New York side now grew slower than on the Pennsylvania side: 2 percent versus 4.5 percent, respectively.

This is where things get murky for all involved.

McMahon wasn't presenting his analysis as the "true" or "correct" one — he was merely demonstrating that questions about what's the best apples-to-apples comparison can profoundly change the result. The public discourse often treats analyses like this as objective in a relatively unique way, because they end in definitive numbers. But the analytical plumbing that produces these numbers inevitably involves lots of subjective judgment calls.

When The Week spoke to the Fed researchers, they didn't find McMahon's logic crazy. (In fact, both sides spoke well of the others' professionalism in the interviews.) They just disagreed. "I don't think it's inappropriate to include Orange county," Jason Bram, one of the researchers, said. "Yes, it's technically part of the New York metro area, but it's about as urban as Erie County on the Pennsylvania side." Since the casino openings, and the business decisions involved, occurred over the same period as the wage hikes, he felt it was proper to include them as well. "The fact that they're opening a hotel a number of years into the ratcheting up of the minimum wage, I don't think that's something you want to ignore," Bram continued. For the casinos, there's also the question of whether you knock out the entire county, or whether you simply realize the spike at the end of the graph above, where job growth on the New York side (the blue line) outpaces the Pennsylvania side (in red), is probably an artifact of the casino openings.

You can also take this logic in the opposite direction: Are those three counties the only ones we should knock out, or are there more? Pike county, on the Pennsylvania side, is a more distant exurb of New York city, but still an exurb. Erie county, on the far west end of the Pennsylvania counties, accounts for the bulk of the jobs on that side, and is more of a rustbelt industrial community than the others.

Remove Pike and Erie from the equation, and job growth on the Pennsylvania side falls to 3.5 percent from 2013 to 2018. That's still above the 2 percent McMahon found on the New York side; but if you put Tioga and Sullivan counties (the ones with the casinos) back in, New York rises to 9 percent. Remove just Orange and Pike, and it's New York with 9 percent and Pennsylvania with 3.1 percent. Remove Erie, Sullivan, and Tioga, and it's 4.4 percent for New York and 6.7 percent for Pennsylvania.

Which version is the best version of this natural experiment? Who knows.

"Starting to remove counties when one is considering a small number of counties to begin with is a slippery slope," Arindrajit Dube, another economist who's done well-regarded work on the minimum wage, told The Week. "Better is to recognize the limitation of any single case study and instead consider it as one additional data point in the rather sizable and growing minimum wage literature." In other words, no natural experiment will be perfect, and the better approach is to just find as many natural experiments as possible, and see where the pattern converges.

In that respect, it's worth noting that Dube himself has been involved in some of the most comprehensive research in this regard. A recent study he co-authored, looking at the local results of over 130 minimum wage hikes over the last three decades or more, found that "the fall in jobs paying less than the new minimum wage had been fully offset by the jump in new jobs paying just over it," as The Washington Post summarized it. Dube also pointed to another recent study, focusing on 51 natural experiments since 2007 — which also looked particularly at rural areas and featured increases comparable to disruption we'd get from a national hike to $15 an hour. Once again, they found no discernible effect on employment. Meanwhile, a recent overview by the Congressional Budget Office concluded a national $15 an hour minimum wage would cause anything from zero to 3.7 million job losses.

Of course, that's big picture across the whole country. At the local level, there is enormous potential variation.

Beyond that, most everyone's moral priors on the minimum wage are pretty strong. Is your moral focus on preserving the freedom of business owners, or on higher wages for workers? If the latter, how much risk do you tolerate in your minimum wage hikes? How do you interpret the overall pattern of study results in light of theory, and what that says about more aggressive minimum wage hikes than natural experiments so far provide?

Then there's one final point McMahon made, about how the Fed study might play in politics. The Fed researchers themselves clearly intended this post as an interesting, while admittedly crude, intellectual exercise — and one that might hopefully turn into a far more sophisticated and comprehensive study in the future. But McMahon worried those qualifications would get flattened in how the study was treated in the wider discourse. Instead of an acknowledgement that the analysis was one imperfect data point among many, "it'll be 'the Federal Reserve says the minimum wage makes no difference,'" he said. "This thing could have a half life of easily a decade."

How economic researchers should calculate when to release work for public consumption is a difficult question. We've entered a strange new era where everything from crude working papers to finely polished and peer-reviewed studies can get picked up by reporters, scribblers, activists, and politicians. But the rest of us out in the discourse definitely have a responsibility to try to put these things in context.

I have strong opinions about minimum wage policy myself. But consider this column at least one effort to meet that responsibility.

Want more essential commentary and analysis like this delivered straight to your inbox? Sign up for The Week's "Today's best articles" newsletter here.