A Close Reading of Moneyball for Government & Why You Should Be Worried

Moneyball for Government

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

Much has been made of the Oakland A’s 2002 season, where the out-resourced baseball franchise fielded a scrappy team which temporarily silenced its critics with a then record breaking 20 game winning streak.

General Manager Billy Beane is credited with this baseball miracle. How? By breaking with tradition and putting together his team using the power of “data” to acquire undervalued players –an approach which became know as moneyball.

Did the moneyball innovation take the A’s all the way to the World Series? Nope. The winning streak did enabled the A’s to clinch their division title and land a spot in the playoffs, where they were defeated in the first round by the Minnesota Twins.

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

It’s a compelling story, especially in the hands of a writer like Michael Lewis, who coined the term and penned the 2003 bestselling book of that name. At its heart, Moneyball is about crunching numbers and relying on hard evidence-not emotion or tradition-to drive decisions about how to allocate scarce resources. It’s also about determining what data matter and what don’t (in the case of baseball, concluding that on-base percentage matters a lot more than total home runs). When it comes down to it, it’s a way to get more with less.

Which raises important questions: Can data, evidence, and evaluations similarly revolutionize America’s government? Can we provide better services to millions more Americans while actually saving billions of dollars? Can we make this country a better place for children and families by investing in what works, by testing it and retesting it, and by holding ourselves to a higher standard? In short, can government play Moneyball?     Moneyball for Government, pages 3-4

In my opinion, using moneyball to allocate government resources is very similar to managing a fantasy sports team. It’s an imaginary world divorced from the complex, precarious reality most Americans live in. It’s a perfect playground for the managerial elites to work their devious magic, without dirtying their hands with actual face-to-face interactions with the downtrodden citizens they profess to care so much about.

Here’s a critical detail to remember: Professional sports has always been a cut-throat business. Players are treated as things to be inspected, judged, cut, or traded — all based on their numbers. This isn’t an arena where fairness –not to mention social justice — is valued. Just take a look at what happened to Michael Bennett after he decided to take a knee during the national anthem.

I read Moneyball for Government, so you don’t have to. Here’s my list of reasons why allowing politicians to run our government like a fantasy sports team is a very bad idea.

Moneyball is about rationing resources and not providing services to everyone who needs them.

The goal of moneyball is to create a compelling narrative that justifies and even celebrates austerity. Moneyball’s fundamental assumption is discretionary spending must continue to be cut and streamlined in the name of “funding what works”. This trick immediately removes from debate any discussion about cuts to non-discretionary spending –like the 50% of the federal budget that goes to defense.

The authors admit that denying services to everyone who needs them is unfortunate, but there’s always a silver lining: rationing services is a cheap way to create a randomized trial!

Resources are limited, though, and we can’t afford to give the most promising interventions to everyone who wants them. This is unfortunate, but it regularly creates a perfect research opportunity. If there are five hundred slots available in a new program, then instead of enrolling the first five hundred eligible people to sign up, we can let a thousand eligible people sign up, and hold a lottery to determine who among them participates. Just like that, we’ve created a randomized trial….         Moneyball for Government, page 18

Moneyball is about funding low-cost interventions with high rates of returns.

Ever wonder why reducing class size isn’t an intervention embraced by philanthro-capitalists like Bill Gates — even though there’s solid research supporting it?

Simple, lowering class size is expensive and takes a lot of real teachers to make it happen. This isn’t the moneyball way, which is low cost interventions with a high rate of return.

This also explains why Moneyball for Government celebrates the work of organizations like KIPP, City Year – Americorps, and TFA. Organizations that provide low-cost teachers and no-cost volunteers, and by doing so, offer interventions which don’t cut into the bottomline.

Moneyball is pseudo-scientific and far from the rigorous kind of research it claims to create.

Low cost interventions require low cost measurements of success. Remember how rationing access to services provided an opportunity to create a lottery –sorry– a randomized trial? Well, there’s plenty of pseudo-scientific short cuts used to cook up moneyball’s version of “rigorous evidence”.

Another frequently noted problem for the most rigorous kinds of research is cost….     Moneyball for Government, page 19

 

Still, the truth is that randomized trials aren’t always feasible….                                         Moneyball for Government, page 19

 

There are some great recent examples of research that have used low-cost methods to study low-cost interventions that have turned out to make a real difference in people’s lives….                                                                                                                        Moneyball for Government, page 20

Strong scientific research requires well designed studies which attempt to reduce all possible causes to the one variable being studied. How studies are conducted are just as important as the numbers plugged into them. That’s why studies are published so other scientists –who have no vested interest in the outcome– can critique the study’s design and publicly discuss how unintended bias could have been introduced into the results.

None of this happens with moneyball, if you can attach a number to something, it automatically becomes valid.

Moneyball creates a surveillance state and privacy nightmare. Citizens shouldn’t be experimented on by their government, without their knowledge or consent.

Again, for moneyball’s low cost interventions to be financially profitable, these programs require low-cost research, which would ideally run on no-cost data.  Preferably, this data would be collected and shared by federal, state, and local governments.

Have you noticed a lot of talk about interoperability and student data? Ever wonder what it’s all about? Here’s the definition of interoperability: The ability of computer systems or software to exchange and make use of information.

Here’s Recommendation 6 on how to get the bipartisan moneyball agenda rolling: Build cosscutting data systems that also protect privacy. (page 126) More detail can be found under Pillar 1: Relentlessly use data and evaluation to learn from experience. (page 116)

What does it all mean? I’ll let the authors explain:

Without a way of identifying what works and what doesn’t, progress in social policy is impossible. Until recently, the most sophisticated evaluations required a lot of time and money. Sometimes that’s still true, but not always. With modern data systems, we can do quick, sophisticated tests of different program designs. Think about a store chain testing different product placements in different stores –or a social-services agency testing different intake routines in different offices. To figure out cheaply what works, we can often use data that governments already collect. Think about a new textbook, rather than setting up a whole new approach to collecting data, we can just assign the book to half the classes (selected at random) in a district and compare the scores of kids who used the new text with the scores of those who didn’t, on tests the kids already take. And once we learn the best interventions, we can subject them to financial analysis to compare benefits and costs -and thus give policy makers an important tool to help make tough choices about different ways to spend limited resources.  Moneyball for Government, pages 116-117

Does this sound like the way to go about designing a rigorous scientific study? Hardly.

Did you get a hint of any concern about the protection of privacy? Absolutely not.

To me, this approach is more like the Silicon Valley startup mentality of code and release. A very profitable approach which usually runs on free data and lets the end users discover any flaws or bugs in the program – and suffer all of the consequences. Of course, the business may or may-not choose to clean up any of these bugs in a future release, if they feel spending time on the fix won’t negatively impact the bottomline.

It’s also important to point out that conducting a scientific experiment using a computer model to decide who does or who doesn’t get access to resources –without the subject’s knowledge or consent — is unethical.

It’s also alarming that the adherents of moneyball want the government to collect, store, and share vast amounts of digital information on its citizens. In short, create the infrastructure for a surveillance state. The Stasi Records Agency was able to wreck many lives with much less.

Moneyball is ripe for abuse and fraud.

Because the numbers used to justify interventions aren’t produced by actual controlled scientific studies, where this data come from creates a hidden opportunity for fraud and abuse.

For instance, numbers can be cherry-picked, others ignored. Unethical service providers could reverse engineer studies to create numbers that justify their intervention –and secure a contract for the services they provide.

Even the authors are worried:

One possible way to prevent the misuse of Moneyball -either through the politicalization of evidence or the use of less-than-rigorous studies as a justification for cuts in services -is to identify an impartial referee to evaluate studies and data that come through the door, wheter that be a nonpartisan office like CBO or a newly created one. Moneyball for Government, page 56

Forgive my cynicism, but I can think of one recent example where an “impartial referee” set up to prevent fraud in a world of data and financial speculation failed spectacularly, ruining the lives of millions of Americans.

Do you remember when all the credit rating agencies gave AAA ratings to a certain complex financial instrument which turned out to be junk? Do you also remember how this triggered the sub-prime mortgage crisis and the recession that followed?

I do.

When it comes to money, greed will find a way to bend, and other times break, the rules. It’s the one thing you can count on.

Now What?

Hopefully, I’ve convinced you that moneyball isn’t all it’s cracked up to be, but here’s a few education specific reasons to oppose the moneyball narrative:

If you want well resourced schools for every child, you can’t support moneyball.

If you want to end standardized testing, you can’t support moneyball.

If you want human teachers for kids instead of devices, you can’t support moneyball.

If you object to kids being used as guinea pigs for education reform, you certainly can’t support moneyball.

In the end, moneyball is just more too-good-to-be-true snake oil packaged in a shiny new Pay for Success bottle.

Don’t fall for it.

-Carolyn Leith

 

 

 

 

 

 

 

 

 

4 comments

  1. Thanks for this analysis. I think the whole data-driven mantra in education is a farce. The ROI enthusiasts are counting on artificial intelligence solutions to major problems for HUMANITY.

    Witness the absurd claims of the founders of The “Partnership on Artificial Intelligence to Benefit People and Society.” This outfit began in 2016 as a PR campaign designed to put a positive spin on AI in the face of increasing public and press disapproval of data mining and the use of algorithms to drive online behavior (e.g. Facebook’s profit-seeking from Russian operatives who interfered with our elections).

    The grandiose vision of artificial intelligence as a panacea is betrayed by the overwhelming purpose of these companies: Profit seeking from the personal data of individuals. Here are some briefs from these founding companies.
    —Amazon has customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa.
    —Apple sells iPhone, iPad, Mac, Apple Watch, and Apple TV, has software platforms — iOS, macOS, watchOS, and tvOS with services via the App Store, Apple Music, Apple Pay, and iCloud.
    —DeepMind, based in London, acquired in 2014 by Google (part of Alphabet group) works on AI programs that can “learn to solve any complex problem without needing to be taught how.”
    —-Google is a subsidiary of Alphabet Inc. Google products/platforms include Search, Maps, Gmail, Android, Google Play, Chrome, and YouTube.
    —Facebook. Facebook has AI Research (FAIR) and Applied Machine Learning.
    —-IBM’s Watson is. .”.the most advanced AI computing platform available today, deployed in more than 45 countries and across 20 different industries.”
    —Microsoft says, “More than any other technology that has preceded it, AI has the potential to extend human capabilities, empowering us all to achieve more.”
    Should corporate systems of artificial intelligence be trusted to solve humanity’s most pressing challenges?
    I do not think so.
    https://www.partnershiponai.org/partners/  

    A number of non-profits, think tanks, and programs have signed on as “partners.” The meaning of “partner” is not clear, nor is the cost of a partnership. Most of the partners to date are non-profits. At the website look for updates under “News.” See also
    http://www.undp.org/content/undp/en/home/blog/2018/let_s-talk-about-artificial-intelligence.html

    See also one example of how the education market is being shaped by the idea that AI algorithms constructed from and trained on existing data can be introduced as if a panacea. https://thejournal.com/…/certica-acquires-nweas-formative-assessment-item-bank.aspx
    then https://certicasolutions.com/certica-solutions-acquires-educuity-powers-certica-connect-platform/ and https://certicasolutions.com/certica-welcomes-unbound-concepts/
    ( Active links removed).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s