A Close Reading of Moneyball for Government & Why You Should Be Worried

Moneyball for Government

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

Much has been made of the Oakland A’s 2002 season, where the out-resourced baseball franchise fielded a scrappy team which temporarily silenced its critics with a then record breaking 20 game winning streak.

General Manager Billy Beane is credited with this baseball miracle. How? By breaking with tradition and putting together his team using the power of “data” to acquire undervalued players –an approach which became know as moneyball.

Did the moneyball innovation take the A’s all the way to the World Series? Nope. The winning streak did enabled the A’s to clinch their division title and land a spot in the playoffs, where they were defeated in the first round by the Minnesota Twins.

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

It’s a compelling story, especially in the hands of a writer like Michael Lewis, who coined the term and penned the 2003 bestselling book of that name. At its heart, Moneyball is about crunching numbers and relying on hard evidence-not emotion or tradition-to drive decisions about how to allocate scarce resources. It’s also about determining what data matter and what don’t (in the case of baseball, concluding that on-base percentage matters a lot more than total home runs). When it comes down to it, it’s a way to get more with less.

Which raises important questions: Can data, evidence, and evaluations similarly revolutionize America’s government? Can we provide better services to millions more Americans while actually saving billions of dollars? Can we make this country a better place for children and families by investing in what works, by testing it and retesting it, and by holding ourselves to a higher standard? In short, can government play Moneyball?     Moneyball for Government, pages 3-4

In my opinion, using moneyball to allocate government resources is very similar to managing a fantasy sports team. It’s an imaginary world divorced from the complex, precarious reality most Americans live in. It’s a perfect playground for the managerial elites to work their devious magic, without dirtying their hands with actual face-to-face interactions with the downtrodden citizens they profess to care so much about.

Here’s a critical detail to remember: Professional sports has always been a cut-throat business. Players are treated as things to be inspected, judged, cut, or traded — all based on their numbers. This isn’t an arena where fairness –not to mention social justice — is valued. Just take a look at what happened to Michael Bennett after he decided to take a knee during the national anthem.

I read Moneyball for Government, so you don’t have to. Here’s my list of reasons why allowing politicians to run our government like a fantasy sports team is a very bad idea.

Moneyball is about rationing resources and not providing services to everyone who needs them.

The goal of moneyball is to create a compelling narrative that justifies and even celebrates austerity. Moneyball’s fundamental assumption is discretionary spending must continue to be cut and streamlined in the name of “funding what works”. This trick immediately removes from debate any discussion about cuts to non-discretionary spending –like the 50% of the federal budget that goes to defense.

The authors admit that denying services to everyone who needs them is unfortunate, but there’s always a silver lining: rationing services is a cheap way to create a randomized trial!

Resources are limited, though, and we can’t afford to give the most promising interventions to everyone who wants them. This is unfortunate, but it regularly creates a perfect research opportunity. If there are five hundred slots available in a new program, then instead of enrolling the first five hundred eligible people to sign up, we can let a thousand eligible people sign up, and hold a lottery to determine who among them participates. Just like that, we’ve created a randomized trial….         Moneyball for Government, page 18

Moneyball is about funding low-cost interventions with high rates of returns.

Ever wonder why reducing class size isn’t an intervention embraced by philanthro-capitalists like Bill Gates — even though there’s solid research supporting it?

Simple, lowering class size is expensive and takes a lot of real teachers to make it happen. This isn’t the moneyball way, which is low cost interventions with a high rate of return.

This also explains why Moneyball for Government celebrates the work of organizations like KIPP, City Year – Americorps, and TFA. Organizations that provide low-cost teachers and no-cost volunteers, and by doing so, offer interventions which don’t cut into the bottomline.

Moneyball is pseudo-scientific and far from the rigorous kind of research it claims to create.

Low cost interventions require low cost measurements of success. Remember how rationing access to services provided an opportunity to create a lottery –sorry– a randomized trial? Well, there’s plenty of pseudo-scientific short cuts used to cook up moneyball’s version of “rigorous evidence”.

Another frequently noted problem for the most rigorous kinds of research is cost….     Moneyball for Government, page 19

 

Still, the truth is that randomized trials aren’t always feasible….                                         Moneyball for Government, page 19

 

There are some great recent examples of research that have used low-cost methods to study low-cost interventions that have turned out to make a real difference in people’s lives….                                                                                                                        Moneyball for Government, page 20

Strong scientific research requires well designed studies which attempt to reduce all possible causes to the one variable being studied. How studies are conducted are just as important as the numbers plugged into them. That’s why studies are published so other scientists –who have no vested interest in the outcome– can critique the study’s design and publicly discuss how unintended bias could have been introduced into the results.

None of this happens with moneyball, if you can attach a number to something, it automatically becomes valid.

Moneyball creates a surveillance state and privacy nightmare. Citizens shouldn’t be experimented on by their government, without their knowledge or consent.

Again, for moneyball’s low cost interventions to be financially profitable, these programs require low-cost research, which would ideally run on no-cost data.  Preferably, this data would be collected and shared by federal, state, and local governments.

Have you noticed a lot of talk about interoperability and student data? Ever wonder what it’s all about? Here’s the definition of interoperability: The ability of computer systems or software to exchange and make use of information.

Here’s Recommendation 6 on how to get the bipartisan moneyball agenda rolling: Build cosscutting data systems that also protect privacy. (page 126) More detail can be found under Pillar 1: Relentlessly use data and evaluation to learn from experience. (page 116)

What does it all mean? I’ll let the authors explain:

Without a way of identifying what works and what doesn’t, progress in social policy is impossible. Until recently, the most sophisticated evaluations required a lot of time and money. Sometimes that’s still true, but not always. With modern data systems, we can do quick, sophisticated tests of different program designs. Think about a store chain testing different product placements in different stores –or a social-services agency testing different intake routines in different offices. To figure out cheaply what works, we can often use data that governments already collect. Think about a new textbook, rather than setting up a whole new approach to collecting data, we can just assign the book to half the classes (selected at random) in a district and compare the scores of kids who used the new text with the scores of those who didn’t, on tests the kids already take. And once we learn the best interventions, we can subject them to financial analysis to compare benefits and costs -and thus give policy makers an important tool to help make tough choices about different ways to spend limited resources.  Moneyball for Government, pages 116-117

Does this sound like the way to go about designing a rigorous scientific study? Hardly.

Did you get a hint of any concern about the protection of privacy? Absolutely not.

To me, this approach is more like the Silicon Valley startup mentality of code and release. A very profitable approach which usually runs on free data and lets the end users discover any flaws or bugs in the program – and suffer all of the consequences. Of course, the business may or may-not choose to clean up any of these bugs in a future release, if they feel spending time on the fix won’t negatively impact the bottomline.

It’s also important to point out that conducting a scientific experiment using a computer model to decide who does or who doesn’t get access to resources –without the subject’s knowledge or consent — is unethical.

It’s also alarming that the adherents of moneyball want the government to collect, store, and share vast amounts of digital information on its citizens. In short, create the infrastructure for a surveillance state. The Stasi Records Agency was able to wreck many lives with much less.

Moneyball is ripe for abuse and fraud.

Because the numbers used to justify interventions aren’t produced by actual controlled scientific studies, where this data come from creates a hidden opportunity for fraud and abuse.

For instance, numbers can be cherry-picked, others ignored. Unethical service providers could reverse engineer studies to create numbers that justify their intervention –and secure a contract for the services they provide.

Even the authors are worried:

One possible way to prevent the misuse of Moneyball -either through the politicalization of evidence or the use of less-than-rigorous studies as a justification for cuts in services -is to identify an impartial referee to evaluate studies and data that come through the door, wheter that be a nonpartisan office like CBO or a newly created one. Moneyball for Government, page 56

Forgive my cynicism, but I can think of one recent example where an “impartial referee” set up to prevent fraud in a world of data and financial speculation failed spectacularly, ruining the lives of millions of Americans.

Do you remember when all the credit rating agencies gave AAA ratings to a certain complex financial instrument which turned out to be junk? Do you also remember how this triggered the sub-prime mortgage crisis and the recession that followed?

I do.

When it comes to money, greed will find a way to bend, and other times break, the rules. It’s the one thing you can count on.

Now What?

Hopefully, I’ve convinced you that moneyball isn’t all it’s cracked up to be, but here’s a few education specific reasons to oppose the moneyball narrative:

If you want well resourced schools for every child, you can’t support moneyball.

If you want to end standardized testing, you can’t support moneyball.

If you want human teachers for kids instead of devices, you can’t support moneyball.

If you object to kids being used as guinea pigs for education reform, you certainly can’t support moneyball.

In the end, moneyball is just more too-good-to-be-true snake oil packaged in a shiny new Pay for Success bottle.

Don’t fall for it.

-Carolyn Leith

 

 

 

 

 

 

 

 

 

Advertisements

Making Childhood Pay: Arthur Rolnick, Steven Rothschild, and ReadyNation

Reposted with permission from Wrench in the Gears.

Pre-K Teachers Heart Tech

The push for early childhood education access is NOT being driven by a desire to meet the basic human needs of children. Rather financial interests that view children as cogs in a national workforce development program are pushing it; and they see preschoolers as lumps of human capital to be plugged into economic forecasts. This is all happening at a time when human services are being privatized in the name of scalable, outcomes-driven social entrepreneurship. The trailer for a new documentary, The Invisible Heart, on social impact bonds indicates how much capital is flowing into this new market.

This post provides additional background on the ReadyNation Global Business Summit on Early Childhood Education that will take place at the Grand Hyatt hotel in New York City November 1-2, 2018. No U.S. educators or policy advocates may attend unless they come with at least four pre-approved business sponsors. Review the draft agenda here.

This is the second in a series. Read part one here.

Where did ReadyNation come from?

The idea emerged from a conversation three men had on a conference call during the summer of 2003:

  • Arthur Rolnick, senior researcher at the Minneapolis Federal Reserve
  • Robert Dugger, financial policy analyst and venture capitalist
  • James Heckman, University of Chicago economics professor

Its first incarnation, the “Investing in Kids Working Group,” focused on researching returns on early childhood investments, developing finance mechanisms, and crafting policy recommendations. Over the past fifteen years Dugger, in consultation with Heckman and Rolnick and with support from the Pew Charitable Trusts, gradually built a structure to undergird a global investment market fueled by debt associated with provision of early childhood education services.

The push for early childhood education access is NOT being driven by a desire to meet the basic human needs of children. Rather financial interests that view children as cogs in a national workforce development program are pushing it; and they see preschoolers as lumps of human capital to be plugged into economic forecasts. This is all happening at a time when human services are being privatized in the name of scalable, outcomes-driven social entrepreneurship. The trailer for a new documentary, The Invisible Heart, on social impact bonds indicates how much capital is flowing into this new market.

Arthur Rolnick, Steven Rothschild, and Pay for Performance

Much of my research has focused on the Boston area (global finance), the Bay Area (tech), Chicago (blockchain), and New York (urban policy). So I was surprised to find what may be a key piece of this puzzle actually comes out of Minneapolis Minnesota. Though perhaps the fact that Minnesota is home to the nation’s first charter school, City Academy that opened in St. Paul in 1992, indicates local conditions favor neoliberal reforms.

Arthur (Art) Rolnick spent his 40-year career as a senior economic researcher at the Minneapolis Federal Reserve Bank. During that time he also served as an associate professor in the economics department of the University of Minnesota and was co-director of the Human Capital Research Collaborative in the Humphrey School of Public Affairs. The Collaborative houses the Chicago Longitudinal Study whose researchers are tracking the short and long term effects of early intervention on 1,000 students who attended Chicago’s Child-Parent Centers in 1984-85.

The Chicago Child-Parent Centers were service providers for one of the nation’s first two early childhood social impact bonds, begun in December 2014. The Chicago SIB included payout metrics tied to third grade literacy scores. Thus far the program has issued maximum payments to investors including Pritzker, Goldman-Sachs and Northern Trust. According to this report from the Institute for Child Success, it is possible that over the seventeen-year time horizon for the SIB, $34 million could be paid out on the initial $16.9 investment.

Click here for the interactive version of this map.

Rolnick connected with Steven Rothschild, a former vice president at General Mills who left the corporate sector and launched Twin Cities RISE!, an “innovative anti-poverty” program that provided workforce training for low income adults, in the mid 1990s. Rothschild arranged with the state of Minnesota to provide services via an outcomes-based contracting arrangement where the organization was only paid when the “economic value” they provided to the state by increasing taxes (paid by those placed in jobs) and decreasing state expenditures (reduced costs for social services or incarceration) met approved targets.

Arthur Rolnick and Gary Stern of the Minneapolis Federal Reserve worked with Rothschild and Twin Cities Rise! to develop the economic analysis in support of the outcomes-based contracting initiative. Rolnick’s work with Rothschild eventually led him to examine the economic implications of early childhood interventions using data from the High/Scope Perry Preschool Study. In 2003, the year Rolnick had that auspicious phone call with Robert Dugger and James Heckman, he and and Rob Grunewald, regional economic analyst, put out the following report for the Minneapolis Federal Reserve: Early Childhood Development: Economic Development with a High Public Return.

In a 2006 profile of Rolnick, Minnesota journalist and blogger Kevin Featherly notes that report catalyzed $1 million in seed money for the Minnesota Early Learning Foundation, a project of the Minnesota Business for Early Learning. It also put Rolnick and Grunewald on the lecture circuit for the next several years where they touted early childhood education as a prudent economic investment. Weatherly likened Rolnick’s schedule after the release of the report to that of a presidential candidate, sharing the stage with Jeb Bush at the National Governor’s Convention, the head of the Gates Foundation at the National Council of State Legislatures, and presenting to a global audience at the World Bank.

Rothschild who served on the boards of the Greater Twin Cities United Way and Minneapolis Foundation, went on to found the consulting firm Invest in Outcomes and write the Non Non-Profit, a book that exhorted non-profits to focus on the Return on Investment (ROI) and measurable economic outcomes of the services they provide. These ideas eventually led the Minnesota legislature to adopt the “Pay for Performance Act” in 2011 that appropriated $10 million for a pilot program to develop Human Capital Performance Bonds or HuCaps.

Rothschild provides a detailed explanation of how HuCaps function in a 2013 article for the San Francisco Federal Reserve’s publication Community Development Investment Review. HuCaps differ from social impact bonds in that they are true bonds and tap into the state bond markets; which, in theory, could give them access to significantly more capital-trillions of dollars rather than millions. In this podcast with the St. Louis Federal Reserve, Rothschild describes the model developed by Twin Cities RISE! as the basis for much of the social impact investing activities that have emerged over the past decade.

Source for this slide.

As structured in the Minnesota legislation, the service provider is the one that takes the risk rather than the investor. If the provider is not able to meet the target metrics they are the ones who will not be paid. As a consequence, HuCaps have not yet taken off; see Propel Nonprofit’s analysis here.

Source for this slide.

Nevertheless, there are those who have not given up on the Human Capital Performance Bond approach. Arnold Packer, former director of the education reform and workforce development SCANS 2000 Center based out of Johns Hopkins University, wrote about HuCaps for the Brookings Institution in 2015 (the co-chair of the Commission on Evidence-Based Policy Making is Bruce Haskins also of Brookings). He noted that Milton Friedman was among the first to float the idea of leveraging private investment in human capital development. Take a minute to watch this one-minute video, from Institute for the Future, that portrays a college student contemplating entering into an income-sharing arrangement in exchange for tuition.

The idea that states could issue bonds for human capital in the same way they do for infrastructure like bridges, and that future savings will be created as people attain higher paying jobs due to their improved human capital, is central to the HuCap premise. In order to justify future cost savings, those receiving services must be tracked, so their “outcomes” can be measured over time. According to Arnold:

“This reform requires a shift in thinking on all sides, investors in human resources (early childhood education falls into this category) will have to consider statistically estimated benefits in terms of future cost savings and revenue as equivalent to projected revenue from a toll road. Government agencies will have to coordinate in order to structure attractive Human Resource bonds, since different agencies at different levels of government, benefit from the savings resulting from earlier investments.” Source

This model of finance, if ever widely adopted, would demand all recipients of public services (including education) be part of the government’s statistical estimate. Because many early-intervention services are directed at families, a person’s predictive profile would likely start to be amassed prenatally; babies assigned a Decentralized Identifier (DID), before they are even born. Estimates would be made about the likelihood a person would need to access services in the future, what those services would be, and what they would cost. Assessments would be made about the anticipated tax revenue a person would in turn generate over their lifetime. All of this data would need to be calculated in order to determine the impact metrics for the investors and structure “attractive human resource bonds.”

Before the rise of cloud-based computing, such a level of tracking would have been impossible. Having access to data to make those predictions would have been difficult to obtain. But that is rapidly changing in this world of Big Data, digital identity and “moneyball for kids.” The bi-partisan Commission on Evidence-Based Policy Making concluded public hearings in February 2017, and the vast majority of those providing testimony favored creating enormous pools of data to inform public policy decisions.

Evidence Based Policy Making

Read the report.

Responsibilities of the Commission on Evidence Based Policy Making:

Things seem to be on hold for the moment with Human Capital Performance Bonds, but I feel strongly they may be simply waiting in the wings until Blockchain sovereign identity is normalized. An Illinois state Blockchain task force (note Pritzker, backer of early childhood SIBs is running a well-funded campaign for governor of Illinois now) has developed preliminary recommendations linking public service benefits to citizens using Blockchain technology. They even envision building in behavioral incentives tied to the provision of services through digital economic platforms. See the diagram below for an illustration of how they might incentivize food purchases.

Read the report.

Of course the implications of this type of manipulation for people who live in food deserts with limited access to fresh produce remains unaddressed. And it doesn’t take a stretch of the imagination to see how other choices might be economically incentivized: which online course to take (the evidence-based one); which training program (the evidence-based one); which therapy provider (the evidence-based one); which medical treatment (the evidence-based one). But by whose measure? Who sets the metrics? Who profits when “evidence-based” standards are imposed?

How will independently-owned, neighborhood-based child care centers fare in this new landscape? If they are shuttered, what will the economic impacts be for communities, especially in economically distressed neighborhoods where such businesses are important sources of employment? Will small-scale providers be willing to collect the “human capital” data required to take advantage of pay for success investments? If they are willing, would they even have the money to purchase the technology (smart tables, anyone?) required to gather their “impact” evidence?

Rob Grunewald, Rolnick’s collaborater on the Federal Reserve Early Childhood paper, is on the ReadyNation Summit planning committee. Rolnick is part of a workshop, “Scalable Success Stories in Early Childhood Programs,” at 11:45 on Friday, November 2nd.

The “pay for performance” finance mechanism dreamed up by Rothschild and Rolnick in the 1990s is particularly well-suited to this age of Internet of Things data collection, surveillance, predictive analytics, financialization, and economic precocity. This is why we should all be very concerned about ReadyNation’s Global Business Summit on Early Childhood; especially because it so clearly discourages early childhood educators and policy advocates from attending.

Next up, Dr. James Heckman and the Institute for New Economic Thinking.

-Alison McDowell