Minding Our Health: The Nudge, Part Two

Reposted with permission from Wrench in the Gears.

Center for Health Incentives

The topic of the hearing was reducing healthcare expenditures on chronic illness, which they claimed would amount of hundreds of billions of dollars in “savings.” Given the amount of money on the table, it seems clear this sector is ripe for outsourced, outcomes-based contracts that will deploy emerging technologies like health care wearables. Six measures of good health were identified during testimony: blood pressure, cholesterol level, body mass index, blood sugar, smoking status and either the ability to meet the physical requirements of your job (or on this one the Cleveland Clinic person said unmanaged stress.)

This piece expands upon my prior post about digital nudging and behavioral economics. Disruption in the healthcare industry mirrors the ed-tech takeover that is well underway in public education. If you explore the webpage for Catalyst, the innovation PR outlet for the New England Journal of Medicine (remember, social impact policy makers and many investors are based in Boston), you’ll notice the language being used to direct health care providers towards big-data, tech-centered solutions is eerily similar to the language being used on educators and school administrators.

The FCC’s “Connecting America: The National Broadband Plan” of 2010 outlined seven “national purposes” for broadband expansion. Healthcare and education were the first two topics covered in that report. Both chapters focus on “unlocking the value of data.” Who will the big winners be as we further digitize our lives? My assessment is the telecommunications industry and national security/police state will come out on top. Locally, Comcast and Verizon are key players with interests in both sectors.

Education and healthcare fall under the purview of Lamar Alexander’s Senate HELP (health, education, labor and pensions) Committee, so the similarities in tactics shouldn’t come as a surprise. In researching the $100 million federal Social Impact Partnerships Pay for Results Act (SIPPRA) launch I attended in Washington, DC last month, I noticed one of the Republican Senators who presented, Todd Young of Indiana, had attended the Booth School of Business MBA program at the University of Chicago. Recent Nobel Prize winner in behavioral economics Richard Thaler teaches there, and I was curious to see if Thaler’s thinking had influenced Young. Interactive version of Young’s map here.

I located C-SPAN coverage of a Senate hearing on healthy lifestyle choices, which Young participated in on October 19, 2017 (transcript follows). Lamar Alexander and ranking member Patty Murray, who inserted Pay for Success provisions into ESSA, chaired that hearing. Behavioral economics was discussed extensively. Young’s remarks start at timestamp 34:00.

https://www.c-span.org/video/standalone/?435978-1/senate-panel-explores-healthy-lifestyle-choices

The topic of the hearing was reducing healthcare expenditures on chronic illness, which they claimed would amount of hundreds of billions of dollars in “savings.” Given the amount of money on the table, it seems clear this sector is ripe for outsourced, outcomes-based contracts that will deploy emerging technologies like health care wearables. Six measures of good health were identified during testimony: blood pressure, cholesterol level, body mass index, blood sugar, smoking status and either the ability to meet the physical requirements of your job (or on this one the Cleveland Clinic person said unmanaged stress.)

The claim was that if an insured person met four of the six measures, saw a doctor regularly, and had their vaccinations up to date they would avoid chronic illness 80% of the time. Of course the conversation was entirely structured around individual “choice” rather than economic and racial systems that make it difficult for people to maintain a healthy lifestyle.

This neoliberal approach presumes people have free time for regular exercise, not considering they may be cobbling together several gigs to make ends meet. It presumes the availability of healthy food choices, when many black and brown communities are food deserts with limited access to fresh produce. It presumes the stress in people’s lives can be managed through medicalized interventions and does not address root causes of stress in communities steeped in trauma. It presumes ready access to a primary care physician in one’s community.

It is a gross simplification to push responsibility for chronic health conditions solely onto the individual, giving a free pass to social systems designed to harm large subsets of our communities. By adopting a data-driven approach to health outcomes, as would seem to be the case with the above six measures (check a box health), the federal government and health care systems appear to be setting health care consumers up to become vehicles for data generation in ways that are very much like what is happening to public education students forced to access instruction via digital devices. Imagine standards-based grading, but with health measures.

The people who provided testimony at the October 19 hearing included Steve Byrd, former CEO of Safeway, now at Byrd Health; Michael Roizen of the Cleveland Clinic; David Asch Director of the Wharton School’s Center for Health Care Innovation; and Jennifer Mathis of the Baselon Center for Mental Health Law and representing the Consortium for Citizens with Disabilities. Mathis was the only one who testified strongly on behalf of the rights of the insured to withhold personal information and was very concerned about the discriminatory nature of incentivized medical insurance programs, particular with regards to people with disabilities.

In his testimony, David Asch, director of the Center for Healthcare Innovation based in the University of Pennsylvania’s Wharton Business School, described effective designs for health incentive programs, noting that concerns about losing money were more effective from the insurer’s point of view that interest in receiving financial rewards. For that reason Asch said taking away money from someone should be considered before offering a reward. Asch also noted that effective programs included emotional engagement, frequent rewards (tweaked to people’s psychological foibles to they didn’t have to be too large), contests and social norming, including the use of public leader boards.

The date of the hearing is interesting, because right around the same time, public employees (including the teachers) of West Virginia were facing dramatic changes to their insurance plans. These changes included compulsory participation in Go365 an app-based health incentive program that imposed completion of intrusive surveys, wearing a fit bit (if you didn’t there was a $25 fee imposed each month), and meeting a certain step count per day. I include a transcription of testimony from one of these teachers, Brandon Wolford, given at this spring’s Labor Notes conference at the end of this post.

The incorporation of mHealth (mobile health) technologies is a key element of the healthcare disruption process. Increasingly, wearable technologies will transmit real-time data, surveilling the bodies of the insured. mHealth solutions are being built into healthcare protocols, so private investors will be able to track which treatments offer “high-value care.” The use of wearables and health apps also permits corporate health systems to insert digital “nudges” derived from calculated behavioral economic design, into the care provided, and monitor which patients comply, and which do not.

At the moment, the tech industry is working intently to integrate Blockchain technology and Internet of Things sensors like fit bits and health apps on smartphones. Many anticipate Blockchain will become a tool for securing IoT transmissions, enabling the creation of comprehensive and supposedly immutable health data logs, which could be key to mHealth expansion. Last summer the Medical Society of Delaware, a state that touts itself as a Blockchain innovator, announced a partnership with Symbiont, to develop healthcare records on Blockchain. Symbiont’s website claims it is the “market-leading smart contracts platform for institutional applications of Blockchain technology.” The company’s initial seed round of funding took place in 2014 with a second round raising an additional $15 million in May 2017 according to their Crunchbase profile.

The July/August 2018 issue of the Pennsylvania Gazette, the alumni magazine for the University of Pennsylvania, features Blockchain as its cover story, “Blockchain Fever.” The extensive article outlines use cases being considered for Blockchain deployment, including plans by a recent Wharton graduate to develop an application that would certify interactions between healthcare agencies and Medicare/Medicaid recipients for reimbursement. The University of Pennsylvania Health System is deep into innovative technologies. David Asch, director of Penn’s Center for Health Innovation, testified at the October 2017 hearing. The Penn Medicine integrated health system was created in 2001 by former UPenn president Judith Rodin in collaboration with Comcast Executive David Cohen. Rodin went on to head the Rockefeller Foundation, and in the years that followed the foundation spearheaded the creation of the Global Impact Investment Network. GIIN fostered growth of the social impact investing sector, at the same time healthcare began to transition away from a pay-for-service reimbursement towards a value-based model predicated on outcomes met.

Below is a relationship map showing the University of Pennsylvania’s involvement in “innovative” healthcare delivery, which I believe stems from Rodin and Cohen’s connections to Comcast. It is important to note that the Center for Health Innovations claims to have the first “nudge unit” embedded within a health system. Asch is an employee of Wharton, and Wharton is leading initiatives in people analytics, behavior change via tech, and Blockchain technologies. Interactive version of the map here.

New types of employer-based health insurance systems have started to emerge over the past six months. Based on this New York Times article, it seems employees of Amazon, JPMorgan and Berkshire Hathaway will have a front row seat as these technological manipulations unfold. Last fall Sidewalk Labs, the “smart cities” initiative of Alphabet (parent company of Google), announced an expansion into managed healthcare. City Block(read Blockchain) will tackle “urban health” and populations with “complex health needs.”

Reading between the lines, it appears Alphabet aims to use poor black and brown communities that have experienced generations of trauma as profit centers. Structural racism has created a massive build up of negative health outcomes over generations. Now, with innovative financial and technological infrastructures being rapidly put into place, these communities are highly vulnerable. Ever wonder why ACES (Adverse Childhood Experiences) has scores? I expect those numbers are about to be fed into predictive profiles guiding social investment impact metrics.

How convenient that the “smart city” solutions Sidewalk Labs is likely to promote will come with IoT sensors embedded in public spaces. How convenient that healthcare accelerators are developing emerging technologies to track patient compliance down to IoT enabled pill bottle caps; sensors that allow corporate and government interests to track a person’s actions with precision, while assessing their health metrics in excruciatingly profitable detail. Technology platforms are central to City Block’s healthcare program. Many services will take place online, including behavioral health interventions, with the aim of consolidating as much data as possible to build predictive profiles of individuals and facilitate the evaluation of impact investing deals.

Interesting aside, I have two friends who had emergency room visits at Jefferson Hospital this summer and were “seen” by doctors on a screen with an in-room facilitator wielding a camera for examination purposes. This is in a major East Coast city served by numerous research hospitals. Philadelphia is not Alaska. Where is that data going? Where were those doctors anyway?

As these surveillance technologies move full steam ahead, it would be wise for progressive voices invested in the “healthcare for all” conversation to begin considering strategies to address the serious ethical concerns surrounding wearable technologies, tele-health / tele-therapy, and value-based patient healthcare contracting. If guardrails are not put in place that guarantee humane delivery of care without data profiling, the medical establishment may very well be hijacked by global fin-tech interests.

As someone who values the essence of the platform put forward by Alexandria Ocasio Cortez, I worry supporters may not understand that several key elements of her platform have already been identified as growth sectors for Pay for Success. If public education, healthcare, housing and justice reform are channeled by global financial interests into outsourced-based contracts tied to Internet of Things tracking, we will end up in an even worse place than we are now. So, if you care about progressive causes, please, please get up to speed on these technological developments. You can be sure ALEC already has, and remember that Alibaba (Sesame Credit) joined in December. It’s not too much of a stretch to imagine patient rating systems regulating healthcare access down the road if we’re not careful.

Senator Todd Young was the first person to respond to witness testimony during the hearing, and his line of questioning revealed he is a strong advocate of Thaler’s “nudge” strategies. The “nudge” is a key feature of “what works” “Moneyball” government that deploys austerity to push outsourcing and data-driven “solutions” that embrace digital platforms that will gather the data required prove “impact” and reap financial returns. See this related post from fellow researcher Carolyn Leith “A Close Reading of Moneyball for Government and Why You Should Be Worried.”

Young asked David Asch of Wharton’s Center for Innovative Health, what employers could learn from behavioral economists? He also posed several specific suggestions that would scale such programs within the federal government namely: embedding units charged with experimenting with behavioral economics into federal government programs; developing a clearinghouse of best practices; and bringing in behavioral scientists into the Congressional Budget Office.

Asch, a doctor employed by the Wharton Business School, runs UPenn’s Center for Health Care Innovation created in 2011 to test and implement “new strategies to reimagine health care delivery for dramatically better VALUE and patient OUTCOMES” (emphasis added). The 28,000-foot facility houses simulation learning labs and an accelerator where research on use of “smart” hospital systems, social media, and emerging technologies in healthcare is conducted. The accelerator aims to rapidly prototype and scale “high impact solutions,” read Pay for Success.

Besides the Acceleration Lab, the Center also contains the Nudge Unit, which according to their website is the world’s first behavioral design team embedded within a health system. The goal of the unit is to “steer medical decision making towards HIGHER VALUE and improved patient outcomes (emphasis added).” Sample healthcare nudges include embedded prompts in digital platforms (for screenings), changing default settings (to generic prescriptions), framing information provided to clinicians (not sure what this means), and framing financial incentives as a loss.

This is longer than intended and hopefully provides some food for thought. This life datifying impact investing machine we are up against isn’t just coming for public education; it’s coming for ALL human service. We need to begin to understand the depth and breadth of this threat. I’m still mulling over a lot of this myself, and my knowledge base in healthcare is much shallower than my expertise in education. I’d love to hear what folks think in the comments or if you know of others writing on blockchain and IoT in medicine with a critical lens send me some links. Below are transcripts from West Virginia teacher Brandon Wolford about Go365 followed by the Senator Young / David Asch hearing exchange.

-Alison McDowell

Go365 Transcript

Brandon Wolford, West Virginia Teacher: When I first began teaching in 2012 the insurance, in my opinion, was excellent, because I had worked for one year in Kentucky and I had known that the premiums were, although they were being paid five to seven thousand more than we were, they still had to pay much more for their insurance. So it balanced out. However, after the first year or two I was there, that was when they started coming after us with the tax on our insurance. First of all the premiums, we started to see slight increases for one, and another was they started to enforce this “Healthy Tomorrows” policy.

So, the next thing you know, we get a paper in the mail that says, you know, you have go to the doctor by such and such a date. It must be reported. Your blood glucose levels must be at a certain amount. Your waist size must be a certain amount, and if it is not, if you don’t meet all of these stipulations then you get a $500 penalty on your out-of-pocket deductible. So, luckily for me, I eat everything I want, but I was healthy. My wife on the other hand, who eats much better than I do, salads at every meal, has high cholesterol, so she gets that $500 slapped on her just like that.

Okay so, that was how they started out. In the mean time, we have been filling these out for a year or two, and they keep saying you know you have to go back each year and be checked. And then comes the event that awoke the sleeping giants. The PEIA Board, which is the Public Employee Insurance Agency that represents the state of West Virginia, they, um it’s just a board of four to five individuals that are appointed by the governor, they are not elected. They have no one they answer to; they just come up with these things on their own.

So they come to us and they say we’re raising your premiums. This was somewhere between November and December of last year. We’re raising your premiums. You’re going to need to be enrolled in a program called Go365, which means that you have to wear a fit bit, as well record all of your steps. You have to check in with them, and it included private questions like how much sexual activity do you perform, and is it vigorous? All of these things that they wanted us to report on our personal lives, and that was all included. In addition to that we had to report all of those things, and if we refused to wear that fit bit and record all of our steps, or if we didn’t make our steps, we were going to be charged an additional $25 per month.

So, when everyone sees this along with the increased premiums, then they’ve also introduced a couple more bills to go along with that, because the PEIA Board, they have the final say. Whatever they do, it’s not voted upon by the legislature. It’s basically just law, once they decide it. But in the meantime our legislature was presenting these bills. We were currently on a plan of sixty, uh excuse me, eighty/twenty we were paying out of pocket. Well, they had proposed a bill that would double that and make us pay sixty/forty.

So, they presented that along with charter school bills and a couple of other things that were just direct attacks on us. We had been going by a process of seniority for several years; and they also introduced a bill to eliminate seniority to where it was up to the superintendent whether or not you got to stay in your position. It was up to the principal and regardless if you were there thirty years or you were there for your first or your second year…they were trying to tell us you know, it’s just up to your principal to decide. The superintendent decides. They don’t want you to go, you’ve been there for thirty years and you have a masters degree plus forty-five hours, you’re gone. It’s up to them. Your seniority no longer matters. So those things combined with the insurance is actually what got things going in our state.

Excerpted Testimony Healthy Lifestyle Choices, Senate HELP Committee 10/19/17

Lamar Alexander: We’ll now have a round of five-minute questions. We’ll start with Senator Young.

Senator Todd Young: Thank you Chairman. I’m very excited about this hearing, because I know a number of our witnesses have discussed in their testimonies behavioral economics and behavioral decision-making. I think it’s really important that we as policy makers incorporate how people really behave. Not according to an economist per se, or according to other policy experts, but based on observed behaviors. Often times we behave in ways that we don’t intend to. It leads us to results that we don’t want to end up in.

So, Mr. Asch, I’ll start with you, with your expertise in this area. You’ve indicated behavioral economics is being used to help doctors and patients make better decisions and you see opportunities for employers to help Americans change their behaviors in ways they want from tobacco mitigation to losing weight to managing blood pressure and you indicate those changes are much less likely to come from typical premium-based financial incentives and much more likely to come from approaches that reflect the underlying psychology of how people make decisions, encouraged by frequent rewards, emotional engagement, contests, and social acceptance and so forth. And you said in your verbal testimony you haven’t seen much of this new knowledge applied effectively by employers, but there’s no reason why it cannot be. So, my question for you sir is what might employers learn from behavioral economists. Just in summary fashion.

David Asch, Wharton Center for Health Care Innovation: Sure. Thank you senator. I think I’ll start by saying there is a misunderstanding often about behavioral economics and health. Many people believe that if you use financial incentives to change behavior you’re engaged in behavioral economics, and I would say no, that’s just economics. It becomes behavioral economic when you use an understanding of our little psychological foibles and pitfalls to sort of supercharge the incentives and make them more potent so that you don’t have to use incentives that are so large.

So I think that there are a variety of approaches that come from behavioral economics that can be applied in employment setting and elsewhere. I mentioned one, which is capitalizing on the notion that losses looms larger than gains, might be a new way to structure financial incentives in the employment setting in ways that might make it more potent and more palatable and easier for all employees to participate in programs to advance their health. The delivery of incentives more frequently for example. Or using contests or using certain kinds of social norming where it’s acceptable to show people on leader boards in contests and get people engaged in fun for their health. All of these are possibilities.

Senator Todd Young: Thank you very much. You really need to study these different phenomena individually. I think to have a sense of the growing body of work that is behavioral economics. Right, so we need the increased awareness, and I guess the education of many employers about some of these tics we have. That seems to be part of the answer. In fact, Richard Thaler who just won the Nobel Prize for his ground-breaking work in this area indicated that we as policy makers ought to have on a regular basis not just lawyers and economists at the tables where we’re drafting legislation, but ought to have a behavioral scientist as well.

And the UK, they have the Behavioral Insights Team. The United States, our previous administration, had a similar sort of team that did a number of experiments to figure out how policies would actually impact an individual’s health and wellness and a number of other things. Some of the ideas that I think we might incorporate into the government context, and tell me if any of these sort pop for you; if you think they make sense?

We need to continue to have a unit or units embedded within government that do a lot of these experiments. We need to have a clearinghouse of best practices that other employers included might draw on. This doesn’t have to be governmental, but it could certainly be. We on Capitol Hill might actually consider aside from having a Congressional budget office than an official budget office, we might have an entity or at least some presence within the CBO or individuals that understand how people would actually respond to given proposals. Do any or all of those make sense to you?

David Asch: Thank you for your remarks. Yes, I think they all make sense. And one of the lessons that I guess I have repeatedly learned is that seeming subtle differences in design can make a huge difference in how effective a program can be and how it is perceived and that will ultimately care about the impact of these programs. So, I am very much in favor in the use of these programs, but in addition, greater study of these programs, because I think we need an investment in the science that will help all of us in delivering these activities, not just in healthcare, but in other parts of society.

Senator Young: That makes sense. I am out of time. Thank you.

 

Advertisements

A Close Reading of Moneyball for Government & Why You Should Be Worried

Moneyball for Government

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

Much has been made of the Oakland A’s 2002 season, where the out-resourced baseball franchise fielded a scrappy team which temporarily silenced its critics with a then record breaking 20 game winning streak.

General Manager Billy Beane is credited with this baseball miracle. How? By breaking with tradition and putting together his team using the power of “data” to acquire undervalued players –an approach which became know as moneyball.

Did the moneyball innovation take the A’s all the way to the World Series? Nope. The winning streak did enabled the A’s to clinch their division title and land a spot in the playoffs, where they were defeated in the first round by the Minnesota Twins.

But the idea of using “data” to ration resources struck a cord with both Democrats and Republicans. Politicians couldn’t resist the opportunity to use a real David vs. Goliath baseball story to sell the American public on lowering their expectations of what government could deliver. And it sounds scientific too!

It’s a compelling story, especially in the hands of a writer like Michael Lewis, who coined the term and penned the 2003 bestselling book of that name. At its heart, Moneyball is about crunching numbers and relying on hard evidence-not emotion or tradition-to drive decisions about how to allocate scarce resources. It’s also about determining what data matter and what don’t (in the case of baseball, concluding that on-base percentage matters a lot more than total home runs). When it comes down to it, it’s a way to get more with less.

Which raises important questions: Can data, evidence, and evaluations similarly revolutionize America’s government? Can we provide better services to millions more Americans while actually saving billions of dollars? Can we make this country a better place for children and families by investing in what works, by testing it and retesting it, and by holding ourselves to a higher standard? In short, can government play Moneyball?     Moneyball for Government, pages 3-4

In my opinion, using moneyball to allocate government resources is very similar to managing a fantasy sports team. It’s an imaginary world divorced from the complex, precarious reality most Americans live in. It’s a perfect playground for the managerial elites to work their devious magic, without dirtying their hands with actual face-to-face interactions with the downtrodden citizens they profess to care so much about.

Here’s a critical detail to remember: Professional sports has always been a cut-throat business. Players are treated as things to be inspected, judged, cut, or traded — all based on their numbers. This isn’t an arena where fairness –not to mention social justice — is valued. Just take a look at what happened to Michael Bennett after he decided to take a knee during the national anthem.

I read Moneyball for Government, so you don’t have to. Here’s my list of reasons why allowing politicians to run our government like a fantasy sports team is a very bad idea.

Moneyball is about rationing resources and not providing services to everyone who needs them.

The goal of moneyball is to create a compelling narrative that justifies and even celebrates austerity. Moneyball’s fundamental assumption is discretionary spending must continue to be cut and streamlined in the name of “funding what works”. This trick immediately removes from debate any discussion about cuts to non-discretionary spending –like the 50% of the federal budget that goes to defense.

The authors admit that denying services to everyone who needs them is unfortunate, but there’s always a silver lining: rationing services is a cheap way to create a randomized trial!

Resources are limited, though, and we can’t afford to give the most promising interventions to everyone who wants them. This is unfortunate, but it regularly creates a perfect research opportunity. If there are five hundred slots available in a new program, then instead of enrolling the first five hundred eligible people to sign up, we can let a thousand eligible people sign up, and hold a lottery to determine who among them participates. Just like that, we’ve created a randomized trial….         Moneyball for Government, page 18

Moneyball is about funding low-cost interventions with high rates of returns.

Ever wonder why reducing class size isn’t an intervention embraced by philanthro-capitalists like Bill Gates — even though there’s solid research supporting it?

Simple, lowering class size is expensive and takes a lot of real teachers to make it happen. This isn’t the moneyball way, which is low cost interventions with a high rate of return.

This also explains why Moneyball for Government celebrates the work of organizations like KIPP, City Year – Americorps, and TFA. Organizations that provide low-cost teachers and no-cost volunteers, and by doing so, offer interventions which don’t cut into the bottomline.

Moneyball is pseudo-scientific and far from the rigorous kind of research it claims to create.

Low cost interventions require low cost measurements of success. Remember how rationing access to services provided an opportunity to create a lottery –sorry– a randomized trial? Well, there’s plenty of pseudo-scientific short cuts used to cook up moneyball’s version of “rigorous evidence”.

Another frequently noted problem for the most rigorous kinds of research is cost….     Moneyball for Government, page 19

 

Still, the truth is that randomized trials aren’t always feasible….                                         Moneyball for Government, page 19

 

There are some great recent examples of research that have used low-cost methods to study low-cost interventions that have turned out to make a real difference in people’s lives….                                                                                                                        Moneyball for Government, page 20

Strong scientific research requires well designed studies which attempt to reduce all possible causes to the one variable being studied. How studies are conducted are just as important as the numbers plugged into them. That’s why studies are published so other scientists –who have no vested interest in the outcome– can critique the study’s design and publicly discuss how unintended bias could have been introduced into the results.

None of this happens with moneyball, if you can attach a number to something, it automatically becomes valid.

Moneyball creates a surveillance state and privacy nightmare. Citizens shouldn’t be experimented on by their government, without their knowledge or consent.

Again, for moneyball’s low cost interventions to be financially profitable, these programs require low-cost research, which would ideally run on no-cost data.  Preferably, this data would be collected and shared by federal, state, and local governments.

Have you noticed a lot of talk about interoperability and student data? Ever wonder what it’s all about? Here’s the definition of interoperability: The ability of computer systems or software to exchange and make use of information.

Here’s Recommendation 6 on how to get the bipartisan moneyball agenda rolling: Build cosscutting data systems that also protect privacy. (page 126) More detail can be found under Pillar 1: Relentlessly use data and evaluation to learn from experience. (page 116)

What does it all mean? I’ll let the authors explain:

Without a way of identifying what works and what doesn’t, progress in social policy is impossible. Until recently, the most sophisticated evaluations required a lot of time and money. Sometimes that’s still true, but not always. With modern data systems, we can do quick, sophisticated tests of different program designs. Think about a store chain testing different product placements in different stores –or a social-services agency testing different intake routines in different offices. To figure out cheaply what works, we can often use data that governments already collect. Think about a new textbook, rather than setting up a whole new approach to collecting data, we can just assign the book to half the classes (selected at random) in a district and compare the scores of kids who used the new text with the scores of those who didn’t, on tests the kids already take. And once we learn the best interventions, we can subject them to financial analysis to compare benefits and costs -and thus give policy makers an important tool to help make tough choices about different ways to spend limited resources.  Moneyball for Government, pages 116-117

Does this sound like the way to go about designing a rigorous scientific study? Hardly.

Did you get a hint of any concern about the protection of privacy? Absolutely not.

To me, this approach is more like the Silicon Valley startup mentality of code and release. A very profitable approach which usually runs on free data and lets the end users discover any flaws or bugs in the program – and suffer all of the consequences. Of course, the business may or may-not choose to clean up any of these bugs in a future release, if they feel spending time on the fix won’t negatively impact the bottomline.

It’s also important to point out that conducting a scientific experiment using a computer model to decide who does or who doesn’t get access to resources –without the subject’s knowledge or consent — is unethical.

It’s also alarming that the adherents of moneyball want the government to collect, store, and share vast amounts of digital information on its citizens. In short, create the infrastructure for a surveillance state. The Stasi Records Agency was able to wreck many lives with much less.

Moneyball is ripe for abuse and fraud.

Because the numbers used to justify interventions aren’t produced by actual controlled scientific studies, where this data come from creates a hidden opportunity for fraud and abuse.

For instance, numbers can be cherry-picked, others ignored. Unethical service providers could reverse engineer studies to create numbers that justify their intervention –and secure a contract for the services they provide.

Even the authors are worried:

One possible way to prevent the misuse of Moneyball -either through the politicalization of evidence or the use of less-than-rigorous studies as a justification for cuts in services -is to identify an impartial referee to evaluate studies and data that come through the door, wheter that be a nonpartisan office like CBO or a newly created one. Moneyball for Government, page 56

Forgive my cynicism, but I can think of one recent example where an “impartial referee” set up to prevent fraud in a world of data and financial speculation failed spectacularly, ruining the lives of millions of Americans.

Do you remember when all the credit rating agencies gave AAA ratings to a certain complex financial instrument which turned out to be junk? Do you also remember how this triggered the sub-prime mortgage crisis and the recession that followed?

I do.

When it comes to money, greed will find a way to bend, and other times break, the rules. It’s the one thing you can count on.

Now What?

Hopefully, I’ve convinced you that moneyball isn’t all it’s cracked up to be, but here’s a few education specific reasons to oppose the moneyball narrative:

If you want well resourced schools for every child, you can’t support moneyball.

If you want to end standardized testing, you can’t support moneyball.

If you want human teachers for kids instead of devices, you can’t support moneyball.

If you object to kids being used as guinea pigs for education reform, you certainly can’t support moneyball.

In the end, moneyball is just more too-good-to-be-true snake oil packaged in a shiny new Pay for Success bottle.

Don’t fall for it.

-Carolyn Leith