McKinsey at Rikers Island: The $27.5m Violence Plan

A pilot is a measurement instrument

Most organisations treat a pilot as a first step in delivery. In practice, a pilot is also an experiment, and its results will be used to make decisions about scale, budget and reputation. That means its design determines what the results can show. If the people entering the pilot are chosen differently from the people outside it, a better outcome may reflect who was selected rather than what the programme did.

ProPublica’s reporting on the early Restart units turns on exactly that point. According to the investigation, people selected by the HUB model and considered troublesome were replaced with alternates believed to be more accommodating, and McKinsey acknowledged that potentially unruly inmates were intentionally excluded from the earliest units. McKinsey and the former commissioner said the filtering later stopped; seven officials told ProPublica it continued. Whichever account is closer to the truth for the later units, the early comparison could not cleanly measure the method.

Fix the selection rule before the results arrive

The practical safeguard is to write the selection rule down before the pilot starts and to log every exception. If the rule says participants will be allocated by a model, any manual substitution should be recorded with a reason and an approver. If exclusions are necessary for safety, the comparison group should be adjusted in the same way, or the analysis should say plainly that the groups differ.

This is not a technical nicety. Once a striking number such as “down by over 70%” is presented publicly, it becomes difficult to revisit. Officials, advisers and the media begin to treat it as a fact about the programme. A selection log is the evidence that allows a later reviewer to check whether that number was comparing like with like. The Allstate claims case offers a different example of how internal documents later become the record against which public claims are tested.

Baselines can move the result as much as the programme

City Hall’s acknowledgement, as reported by ProPublica, that coding errors had inflated the starting level of violence by 10% to 15% shows a second risk. A pilot’s improvement is measured against a baseline. If the baseline is overstated, the improvement looks larger than it is. Changes in how incidents are classified, who records them and which records are missing can all shift the numbers without any change in behaviour on the housing units.

The federal monitor’s observation that the Restart facilities accounted for the largest amount of missing documentation, as reported by the New York Daily News, belongs in the same category. Missing records do not prove that results were manipulated. They do mean the results cannot be fully verified. A pilot dashboard should show data completeness alongside the outcome measure, so that a decision-maker can see whether a rise or fall is supported by complete records.

People closest to the data see problems first

The reported internal emails, including the June 2017 note that the validity of currently reported data was questionable, are a familiar pattern. The staff who record incidents, reconcile spreadsheets and answer follow-up questions often notice inconsistencies before leaders do. When several of them raise concerns, that is a signal worth pausing for, even if their concerns later prove resolvable.

A good engagement creates a route for that signal. The route should be known to the client’s staff and the consultants, should not depend on the people who will present the results and should produce a written response. The response may confirm the data, correct it or qualify the claim. Any of those outcomes is better than an unrecorded concern. The Enron case is a reminder that organisations can reward confident narratives while discouraging the people who question them.

Be clear about why the adviser is there

ProPublica reported that three officials believed mayoral aides saw hiring McKinsey as a way to head off a federal monitor and as political cover; the mayor’s office denied it. The claim is contested, and this article does not adopt it. But the general risk is real in public-sector work. An adviser can be valued for reassurance as well as for analysis, and the two purposes can pull in different directions when the evidence becomes uncomfortable.

The defence against that risk is to agree measurable outcomes at the start, even when the client’s motives are mixed. If an engagement is meant to reduce violence, it should state how violence will be measured, against which baseline, over what period and with what data checks. Those commitments protect the adviser as much as the client, because they make it possible to show what the work did and did not achieve.

Outcomes for the whole system need careful attribution

ProPublica’s comparison of 2019 with 2014 found inmate attacks and use of force up 25% and assaults on staff up 15% by the city’s data, and violence up nearly 50% using McKinsey’s own metrics. These are serious findings about the jail system. They are not, by themselves, a measurement of what any single programme caused, because staffing, population, policy and leadership all changed over the same period.

That limitation cuts both ways. It means the later rise cannot simply be blamed on the advisers. It also means early pilot gains could not be credited to them without a clean comparison. Reporting a whole-system outcome next to a programme’s fee invites a causal reading that the data cannot support. The ICE detention-budget case raises a related point about distinguishing what an adviser recommended from what a public body implemented.

Spend analytics money only on tools that will be used

The IntelWatch tool, as reported, was paid for but never used. That outcome is common in analytics projects and is not unique to this engagement. It usually reflects a gap between what a technical team can build and what the client organisation is ready to operate, maintain and trust. Before funding a new model, ask who will use it daily, what decisions it will change, what data it needs and who will fix it when it fails.

Write those answers into the business case and test them before the build is complete. The business case calculator and the management consulting presentation guide can help set out the costs, the decision the tool supports and the evidence that will show whether it works. In a setting as consequential as a jail, an unused tool is not a neutral outcome. It is money and attention that could have gone elsewhere.