Better Than the Code
Why tracking who worked on each part of Windows Vista and who left helped researchers predict failures.

Harvard Business Review wanted proof. Melvin Conway had an example.
In 1967, Conway sent the journal a paper called "How Do Committees Invent?"
The editors rejected it. As Conway later recalled: "HBR rejected it on the grounds that I had not proved my thesis."
His idea was simple: a system's design copies the way its builders communicate. The boundaries between people tend to become boundaries in what they build.
Windows Vista binaries touched by more organizations and engineers who had left the company were more prone to failure after release. Teams struggle to investigate handoffs because the code is there to read, but the conversation behind it is not.
Datamation published the paper in April 1968. In a 2010 note, Conway pointed out that his central idea sat third from last in 45 paragraphs.
His paper offered a small example of the link. The example was clear enough to count, but much too small to settle the question.
Eight people
A contract research organization had eight people available to build two compilers.
Five went to the COBOL compiler. Three went to ALGOL.
The COBOL compiler ran in five phases. The ALGOL compiler ran in three.
That neat match did not prove that every team would leave its shape in its software. It showed what Conway meant without asking anyone to accept the idea on faith.
Conway argued that some design choices were already made when the team was organized. Once the team split, a design choice needing close contact between the COBOL and ALGOL halves had no path forward.
Conway argued that some design choices were already made when the team was organized.

Click a numbered dot to see what it stands for. Start with 1.
The constraint was in how people could work together. A reporting chart could help reveal that, but the boxes alone were not the mechanism.
Counting the seams
In 2008, forty years after Datamation published Conway's paper, Nachiappan Nagappan, Brendan Murphy and Victor Basili studied Windows Vista.
Their dataset contained 3,404 binaries, spanning more than 50 million lines of code. They measured failures during the first six months after release.
They also gathered eight measures of the organization behind each binary.
These included engineers who had worked on it and remained employed, engineers who had left, and the different organizations contributing substantial edits.
They predicted failures better than every code measure the researchers tested.
The organizational model achieved 86.2% precision: that share of the binaries it flagged were failure-prone. Its recall was 84.0%: it found that share of the failure-prone binaries.
Code churn achieved 78.6% precision and 79.9% recall. Complexity, dependencies, coverage and pre-release bugs also fell behind the organizational model on both measures.
The engineers did not know the study was running. Nagappan later said: "We didn't expect it to be that accurate."
Departures were among the measures.
A record of who left gives you somewhere to investigate the information lost at a handoff. A departure count, though, cannot tell you what knowledge went missing.
The evidence also extended beyond Vista. In 2012, Alan MacCormack, Carliss Baldwin and John Rusnak compared matched open-source and commercial products doing the same job.
In every pair, the product from the more loosely connected organization was more modular. The differences reached a factor of six.
Where to look first
Vista was still one system, at one company. Its results showed that organizational measures and failures traveled together, not which caused which.
The broader record also resists the word law.
In 2016, Lyra Colfer and Carliss Baldwin reviewed 142 studies. In firm and industry studies, they found mirroring, their term for designs copying how builders communicate, was "a prevalent pattern but not universal."
Of those, 70% offered strong support, 22% partial support and 8% none. Among studies of open collaborative projects, mostly software, 56% did not support it.
Organizations can deliberately break the correspondence. And correspondence alone does not establish a direction of cause: the organization may shape the system, or the system may shape the organization.
The organization may shape the system, or the system may shape the organization.

Click a numbered dot to see what it stands for. Start with 1.
Structure can limit what people build, though Vista gives us evidence from just one system. The wider studies show a common pattern with exceptions.
I would start by asking which service has the most owners and the most departures. Then I would compare that history with the failures customers met after release. I would not ask for that comparison as a report. The service owner should read it and decide which fix ships before customers meet the next failure.
In Vista, histories like those helped predict trouble.
In your system, they may not.
I send one idea in my letter when I publish something new.