Characterizing a high-functioning team

This post was originally published in Engineering Enablement, DX’s newsletter dedicated to sharing research and perspectives on developer productivity. Subscribe to get notified when we publish new issues.

I read Software Development Is a Team Sport, the latest in Google’s “Developer Productivity for Humans” series. In this paper, the authors describe how they went about measuring team functioning and share their findings about its impact on productivity.

Developer productivity research often focuses on the individual when it comes to measuring velocity or effectiveness. However, a team’s ability to work together is just as critical a part of their contribution to the business. That’s why Google’s Engineering Productivity research team decided to study teamwork.

In order to understand how individual efforts combine to determine group outcomes like productivity, the authors conducted their research in multiple phases. First, they developed a way to measure collaboration (i.e., who works with who, and when and how they interact). Next, they characterized what it means for a team to be “high functioning.” And finally, they explored the impact of team functioning on other outcomes such as productivity and satisfaction.

Here’s how they conducted their research and what they found.

1. Measuring collaboration

To measure collaboration, the researchers tracked how often engineers interacted with coworkers using tools like chat, meetings, documents, and code reviews. (Note: it helps that Google has strong infrastructure for collecting this kind of data.) Since some tools are used more frequently than others (for example, people send lots of chats but have fewer meetings), they adjusted the numbers to reflect those differences.

Then they used that data to identify each engineer’s top collaborators. They filtered out noisy signals that weren’t strong signs of real collaboration, like large meetings or document comments made months apart. They also acknowledged that this approach wouldn’t capture interactions happening outside these tools.

To check for accuracy, they asked engineers to review a list of the top 13 collaborators identified by the metric and add any missing names. The results showed that the metric they developed had high accuracy.

While the main goal of building this collaboration metric was to support the rest of their research, they did identify an interesting pattern along the way: senior engineers tend to work with more people and collaborate more frequently, while junior engineers have fewer collaborators and interact less often. This is consistent with their job expectations, but it was valuable to see the data back it up.

2. Characterizing a high-functioning team

The new metric enabled the researchers to know when collaboration was happening, but it didn’t provide a notion of whether that teamwork was good or bad. To capture this, they set out to develop a reliable survey instrument to provide a picture of team dynamics, including what helps teams thrive or holds them back.

The researchers followed established best practices to develop a survey that would allow employees to self-report about their team’s functioning: this process included a comprehensive literature review, interviews with subject matter experts, multiple iterations of refining survey items, and a factor analysis.

From this, they developed a four-factor scale for measuring team functioning:

• Team processes and visibility: Are tools and processes helpful and transparent?
• Team culture: Is the environment respectful, safe, and open?
• Strategic alignment: Does the team understand and connect to broader goals?
• Balanced team workload: Is the team’s workload sustainable?

3. The impact of team functioning

By using the survey instrument and triangulating results with their primary engineering satisfaction survey, the researchers measured the impact of team functioning on other outcomes. They found that those who reported higher levels of team functioning also reported feeling more productive at work and more satisfied with their roles.

Developers on higher-functioning teams were also more satisfied with the quality and speed at which their teams ship products to users (i.e., team velocity).