How Pipedrive enhances developer productivity with a data-driven DevEx strategy

Yoda’s lesson, “Do or do not. There is no try,” doesn’t really work for developer productivity and experience.

Engineering is a science that can be improved only through experimentation. You must measure, ideate, experiment and measure again. Developer productivity metrics in isolation are useless without a plan of action.

“It’s metrics-backed intention,” Diogo Correia, developer-experience product manager at Pipedrive, told The New Stack. By applying that developer-experience (DevEx) strategy across your organization, he contends, you foster continuous improvement.

Here’s how Pipedrive works to achieve just that.

Invest in developers

In 2020, Pipedrive looked to elevate its platform engineering strategy and found it needed a product manager to meet developers’ expectations as it embraced a platform as a product mindset.

“There was an increase in dissatisfaction by our developers in terms of the tools that we were offering,” Correia said, “and in particular with the quality of our local development environment.”

As the Estonian Software as a Service (SaaS) company scaled to hundreds of microservices and hundreds of deployments per day, the local development environment simply couldn’t cope with the growing complexity. They needed more than just tools and the libraries. “Because there was no one actually seeing what the needs of the teams were and building those common libraries, they started reinventing different things” and building ad-hoc solutions to deal with the problems, Correia explained.

Engineering leadership needed to identify what was working and build golden paths that help steer developers in the right direction.

“There was a need for someone to take care of engineering productivity [and] engineering experience to make sure that things are very easy,” he said. So, he was promoted from principal engineer to DevEx product manager. It’s his job to look at the bigger picture, interview developers, measure the impact and plan.

At first, he interviewed individual developers to uncover and solve known problems — like completely rebuilding the development environment, standardizing libraries and improving the CI/CD pipeline.

This worked well, but they reached a point where they couldn’t surface what was left to fix.

Scale developer productivity metrics past the interview

In 2022, they decided it was time to look at continuous improvement processes within the teams “to provide them with the data and tools to evidence the problems and then work on them,” Correia said.

Pipedrive began to test out developer insights platform DX for the less easily measurable aspects of developer experience. DX was rolled out across the organization in March 2023.

While he had run developer surveys before, “crafting the right questions is very hard,” he said. Then it’s especially challenging to visualize the results “and put it in a way that makes it easy for developers or the teams to use.”

DX uses a mix of research-backed questions drawn from DevEx metrics, the SPACE framework and other qualitative metrics. But it’s not just about asking the right questions; it’s getting developers to respond to them.

“Typically, the problem with a manually created survey is that I need to constantly ping developers to fill it out, and then I need to process the data. And then there’s a huge delay between the data becoming actionable,” Correia said.

DX pings developers daily until they fill out the survey, then it generates a report within a week. But the report is not just for him. “That report is fully transparent to developers,” he said, “and they get immediate access to it, as much access as me or as engineering leadership.” This builds trust for the developers, who are more willing to respond to the next survey.

Steady at a 95% participation rate within the first week of each quarterly DevEx 360 survey, Correia said this “confirms that developers here continue to see the value of these surveys and are happy to engage with them.”

When Pipedrive launched the quarterly surveys in April 2023, they were below the 50th percentile against industry standards. If trends continue, they will be in the 90th percentile 15 months later.[/caption]

Pipedrive uses DX’s Developer Experience Index, a comprehensive metric that enables organizations to compare their developer experience against industry standards. DX’s benchmarking features allow teams to see how they compare to others within Pipedrive and companies across the DX user base — a strong motivator, Correia said, for continuous improvement.

“We started a year ago slightly above the industry 50th benchmark,” Correia said. “Since then we’ve steadily improved. We’ve crossed the p75 benchmark and are well on our way to cross p90,” meaning their DevEx should soon be among the top 10% of the industry.

Take action with a developer productivity metrics workshop

A developer-experience strategy can never just be about the data, but the actions planned because of it.

“Since we deployed it, I’m seeing the impact of engineering managers leveraging the tool, doing workshops inside the teams, identifying problems like tackling documentation and technical debt,” Correia said.

Correia has developed and open sourced a one-hour developer-experience workshop to help teams identify actions based on the DX metrics, focusing on the three lowest sentiments and the three top priorities, as rated by developers. Each step takes about 15 minutes:

1. Identify problems. Examine individual team trends and compare them with the company average before writing ideas on sticky notes.
2. Group similar topics and then vote.
3. Ideate over solutions.
4. Clarify the action items for the team to focus on over the next quarter.

For the teams that have done this workshop, Correia said, “it has an immediate impact in terms of the sentiment and priorities that they report in the next quarter.”

For example, one team’s top three priorities were:

1. Test efficiency
2. Cross-team collaboration
3. Documentation

And its lowest sentiments were:

1. Codebase experience
2. Test efficiency
3. Documentation

“They’re not saying that they feel bad about cross-team collaboration, but for them it’s important,” Correia explained. On the other hand, test efficiency and documentation impact developer experience and are top priorities, which the team should aim to improve.

DX’s developer surveys capture the changes in sentiment and actions based on subquestions, quarter over quarter. These sentiment metrics — called drivers — allow developers to respond with Bad, So-so or Good, based on the SPACE framework and other developer productivity research. DX also includes workflow questions, like: How much time do you usually spend waiting for your service to deploy? This allows teams to reflect on “fishy” numbers, he explained, and then do a workshop to investigate further.

This empowers them to improve their own developer experience at the team level, Correia said, whether it’s around automation, lax documentation practices, flaky tests or paying down technical debt. Previously unknown, company-wide problems, including speed of testing and deploying, also emerged.

“With DX, you’re providing these tools, the evidence of these problems,” he said, and Pipeline’s team managers provide “the time to tackle them at work.”

Prioritize technical debt smartly

Based on the DX 360 survey results, engineering and nontechnical product managers have had to redirect some focus away from releasing new features toward paying down technical debt.

“Developers complain about technical debt, but most don’t act on it proactively,” Correia said. It’s more bubbling up in the background. “That’s where I come in, where I try to explain what the impact is and try to detail technical debt” identified in the DX results during the workshops.

With the combination of the qualitative and quantitative data collected in DX and the workshops, they can identify exact problems and actions to take and estimate how long it’ll take to implement them.

Right now, several Pipedrive development teams are doing a lot of refactoring and rewriting of code, while still shipping value to customers.

Pipedrive has an organization of 240 engineers plus engineering managers, organized into 19 teams of about 14 each. Within the teams, about every two months they intentionally rotate who gets to focus on features versus who gets to work on technical debt and maintenance.

Another change is that some teams have removed some meetings from their calendars, while others, he said, have become more intentional about who is invited to meetings or sharing schedules ahead.

Across the organization, teams are reducing technical debt by refactoring services from scratch, rewriting and optimizing tests, and improving or adding missing documentation, among other things.

“The workshops help by identifying the concrete services or libraries that any given team owns that most developers in the team are feeling pain with,” Correia said. This helps the team prioritize and plan the refactor promptly, “instead of suffering through it for years on end, as before.”

The proof is in the productivity

Following any workshop, Correia continues to observe improved sentiment and increased developer productivity at the team level in DX.

Specifically, there’s a correlation between teams that take action based on DX results and overall improved team sentiment, as well as aspects of it, like around the codebase. Pipedrive has seen a 69% increase in developer sentiment scores due to planned improvements over the past year.

Quarter over quarter, 13 teams have driven improvements in the last two surveys. For one team, “improving realistic timelines” arose as a priority in the first quarter, but after being addressed, it dropped way down the next. This repeated across several teams with varying priorities — not only did the overall sentiment increase, those actions dropped from the top three priorities.

Correia’s next step is to examine what could be replicated and shared across teams to better improve the organization’s overall sentiment.

He emphasized, “It’s not just that the numbers in the survey increase — there’s a real sentiment change.”

This article was originally published on The New Stack.