Paving the way for increased speed and quality at MacPaw

MacPaw, led by their CTO Vira Tkachenko, faced a common yet complex problem: how to measure and improve the engineering organization’s speed and quality of delivery. This goal led Tkachenko to explore various tools and methods to measure productivity.

“Speed and quality of delivery are big problems for every CTO. I needed a way to understand the major blockers to engineers so we could solve them and allow developers to deliver higher quality software quicker,” says Tkachenko.

Initially, the company experimented with a range of solutions to get insights, including basic reporting from tools like Jira, custom Grafana dashboards and Google Sheets, and surveys through Google Forms for feedback from developers. However, these methods had some drawbacks. First, they were used inconsistently across different teams, so there wasn’t a single place to view metrics at the organizational level. This made it hard to confidently measure productivity and identify broader problems. Additionally, the effort required to maintain and analyze these dashboards and surveys was substantial and not sustainable long-term.

During this time, Tkachenko became interested in the DORA and SPACE frameworks for measuring developer productivity, which led to considering DX. DX’s research and holistic measurement approach resonated with Tkachenko: “Measuring productivity is not just about activity. We liked that DX takes a more comprehensive approach.”

During their evaluation, MacPaw was initially hesitant about whether DX would justify its costs. They were also unsure of whether the DevEx 360 product (one of two DX products MacPaw now uses) would see the participation rates that were promised. Both of these concerns were addressed. By using both DevEx 360 and Data Cloud, MacPaw could get a complete picture of what’s impacting developer productivity, which they were unable to capture previously. This provided Tkachenko with the information necessary to make improvements that would increase speed and quality. This also eliminated the time teams were spending maintaining various metrics dashboards. Also, as for their concerns about participation, the company’s first DevEx 360 snapshot saw 99% of developers participate, a figure that has been maintained.

Several other factors went into MacPaw’s decision to choose DX. For example:

• Reception from developers. “We were looking for something engineers wouldn’t be annoyed with,” Tkachenko explains. “I wanted them to see it as not only my instrument, or an instrument of someone from engineering leadership, but as their instrument.”
• Integrations. Developers at MacPaw use many different systems, and it was imperative that the chosen solution would integrate with everything they were using.
• Transparency and delivery to teams. MacPaw wanted a solution that teams would use. DX provides insights to teams transparently so they can take action on their own results. “Usually with surveys, you just fill something out and wait for a presentation or report back. You don’t see the raw data. Engineers have more trust in something that’s more transparent, and I think the same is true for quantitative metrics.”
• Usability. During their evaluation, Tkachenko appreciated how easy DX is to use: “It’s simple to use, and I enjoy the UI because it’s very understandable.”
• Ongoing product evolution. “DX is constantly evolving,” Tkachenko says. “In just one year, we saw so many new features. For us, this was very important. I’m happy to see that this has continued.”

MacPaw is committed to using DX at the team and organizational levels. Each team uses DX to identify and select focus areas, and take action. Additionally, problems that span multiple teams are selected quarterly and tackled with temporary working groups.