Grammarly uses DX to optimize AI tool adoption and measure its impact on productivity
Grammarly, the trusted AI assistant for communication and productivity that serves over 40 million people, has embraced a dogfooding approach to understanding how AI tools impact developer productivity within its own engineering organization. As an AI company itself, Grammarly recognized the importance of understanding how AI coding assistants could drive engineering velocity, while also providing insights applicable to improving its own AI products.
“We want to stay on the cutting edge,” says Maryna Veremenko, who leads the Developer Experience team responsible for driving AI’s impact in engineering. “But AI tools are evolving rapidly. Early on, we wanted to understand: Where do we stand compared to other companies? What are our areas for improvement? And what impact are these tools having on productivity?”
When Grammarly first began adopting AI tools, the approach was entirely decentralized. Teams organically started using different AI assistants, requesting various tools, and experimenting with multiple solutions without centralized oversight. While this fostered innovation and natural adoption, it created challenges in understanding the actual impact and ROI of these investments. “We don’t want to spend money on tools that are not used,” says Veremenko. “This is a really fast and aggressive environment, so we want to avoid spending on something that doesn’t deliver value.”
By partnering with DX, Grammarly was able to create a unified measurement approach across its diverse AI tool ecosystem. The Developer Experience team focused on standardizing metrics and creating visibility into tool usage patterns across their engineering organization. “One of the key benefits of DX is having surveys and system data in one place, which we didn’t have before,” explains Veremenko. "It’s super powerful to have the ability to query, slice, and dice metrics from different angles and learn from the data, especially when it’s tied to self-reported data from developers.”
Today, Grammarly uses DX to understand where friction exists (and where AI could be implemented), compare AI tools against each other, and track the impact of increased usage on various productivity metrics. One capability that has been particularly helpful for Grammarly has been DX’s Data Studio product, which allows the team to create custom metrics and dashboards. Veremenko explains: “To validate that, we’re pulling metrics from our version control systems into DX, like the time from merge request creation to deployment. We want to surface those correlations, and we rely heavily on this part of the platform to do it.”
One key insight Grammarly has identified is that when developers transition from occasional to frequent AI usage, they see productivity improvements across the board. That’s why Veremenko’s team is now focused on turning more users into power users. “DX is super powerful,” she adds. “There’s still a lot more we know we can get out of the platform.”
