SiriusXM quantifies the impact of platform engineering initiatives

In 2022, SiriusXM kicked off an ambitious plan to completely rebuild its digital streaming offerings in order to improve the listener experience, increase the pace of innovation, and modernize its technology platform. The company’s Platform Engineering team was tasked with building out a new cloud footprint and internal developer platform from scratch to serve as the foundation for everything. They adopted a ‘platform as a product’ approach to build the ideal developer experience but found that effectively measuring the success and impact of their efforts proved to be a challenge.

“We’re working towards building a platform portal, unified CLI tooling, and creating more abstraction in our infrastructure as code strategy, among other things,” says Jared Wolinsky, VP of Platform Engineering. “It’s important for organizations like ours to measure how successful these initiatives are so we can show the impact we’re having on the company. We also need metrics to inform our own efforts. Are we investing in the right areas? Should I be requesting more headcount on the observability team because it’s a source of pain for developers?”

While evaluating DX, SiriusXM explored other ways of getting insights about developer pain points and needs, including using Slack polls, internal user channels, and interviews. These methods were useful and are still in use today, but they weren’t scalable. “It became obvious that trying to replicate DX’s DevEx 360 product would require multiple full-time employees working pretty much all the time, creating surveys, and working with data scientists to make sense of the data,” Wolinsky explains. “And even then, we probably wouldn’t achieve the same level of polish and insights that DX provides.”

The business case for adopting DX centered on reducing Weekly Time Loss for engineers. “If we can use DX data to cut Weekly Time Loss from 23% to 21%, that’s equivalent to about 20 full-time employees every week at our scale,” Wolinsky says. Two other factors that played into their decision were DX’s ability to provide the platform organization with insight into whether they were spending time in the right places, and to deliver insights to product engineering teams. Wolinsky adds that he used DX to justify another tool they were considering, LocalStack, by showing how much time developers were losing due to their local development environments.

“DX gives us a way to quantify and track important metrics in areas that are hard to measure. In the past, we relied on gut feelings and feedback from engineers about issues like how long it takes to create or deploy a service, or that the feedback loop for development and testing is too long. DX gives us the actual numbers.”

Looking forward, SiriusXM plans to continue using DX to measure the success of their platform initiatives and share that story with the rest of the engineering organization.