How Splunk revamped their developer onboarding program

This is a recap from an interview with Gail Carmichael, Principal Instructional Engineer at Splunk, for the Engineering Enablement podcast. Listen to the full episode here.

The challenge at Splunk

Splunk’s Engineering Enablement and Engagement team realized that their onboarding program was too generic and didn’t cater enough to the specific needs of engineers. Given that Splunk’s engineering organization is very large and distributed, the Engineering Enablement team saw onboarding as an area they could focus on to meaningfully improve productivity and long-term retention.

Splunk identified issues with their onboarding process when they observed that engineers, despite being proficient in their specific roles, struggled to connect their work to the broader company ecosystem. This disconnect was highlighted through feedback from recent hires, indicating a need for a more tailored onboarding approach to help engineers better integrate and understand the larger organizational context.

Hypothesis

The hypothesis behind the new onboarding program was that by tailoring the process specifically for engineers, Splunk could improve their integration into the company. The focus was on providing a comprehensive understanding of Splunk’s technical environment, which was expected to reduce ramp-up time, enhance engagement, and ultimately contribute to building a world-class engineering organization.

Metrics

Time to First Merge Request and Time to 10th MR: At the outset of the project, the team started tracking metrics like Time to First Merge Request and Time to 10th MR by connecting to their GitLab instance. These metrics served as a proxy for “ramp-up time,” with the goal of reducing them as much as possible. They focused particularly on the 50th percentile for Time to First MR and Time to 10th MR for all developers who onboarded within a given month.

NPS and Confidence score: Another primary goal of the bootcamp was to make sure new hires felt confident in their roles within the first 90 days. Splunk had already been running a survey at the end of every onboarding session for employees, asking them to rate the quality of each session and give feedback on what could be improved. However, the Engineering Enablement team noticed that participation in these surveys was low, so they decided to make a few changes:

They continued running just one NPS survey immediately after the engineering onboarding program, with the ability to provide feedback
They also introduced a new survey that’s sent out 45 days after the program. This one asks, “Would you recommend how you onboarded to other people?” and also includes a key question: “How confident do you feel in your new role, on a scale of 1 to 5?”

The latter “confidence score” question is used as a top-level metric for measuring how well the onboarding process is working. They pair this alongside their Time to First and 10th MR metrics.

Solution

Splunk’s Engineering Enablement team started by gathering feedback from developers who had gone through the generic, company-wide onboarding process. They also sat down with developers who had just finished onboarding to hear firsthand what they felt was missing. This feedback shaped the first version of their solution: a week-long engineering bootcamp designed to address the gaps identified by gathering feedback from developers.

The engineering bootcamp is a week-long program for developers during their first week at the company. Here’s how the bootcamp works:

1. Structure

When it happens: The bootcamp takes place during the first week of a new engineer’s tenure at Splunk, following their initial HR onboarding.
Format: The bootcamp consists of live, instructor-led sessions spread over four days (with a bit more learning extending beyond the week through self-paced modules and activities).
Cohort-based: Engineers who join the company around the same time participate in the bootcamp together, creating a cohort. This helps in building a network of peers from the start, fostering a sense of community.

2. Content

Here are some examples of the sessions included in the bootcamp:

Technical overview: The bootcamp provides a high-level abstracted view of Splunk’s systems, products, and technology stack. This includes an introduction to the company’s technical history and a systems-level understanding of how different products and services fit together.
Voice of the Customer: One of the sessions is dedicated to understanding Splunk’s customers—who they are, what their needs are, and how Splunk’s products serve them. This is designed to get engineers excited about the product and align their work with customer impact.
Product ecosystem: A significant portion of the bootcamp is devoted to explaining Splunk’s product ecosystem. This involves a high-level overview of how various products communicate with each other, the deployment methods, and the different platforms and services used within the company.
Software development lifecycle: The bootcamp also covers how software development is conducted at Splunk, including the tools, processes, and best practices engineers need to follow.

3. Delivery

Live vs. self-paced: While the bootcamp is primarily delivered through live sessions to foster interaction and engagement, there has been a self-paced version as well. The self-paced version includes recorded sessions and is used when live sessions aren’t feasible, such as when hiring slows down or during the pandemic.
Interactive: The live sessions are designed to be interactive, allowing new hires to ask questions and engage with instructors in real-time. This helps in addressing doubts immediately and tailoring the learning experience to the cohort’s needs.

5. Post-bootcamp activities

Ongoing learning: After the initial week, engineers continue their onboarding with more team-specific and product-specific learning. This includes deep dives into the particular technologies and products they will be working on.
Buddy program: New hires are paired with a buddy—an experienced engineer—who helps them navigate their team-specific onboarding and further integrates them into the company’s culture and practices.

Rollout

Splunk’s Engineering Bootcamp was launched gradually, adjusting based on feedback from new hires. The team fine-tuned the program over time, incorporating both live and self-paced learning to accommodate different needs, including during the pandemic. With consistent feedback and improvements, the bootcamp evolved into a core part of the onboarding process, ensuring engineers felt integrated and aligned with Splunk’s goals.

Outcomes

Since iteratively rolling out the engineering bootcamp, the EE team has observed some improvements:

Improved onboarding metrics: Significant improvements in Time to First Merge and Time to 10th MR, showing faster productivity for new engineers.
Increased confidence: Post-onboarding surveys indicated that engineers felt more prepared and aligned with Splunk’s broader goals.
Higher engagement: Engineers reported higher levels of engagement and connection to Splunk’s mission.
Faster ramp-up: New hires were able to integrate more smoothly and contribute more quickly.

Listen to Splunk’s full story here → How “instructional engineers” improve developer onboarding at Splunk