My previous article, Design Thinking — Not Just For Designers, has gotten over 5,000 views since I wrote it last summer. The article is also still getting a steady stream of readers, some of whom have highlighted the content. Now that a year has passed since the completion of the on-boarding program described in that article, it seems like a good time to playback the outcomes.
Playbacks are key moments during design thinking. They are an opportunity to share with your stakeholders or your team what you’ve learned and what your team is planning. Playbacks are used to create alignment and share information by sharing what you’ve observed, reflected and made by telling a story about your users.
In my previous article, I concluded by saying:
Ultimately, deployment to their teams will be the true test of how well this new developer track has prepared new developers to join their teams. Once deployed, we will ask their managers to compare their level of preparedness to the level of their previous new developers that were deployed directly to their teams; this feedback will be the ultimate determination of the success of this new developer track.
Before (or the As-Is Experience)
In preparation for the design thinking workshop, we surveyed front-end developers (FEDs) who went through IBM Design Bootcamp (the as-is experience) and asked them to rate how well they felt their experience prepared them for deployment to their team. We also asked managers of FEDs who went through bootcamp to rate how prepared they felt their FEDs were who were deployed to their team.
We asked them to answer this question on a scale of 0 to 10 so that we could determine a Net Promotor Score (NPS). NPS is an index value ranging from -100 to 100 that measures the willingness of customers to recommend a company’s products or services to others. NPS has become a widely-adopted measurement so these scores can be quickly understood.
NPS scores for front-end developers in the previous IBM Design on-boarding program.
While responses would have been very different for designers responding to this question since the program was designed for them, the scores for front-end developers were not so great. They rated their experience overall at an NPS Score of -42.9. Managers of those developers rated the preparedness of their developers even lower at an NPS Score of -75.
Some of the feedback participants gave along with their ratings were:
“Have projects for FEDs to actually do FED work. During my bootcamp, I spent maybe a week total time coding, and during the rest of the time I was acting a research assistant. In the second half of the program, I think FEDs would benefit more from working on teams together and having projects that are FED-ready to complete.”
“What would even be better is to drop them into a project that already exists — give some experience to working with other people’s code. Or experience using a component library.”
“Either pick projects that allow for FEDs to code and advocate for them pushing code or set aside time for them to code and bake it into the program.”
And from their managers:
“Bootcamp doesn’t prepare any of the participants for anything other than a programmed view for playbacks. Perhaps working in a team, but ill-prepared for anything else.”
The Prototype
Taking this feedback into the Design Thinking activities described in Design Thinking — Not Just For Designers, we prototyped a new experience which allowed the front-end developers to spend four consecutive weeks building an application. The prototypes for the application were designed to assist the IBM Design recruiting team with collecting feedback from interviewers during on-site interviews. Their system had relied on a series of spreadsheets that were fraught with problems — from data overwriting to controlling who could access the feedback and ratings given for a candidate.
The prototypes for the new application were created by Bethany Sonefeld, a designer on the Carbon Design System and provided an opportunity for the front-end developers to work with the design system and components that have been widely adopted by IBM product teams. The front-end developers built and launched their application as new features in an existing project, allowing them to work within an existing codebase. They gained experience implementing a database as well as developing and deploying their application using IBM Cloud platform and services.
Another key component of the prototype included leveraging senior developers to present topics of interest to the new-hire developers. We created a series of sessions covering topics such as:
• Getting Started with Carbon Design System
• GitHub Workflow
•
Deploying to IBM Cloud
• Node
• Coding Standards
While I felt strongly that this new experience would serve to provide the new hire front-end developers with a much richer and specialized on-boarding experience and prepare them to be a developer at IBM, we were constrained by the framework of the existing program. Working as whole team with designers and offering managers and learning to practice design thinking, would come after this specialized front-end developer track.
The first playback to the rest of the new hires who were working in the design phase, doing user research and playing back their findings to stakeholders and IBM Design leaders, was atypical. Rather, their first presentation was a developer demo, showing what they had begun to implement and build based on the prototypes they were given. Not surprisingly, there was a great deal of questions asked about whether they had talked to the users of their application and feedback was given to the team that they should not overlook this practice.
The team took that feedback and in their following weeks of the 4-week program, they incorporated user research into their future iterations. By the end of the track, having built a functioning application that could immediately be used by the recruiting team, they were ready to join their design and offering management colleagues for the remaining 6 weeks of the on-boarding program with all the tools in their toolbox needed to develop front-end prototypes and even in some cases working applications.
One year later
A year has passed since the developers in this prototyped FED track joined IBM as new-hire developers. Immediately following the three-month on-boarding program, they were deployed to product teams working in IBM Storage, IBM Cognitive Systems and IBM Security.
Using the same survey as before, I surveyed the participants of the prototyped FED track and their managers. I was astonished at the huge jump in their ratings:
NPS scores for front-end developers in the prototyped FED track in IBM Design on-boarding program.
Based on the responses of the front-end developers who participated in the prototype, they rated the experience overall at an NPS Score of 33 - a jump of 75. Managers of those developers rated the preparedness of their developers had an even larger jump — responding with an NPS Score of 50, a jump of 125!
In support of their ratings, they said:
“Learned about ideal FED stack and workflow. Actually got to code and practice skillset. Still a lot of domain specific/general IBM product knowledge to learn after joining permanent team.”
“Being able to start coding right away for the micro project, plus meeting and learning from other FEDs via the FED sessions, helped me a LOT. It helped me quickly prototype for the longer 6 week “max” project and also prepared me to jump right in with my product team when Bootcamp was over.”
“Prepared me well for collaborating with other FEDs and other designers, but didn’t prepare me well for working with those outside of the studio (mainly tools and systems that might be used by non-studio developers).”
And one of their managers said:
“Collaborating and working with other developers and designers is his biggest takeaway from the Bootcamp. He didn’t have much experience in working with other developers or designers before Bootcamp. He also thought bringing the FEDs in for talks was helpful during the Bootcamp.”
Since this prototype, the entire IBM Design on-boarding program has been overhauled into a new experience. Instead of focusing strictly on designers and teaching the practice of design thinking, the program has evolved into a rich experience combining design, technology and business.
Through that overhaul, the team that developed the new program also practiced design thinking by conducting user research through surveys and interviews with past participants and managers and business leaders throughout the company. They developed an experience which incorporated the specific outcomes of the FED track — the seedling for the technology focus of the new program. In addition, they expanded the concept I prototyped of presentations by senior developers to include all disciplines — content, visual, user experience and user research — delivering over 130 sessions to both interns and new-hires.
Through my practice of using design thinking to create a better experience for developers hired through IBM Design, from initial research, to workshopping, to writing hills, to executing a prototype, to playing back a measurable outcome, I have been able to see how design thinking really is not just for designers. Instead of focusing on the word “design” in the term “design thinking,” I like to focus on the “thinking” part. After all, design thinking is a way of solving problems and is a technique that anyone can learn and use to achieve better outcomes for their users.