Recently, a person involved with soft skills programs described an issue she was facing. She had communicated the value of various soft skills programs to her executives using anecdotes and comments. She was proud of the stories about these programs, but one nagging question remained: Do you have actual data? She also received feedback that her stories needed structure. She needed a thought out process with stories and data.
Our ROI Methodology provides a framework for categorizing, collecting, analyzing and reporting six types of data (reaction, learning, application, impact, ROI and intangibles). This framework presents a profile of success with qualitative and quantitative data often collected at different time frames from different sources. Numbers, by themselves, can be boring. When collecting data, always provide a place for respondents to provide comments, which will provide the basis for the narrative. A narrative in a story format makes the results interesting, engaging and memorable. The key is to have both numbers and narrative.
Reaction data will have measures to show that participants perceived the program to be relevant, important to their success, and something they would use and recommend to others. In a place for comments, one participant in a program on building trust noted, “This is the most significant professional development program I have attended in my 22 years of work. It was timely and just what I needed.” That brief but powerful story leaves a lasting impression.
Learning data provide measures of learning, whether it is self-assessment, role-playing, test scores or simulations. While important, executives may not be very interested in learning data; however, a good story about learning could be helpful. For example, a participant in a sales training program commented, “I came to this program expecting it to be difficult and confusing. Instead, I’ve learned a tremendous amount and now have the confidence to face my first real customer.” This story brings the participant’s experience to life.
Application data often show the extent, frequency and success of use, as well as the barriers and enablers. Comments collected at this level will often be free flowing, presenting a great opportunity for stories. In a leadership for high-performance teams program, a participant said, “This was much easier to use than I thought. The reaction of my team was much better than expected and, for me, it is working. The support, tools and job aids provided made it much easier. This will be my new approach going forward.” This story presents a vivid image of success.
Impact data are tangible (output, quality, costs and time) and intangible (teamwork, engagement and collaboration). Impact data capture executive and sponsor attention. Displaying those measures requires data and more data. A story can personalize the success. In a program on improving work habits of the team, one participant wrote, “After using this material, I noticed unplanned absenteeism began to reduce quickly. This validated what I learned in the program. I am amazed at the results. Not only did I meet my goal of reducing unplanned absenteeism, but I exceeded it.” The words personalize the impact.
A story at the ROI level is also possible, even if the participant hasn’t seen the ROI calculation. One nurse manager, responding to the success of a safe workplace program, said, “I was amazed at how we improved our safety performance. When we evaluated one year with this program, it exceeded our wildest dreams. We avoided some very costly patient accidents. No doubt, this is a positive ROI for the hospital. It’s also an eye-opener of how we can lower costs in the hospital.” This story responds to ROI, even without the actual ROI calculation. This is powerful.
Intangibles are measures not converted to money because of time constraints and credibility concerns. Great stories often surround intangibles. One participant in a leadership program said, “This program has enhanced my team’s ability to collaborate, communicate and work together. It’s made an amazing difference in our work and the efforts of my team.” This story connects important intangibles to the program.
It is also possible to follow one participant’s story completely through the evaluation levels. Although this may take more time, it shows one person’s experience throughout the process.
In short, to be effective, present both numbers and narrative within a framework, through which the audience can logically see how the stories emphasize the data and illustrate the overall value chain.