Recently while presenting to a local ATD group, I asked the attendees if they felt measuring learning was an important part of professional practice. All 50 learning professionals in attendance raised their hand. Next I asked, “How many of you conduct learning measurement at your company?” Only half the attendees raised their hand.
That result immediately revealed one thing: We believe measurement is important, but we don’t do it. Hmm.
I asked the group that conducted measurement to raise their hands again because I wanted to see what they gained from their measurement programs. I asked them to keep their hands raised until they heard a result they did not get from their measurement program. My questions covered 10 benefits that I know the ROI by Design model provides. This is how it went:
Question 1: Do your measurement methods allow you to confirm alignment between your learning programs and organizational needs? One hand went down.
Question 2: Does your measurement tell you which objectives were achieved, and which ones were not? A few more hands went down.
Question 3: Does your measurement tell you which objectives participants applied on the job? Again, a few more hands went down.
Question 4: Does your measurement tell you what participants knew prior to learning? And again, even more hands went down.
Question 5: Does your measurement tell you which of your design decisions were successful and which were not? Now only three raised hands remained.
Question 6: Does your measurement provide you with the information necessary to repair poor design decisions? The three raised hands remained.
Question 7: Does your measurement provide you the number of skills you added to the organization? Still, three raised hands remained.
Question 8: Does your measurement provide you the type of skills you added to the organization? Now two hands went down, leaving only one hand raised.
Question 9: Does your measurement tell you which department managers supported/reinforced training back on the job? Still one hand raised.
Question 10: Does your measurement tell you which organizational metrics were targeted or changed by your learning program? At the end of my tenth question, this one person still had her hand raised.
I asked her what measurement model or process she used. She told me she was a consultant and used a variety of methods.
Why I Asked These Questions
While I decided to do this exercise at the last minute, I often address groups about measurement and to explain the ROI by Design measurement model. I was tired of the “this is what we do” speech and wanted a more pleasurable way to convey the messages. This seemed like a new way to engage the audience and make the message real and interesting for them.
I believe this worked because participation was high, but I also think the message homed in more for me than the attendees.
I spend every day helping organizations design measurable programs, evaluate learning results, and convey the impact learning programs have on their organization. Daily I see how failure to understand cause and effect affects learning. I witness the frustration learning professionals feel when they think the poor measurement methods of the past are all that’s available. I watch the apathy set in as learning professionals fail to gain credibility from their lackluster measurement methods.
How Did Professionalism In Learning Take This Turn?
Even with all this experience, the responses from these 50 learning professionals shocked me.
How did things get this bad? How did learning professionals (who, by definition, want to help others) completely skip over the “how” we help others? Why have we focused our professionalism in learning on conveying information (having content and facilitation expertise)? Instead, why haven’t we focused on expertise in enabling learning through instructional design, instructional methods, and adult learning principles?
Why have we focused measurement on activities (number of courses developed/attended and how much learners interact with learning materials)? Why haven’t we instead focused on outcomes (competencies added and changes in job behavior)?
When I saw the complete disconnect between knowing measurement is important (as a means to ensure we are doing our job well) versus who actually conducted measurement, I was stunned.
Asking A Hard Question
It comes down to this: Do learning professionals really want to know if, or how, they make a difference to an organization?
Do learning professionals really care if they make a difference? Do they really want to know which methods of their practice work and which methods do not? Is getting better at helping people learn and grow their competencies their goal? Or, are they simply satisfied to provide content and engaging experiences?
It’s not about measurement. Measurement is a tool. Measurement shows what works and does not work. Instead, it is about gaining the information necessary to do your job well. It is about caring that you do your job well.
The absence of measurement practice and lack of rigorous measurement said a lot about the failure of professionalism in learning. I need to spend time considering the implications of this realization.
Am I missing something? Do you interpret these conditions that same way? I would love to hear your thoughts on the state of professionalism in learning.
Photo copyright: michaeljayfoto / 123RF Stock Photo
Enter your information information below to subscribe to our blog