The Measurement Method Harming Our Profession, Part 1

learning professional upset over choosing wrong measurement method

I’ve written often about the value of the learning profession. I’ve described how measurement is essential to show our value. But recently I read an article that said measuring ROI was detrimental to the learning profession. I was immediately skeptical, but soon I understood the author’s point. His argument was that the real goal of learning was not to show a financial result. Instead, he said, the goal was to show a “return on expectations.” In other words, we need to show how learning created an expected change in behavior. I can agree with that conclusion because the point of organizational learning is always to improve performance (behavior). Behavior change results directly from our work as learning pros.

But then I thought, if our goal is to show a return in the form of behavior change, why are so many of us using surveys to measure learning? Surveys are the wrong measurement method if we want to show behavior change. Is trying to measure ROI really the problem? Isn’t measuring by survey even more detrimental to our profession?

A Popular — Yet Faulty — Measurement Method

Much as been written about how ineffective it is to use surveys to measure learning success. Most learning professionals now see that using a survey to measure learning program success is less than ideal. Numerous studies have confirmed that survey results do not accurately indicate learning outcomes. Analyzing data from surveys does not provide true performance results. Still, many businesses use surveys to justify training budgets because they are easy to implement.

Working in L&D, I understand how surveys became a popular choice. Unfortunately, I also see how survey use has damaged our industry. Their use to validate learning results had disturbing effects on our profession. Specifically, survey use made us vulnerable. It shifted the learning professional’s focus away from effective practice, and erected real barriers between learning teams and other business units.

The maxim “Tell me how you measure me and I will tell you how I’ll behave” holds true in learning measurement. Surveys don’t gather actual behavior data; they gather perceptions and opinions. Using surveys as the main measurement method resulted in learning professionals chasing the “please like me” vote. By design, surveys ask the opinions of respondents. Opinions are influenced by emotions. A person’s opinion of learning program value can be predisposed by much that has little to do with how effective the learning was. When a learning professional is judged by participant opinion then the evaluation hinges on things that can influence learner opinion. This has resulted in learning professionals focusing on the wrong things. To make the participants happy (and thereby drive positive reviews), learning professionals have focused on things like:

  • The food and venue
  • Keeping learners engaged/entertained
  • Conducting learning programs in the least amount of time possible

Content Rather Than Capable

To prompt positive responses, learning professionals turned their attention toward keeping their audience contented instead of making them more capable (by giving them new skills and changing their behavior). As long as learners “liked” their experience, the learning professional felt safe and valuable.

Using survey response as our success measure let us ignore the need to ensure learners gained skills. Instead of verifying our results through outcomes (knowledge and skill gains), we relinquished our “power” to others. Instead of measuring our own value with reliable, accurate measurement methods, we instead asked others to tell us our value. Our value relied solely on what others thought of what we did.

You only resort to asking others your value when you do not know how to value yourself. We became vulnerable to others’ opinions. This vulnerability chipped away at the confidence of learning professionals because many of the things needed to “be liked,” to achieve positive survey scores, were out of our control.

In my next post, I’ll give more detail about why surveys are the wrong measurement method to pursue.  I’ll also describe how to undo all the harm they have done to our field.

Are surveys still part of your measurement methods? If you want to learn the right way to design and measure learning, then contact us here at eParamus. We’d love to help.

Please follow eParamus on LinkedIn and feel free to connect with me, Laura Paramoure, PhD to discuss the learning challenges you face.

Photo copyright: fizkes / 123RF Stock Photo

Enter your information information below to subscribe to our blog

[gravityform id=”14″ title=”true” description=”false” ajax=”false”]

Talk to a Training Evaluation Professional

Thank you for your interest in eParamus. We look forward to helping you meet your design and measurement goals.

Contact Us - Global