Theme 1: They just don’t get it

Look back at some of the comments above, and you’ll see a general sentiment arise that I’m labeling “they just don’t get it”. It is that exasperated, jaded, and somewhat depressing playmate of “I told you so” and perhaps the distant cousin of “not my job”. I believe this collection of thoughts arises from repeated frustration with stakeholders around what learning designers tend to care about most: helping people learn and develop.

I know I’m not alone in judging stakeholders fairly swiftly as just not grasping the absurdity of their propositions and actions related to the design and evaluation of learning programs.

For example, “they” (stakeholders):

• Want to move forward with new requests and not backward for evaluations and post mortems
• Resist, avoid, or deprioritize defining success of a proposed learning event
• Just don’t get how people work when it comes to learning new skills

Other than a lot of frustration from well-intentioned people, I’m struck by the focus of these statements from other designers and myself. The source of the problem is on “them” and the lack of knowledge and perspective of the stakeholders. Could it be that if all stakeholders were up to speed on the latest learning evaluation models then most of the barriers to measuring the impact of learning would go away?

I don’t think so. Indeed, I believe this problem is largely one of our own making. By putting the responsibility on my stakeholders to sponsor and execute evaluation work, I am cutting out the role I might play as the designer of that learning. I’m offloading the responsibility and accountability for my designs.

You may be thinking. “Are you saying that this barrier of working with stakeholders is actually just wrong-headed thinking by learning professionals?”

Partially, yes. There are real and unavoidable challenges in working with stakeholders which are addressed in the next section, but hold on for a moment! This theme of stakeholders “just not getting it” is really important, because here is the truth: they don’t! They aren’t the learning design and evaluation experts. You are!

If we turn the tables and ask why you don’t understand the details of the operations plan or the dependencies of a marketing campaign at the level that an operations or marketing professional does you would rightly declare something like, “that’s not my expertise.” Or admit “I just don’t get it.”

A key part of a learning professional's role is evaluation. But the “E” for evaluation in ADDIE is often minor or even kicked off the end of the process entirely. This needs to change.

Shifting the Focus

When I start approaching measurement and business impact evaluation from a different angle in which I am responsible for ensuring this thing gets done vs. my stakeholders somehow “getting it”, things really take on a new shape.

The solution to overcoming this barrier can be summed up as taking responsibility to ensure that measurement and evaluation of your learning designs take place. How that actually looks will be different based on your role, the current degree of influence on your team or in your organization, and the nature of the learning products you have charge over.

This isn’t easy. It may not be entirely possible in your current role or organization right now. I get that it isn’t as simple as deciding that you will now evaluate everything and the problem is solved. What can change almost immediately though is viewing your stakeholders as people rather than obstacles and reframing your role as chief evaluation advocate on all your designs.

Reframing the designer’s role

In shifting the responsibility of evaluating the value of your designs to yourself you are reframing the challenge. Here are a few next steps you might consider as a way to get started with this reframe:

• Write down a list of all the things that you wish were being done as it relates to measurement or evaluation with your learning designs. If people and resources weren’t a problem when it comes to providing evidence of the value of your programs, what would your ideal state look like?
• Next, look at each item on the list and write down why, to your best knowledge, that currently isn’t happening.
• Finally, put on the lens of responsibility. Is there anything you can do right now within your own realm of influence to begin moving towards some or multiple items on this list?

I hope you found at least one thing you can begin to do differently. However, it is likely that you can’t move forward with a few things on your list without deep collaboration with others.

Theme 2: Stakeholders make measuring impact difficult

In recent years there has been a movement toward human-centered design in a host of industries including learning design. At the heart of this approach is empathy which I think everyone will agree is a critical attribute of any designer or emotionally intelligent professional. Empathy is important, and I don’t think you’ll find many convincing arguments that it isn’t. Few will be surprised when you advocate for empathizing with a customer.

What I don’t hear spoken of as much is empathy for stakeholders.

For example, do these comments sound familiar?

• “They just don’t get it”
• All they care about is making a profit
• They waste so much time and resources on awful training initiatives
• As long as you are giving them what they want, they’re happy. But don’t ask about what the impact is! What a waste of time.

Can you imagine for a moment that you said these kinds of things or took this tone about your learners or customers in a team meeting? My hope would be your team wouldn’t allow it, or would at least challenge you to produce evidence before considering such a blanket statement. And yet when the target is a stakeholder other than the learner it is rare for folks to hold back grievances and annoyance. People are people, and whether they are your learners or your project sponsor they have a complex set of interests, challenges, and characteristics. No one deserves to be reduced to a label.

Ok, be nice. But what about those really difficult stakeholders?

There are and probably always will be people who really don’t want to look deeply into the nature of their training request or learning program they sponsored. As you likely already know, just “being nice” isn’t going to solve this. Being nice can actually get you in a lot of trouble! For example, to be nice I might say YES to everything that comes my way. To be nice, I might not push my stakeholders to define success from the outset of a project.

A better way of viewing this is by being kind. The common maxim, “clear is kind”, is the goal.

First, look at people as people and not some other label that contributes to a false narrative. And second, commit to driving clarity with your stakeholders both learner and otherwise. A fantastic tool for making this work comes from project management; stakeholder analysis.

Stakeholder Analysis

Theoretically, every major project has a project management plan which includes a stakeholder management plan. A stakeholder management plan includes many pieces, but the most relevant here is the stakeholder analysis. You may be familiar with the concept of personas, a process in which a design team creates fictional characters with key attributes to help guide design decisions during development so that the final product resonates with the learner or product end-user. A stakeholder analysis is analogous to this, but rather than creating representative personas you are gathering the actual people’s preferences and attributes and mapping that to a plan of action.

A stakeholder is any person that will be directly connected to the creation of or outcomes from your project. This includes project team members, approvers, sponsors, key partners, learners, learner managers, LMS administrators (or other technology partners), and more. By extension, stakeholders can be more broadly considered as anyone who may be impacted in any way by your designs. While comprehensive consideration for all of these people may not be possible, acknowledging the various groups is still important.

Essentially this looks like identifying all the stakeholders related to your learning project or initiative and writing them down and determining to what degree their role factors into the success of your program, and how best you can involve them in your project.

The influence/interest grid is a helpful place to start thinking about your stakeholders. Used often in the project management discipline I like this grid as it helps me begin to make sense of the landscape of stakeholders who I should consider not only involving in the project but also understanding how they measure success and to what degree I should communicate with them.

Using this grid, consider these steps to completing a basic stakeholder management plan:

1. With your team or by yourself, write out all the stakeholders you can think of related to the project. In just a few minutes you may generate a big list
2. Assign each stakeholder a place in the grid.
3. If you aren’t confident where a given stakeholder might fit on this grid, set up a time with them to gauge their interest and level of influence on the project.
4. For those under “manage closely” set up a time with them to talk through your project and begin learning their language and their explicit and implicit definitions of success. Depending on your project it may be worth getting time with your “keep informed” and “keep satisfied” stakeholders as well.
5. Consider asking them what their ideal state looks like as it relates to your project. What evidence would be most convincing to them that the project was successful?

The most ideal situation is that before a project is ever identified you actually reach out to stakeholders to begin developing a relationship with them before you have any explicit agenda. Design is inherently a social activity, built on foundations of trust and service to others. If you are designing for a set of stakeholders too narrow in scope then you face the dreaded “swoop and poop” scenario. When you fail to identify stakeholders early on who turn out to be highly influential and highly interested in your project, or even highly influential with minimal interest, your design can face major setbacks or all out destruction. They could “swoop” in at any time and all is lost.

Don’t let that happen!

Ok, so I’m taking responsibility now for measuring the success of my learning design and am committed to making it happen. I’ve worked to identify the various stakeholders and their level of interest and power as it relates to my project. But what if I can’t get time with them? Or what if I don’t have access to what really matters at higher levels of the organization? These barriers are real and painful especially if I’m viewing return on expectation as one of the key indicators of success.

You can’t really evaluate impact if you don’t have a certain degree of access to what currently is going on in the business and what impact would actually look like. You need that coveted “seat at the table”.

Theme 3: A Seat at the Table

I’ve often heard learning professionals (and myself) pine for “a seat at the table”, to be treated like a business partner rather than an order taker. This intention is closely related to the previous sections. If we take responsibility for building business partnerships with stakeholders rather than waiting for someone somewhere to make it happen for us, then the path forward becomes one we create for ourselves.

Easier said than done, right? Some challenges along this path emerged when I talked with other designers. A few barriers to getting the “seat at the table” include:

• Difficulty understanding the problem without just saying “Yes” all the time
• Pushing back on requests in order to first align with stakeholders on the goal and definition of success of the learning intervention, even when it is uncomfortable
• Dealing with significant dependence on stakeholders for data access, reporting, or strategic engagement in general

Much has been said and written about the first two items and while they are very difficult there are answers out there with tactics and interpersonal tips to doing so. But the last item points to a deeper issue that underlays all three of these items. This issue is the fundamental thing keeping L&D away from the elusive “table” of influence. This critical element is something I’ve rarely heard talked about explicitly in L&D.

I’m talking about power.

Power and Influence

Martin Luther King Jr. said it best:

“Power properly understood is nothing but the ability to achieve purpose. It is the strength required to bring about social, political, and economic change.”

My belief is that when learning professionals declare a desire to have a “seat at the table”, what they are really saying is “we have no power, and we want some in order to help learners be successful.”

In their book, Designing Your Work Life, Stanford Professors Dave Evans and Bill Brunett have a helpful formula that illuminates the murky halls of workplace politics. They use influence here, which I think is a proper synonym for power.

Influence = Value + Recognition

Learning professionals will not get power or exert influence until they can demonstrate genuine value to the organization in a way in which those stakeholders recognize that value.

Why power and influence?

For learning designers it means:

• An opportunity to bring about change for learners in positive ways
• Funding to properly carry out the mission of learning in the organization
• A say in the strategic decisions of the company or business unit (including saving lots of wasted time and resources on training programs that are doomed to fail from inception)
• And related to our topic here: the ability to generate and access data to measure our impact

You are likely in the learning design discipline because helping others grow and develop is one of the most important parts of your work life. It is frustrating when you can’t achieve this purpose, but with increased power and influence you can make huge strides towards realizing this goal.

So look back at the formula above and ask yourself:

1. What is the genuine value that your learning designs generate? For the learners? For the organization?
2. Is that value recognized for what it is by your stakeholders? By you?

This gets tricky when you start to translate the value into terms that make sense both to you and others. “People like my training” is one indicator of a certain kind of value, but you can bet that it pales compared to something like “our customer churn rate decreased by 3% over the last 2 quarters due to the new customer education initiative.”

Some value is extremely difficult to operationalize into a number, for example, I’m thinking of a project I worked on last year creating a single experience that gets every new employee up to speed on the basics of our product on their first day. The time saved and value gained from this technically could be boiled into a number. The problem was getting the data insights from all the groups needed in order to do that, including in areas where zero measurement was already taking place. In my case, the actual feedback from director-level people who had been newly hired (including the new CEO) has been more than enough to translate this value to the existing executive team. They get it: this learning experience is valuable to the organization. The return on expectation was more than enough.

This might not have been the case at a different company, however, so learning what “counts” as valuable to your stakeholders and translating your work into that language, and generating recognition for it is the only way to go.

Determining value (return on expectation)

How do you create value then?

In their book, Kirkpatrick’s Four Levels of Training Evaluation, Jim and Wendy Kirkpatrick explain how return on expectation is the ultimate indicator of value:

Stakeholder expectations define the value that training professionals are responsible for delivering. Learning professionals must ask the stakeholders questions to clarify and refine their expectations…

Determining the leading indicators upon which the success of an initiative will be measured is a negotiation process in which the training professional ensures that the expectations are satisfying to the stakeholder and realistic to achieve with the resources available (p. 34).

If you really want to know what is valuable, you will need to uncover that yourself through conversation and negotiation with your stakeholders. You can’t deliver a valuable learning experience based on your own definition of value, it must come from those you are serving (learners and stakeholders alike). By clarifying expectations, you are then in a position to meet or exceed them which in turn allows you to generate value and take steps towards building the influence you need to bring the perspective of the learning design department to where you want it to be.

Generating value recognition

This value must be recognized or else it makes no difference. If you designed the most amazing learning experience ever and the value it generates is massive, people might connect the dots and recognize your program as the contributor. But what if you proactively did this yourself? Regularly share the value that your program is generating in a language, setting, and method that stakeholders understand.

“I don’t toot my own horn, Greg” you might say. I get it. So don’t! But DO toot the horn of the value you have created, your team members that helped bring that about, and the leaders that cleared the path or partnered with you to make that value a possibility. Make it easy for stakeholders to see the connection your learning initiative has on the success of outcomes they are deeply invested in.

Your learners may be aware of the value that your learning program created, and so might middle management or other stakeholder groups. That said, do you want to leave the success of your program up to chance?

If you’ve created a learning intervention that is helping people achieve impactful results on the organization, you must make this known. Here are a few ways you might do that:

• From the beginning of the project, identify a communication plan whereby you will share updates on the project. Use this same communication channel to share the success of that project both immediately after completion and as you gather evaluation results over time.
• Take time to meet your stakeholders before there is a project connecting you together, or when there isn’t a specific reason for meeting with them beyond your genuine curiosity to better understand their world. Leverage these relationships when the value you’ve generated is clear. Seek feedback, ask for advice on how to make the results more visible to more stakeholders and the language and methods by which you can share those results.
• Work to understand deeply the metrics that your manager, director, VP, and executive leadership team are watching. Connect the value your learning intervention has created directly to one or more of these metrics in venues such as 1:1s, skip-level meetings, all hands (ask to have time on the agenda to share successes), and/or Slack/Teams channels.

As you create value for learners and the organization, and strategically garner the recognition of that value it is only natural for you to gain the ability to achieve purpose and effect change. This power and influence gives you a seat at the table to advocate for your learners, influence the business for good, and shape the culture of learning at your organization.

In order to provide value, you have to create it first! Going back to my earlier example about the company-wide product learning program I created, gathering the needed data was so difficult that I ultimately decided it wasn’t worth it. Why? Because the opinions of the new CEO and director-level new hires were strong enough data points to carry the message of value to the rest of the company. But if that hadn’t worked, I had a fallback plan in that I had cultivated relationships across various teams who had access to data that we determined would be compelling in the long run in providing evidence of the value of what we were doing. While I’ve captured some of that data and story, I’ve not had to rely on it to convince other key stakeholders of the business impact of this project.

A key part of your design process should always include alignment with stakeholders, with evaluation plans and execution woven throughout the whole project.

Wrap up

My assumption was that in exploring the question of measuring learning impact on business or organizational goals I’d find quantitative and data literacy as the primary culprit. While I think these things certainly do matter, and most of us in the learning design discipline (me especially) could use additional development with these skills, none of that matters much in the face of working with stakeholders and the associated challenges in doing so.

As learning designers, we can take responsibility for ensuring some level of measurement and evaluation takes place for the work we produce. We can meet stakeholders where they are and determine what success is for them and what that looks like in ways we can provide a high return on expectation. We can identify what is truly valuable to our organization and create strategic connections to our work and that value. In doing these things we will make a real and impactful influence on our learners and other stakeholders! Having those contributions recognized repeatedly will pave the way to develop influence for you in your organization and for the discipline of learning design in general.