I don't know about you, but the word evaluation can send a shiver down my spine. For many of us in learning and development it's a word that can have so many negative connotations, we sometimes fudge or avoid thinking about it completely.

I think these negative associations are because, typically, we take too narrow a view of the word.  For most people evaluation is about whether or not the learners liked the course or the trainer - or the chocolate biscuits served up at break time.

This kind of evaluation really gives us little more than broad, hard to quantify opinions about something. What we really need to do is start  thinking  about evaluation as a means to really identify what's effective about a piece of learning. And what's not.

If we adopt this broader view of evaluation, then it has a place through the entire design and development process, not just at the end. This is true for any kind of learning, but is especially true for e-learning.

I say that because unlike classroom training, e-learning is more time-consuming and more expensive to refine once it's been created. If you are evaluating its potential effectiveness at every stage in the design process, it's much more likely to hit its target first time, avoiding the need for costly revisions.

We can take a leaf out  of the usability designers book here.  They do something very close to what I'm about to describe with website design. It's a simple, practical exercise which frequently gets overlooked or skipped over in a typical e-learning design process.

Work 1-to-1 with some typical learners
This is something you should do while you are still in the prototype or storyboard stage of development. The only difficult parts are getting access to a learner or two and co-ordinating diaries. I say 'only'. I know those can be two major difficulties. But it's worth persisting, because the dividends this exercise  pays are tremendous.

Sit with the learner. Have them evaluate the prototype or storyboard and give you their feedback. There are various things you can look at. How clear or understandable is the content?  Are the proposed interactions or activities relevant and meaningful? Can they make sense of the overall interface and the specific navigation?

Do this with a handful of learners and you'll very quickly get a sense of what is problematic or confusing for everyone and what is just a subjective opinion held by a single individual.

You'll need to be a good note taker. Because you'll usually get plenty of valuable comments which you won't want to forget. Better still, (with the learner's permission) you might consider recording what they have to say.

Jakob Nielsen tells a funny story about how website designers react the first time they do an activity like this. The first user is wheeled in and starts to look at the design. Some things just don't make sense. 'They must be a particularly stupid user, not to get that" thinks the designer.  Then the second user is wheeled in. Same problem with the design. Then the third. Same problem again. And so on. Until the designer 'gets it' and the penny drops: their design is the problem. Not the intelligence of the users.

And that's the beauty of carrying out an exercise like this, during your e-learning development. It strips out any ego that might've found its way into the design. It forces you as the designer to really see how the learners react to it. In the end, this helps you make changes that your learners will only thank you for.
 

 

We can all feel our pulse quicken, our emotions rise when we get chance to talk or write about a topic that engages us totally.

And we usually know lots about this topic. We can frequently talk about it for hours without getting bored. We can tell anyone willing to listen about its every last detail. In that sense, we are all subject matter experts (SMEs) in something.

As instructional designers, when we have a talkative SME in front of us and limited time to get the information we want, it's worth remembering how our own passion for a particular subject matter can allow us to get carried away.

So aside from being more empathetic to a talkative SME, is there anything else we can do to make our time with them more productive? I think there are four areas to consider when gathering content from SMEs. By the way, the greater the quantity of content you need to gather, the more you are likely to want to formalise the approaches below.

Ownership
Before any information gathering even happens, you need to take ownership of the process. This may involve becoming more assertive than normal: be quite specific about how you want the process to unfold, including the number of meetings you'll need, how long each meeting should be and how much time you'll need between meetings for reviewing and feedback.

Planning
Tempting as it might be to go into your early meetings knowing nothing, better to do research to familiarise yourself with the subject matter area. Spend time creating a basic project plan. Clearly define your and their roles in the whole process. Formally identify the risks of not getting the required information in a timely fashion and communicate this to the project sponsor.

Connecting
Your initial research can pay dividends once you start interacting with your SME. Exhibiting some knowledge of his/her topic can help build rapport and, more important, establish your credibility. Earn trust by emphasising the confidentiality of your information gathering sessions and the promise of a review of content before making it more widely available.

As the content gathering progresses, aim to establish points of shared interest both within the subject matter area and outside. Most people appreciate a little interest in their life outside work.

Focusing
Set an agenda in advance of the meeting clearly stating goals and expectations.
During your content gathering sessions, regularly paraphrase, clarify and summarise what you have covered; use closed questioning techniques if your SME has a tendency to go off on tangents. After the session, collate the content into a structured document you can share with your SME for review and feedback.


It's easy to dismiss some of the subject matter experts we deal with in our professional capacity as out of control windbags who want to bore us and our learners with every last detail of their knowledge.

That may be true. But let's not forget, given the right topic and the opportunity, many of us can happily do the same.

So with a bit empathy and some detailed preparation and work before, during and after your content gathering, the analysis phase of your project need not be an out of control nightmare.

If out of control SMEs are your current nightmare, check out our the Analysis and Planning modules in our instructional design programme for help on dealing with this problem.

 At a conference a year or so ago, I noticed a seminar that drew a good crowd was entitled, "Who says e-learning compliance training has to be boring?". Well not me for sure.

Perhaps, I'm a bit naive, but even now (after many years in the world of learning) it shocks me that some people can shrug their shoulders and say., "well this material is pretty dry and boring, so we'll just have to accept that the way we deliver it is dry and boring". To me that's a bit like the designers at a car company saying, "well it's a bit difficult to design a really comfortable car seat, so we'll just fit the car with uncomfortable wooden benches and the passengers will have to lump it."

Perhaps the acceptance of poor quality compliance training is linked to the box ticking mentality that often accompanies the dreaded 'c' word. We have to do the training  -  even though nobody wants to, so let's just collectively hold our noses and all be bored together - designers, trainers and delegates. Oh yes, and let's make it even worse, by delivering it as the most boring, sleep-inducing piece of e-learning you have ever seen.

From this point of view, you'd think that compliance in a particular job role or organisation wasn't something anybody really needed to know or do. Yet it most definitely is.

So rather than treating it as a dry abstract topic, why not relate it back to the context or contexts in which learners need to be compliant? Why not provide the learners with challenging, life-like scenarios and activities that require them to think about what they actually need to do to be compliant. How about some intrinsic, contextual feedback that vividly demonstrates the consequences of not being compliant or trying to cut corners.

If you find yourself nodding your head in agreement but you are feeling a bit unsure about the effectiveness of your e-learning in general and your compliance e-learning in particular, take a couple of minutes to complete our E-Learning Impact Scorecard which can help you benchmark what you are doing against some of our recommended best practices. 

It was the late Steve Jobs who said: "Design is not what it looks like and feels like. Design is how it works." Classic Steve. In just a couple of sentences he sums up one of the fundamental factors in Apple's phenomenal success.

But you don't have to be the producer of uber-cool computers and gadgets to find relevance in his words. They could be equally applied to aspects of e-learning - particularly with regard to the interface we devise for learners.

In my view, there's an interesting distinction to be made here between User Interface Design (UID) and Learner Interface Design (LID). The first is probably a more familiar term than the latter.

In an e-learning context, UID is about total simplicity and focusing on the ease of use of a course. Good UID shouldn't leave people pondering the outcome of alternative actions. It generally aims to minimise mental involvement. The overriding imperative of good UID? "Don't make the user think".

Now here's the difficult bit. LID, by contrast, has exactly the opposite imperative. It's totally about helping learners to think, learn and perform. Designing a learning interface is about getting learners to engage their mental faculties in order to learn. And this might involve confronting them with problems, challenges and issues.

In designing a learner interface, the focus should be on making it relevant and motivating. Something that will help the learner remember and implement what they learnt long after the learning event.

So, the challenge for e-learning developers is two-fold. First, being clear about the difference between the two concepts. Second, understanding where and when to apply one or the other.

If e-learning design and its overall effectiveness is something you are concerned with or responsible for, why not take our E-learning Impact Scorecard to help you benchmark where you might be with your overall e-learning design approach and any associated tools and techniques. It only takes a couple of minutes to complete and it's entirely free.