I don't know about you, but the word evaluation can send a shiver down my spine. For many of us in learning and development it's a word that can have so many negative connotations, we sometimes fudge or avoid thinking about it completely.
I think these negative associations are because, typically, we take too narrow a view of the word. For most people evaluation is about whether or not the learners liked the course or the trainer - or the chocolate biscuits served up at break time.
This kind of evaluation really gives us little more than broad, hard to quantify opinions about something. What we really need to do is start thinking about evaluation as a means to really identify what's effective about a piece of learning. And what's not.
If we adopt this broader view of evaluation, then it has a place through the entire design and development process, not just at the end. This is true for any kind of learning, but is especially true for e-learning.
I say that because unlike classroom training, e-learning is more time-consuming and more expensive to refine once it's been created. If you are evaluating its potential effectiveness at every stage in the design process, it's much more likely to hit its target first time, avoiding the need for costly revisions.
We can take a leaf out of the usability designers book here. They do something very close to what I'm about to describe with website design. It's a simple, practical exercise which frequently gets overlooked or skipped over in a typical e-learning design process.
Work 1-to-1 with some typical learners
This is something you should do while you are still in the prototype or storyboard stage of development. The only difficult parts are getting access to a learner or two and co-ordinating diaries. I say 'only'. I know those can be two major difficulties. But it's worth persisting, because the dividends this exercise pays are tremendous.
Sit with the learner. Have them evaluate the prototype or storyboard and give you their feedback. There are various things you can look at. How clear or understandable is the content? Are the proposed interactions or activities relevant and meaningful? Can they make sense of the overall interface and the specific navigation?
Do this with a handful of learners and you'll very quickly get a sense of what is problematic or confusing for everyone and what is just a subjective opinion held by a single individual.
You'll need to be a good note taker. Because you'll usually get plenty of valuable comments which you won't want to forget. Better still, (with the learner's permission) you might consider recording what they have to say.
Jakob Nielsen tells a funny story about how website designers react the first time they do an activity like this. The first user is wheeled in and starts to look at the design. Some things just don't make sense. 'They must be a particularly stupid user, not to get that" thinks the designer. Then the second user is wheeled in. Same problem with the design. Then the third. Same problem again. And so on. Until the designer 'gets it' and the penny drops: their design is the problem. Not the intelligence of the users.
And that's the beauty of carrying out an exercise like this, during your e-learning development. It strips out any ego that might've found its way into the design. It forces you as the designer to really see how the learners react to it. In the end, this helps you make changes that your learners will only thank you for.
I think these negative associations are because, typically, we take too narrow a view of the word. For most people evaluation is about whether or not the learners liked the course or the trainer - or the chocolate biscuits served up at break time.
This kind of evaluation really gives us little more than broad, hard to quantify opinions about something. What we really need to do is start thinking about evaluation as a means to really identify what's effective about a piece of learning. And what's not.
If we adopt this broader view of evaluation, then it has a place through the entire design and development process, not just at the end. This is true for any kind of learning, but is especially true for e-learning.
I say that because unlike classroom training, e-learning is more time-consuming and more expensive to refine once it's been created. If you are evaluating its potential effectiveness at every stage in the design process, it's much more likely to hit its target first time, avoiding the need for costly revisions.
We can take a leaf out of the usability designers book here. They do something very close to what I'm about to describe with website design. It's a simple, practical exercise which frequently gets overlooked or skipped over in a typical e-learning design process.
Work 1-to-1 with some typical learners
This is something you should do while you are still in the prototype or storyboard stage of development. The only difficult parts are getting access to a learner or two and co-ordinating diaries. I say 'only'. I know those can be two major difficulties. But it's worth persisting, because the dividends this exercise pays are tremendous.
Sit with the learner. Have them evaluate the prototype or storyboard and give you their feedback. There are various things you can look at. How clear or understandable is the content? Are the proposed interactions or activities relevant and meaningful? Can they make sense of the overall interface and the specific navigation?
Do this with a handful of learners and you'll very quickly get a sense of what is problematic or confusing for everyone and what is just a subjective opinion held by a single individual.
You'll need to be a good note taker. Because you'll usually get plenty of valuable comments which you won't want to forget. Better still, (with the learner's permission) you might consider recording what they have to say.
Jakob Nielsen tells a funny story about how website designers react the first time they do an activity like this. The first user is wheeled in and starts to look at the design. Some things just don't make sense. 'They must be a particularly stupid user, not to get that" thinks the designer. Then the second user is wheeled in. Same problem with the design. Then the third. Same problem again. And so on. Until the designer 'gets it' and the penny drops: their design is the problem. Not the intelligence of the users.
And that's the beauty of carrying out an exercise like this, during your e-learning development. It strips out any ego that might've found its way into the design. It forces you as the designer to really see how the learners react to it. In the end, this helps you make changes that your learners will only thank you for.
