Last updated:  
Apr 10, 2018 @ 7:02 PM
Usability Testing Data: Don't Limit Your Data Collection Options

If you’re new to testing, usability testing data can be very technical and kind of confusing. It’s hard to know what you’ll need to invest, and what you can expect to get out of it at the end. Obviously, you want to test with real users so that you can get information about their needs and experiences, but what form will that information take?

One thing is for sure: If you decide to collect one type of feedback, a month down the road you’ll think of another thing you wish you had collected. That’s why it’s better to get everything you can the first time. Being prepared to go back and glean additional insights from the testing you’ve already done will add value and help keep your budget in check.

So what data should you get from your testers? Here are some ideas.

What usability testing data to collect:

1. Recordings

usability testing data videoEven if the system you’re working with logs interactions, try to get at least one more layer of recording, not only as a backup but to give you more flexibility at the end of the study. For some special sessions, we’ve recorded video of testers reacting to the software as well as on-screen recordings. You might think this isn’t relevant to you because the product you’re testing doesn’t have a visual or even an audio element. But recording is so inexpensive now that there’s really no reason to skip it, and you just never know how it might turn out to be useful.

2. Numbers

usability testing data numbersFind a way to generate a number, even if you’re more focused on user experience and feelings. Rating satisfaction on a number scale can work. Look for things you can count as pass or fail, the number of issues reported per session, and other numbers that describe performance. Grab whatever statistical information you can as well, not just counting the number of people who come through the door but finding out a little bit about them as well (without violating their privacy, of course!).

3. Written feedback

usability testing data feedbackTry letting testers take notes while they work, and be sure to hang on to those notes. Not only does note-taking seem to focus many people, it also helps them remember important things to relay at the end of the test. Once your testers have finished the technical side of what they’re being asked to do, they probably want to share their thoughts and recommendations with you – give them a forum to do that, either through a free-response question on a survey, a text document – whatever works.

Next steps

With usability testing, it’s great to collect lots of data, and there are tons of ways to do that. Just remember that the work you put into usability testing data collection is totally wasted if you don’t put in an equal amount of work organizing and analyzing it afterward! The value you get from mining that data in different ways will surprise you, and will make your project better, long after the testing itself is over.

We’ll have a follow-up post about creative ways to use some of this data soon. Other questions about usability testing? Share them in the comments!


globalme testing services

2 Comments

  1. Thanks for the great article, Emily!

    I agree – I’m never sorry that I captured too much information, only not enough. A few years ago I used a program called Morae for conducting in-person user studies, which has some great features like webcam, audio and screen recording, timed tasks and end-surveys. I’ve never used it for remote testing, but I think it has the capability to do so. It can be really helpful for designers and stakeholders to be able to watch the videos if they can’t attend the test when it occurs. For remote testing done by an external usability testing company, recording everything you can is possibly even more important, since no one is really there real-time. Userlytics.com, a usability testing company I’ve just started doing some work for, has options to record webcam, audio, screen (including mobile), as well as written and quantitative data via an optional test-end survey tool. Another company, Tobii, even makes a really cool product that allows you to track eye movements of your users!

    When creating a testing protocol and methodology, I always put in more questions and forms of recording and quantifying data than I might need. You never know! I have had things come in handy that I might never have guessed at the start of the test. Better to have it and not need it.. 🙂

    • Good info, thanks for chiming in! It’s true, you never know what will come in handy until you’re done testing, and it’s nice to be prepared for whatever questions come up.

Comments are closed.