10 Simple Steps to Start User Testing by Tomorrow (and Why You Should)

Written by Craig Morrison - Get free updates on new posts here.


User testing is the only way to know how people are interacting with your product. One simple test can reveal almost 100% of the problems that are slowing your sales, sign ups, and retention rates.

Imagine I showed you two layouts for a pricing page and asked you to tell me which one would produce more sales.

What would you do?

You could do some research on the best way to create a pricing page.

Then, based on that, you would tell me which version you think will do better.

But can you be 100% sure it will? No.

So what do you do? You test both versions at the same time, and discover that version B performed 50% better. Perfect! Now you’re all finished right?

Nope.

Version B of the pricing page performing better doesn’t actually tell you how effective version B is at all, it just tells you it’s better than version A.

So how do we really figure out if our pricing page design is working? By asking people to use it, and watching them while they do.

This is called user testing. It’s the number one most effective way to build a highly profitable, highly engaging and highly shared product.

Do you want to learn how to get more sales, more sign ups, and more extremely satisfied users?

Why is User Testing So Important?

To start, consistent user testing has been shown to increase customer satisfaction, decrease customer support needs, and save a butt load of money in design and development costs.

But, when it comes down to it, user testing is important for one hugely important reason. Time.

User testing immediately shows you real problems with your product. Not problems you think you have.

This means you spend your time more effectively by solving problems that have a direct impact on things like sales, sign ups, growth and retention.

For example, let’s say I pinpoint a problem with my web app. Users are signing up, using the service, but they aren’t inviting any new users.

I need to know the reason for this. If I can make it easier for my users to invite people, then my product will grow faster.

I could approach this in two ways. First, and most commonly, I‘d try to figure out the problem myself.

I’d setup a meeting with my designer, head to the whiteboard where we’d put our thinking caps on and try to hammer out what’s wrong.

Once we think we know the problem, we create a brand new layout and study the analytics to see if it performs better. When it doesn’t, we head back to the drawing board and start the whole process over again.

By the time we even begin to make a dent, we’ve spent 300 man hours and burned up $50,000.

ZOINKS!

6 more packs of markers and we'll have it solved

6 more packs of markers and we’ll have it solved

The other way I could, and should, approach this is by running a number of user tests and letting the problems reveal themselves.

Just asking a user to invite someone new and watching them struggle is probably enough to show me almost all of the problems I need to fix.

User testing is cheap, simple, and extremely effective.

That’s why you need to start now. The 10 steps below will give you all the info you need to run your first user test tomorrow.

10 Steps to Start User Testing by Tomorrow

Step 1 – Setup Your Testing Zone

The great part about user testing is that it can really be done anywhere, at anytime, as long as you have laptop and a person to test on.

But, to get the most out of your user test, you want to make sure you follow some basic guidelines.

First, you want the space to be quiet and free of distractions. Your subject should be focusing on what their doing. Distractions can skew the results of the test, so this is pretty important.

Second, you’re going to want a laptop with a camera, or a webcam attached to your desktop, and a microphone. We’re going to be recording our subjects voice and facial expressions. Body language can tell you a lot about the user’s frustration level when attempting to perform a task.

Third, you want to install software that can do all of the following things:

  1. Record your screen as the user navigates
  2. Record video from the webcam
  3. Record audio

Personally, I use a free Chrome extension called Screencastify. It works great, records all three of these things, and is free. The downside is it only records in the browser, so if you’re testing software, you’ll want to check out another screen recording program like Jing.

Lastly, make sure to test your setup before running your first user test. You don’t want to be fiddling with the computer while your user gets bored.

Please take a seat in the pile of wet leaves and we'll get started.

Please take a seat in the pile of wet leaves and we’ll get started.

Step 2 – Identify Your Target Audience

Not everyone is going to have a product full of dedicated users. Maybe you run an online store for men’s grooming products, or maybe it’s a mobile app. Either way, it’s important to know who your audience is, and recruit people from it.

Many products will also have multiple sets of users. Remember to consider this as well.

For example, maybe I’m trying to find out why people aren’t buying women’s shoes like I think they should. Would it make sense for me to recruit a man for this user test?

Make sure the people you recruit relate to the problems you’re trying to solve.

It’s important to keep in mind that a user test is not the same as asking for feedback. With this in mind, here are some commandments to follow when choosing test subjects:

  1. Thous shalt not test on fellow employees, whose mind doth contain too much insider information for valuable results.
  2. Thous shalt not test on any team member significant others. Bias shall be present in the form of false flattery.
  3. Thou shalt not ask social media sources to “try it out and tell me what you think”. Bias shall also be present in the form of false flattery, or, false condemnation!
Step 3 – Identify Problems and Hypothesize Fixes

Hypothesize! That’s right, I just used that word. Take THAT Grade 10 science teacher that gave me a D.

In order for a test to be a test, you need to have something you’re testing. Watching someone use your app isn’t a test, it’s just observation.

The best way to approach a user test is by using the scientific method. This means that once you identify a problem, you hypothesize a solution, implement it, and test to see if your solution passes or fails.

For example, let’s assume I notice that people aren’t completing the checkout process on my online store. They’re adding things to a cart, but never paying for them. So how do I find a solution?

I could just guess at what I think it is, but that would be wrong.

Instead, I would run a user test, asking a user to buy something. Once I noticed them get stuck, and spoke with them about it, I would probably have a pretty good idea of the problem.

The next thing I would do is form a hypothesis. In this example, I’m going to hypothesize that “Users are not checking out because they can’t find “checkout” button easily enough. Making the button larger and easier to see will increase checkouts.”

Next I would re-run the same user test and gauge the new subjects reactions. Did the next people I test have a much easier time checking out? If yes, then my hypothesis was correct and it passed, unlike myself in Grade 10 science.

If it failed, then I would go back and make another hypothesis, test, and repeat.

Exactly.

Exactly.

Step 4 – Create a List of 5 – 10 Tasks

This is where we get into the real meat of the test, telling the user what to accomplish.

You need to keep in mind that the way you ask the user to do things has an enormous impact on the outcome of your tests. In my opinion, you want to keep your instructions very vague.

For example, I could say, “Navigate to the women’s shoe section, add a pair of heels to your cart and then checkout”. But if I say that, I’m basically giving them all the directions.

Instead, say something like “Navigate the site and buy a pair of shoes that you like with this credit card number”.

This is much closer to the way a real user would think when using your site. They wouldn’t have someone telling them what to do, it’s up to them to figure it out.

Generally, you want to split tasks into sections. If you want more people to sign up or buy, you can focus your tasks on the most common conversion goals. Things like signing up, purchasing, adding to cart, and paying.

If you’re focusing on improving usability, focus on the most common user goals. Things like adding a profile photo, changing email settings or deleting something.

The most important thing to remember when writing your tasks is to put yourself in the mind of a regular user. If you do this, then watching your test subject will be as close as you can get to secretly watching a real user.

Be sure to set clear goals that define the success of these tasks. If you ask the user to sign up for your service, make a note that this task isn’t completed until the user reaches the end of the entire process and is back on the homepage (this could include activating within an email).

Step 5 – Write a Script for Everything

This is going to seem really really strange but, you need to script everything you’re going to say to the user. And you need to follow that script exactly.

Testing works the best when all the subjects have been exposed to the same constant variables. To get accurate results, you need to control the environment, and replicate it every single time you run a test.

For example, if you’re having a rough day, and you perform a user test and come across as cold or harsh, the user doing the test may be extra nervous.

This could cause them to be less focused on their assigned tasks, making you think there was a problem with your app when in reality, it was you causing the problem.

At the same time, you might accidentally emphasis different parts of what you’re trying to accomplish to different users. This could mean the results you get back are all over the place.

As you continuing reading this article, the things that you will need to script out will become more obvious.

Step 6 – Find People to Test On

By this point, you’ve got your testing area ready, you’ve written some tasks out that will attempt to find the root of your problem, and you know the audience you want to target.

Next, we need to find some people to test on!

There are a variety of ways to do this, the easiest and most obvious is finding people who are already users of your product. This works best when you have a site that has repeat users that requires them to log in, such as a social network or mobile app.

You can simply reach out to your users through email, personally, and ask them if they’d like to participate in a paid user testing study to help improve the app.

Yes, paid. The only way you’re going to get these people on board with giving you a decent chunk of their time and attention is by paying them. It doesn’t need to be much, especially if you can find users in your area.

You can also offer gift cards or discounts to your service in return.

You can also find subjects on craigslist and Kijiji, just by posting ads and listing the user requirements (over 60, female, etc.) or even ambush people in Starbucks and see if they’d mind taking part.

Or you can try online user testing. I’ll cover this in full detail at the end of this article.

Step 7 – Welcome the User and Explain Things

The moment has arrived. You’re on your way downstairs. An actual person has arrived at your office and is willing to be a subject in your user test.

Holy shit. How did you get here? You’re not a psychologist, what are you doing conducting psychological experiments?

Calm down! We’ll get through this together.

The best thing you can do is be extremely welcoming to your test subject. You want the person to feel relaxed. Welcome them into your office like it’s your home (even better if it is your home).

Offer them a drink. If you’re funny, make a joke.

If you aren’t funny, don’t make any jokes.

I wore this costume to make you feel more comfortable!

I wrote this costume to make you feel more comfortable!

Have them sit down at the computer. It’s important to load a simple and non distracting webpage, like Google, on the computer to start.

Don’t have your website loaded yet. We want to see the person react with fresh eyes.

Next you’re going to explain to them how the test is going to work. It’s very important that you script this part out ahead of time and read the exact same script to every user. Explaining this in different ways can really skew your test results.

A really important part of user testing is understanding what the user is thinking. Since we can’t read minds, you’ll need to explain to your test subject that you want them to think out loud.

This is going to be weird for them at first, but if you find them being quiet during the test, keep asking them “what are you thinking?” They’ll get it after a while.

Remember the last time your friend asked you to search for something while they sat beside you? For that split second you thought, oh my god, I can’t remember how to spell pizza…

People are generally pretty uncomfortable with being watched, especially when using a computer. To take some pressure off your subject, explain to them that you’re testing the software / website, and that you’re not testing them.

Tell them there are no wrong answers, and that they’re simply a tool for seeing how well the website works.

Here are a few more important things to mention:

  1. Tell them not to be be polite. They should be completely honest and open about how they feel.
  2. Tell them not to start clicking around to other pages unless they’re told it’s ok.
  3. Explain that they can ask questions but they might not get answered during the test.
Step 8 – Get Them Talking and Relaxed

Now that you’ve explained to them how things are going to work, they’re probably feeling pretty anxious. Especially with all that robotic script reading you’ve been doing.

Get them talking to relieve some stress, but also to get them used to speaking out loud.

Ask them some questions such as:

  1. What do you do for a living?
  2. Why do they do that job?
  3. How often do they use the internet?
  4. What do they primarily use it for?
  5. What are some of their favourite sites?
  6. Do they use mostly email or do they browse or play games?

Their answers don’t really matter, the point is to just create a relationship with them. By the time it’s time to start testing, they should feel comfortable enough to speak out loud candidly.

Step 9 – Ask Them to Perform The Tasks

By now the user should be comfortable and ready to begin the test. This is where we ask the user to complete the tasks we wrote beforehand.

I write each task down on a separate sheet of paper and hand them to them one at a time. This way, they always have something to refer back to in case they get confused about what they’re supposed to be doing, but they can’t skip ahead and get distracted.

Begin the test by asking the user to take a look at your homepage. Don’t get them to perform any tasks yet, we just want to get a “first impression” from them.

Try asking them questions like:

  1. What are your first impressions?
  2. What strikes you about it right away?
  3. Who do you think this site is for?
  4. What do you think you can do on this page?

The important part is to get their first impression, but to also keep them talking. This is a good introduction to how you want them to behave for the rest of the test. Repeat that they should think out loud. Correct them if they try to click around.

It’s important to set up a “user scenario” for the user as you give them each task in order to get most accurate results.

For example, “You spend a lot of time outdoors in the winter, and because of this, you’ve found your feet are often cold. Buy a pair of good winter boots that will keep your feet warm from our online store.”

This puts the user into the mindset of someone who is actually looking to buy boots, rather than the mindset of a person performing a test.

Remember, try not to phrase the tasks too much like instructions. You want to match the mindset of an average user to your site, and no one is standing over their shoulder telling them what to do next if they want to buy a pair of shoes.

Step 10 – Analyze Your Results and Repeat

The test is over and you’ve sent your test subject on their way. You’re either feeling very happy or ready to break down and cry. Either way, you’ve got a boatload of results that need summarizing.

The goal here is to narrow down what worked and what didn’t, and decide what actions to take.

If you set the “success goals” of your tasks, this shouldn’t be too difficult. If the user reached the “success” mark, then it passed, if they didn’t then it failed.

For example, I made a hypothesis earlier that users wouldn’t checkout because the “checkout” button wasn’t big enough or easy enough to see. I hypothesized that making the button larger would increase the amount of people checking out.

My success mark would be a user reaching the point where the system sends them an email receipt.

After testing 5 subjects, I would decide if this was a success based on two observations. First, how many of the 5 users passed and how many failed? Second, based on observing the users behaviour, how easily did they find the checkout button?

If the test was a success, I would push the larger checkout button live to my site. If it failed, I would create a new hypothesis that was based on the users behaviour and thoughts while attempting to complete the task.

I know this sounds a lot like A/B split testing, and it is, but I firmly believe it’s smarter to do both at the same time. Let me explain.

Traditional A/B testing relies on someone taking a guess at changing something they believe will do better, then testing to see if it does or not.

On the other hand, user testing relies on listening and recording user behaviour in order to learn what changes will perform better, and then testing to see if they do or not.

User testing can reveal an unbelievable amount of detail about your users. You can then use that info to influence your decisions on your A/B testing. This leads to more efficient testing.

Bonus Step – Online User Testing

User testing is a fairly simple process, but sometimes you just don’t have the resources or time to be performing live, in person user tests. The solution? Online user testing.

I have used online user testing for my last two startups, and let me tell you, the first time I ran one, I was blown away.

Companies like UserTesting.com employ a huge variety of people willing to perform user tests on demand. This means it’s easy to request the testers match from your demographic. Age, gender, income level, even specific requests like “knows how to use a DSLR camera”.

You specify your tasks and the users screen is recorded as well as their voice, speaking their way through the test.

Just to be sure UserTesting was legit, I signed up as a “tester” and was approved. I spent about 2 or 3 weeks performing user tests, and I discovered that the majority of tests I was offered, I failed to qualify for.

That’s pretty awesome, because it means the testers they give you are exactly what you want.

If you want to try a free version of online user testing, check out Peek. Submit your URL and they will send you back a free 5 minute video of a tester answering some generic questions about your site. Even this can be eye opening.

UserTesting’s full tests run for $50 each.

Let’s Summarize

User testing is essential because it clearly illustrates the way your users are interacting with your product.

One simple test can reveal almost 100% of the problems that are slowing your sales, sign ups, and retention rates.

Start user testing tomorrow, or as soon as possible. To get started, download my user testing cheat sheet that summarizes all 10 steps to easily conducting your first user test.


  • Christian Contemori

    Dear Craig, first of all thank you for your very helpful article.
    I would like to ask you something about pitching businesses to increase the chances to get hired as UX designer by showing them usability tests conducted on their service.
    As you wrote in another article this is the best way to show a business that you have spotted a problem, that it could be measured and that it can be fixed.
    Now my question is: how do I run a test on a website/app that I don’t work for and how do I measure the key metrics that address the issue? Since I am not able to access that data I am a little bit stumped on how to make this process effective.

    Thanks

    Christian

    • Hi Christian, sorry for the late reply. I missed your comment.

      The idea behind this strategy is to keep track of the data yourself. You won’t be able to make actual changes to their live product, but you can create mock ups in a variety of different ways.

      For example, let’s say you run a test on a checkout page, and you see the 9 out of 10 people you tested struggled to find the “checkout” button when instructed to.

      You make your proposed change to the page, and say build a quick interactive prototype using screenshots + Photoshop + an interactive mockup tool like Invision.

      Run 10 more tests with the changes you’ve made and see if now only 1 out of 10 people struggled to find the checkout button.

      Get it?

      It’s a lot of work, but it works.

  • Crystal Roberts

    Very helpful article! But I’m having trouble downloading the user testing cheat sheet. I tried several emails in several browsers but keep getting an error.

  • EJ

    I can’t open the user testing cheat sheet… would you send it to me please? eipkkuk@gmail.com