The use of A/B testing, where customers get different versions of the same live site to interact with, is an increasingly popular tool for website owners. However, while the company may get useful feedback and data from this testing is it really the best way forward?

While the results might look black and white, A/B testing still leaves a lot of unknowns. The test designs can be subject to bias from the person setting up the test, while customers may not get the best service they were expecting. It also leaves the question of how many testing iterations do you go through, and what impact will this have on the goodwill of your customers.

Pros of A/B testing

Get clear evidence
It’s easy to see how many users complete a transaction with site A over site B. The evidence is based on real behaviour, so is hard data of the type that money men love (and can be presented in a simple-looking, hard hitting chart).

Test new ideas
If you have an innovative idea for an existing site, A/B testing provides hard proof as to whether it works or not. However, you will need to implement that big idea in hard code before you can test it this way.

Optimise one step at a time
If you run a large site, or many sites, then A/B testing is a fantastic opportunity to “patch” test, by starting out in a small corner and then working up to the main pages of the site. However, can smaller site users with less traffic afford to gamble with real users by giving half of them a site experience that might not be optimal?

Answer specific design questions
Are green buttons better than red ones for your site design? This and many other questions can be answered by A/B testing as they allow the designer to test different colours, placement of buttons, page layouts, different images which are all good areas to slowly improve a website.

Cons of A/B testing

Can take lots of time and resources
A/B testing can take a lot longer to set up than other forms of testing. Setting up the A/B system can be a resource and time hog, although third-party services can help. Depending on the company size, there may be endless meetings about which variables to include in the tests. Once a set of variables have been agreed, designers and coders will need to effectively work on double the amount of information. In addition, in order to get conclusive results, tests can take weeks and months for low-traffic sites.

Only works for specific goals
This kind of testing is ideal if you want to solve one dilemma, which product page gives me the best results? But, if your goals are less easy to measure pure A/B testing won’t provide those answers.

Doesn’t improve a dud
If your site had usability problems to begin with and the variations are just an iteration of that, it is likely to still have the fundamental flaws that your other site contained. A/B Testing won’t reveal these types of flaw or reveal user frustration and you won’t be able pick up on the reasons behind the site’s problems. Just because A resulted in more sales, it is only in relation to B. Removing the original usability issue could be much quicker to identify and result in much better results.

Could end up with constant testing
Once the test is over, that is it. The data is useless for anything else. Further A/B tests will have to start from a new baseline and other types of testing will only likely be applied to the more successful site, when they could have found equally useful information from the rejected version.

The best use of A/B testing

When used with other testing methods, A/B testing provides a valuable tool in refining a working design and finding out what attracts your users or helps them complete the processes on pages. However, it cannot measure ease of use, frustration or other elements, so cannot be relied on as the total solution. Therefore, utilise some form of usability testing to better understand the users’ frustrations and issues, then use A/B testing to test the different solutions.

What are your experiences of A/B testing, were they as useful as you had hoped?

Related services: eCommerce Usability &  Usability Testing

Most website owners are never happy with their sites. It could be doubts from a design or content perspective, or merely not seeing the metric results that were anticipated. Whatever the issue, site stress is becoming a common ailment among the Web fraternity.

Our research has shown that many site owners are actually some six months behind where they want it to be. Some react by throwing time and effort at the site without considering the underlying causes, leading to wasted effort and yet more stress.

Others look for new ideas and features to add, which can just over-complicate the site and further exacerbate the problem. If any of this sounds familiar, there are some easy ways to fix your site. They are not instant solutions, but can make for a quick turn around and a lot less stress.

1. Opinions are like tummy buttons, everybody has one, but they’re full of fluff!
Many sites draw influence from a range of sources. This may be from rival sites, well known ones, input from designers, design agencies, even colleagues and peers. Designing a site based on opinion is a recipe for disaster, the design for your site should be pure, simple and achieve the tasks behind your business plan. If you have a live site that came to life through opinions without speaking to users, it is critical to validate them with research rather than assuming everything is fine. More on how to deal with opinions about your website

2. His vital stats are dropping, get the crash cart!
Following the metrics religiously in the early stages of a site’s life is a sure way to get a headache. Consider the first few months as the creation of a baseline which you will compare against in the future. As your site finds its feet, you might get a spike of interest that will tail off, or you might see cyclical peaks and troughs (beyond the usual weekly rhythm). Use this data to gradually better your site and not induce a panic attack.

3. You don’t need a crash test dummy, but a crash test smarty
Using website testing is the best way to find out what is wrong with a site that is not performing as expected. Usability testing can methodically go through the site and point out where real users will have trouble, find a problem or get frustrated. Fixing those problems found in testing is the only way to ensure that your users get the best experience and keep them coming back.

4. Innovation can save the day
If you start to worry that your site is stagnating then throwing bells and whistles at the problem will not solve it. Instead get creative with what you offer your customers or clients. Make your content more valuable, insightful or add unique elements. Reward loyalty from repeat customers and create offers that add value for customers rather than just slashing prices which will only serve to reduce your own income.

5. Check what’s going on around you
Watching competitors is always a good plan. The trick is not to just blindly follow what they do (we’re staggered at the number of designers who are asked to build a clone of a successful site from people who think that’s all that is needed). Do what they do better, or with your own unique twist, or just do something that makes you look different to build up your own customer base. If you’re always following your competitors lead, you’ll always be one step behind.

Keeping it together
When your site is struggling, the first thing is step outside the problem(s) and look at your site as a whole. Often, getting an independent voice is the best solution and this is where usability testing can play the most beneficial role.

Have you had website anxiety? how have you dealt with it?

Related services: Usability testing

Nothing will turn a visitor away from your website faster then coming across a problem or issue that they cannot see an instant way around. Even a pet peeve can see a potential customer going elsewhere at the speed of a mouse click. Common issues with websites can often be solved in short order and radically increase the overall friendliness to keep people coming back.

 
1. Why must I sign up for something you say is free?
Many visitors are led to a website with the promise of some free content, information or other goody, only to run away when they see the dreaded sign-up screen. Trading their personal details makes the deal not free in the eyes of many users.

Solution: Remove the sign-up screen entirely, or limit it to an optional request for a name and email address. When the visitor sees the value of your content they are more likely to come back.

 
2. Why enter my e-mail address twice?
Endless duplication is a bane in many lives. But, this step, seen over and over on sites throughout the ages with no attempt to fix the issue, can stop some users in their tracks.

Solution: Instead of asking them to enter an address twice, let them enter it once. Then, show it to them on the final confirmation page in a larger than normal font size with a tick box to confirm the address.

 
3. Why is there so much text on a page?
Most users are goal orientated, so they scan the page to find the next step on their path. When there is too much text on a page, particularly during the first pages on a site, when they need the most guidance, visitors get frustrated and tend to leave because the site is too much effort.

Solution: Employ a copywriter (either get a freelancer or use an agency), or be ruthless in editing and remove everything that isn’t needed. Use bullet points where possible to condense text and get your points across.

 
4. Your site is confusing, why are there so many icons and too much information?
If a site has too many options, buttons and icons – users get overwhelmed with the choice and leave. Some homepages can be so crammed full of links, text buttons and banners that users really struggle to find what they are looking for. The irony is that site owners/designers often put all these options there to help users. But, how is a typical user expected to know what all those icons mean or trawl through vast amounts of text on the first page?

Solution: Prioritise key users (who is most likely to visit your site) and prioritise their core information needs, the reason they are visiting your site. Offer the high priority options and allow users to drill down to more detailed information rather than offer everything up front.

 
5. Why is your site full of strange words and acronyms?
Just because you and your web-designer friends can have an entire conversation in TLAs, doesn’t mean the rest of the world does. Nothing will put a visitor off a page faster than a wall full of jargon, industry terms and buzzwords.

Solution: Ask a non-technical person to read through a print out of a site and highlight anything they do not understand. Replace any highlighted terms with plain English, if some terms or acronyms are necessary, ensure the site has a highly visible panel defining any acronyms or terms.

 
Trouble behind the scenes
One or more of these problems can appear on almost any website. It may be that the designer is simply following a previous design, or that the site owner has seen them elsewhere and considered them “de rigueur” or good looking. Either way, they put up additional barriers to a good user experience but, fortunately, can be effectively pruned to improve the site.

Are there any other common issues you can think of and how would you address them?

Related services: Usability Testing & User Experience Audit