Landing Page Testing

A couple of experts at a conference in Dublin put together a compelling case for multivariate landing page testing. If you don’t have someone doing landing page testing for you, then it’s pretty clear that even smaller companies are losing out on getting to grips with your website users.

My company, Receptional Internet Marketing are now making use of a new Google product called Website Optimizer. I am at a conference in Dublin where Jon Myers and Russell Sutton from Conversion Works are talking about landing page optimization.

I know that Russell is one of the few Google Optimizer professionals in the country (we have Nick Gaunt at Receptional) but up first was Jon Myers who is from Mediavest. Like Receptional, Mediavest is a Google Certified Adwords company. Their presentations give a compelling case for using this technology.

Jon talked a little about Website Optimiser, but then really started to concentrate on the “quality score” metric in Google Adwords. He pointed out the need to ensure that the quality score of your advert is “good” or “great”.

To improve your Adwords campaign, you can test different landing pages for your adverts. Within the Google Adwords system, you can set Google up to alternately send traffic to first the home page and then alternately the product specific page, for example, and Google helps to tell you which page engages the user better and – assuming you have conversion tracking set up in Adwords – you can work out which page generates a higher return on investment.

Jon also noted a few other tools. was recommended as well as a particular favourite of Receptional – Jon also noted that has all the decent  free tools listed in one place for conversion and landing page testing.

Jon than went on to the main course – A:B Testing and Multivariate testing and used a case study  of Skype, who tested three different layouts for their “buy” page (which is pretty close to their home page). Using Google’s website Optimizer, they were able to randomize the three variants for the users and track the orders and revenues generated from the three layouts. The audience were asked to guess which page worked best… and most got it wrong.

Jon showed some nice screenshots from  Website optimizer that showed how you can easily drill down and understand the effects of what you test, but did warn about the issues of some mathematical principals surrounding test using limited amounts of data.

Jon feels times are changing and that the marketer is really starting to get in control of the message from search, but you only have 1.8 seconds to attract your user from search these days. This means that you now need to start treating Google itself as a landing page. Local business listings in Google can have phone numbers ON the Google results, for example, meaning that  now your customer doesn’t even have to come to your wesbite to convert.

As an aside to Jon’s presentation, Receptional have a number of technologies that allow us to track users by ohone number – one of which can even tie phone numbers into Google Analytics results.

Russell came on next. His business really only concentrates on conversions, not traffic generation. He started by asking the question: “What’s in it for you”? and said that it is all about outcomes – in terms of money or actions or market share or any other chosen metric. Knowing what you want to get “more of” is key to knowing what to test.

Russel; says that when you know what you want more of, you then need to use analytics to find the pain. (Receptional has Google analytics and Yahoo analytics expertise in-house). He took an example from Google analytics which clearly showed two pages that had a much higer bounce rate on a site than any other and chose these to start testing variations. He pointed out that the higher the traffic to those pages, the more the pain for your business.

He recommended starting with a super small test to get used to the Google Website Optimiser technology. You then need a hypothesis of why you think a particular page may be broken. You then should be bold – try some  changes that look really different and test the outcomes.

Russell used a courier company case study. The home page had a huge bounce rate and their hypothesis was that the “get a quote” button was tiny. They then came up with 8 variations on a theme – not just one.  Again – the audience was asked which variation worked best. It turned out again that we all guessed wrong.

The winning result increased the click through to the quote page by 101%.

Russell and Jon’s presentations – together with Receptional’s own experiences – show that Google optimizer can make an incredible difference to your buttom line. The maths is compelling. At Recetional, Nick Gaunt has taken on this specialism and has taken the Google website optimizer exams. If you would like to contact him, you can do so via .


Microsoft “Quality Index” uncovered

Microsoft reveal many of the underlying factirs that affect their AdCenter Quality Index Score, giving you an edge in lower cost adverts, and quicker “submission to Live” times.

Microsoft are to start filtering PPC advertisers by market segment and website type in the UK, so that they can better use this segmentation to improve their “Quality Index”, the equivalent of the Google AdWords “Quality Score”. The changes are already integrated into the US Ad-Center listings but are likely to come to the UK over the next few months.

The news was met mostly with approval at AdChamps Europe. Mary Berk from Microsoft took us through much of their thinking about their Quality Index algorithm and for once I made some notes, which I hope you’ll find useful. I started taking real notice when I realized that Microsoft had already started rolling out “QI” improvements in the US, but that these had not yet come to the UK. This made me realize why chatter on the boards tends to be so inconsistent feedback-wise.

Mary highlighted that some of the things that matter in their interpretation of quality included “Market Segment” and “ad-style” as well as some of the more usual things that we have come to expect from Google. By way of example, we heard that aggregaters may get hit and have to pay more to stay in the running. She also stated that MFA (Made for “Adverts” as she cleverly described it) would also take a big hit. Other sites that effectively rearranged search may find themselves having issues, but she also admitted that this was also hard to get right, and it was likely that unique content would be one of the main differentiators between a site that gets penalized and one that doesn’t.

Interestingly, some searches were done on the fly in the room and on the whole, the US users do seem to be seeing far less of these kinds of results in the paid search results. Expedia was a notable exception, though arguably their content is plenty unique enough.

It seems that the system is able to – both algorithmically and manually – segment sites into types as well as markets and the quality team can then make decisions and set up filters at several levels within the system. I take this to mean that their Quality team will be able to assign rules affecting your bid price either by segment or individually.

Microsoft are not stopping there, however. Mary also went on to say that Microsoft have also developed more sophisticated “user intent” algorithms. The example of where this might be important was if a person typed in “pick up trucks” and the advertiser bid on the phrase “pick up”, but offered an advert about “Pick up lines”. Clearly a mismatch which will not be fixed by the advertiser and would only be fixed by the user by repeating the search… not necessarily using the same engine! It is this propensity for people to jump AWAY from Microsoft’s offering when they get just one bad experience that is really driving this “QI” initiative, it would appear. Mary gave some overwhelming statistics about just what would make people change search engines.

Mary also went on to give examples of sites that they really were targetting – hopefully to the point of elimination. In particular, they are concerned about sites that give deliberately misleading claims. For example “free” offers that might require jumping through so many hoops that they offer really is not free at all.

In fact, she gave quite a few clues as to how to get your adverts up and running without any delay… and how to get your adverts delayed of penalized. When the “QI” improvements take hold in the UK (as they are already in the USA), your history and the keywords you are taergeting or using will be affect your initial ad-launch. Certain keyords or patterns will trigger a manual review – which may happend before or after the advert goes live. If it does happen after the advert goes live, however, their objective will be to carry out the review withing 48 hours. Trigger words will include the word “free”. Whether the advert goes live immediately or not will largely depend on how much of a good submission history you enjoy as an advertiser or agency. Long standing “good guys” are likely to get the benefit of the doubt – at least initially.

The objectives seem to be twofold:

  1. To get 90% of all new adverts live within minutes – not hours.
  2. To make it harder (more expensive) for non-original content to dominate the PPC results.

It will be interesting to see how sites like Moneysupermarket fare under this regime in the UK… as they are a price comparison site and aggregator, yet they are also one of the UK’s largest PPC accounts.

The Quality Index improvements are coming to the UK before the end of the summer.