SEO Werkt!

Posted by thogenhaven

A/B or MVT

MVT Face-off: Full Factorial vs Fractional Factorial

Testing The Test Environment With The A/A Test

Most inbound marketers are quite familiar with A/B tests. But what is less known is the A/A test. The A/A test is useful as a test of the experimental environment, and is worth running before starting A/B or MVT tests. The A/A test shows if the users are split correctly, and if there are any potential misleading biases in the test environment.

 

 

In the A/A design, users are split up like they are in an A/B or MVT test, but all groups see the same variation. We want the test results to be non-significant, and thus see no difference between the groups. If the test is significant something is wrong with the test environment, and subsequent tests are likely to be flawed. But as discussed below, an A/A test is likely to be significant sometimes, due to random error / noise.

 

The A/A test is also a good way to show co-workers, bosses, and clients how data fluctuate, and that they should not get too excited when seeing an increase in conversion rate with 80% confidence. Especially in the early phases of experiments.

 

Statistical Significance

Pro Tip: Ramp Up Traffic To Experimental Conditions Gradually

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

The whole point of A/B testing is to put your feelings aside, and let data tell you if the layout, content, and design of your web page or marketing element is optimized to meet your goals. But what happens when the way you proctor the test messes up that data, leading you toward misleading results and inaccurate analysis?

Just as it’s important to know the best practices of landing page creation and A/B testing to see great results, it’s crucial to know what not to do to make sure your efforts aren’t wasted. Consider these 7 common mistakes marketers make when performing A/B tests, and make sure none of them crop up in your next (or current) round of A/B testing.

The 7 Worst A/B Testing Mistakes Marketers Make

1.) Running an A/B test when you should run a multivariate test. What’s the difference between the two? In a few words, an A/B test evaluates the performance of two different versions of a web page; a multivariate test evaluates the performance of the elements inside of one web page, and offers far more possible outcomes because of the combination of elements that can result. Make sure you’re running the right test for your needs. If you’re not sure which test is right for you, read this explanation breaking down the difference between A/B tests and multivariate tests, and how to know which test to run.

2.) Not establishing the criteria for success. Now that you know you’re running the right test, do you know what your goal is? A successful A/B test will have a specific end it is trying to achieve. Hypothesize what the changes you’re making will result in, and know which metrics will indicate success. Some admirable goals might be to lower bounce rate for new visitors by a certain percentage or to increase click-through rate by 1300%. Whatever criteria for success you choose, remember that you can’t achieve success without knowing what it looks like!

3.) Starting your test with crazy web pages. The design, layout, and copy you choose for your first pages shouldn’t just be a shot in the dark. Base your decisions off of best practices so you’re not wasting your time with designs that, based on hundreds of thousands of tests from other websites, probably won’t work. In other words, they’re called best practices for a reason.

That being said, one reason A/B testing is so useful is because it tells you when you should flout best practices to achieve the best results. But only when testing tells you it’s prudent to do so. So in the meantime, follow best practices from people who’ve learned the hard way, and tweak according to the results of your A/B tests.

4.) Not performing a radical redesign. We just told you not to start with crazy web pages, and now we’re telling you to perform a radical redesign. What gives? The pages you’re testing should follow best practices, sure, but they shouldn’t still look exactly like one another. Move the form from the right side of the page to the left; dramatically change the size of your header; test the response to totally different language; experiment with different images. And do it all at the same time. If you don’t perform radically different tests, you could hit your “local maximum” and start iterating on designs that aren’t as effective as they could be to begin with.

5.) Performing tests on pages with too little traffic. A/B testing is great for new websites because you don’t need a ton of traffic to get meaningful results. But you still need enough traffic for statistical significance. Make sure you run tests on pages that are either highly trafficked, or if you’re running this test on new or buried pages, that you run the tests for longer than you do on your more popular pages to ensure you have enough data points for a meaningful evaluation. Before jumping to any conclusions, make sure you have enough data to make a relevant determination of success.

6.) Not considering how your changes affect other metrics. Have your design changes increased conversions but decreased time on site? Is that okay? It might be. If you’ve established the criteria for success (see mistake #2) and you analyze how all your important metrics are affected when you make design changes, you can be sure what you interpret as improvements aren’t actually having an unintended, negative consequence.

7.) Giving up when you see no results. Just because you didn’t see results with one test doesn’t mean you’ve hit the jackpot page. Consider more iterations you can test — different colors, layouts, copy, images, and proportions — to see if there’s still a better page design out there that you just didn’t consider in the first round of A/B testing.

Have you made any mistakes while A/B testing that adversely affected your results? Share them with the rest of us so we can learn from them, too!

Image credit: orijinal

Connect with HubSpot:

On a previous blog post about marketing tests, a commenter asked for some examples of great landing pages. We’ve received that request more than once and figured it’s about time to deliver! So I set out to find some of the best landing pages out there, and to get started, I went to some of my favorite companies. I figured they might be a great place to start because as a marketer, my favorite companies have to not only have a great product or service; they have to be great inbound marketers. Whether you’re B2B, B2C, a product or a services business, these five companies have created great landing pages from which we can learn some serious lessons.

To help hammer those lessons home, I’ve also included suggestions for how these landing pages can be improved even more. These suggestions are based on landing page best practices, and don’t take into account that sometimes, you can shirk the best practices based on the results of A/B and multivariate tests. I have no insight into whether these companies have run tests to reach these designs (many probably have!) but the lesson for all marketers is to build something according to best practices, then test, test, test until you get the best version possible.

ModCloth

ModCloth is a retailer of retro women’s clothing. This landing page prompts visitors to sign up for its mobile communications and offers.

What they’re doing right: ModCloth’s landing page rocks for two reasons. First, notice the consistency between the page headline, the form headline, and the button; they all mention joining ModMobile (glad to see you’re in on the mobile movement, guys!), so it’s very clear what you’re on this page to do. It’s important to have this consistency in all your headlines so your visitor doesn’t get confused about what action they can execute on that page.

ModCloth is also successfully explaining what happens if you join ModMobile through its page copy. Notice the use of bullets to break up each point they want to convey so the information is digestible.

How they can improve: Two page elements a ModCloth marketer might consider changing on this landing page are the color of the bulleted text and the “Join ModMobile!” button. The bulleted text is awfully light, making it hard to read, and the button could stand out more from the rest of the page, as the blue blends in with its site skin. Or hey, maybe they’re in the middle of some A/B or multivariate tests!

Salesforce

Salesforce is a CRM and cloud computing software company (with whom you may be intimately familiar if they are your CRM of choice!). This landing page offers a free download of a Gartner research report on sales productivity and automation.

What they’re doing right: First, take a look at the top of the landing page. Notice how there’s no navigation? This is a wise move, as it prevents the visitor from getting distracted and abandoning the landing page for another area of the site. Salesforce is also leveraging the use of relevant images on the landing page, including a screenshot of the research report the visitor will receive if they complete their download. They back this up with a relevant quote from the report that continues to engage the visitor and entice them to download the report.

How they can improve: Salesforce should follow ModCloth’s lead and include the name or subject of the report that the visitor will download. This should be in the form headline, and in the button copy. For example, the headline can be modified to read ‘Get Your Complimentary Gartner Report,’ or’ Get Your Complimentary Salesforce Automation Report’; the button can be modified to simply say ‘Download Your Report Now.’ These changes will help solidify the purpose of the page, leading to more conversions.

YouSendIt

YouSendIt provides secure online file-sharing software so anyone can easily send large files and attachments. This landing page lets visitors sign up for a free trial with the software.

What they’re doing right: Like ModCloth, YouSendIt is doing a great job using consistent language from its page headline to its form headline to its button. But also notice that they’ve selected a green button for their form. Using the green helps it stand out from the rest of their site, which is mostly blue and white. Aside from having a remarkably short form to redeem the free trial, there’s one more page element that is probably helping their conversions: the TRUSTe seal of approval. Including verification signs from third parties like TRUSTe, the BBB, or VeriSign helps instill trust in the visitor that they can safely enter their information to redeem the offer.

How they can improve: YouSendIt is rocking a pretty sweet landing page for their free trial, but there’s one area they can definitely improve. They have a navigation along the top and bottom of their site, increasing the likelihood that a visitor will get distracted and click away to another part of their site before filling out the free trial form. You can bring back your navigation and keep the visitor moving through the site with other offers on the thank-you page after the form has been completed.

Jetsetter

Jetsetter is an invitation-only travel community that provides its members with access to exclusive deals and insider information on amazing vacations.

What they’re doing right: Between the headline and the images used at the top of this landing page, it’s clear as day what you’re supposed to do here. Buy a travel gift certificate for someone. They also lay out the steps of the process clearly, highlighting that you are on step one currently, and graying out the next two but still including the copy that explains what happens during those steps. For an ecommerce site, an easy and clear shopping cart experience is crucial to getting your visitors to move through all the stages necessary to complete a purchase.

How they can improve: Jetsetter’s form falls below the fold of the web page, which can impact conversion rates for some sites. If you had a page with a similar layout to Jetsetter, one place they can cut space is the size of the image at the top. Alternately, a two column layout can help condense space and fit everything above the fold. To be really nitpicky, the copy on the “Proceed to Purchase” button is also quite small and light. Making it bigger, bolder, and brighter may help increase conversions.

SEOmoz

SEOmoz is a thought leader in search engine marketing and provides SEO software. This landing page lets visitors sign up for a free trial of their software.

What they’re doing right: SEOmoz is also making use of a third-party verification badge to instill trust in the visitor and using consistent language from its header to the copy in its button. The best part of their landing page, however, is the inclusion of the chat icon, which follows the user as they scroll down the page. Along with removing the navigation, this chat icon helps mitigate the chance of page abandonment by giving visitors the opportunity to get answers to questions that are preventing them from completing the form.

How they can improve: While the red bubble on the top right corner of the landing page explains what the free trial is for, the header could benefit from the inclusion of the software’s name. Visitors will be happy to easily confirm via the page heading that what they clicked through to download is in fact on this page!

When you’re creating landing pages, what best practices do you think contribute the most to a higher conversion rate?

Image credit: Beth Kanter

Connect with HubSpot:

There’s seemingly no end to the things you can test in your marketing, and if you’ve laid a solid framework for your inbound marketing programs, now’s a great time to start optimizing and making what works pretty well work amazingly well. And the best way to get started is to conduct some A/B tests!

Or…multivariate tests? What’s the difference between A/B tests and multivariate tests? Will it affect my results if I choose the wrong one?

Yes, there is a difference, and yes, it will affect the integrity (and thus usefulness) of your results if you choose the wrong test. But no fear! We’re going to break down the difference between A/B tests and multivariate tests in this post, and tell you exactly when to use each so your tests can run smoothly and make your inbound marketing rock even harder than it already does.

What Is an A/B Test?

When you perform an A/B test, you create two different versions of a web page, and split the traffic evenly between each page. You can also perform an A/B/C test that tests three different web page versions, an A/B/C/D test that tests four different web page versions, and, well, you get the picture. In an A/B test, you can change literally any variable you want from page to page, and it is in fact a testing best practice to create two (or three, four, whatever) radically different pages for your test.

What Is a Multivariate Test?

When you perform a multivariate test, you are not testing a different version of a web page like you are with an A/B test. You are performing a far more subtle test of the elements inside one web page. The point of the multivariate test is to give you an idea of which elements on a web page play the biggest role in letting you achieve the objective of that page. The multivariate tests is more complicated and best suited for more advanced marketing testers, as it tests multiple variables (get it? multi-variate?) and how they interact with one another, giving far more possible combinations for the site visitor to experience.

This is a tricky concept, and a visual usually helps clarify complicated ideas. Luckily, Search Engine Land shared an image on its site from Yam Designs that visually illustrates what a multivariate tests looks like.

Should I Use an A/B Test or a Multivariate Test?

A/B testing is a great testing method if you need meaningful results fast. Because the changes from page to page are so stark, it will be easier to tell which page is most effective. It is also the right method to choose if you don’t have a ton of traffic to your site. Because of the multiple variables being tested in a multivariate test, you’ll need a highly trafficked site to get meaningful results with MVT.

If you do have enough site traffic to pull off a successful multivariate test (though you can still use A/B testing if you’re testing brand new designs and layouts!) a great time to use the testing method is when you want to make subtle changes to a page and understand how certain elements interact with one another to incrementally improve on an existing design. You can also use multivariate testing to perform a test that will give you results you can extrapolate out and apply to a larger site redesign.

Just remember that, in order for the multivariate and A/B tests to give meaningful results, it’s not enough to have site traffic overall; the pages being tested also need to receive substantial traffic! Make sure you select pages that people can find and visit regularly so your test actually yields some data to analyze.

Have you performed A/B tests or multivariate tests on your website? Did the results of the tests cause you to make any changes to your site?

Image credit: Guudmorning!

Connect with HubSpot:

by Stoney deGeyter

Sometimes getting conversions is like trying to capture mist in a jar or water with your fingers. No matter how hard you try, you just can’t seem to get a secure hold on them.

There are countless test you can do with your website to help increase your conversion rates. A/B and multivariate tests can help you increase your conversion rate a couple of points, which can often translate into thousands of dollars of increased profits. But in all that conversion testing, trial and error, banging your head against the wall and twisting customer’s arms until they cry “UNCLE!,” there is often one overlooked piece of information that can help you dramatically improve your conversions.

That piece of information is: information. Content, to be exact!

A little information can go a long way when building relationships with your visitors–your would-be customers and bringers of the sacred sale. Building a relationship that is based on a genuine interest and possible dialogue with your customer can lead to more customers, higher sales and significant business growth. No arm twisting or head-banging needed!

Unfortunately, many websites employ the “less is more” philosophy. They see content as the barrier that prevents the customer from getting to the check out isle. Unfortunately, when you remove the content you are not removing barriers; you’re removing the associate that helps the customer find what they need.

I don’t know about you, but I often find myself in the middle of the store looking for someone–anyone–who can help me. Looking up and down isles, not finding an available employee within driving distance, I get the urge to shout, “I’m going to steal something!” just to see if anyone cares!

Your content shows them you do.

How information wins conversions

Unique content on your home, category, sub-category and product pages explains the value in purchasing your products and services. It allows you to provide the relevant information each visitor needs to know your product will satisfy their wants and needs. It gives them the emotional justification needed to commit to buying what you’re offering and make them feel like a puppy just licked them in the face while they do it!

More information on your site creates less resistance in convincing the customer to buy. Content on your site essentially greases the wheels for the potential customer to move through the various phases of the buying cycle. The more customer focused your information is, the easier it will be for your customers to justify a purchase from you.

Essentially, adding quality information to your site helps your visitors make smart purchasing decisions. When customers make decisions on little and/or incomplete information, you may get the sale today, but quite possibly the return will come in the mail tomorrow. Lacking enough product information and purchase justification on the site means you’ll ultimately lose the customer for life instead of gaining a life-long spending buddy.

Detailed information on your products and services gives you greater opportunity to create happy customers. Being up front with both pros and cons, benefits and possible side-effects, allows the customer to weigh each against that of other products or even your competitors. Short of that information you risk having an unhappy customer or no customer at all, when all that was missing was the correct information they needed to pull the trigger on a purchase.

Well-written and customer-focused content creates an open and honest relationship with your customers. The more open you are about your strengths and weaknesses, the more open your customers will be with you, as well. This will give you better opportunities to meet their needs, if not now, at some later point down the road with new product or service innovation. Even if you lose that customer today, you’ve built a bridge to bring them back to you later.

Building up a content-rich website through all levels of the buying process gives your customers reassurance that you are trustworthy and a valuable resource for them to return to. When customers trust you, they hesitate less when making purchase decisions, which ultimately leads to more customer confidence and translates into greater sales and repeat customers.

Any business looking for the long term customers: content can make the difference between no sale at all and a life-long customer. You’re essentially allowing your content to play the role of the store employee roaming the isles asking customers, “How can I help you?” If you want to get conversions on your site, you need to think about how much content you’re giving your visitors.

If you don’t have quality, engaging, explanative and customer-focused content on your site, you’re preventing your customers from getting the information they need to make an informed purchase decision. That click you just heard is the sound of your visitors leaving because they can’t find any text that makes them confident in your products or services. That cha-ching is is the sound of your competitors’ cash registers ringing from the sales you just lost! And that crying? That’s you.

Follow me at @StoneyD, and @PolePositionMkg.

Be sure and visit our small business news site.