Nowadays customization has become a must and many products and services have adopted a “menu” approach similar to what restaurants offer. You can now find product categories in which:

• Different packages including a certain number of features are offered (e.g. Netflix, Software as a Service or SaaS online services)
• Products are offered in “combos” side by side à la carte items (e.g. McDonalds, customizable HP computers, cell phone plans, banking services)
• Customers have the opportunity to build your product from scratch (e.g. Genghis Grill, Build Your Own BMW).

The challenge is to find the right balance of offering to provide options for both Do-It-Your-self and Do-It-For-me customers. The good news is that we have now tools that allow us to mimic and research a “menu” offering faster and cheaper than ever. Welcome Menu-Based Conjoint Analysis.

This technique allows us studying customer choices in a context where customization is present and multiple choices can be made.

In traditional Choice-based Conjoint (CBC) analysis, customers are presented with a set of options and are asked to select one. The problem with this is that some of the features included in an option may not be relevant to the person making the choice.

Adaptive Conjoint Analysis (ACBC) is an improvement by starting with a set of exercises aimed at identifying which attributes are relevant and using them to create options from which participants have to choose one. Still, in both approaches, participants can only choose one option and in full customization scenarios these approaches fall short.

In Menu-Based Conjoint Analysis (MBC) we can create scenarios that match the actual offering, either combos + à la carte items or all à la carte items and let participants choose multiple options within the
parameters of the offering. We can then estimate the probability of choosing each “combo” or indivdual item presented in the menu at different prices.

Some of the business questions for which the MBC approach is a good fit are:

• How should the offering be configured? What items/features should be included?
• What item combinations are more likely to be purchased?
• How many item combinations should be offered to different market segments?
• Which prices by item or item combinations will increase purchase likelihood?
• What are the optimal premium and bundling prices?
• Which items are substitutes and which complement each other?
• Which items can be eliminated from the product line without having a big negative impact on sales?
• How will an offering structure change affect sales and profits?
• What items would increase the conversion rate of a promotion?

Products and Services that could benefits from Menu-Based Conjoint Analysis:

• Automotive
• Banking services
• Hotels
• Pharma
• Restaurants
• Software as a Service (SaaS)
• Subscription services
• Telecom services
• Website Development
• Any service or product offering customized solutions

If your business operates in any of these categories or in any other not mentioned here that offers any degree of customization, the Menu-Based Conjoint Analysis can help you not only to configure new offerings, but also optimize your current product and services to increase revenues and profits.

Customer satisfaction and customer retention, although correlated sometimes, don’t have a straightforward relation.

Customers may continue buying your products and services because of:

• Habit or inertia when the cost and risk of changing is perceived to be higher than the benefits and the customer experience may be “good enough.”
• Competing alternatives are as good or nonexistent.
• Price.
• Clearly superior products and services as well as excellent customer experience.

In customer satisfaction research, we typically ask about overall satisfaction with products or services. When inertia, lack of competition, price and partially good offerings are the main reasons why customers continue patronizing a product or service, an overall customer satisfaction score may be misleading.

To avoid being misguided by an overall customer satisfaction metric, you should include other metrics, such as likelihood to continue being a customer and likelihood to recommend the products and services to others. The fact is that no single customer satisfaction metric alone will be accurate enough.

Before putting a lot of weight on a single satisfaction score, making business decisions or basing employee compensation on it, design a customer satisfaction research plan with metrics that reflect the performance of your business in key areas, customer touch points and how you stand against the competition. Consider monitoring:

• Product/service design and performance
• Customer experience with your product or service
• Customer service
• Usage and satisfaction with competing alternatives
• Price sensitivity

When it comes to customer retention, adopt a holistic approach, use more than one metric and focus on key drivers.

There is a lot of debate about how to use social media content in market research and in which category it fits: Quantitative or Qualitative research?

The massive amount of content generated by social media and the proliferation of sentiment analysis tools attempting to quantify it, give the impression that it can be considered quantitative research and thus quantitative analysis methods (e.g. t-testing, regression, etc.) can be applied to it. However, the issues of data accuracy and representativeness have not been solved yet.

DATA ACCURACY

Anybody who has done coding of open-ended questions in surveys knows how difficult this is. Finding categories to classify the answers is a very subjective process. To attain some level of objectivity and find categories or codes that accurately reflect the content in these questions, we often need several iterations and different coders to achieve consensus.

In social media, as in surveys, more often than not, people write incomplete and grammatically incorrect sentences, and use irony, sarcasm or humor to convey their meaning. In social media it is even more difficult to discern what people are talking about, as there are no survey questions to filter the information by.

Unfortunately, there is not a good text analytic tool yet that can interpret language nuances as well as a human coder. Sentiment analysis tools still need to be fed with codes and definitions of positive and negative content defined by someone in order to count and classify content.

REPRESENTATIVENESS

Linked to data quality is the problem of representativeness of opinions. From customer satisfaction research, we know that people, who decide to provide feedback, are either very happy or very unhappy with the issue they give feedback about. Those who are in the middle often don’t bother to comment.

The level of category involvement also affects representativeness. Not all products command enough attention to be topic of conversation in social media, unless there is something that surprises or annoys the public (e.g. banks imposing debit card fees).

Then there is the issue of how we separate unpaid opinions from unpaid opinions, which are becoming more common.

APPLICATIONS

For the moment, until better text analytic tools allowing more accurate data cleaning and other tools that would help identify who is behind the opinions, are available, I think social media content should be considered as a tool in the qualitative research arsenal to explore and sometimes dig deeper in certain issues. Among its applications, social media content can be used to explore:

• Likes and dislikes about a product, brand or company
• Language used to talk about needs, expectations, barriers
• Problems with products, customer services that need a quick response
• Topics of interest within a product category
• Perceptions about a brand and its competitors

Social media content can be a source of rich insights, but we should be aware of its limitations and do not equal access to large amounts of text to quantitative research.

A client recently asked for advice about usability testing approaches. Her internal client wanted to do a traditional usability lab test, while she was wondering if a quantitative online usability approach was a better fit.

This question got me thinking about the advantages and disadvantages of each approach. As usual, no research method is perfect and each has his strengths and weaknesses. There are many tools available to do quantitative usability testing online faster and cheaper. Although these can also provide qualitative data, usability testing in the lab environment can give you a deeper layer of insights into user behavior on specific issues.

The table below shows what I have learned about usability testing, after managing two usability labs in my past life as a corporate market researcher.

Each usability testing approach has a purpose and complement each other. If timing and budget permit, you should use both. If you can only afford one or the other, think hard about the objectives and the type of decision you will make based on the results.

For detecting major problems or understanding at a deeper level why people find certain tasks difficult to do or what their expectations are, a qualitative approach is a more appropriate. On the other hand, if you need to make a decision about major changes or a redesign and need to know how big of a problem you have in your hands, or how you compare to your competitors, a quantitative approach would do the trick.

Not long ago I got a call from a potential client asking for research to determine why a recent marketing campaign failed to increase sales, despite a significant increase in awareness.

He had conducted an advertising awareness survey, and the results showed that many in the target audience had noted the advertising and gave it high ratings, but didn’t make a purchase. All possible explanations were merely speculations. He couldn’t pintpoint to any particular cause for this. The main problem was that he wanted to obtain evidence of a cause-effect relationship, but the research design was not appropriate for that.

The main method for causal research is experimentation. In experimental-based research, the causal or independent variables are manipulated in a relatively controlled environment. This means that other variables that may affect the dependent variable (e.g. sales) are controlled and monitored as much as possible.

In this case, the client had conducted the survey and analyzed the data without taking into account the effectiveness of the different marketing collaterals used, its market penetration, competitor activity, and some characteristics of the purchase decision makers. After doing some digging around, we uncovered that in some markets, competitors had launched high frequency advertising campaigns which helped the client indirectly by increasing category awareness, but not his sales. Also, the program targeted customers who were recent buyers and probably didn’t have a need for his products at that particular moment.

Surveys that are not designed as part of an experimental approach may show correlations, but not causality. To really connect the dots between cause and effect, we needed to create an experiment including different renditions of the marketing collaterals, different markets, customers at different stages in the purchase cycle, and different actions taken by competitors.

Experimentation in marketing has traditionally taken the form of standard test markets, in which test market are selected, controlled advertising is put in place, and the product is sold through regular distribution channels. The drawbacks of this type of test are that they can be time consuming, are often expensive and may be difficult to administer.

A more palatable solution is what is called simulated test markets in which individual are selected, exposed to the product or concept (e.g. via actual marketing collaterals), given the opportunity to buy the product in real life and if they buy it, they are asked to evaluate the product and state their repeat purchase intent. The trial and repeat estimates are combined with data about promotions, distribution levels, competitor activity and other relevant pieces of information.

Another possibility can be found in the popular freemium model, adopted by many businesses both in B2B and B2C, which mimics this process to some extent. This model can be used for experimentation and as a rich mine for insights at a low cost. The basic principle is to let people try it and observe what decision they make, after which research can follow to understand what drove their decision, controlling for other variables that may be affecting the outcome.

In short, if you want to understand cause and effect, you need to conduct experiments, which may include surveys as data collection method, but surveys in themselves can’t provide the answer. It is the  experimental design what will lead you to it.

Once you get over the AHA moment of the new idea for a product you just got, the question you should ask yourself is how you are going to position it. This is valid also for old products that need a makeover and boost in sales.

New products usually come in:

• A new form but with similar function as other alternatives (e.g. new bottle shape for a hair product)
• A known form with new functions (e.g. car with GPS)
• New form and functions combined (e.g. iPhone)

Product positioning often takes two forms: functional or experiential. Although, you can find a little bit of both in many ads, there is usually a focus on one of them.

In functional positioning, utilitarian benefits take the front seat by focusing on product attributes, how it works, how helpful it is and what needs it meets. An example is the 2011 ad for Boyardee Ravioli below. It uses its long tradition and beginnings to emphasize quality ingredients and non-preservatives as its main product attributes.

Experiential positioning, on the other hand, is about hedonic benefits, how they product makes you feel. A stellar example is the recent years’ Old Spice commercials telling women how they would feel if their men were to use Old Spice body wash: “We’re not saying this body wash will make your man smell into a romantic millionaire jet fighter pilot, but we are insinuating it.”

Recent research (Noseworthy & Trudell, Journal of Marketing Research, Dec. 2011, Vol 48, p. 1008), shows that new products that are moderately incongruent with what consumer expect from it, but positioned with a utilitarian angle like in the Boyardee’s ads, tend to receive more favorable evaluations than typical, congruent products or highly incongruent products. However, in experiential positioning, this seems to work in the other direction. In this study, congruent products were more likely to receive favorable evaluations than products that came in an atypical form.

This research, validated across 5 different product categories, found that “when a product is positioned on functional dimensions, moderately incongruent form causes consumers to perceive more hedonic benefits, whereas when a product is positioned on experiential dimensions, moderately incongruent form causes consumers to perceive less utilitarian benefits.”

These results suggests that consumers put more value on hedonic benefits once they understand what the product does, which seems logic, but not always considered. Have you ever seen a commercial and wonder what is it for? Check the G commercial created when Gatorade did a brand makeover a couple of years ago and left consumers scratching their heads.

Before you decide on how to position your product, I suggest doing research to understand:

• Prior knowledge about the brand, product or product category
• User behavior
• Awareness and usage of competing alternatives
• Perceived risks
• Price perceptions and willingness to pay

With this knowledge, you can create both functional and experiential positioning versions and test them before you decide which version to go with. One test may not be enough. Test a version, refine it based on the research results and test again until you feel confident that the positioning chosen is going to advance your product.

Although brand awareness can boost purchase consideration, the actual buying decision is likely to be done well before a customer may be aware of a brand, recent research indicates.

New findings show that buying decisions are triggered rather by a need which sets the buying process in motion.

After making the decision to buy, potential customers often start researching what options are available, even if they are already aware of certain brands.

Being part of the initial consideration set increases a brand’s likelihood of being purchased, but there is still a chance for low-awareness or even unknown brands to be considered if they are discovered while the customer evaluates different options and the offer is compelling, affordable and inspires trust, among other factors.

Thanks to widespread Internet access, the growth of social media and the explosion of information sources on the Web, the evaluation of different options is easier than ever. This is why search engine optimization (SEO) is so important for new and small businesses.

If your brand is unknown or suffers from low awareness, first invest in researching your target markets before you start spending money in launching an awareness campaign or doing SEO, including:

• Needs and purchase occasions
• Category- and product-specific behavior
• Perceived differences between you and your competitors
• Appeal of your value proposition
• Demographic and psychographic characteristics
• Media consumption behavior, including the Internet and social media

Learning how your potential customers make buying decisions will allow you to invest your marketing dollars more effectively.

For entrepreneurs, it is never too early to start thinking about which marketing metrics to track when running a business.

Some of the most basic but important marketing metrics can be gathered through survey-based Awareness, Attitudes and Usage research.

This type of research provides quantitative marketing metrics about product and service knowledge, perceptions and customer behavior. The research is often set up as a “tracker” used to monitor brand health and trends affecting business growth, but it can be implemented as an ad-hoc research project to gather feedback from the market at any time. Typical question areas include:

• Awareness: Both top-of-mind (“Which brand comes to mind when you think of …?”) and recognition (“Have you heard of Brand X?”). Awareness metrics are important to decide where and how to invest advertising dollars.

• Attitudes: Beliefs and feelings about the brand, company or products/services. We need to understand how our brand is perceived to direct our marketing communication to either strengthen or change those perceptions.

• Usage: What, when, where, how much, how often do you buy/use Brand X and Y? Knowledge about the purchase behavior of customers and noncustomers allows us to quickly become aware of and react to any changes in the market as new marketing strategies, products, and prices are put in place not only by us, but also by old and new competitors entering the category.

As these marketing metrics are gathered, entrepreneurs should also put systems in place to capture transactional data linked to customer profiles (such as demographics) so results can be validated through triangulation with actual sales data. While sales data can tell us what customers do, it doesn’t tell us why they do it. The metrics can fill some of the gaps inherent to transactional data and support better business decisions.

Whether you are starting a business that needs a website to represent it or already have a website that needs a makeover, you should consider testing different design alternatives before launching into development. Don’t assume you know your audience if you have never tested how it would interact with your website. Testing website designs doesn’t have to be complicated or very expensive.

In the concept stage you can do it as simply as using still pictures of different designs showing how it may look like. Do it with real content and as close as possible to a finished look. Even if the interactive piece is missing in this approach, you will learn tons about the first impression a design gives and its capacity to grab attention and invite visitors to come in and explore your website site.

By comparing website designs in a simple concept testing you will learn how different elements speak to your audience, including:

• Color scheme
• Layout
• Content organization
• Navigation expectations
• Potential sources of abandonment
• Brand image elicited by the design
• User identification as a member of your target audience (Is this website for me?)
• User concerns and anxieties about using your website
• User reaction to different calls to actions and labels

Testing these elements with your target audience (there is no such thing as ‘my product/service is for everybody’) early in the design phase will save you time and money in the long run as you will be able to launch a website that will work for your business from the get go. Of course, testing shouldn’t stop there as website interaction is a big factor in the user experience, and for that you may need qualitative and qualitative usability testing. However, if your budget is tight, the more reason to start with a simple design concept test. It would be better than just relying on personal taste or misleading assumptions about your target market.

To learn more how we can help you with this type of research check our Website Design Optimization Service. You can also check the case study about what we did we our website.

I hear questions related to statistical significance on a daily basis. It is usually some variation of “How much sample do we need to be significant?” which often reflects some confusion about the term.

Statistical significance is a concern when we are interested in detecting differences not due to chance between two or more groups (people, objects, ads etc.) being compared.

As sample size increases, the margin of error around a percent or a mean get smaller and we get, not only more precise estimates, but also more sensitivity to detect differences that are not due to chance. In a large sample, a difference of 1 or 2 percentage points may be significant, while in a smaller sample, where there is more variation, we may need to see more than 10 percentage points to detect significant differences.

In survey research, we often talk as if the results are finite point estimates when in fact we should be talking in ranges since there is always a margin of error around any estimate. So if the margin of error is +/-3% and we get a value of 50% for a variable, it means that the true value of the variable should be between 47% and 53%.

Now, if we measure the same variable in another group with a sample size where the margin of error is +/-5% and we get a value of 57% for the same variable, it means that the true value is expected to be between 52% and 62%. Despite the 7 percentage point differences, which seems large, we can’t say that it is statistically significant because there is some overlap between the margin of error range of each group (47% – 53% and 52%- 62%) and the true value of the variable in the second group could be 52% or 53% which are values included in the first group’s margin of error range.

How confident are we about this? We often say 95% confident, which means that if we repeat the study 100 times, we can expect similar results 95 times and be wrong 5 times. This is called Confidence level and the margin of error range is called Confidence Interval. In short, we want to make sure the true value falls within the same range every time we repeat the study. Unfortunately, statistical confidence has an inverse relation with estimate precision. If you want to be 99% certain then you have to allow for a larger confidence interval that will include the true value.

If there is no comparative analysis involved, it doesn’t make any sense to talk in terms of statistical significance. However, we are still concerned about estimate precision of results in total. We want our margin of error to be as small as our budget and tolerance for risk allow. To get greater precision, we need larger sample, which in turn costs more money. To be more certain, you sacrifice some precision. There is always a trade-off to make.

Next time when you considering sample size for a survey get ready to answer these questions:

• What is the desired precision (margin of error)?
• How confident do you want to be?
• Can your budget accommodate the required sample size for the desired precision? If not, what are you willing to settle for?
• Are doing any comparisons between groups? If so, how many?
• Can your budget accommodate the required sample size by group to make meaningful comparisons?

Unfortunately, the difference between the sample you want and the one you can afford is oftent significant (pun intended), so budget questions are always in the mix. For more help on calculating sample size and margin of error, use our Sample Size and Margin of Error Calculators.