Implicit reaction time tests (IRTs) are one of the fastest growing approaches in market research. Online, objective and cost-effective, they capture consumers’ immediate, gut instinct or subconscious responses to brands, campaigns, new product concepts, packaging designs and a vast array of other marketing related outputs. Free from the biases of conscious rationalization and distracting ploys inherent in quantitative and qualitative research, IRTs offer marketers a chance to study consumers at a deep, emotional level and predict their behaviour more accurately than has previously been possible.
But how do we know that IRT’s work? And what should clients look out for when commissioning implicit studies?
Having spent nearly 20 years in the intersection between academe and industry, I have conducted a vast number of IRTs to address a very large number of both academic and commercial marketing related questions. In this article, I will share some of that expertise to help those wanting to adopt these approaches gain a basic understanding of the fundamental principles.
From SMEs to global multinationals, IRTs are set to become a mainstream part of the market research mix, providing the missing implicit piece of the jigsaw, particularly when combined with established explicit approaches. IRTs now permit marketers to understand consumers from a 360-degree perspective and to integrate their explicit verbal feedback with the subconscious thoughts, feelings and influences that are inaccessible to conscious introspection.
Why do we need to measure subconscious responses?
Neuroscience has shown that a vast amount of our behaviour is driven by brain processes that operate below our conscious awareness. While many of these processes are related to survival mechanisms (such as breathing, motor functions and homeostatic regulation), and not of primary interest to marketers, a significant number are nevertheless associated with emotions, storage and retrieval of memories, involuntary attention and perception, semantic processing and decision-making.
Importantly, because all these subconscious influences shape our behaviour and often impact on the choices we make, it seems obvious that marketers need to understand these in order to predict consumer behaviour more accurately. Relying solely on explicit research approaches (e.g. focus groups, surveys etc.) equates to the tip of the iceberg in terms of the entire multitude of emotional and motivational processes (most of which are subconscious) that ultimately determine our behaviour.
Anyone who has worked in qualitative and quantitative market research will be familiar with the potential confounds associated with explicit (conscious) consumer responses:
- people respond even when they don’t know what they think
- people may not tell the truth (e.g. for fear of what others might think)
- people are bad at predicting their own behaviour
IRTs thus provide a way of minimising these confounds.
What are some of the advantages of IRTs?
IRTs belong to a family of neuromarketing tools currently available to marketers today. These include functional MRI, EEG, biometrics, eyetracking, facial decoding, as well as IRTs. Each method has pros and cons. However, IRTs are one of the most (if not the most) rapidly expanding approaches for several reasons:
- they capture subconscious responses online (or on a mobile device)
- they do not require equipment (e.g. electrode caps, MRI scanners)
- they can be turned around quickly
- they are highly scalable
- they are cost effective
Added to which, IRTs provide a level of flexibility and granularity that far outstrips other neuro-methods, making it the neuromarketing method of choice across a very wide range of marketing questions.
What is the science behind IRTs?
The history of reaction time testing can be traced back to the 1860s, when the Dutch physiologist, F C Donders, applied an electric shock to the left or right foot of volunteers and found that the time they took to respond by pressing a button using the corresponding left or right hand was measurably faster when they were told in advance which foot would be shocked than when they didn’t know. This showed for the first time that human mental processes (in this case, the decision about which button to press and the will to make that response) could be measured using reaction times.
Donders also devised the subtraction method to calculate the processing speeds of different mental operations. He observed that participants’ reaction were very fast when they only had to detect a stimulus compared to when they had to recognise that stimulus. Their reaction times were slower still when they had to make a choice about that same stimulus.
By subtracting the reaction times obtained during these 3 different operations, Donders was able to infer the processing speeds of the different mental operations (react, discriminate, choose).
But it wasn’t until the 1970’s when the academic discipline of cognitive psychology really started to take off that psychologists realised the importance of reaction time testing and it soon became one of the mainstream methods for studying human cognition and emotion.
Importantly, reaction time testing provided the means by which psychologists could study subconscious brain processes and discriminate them from conscious thoughts or decisions.
This is because conscious and subconscious mental processes occur within different timeframes, allowing for two distinct routes to decision making:
- The fast route (implicit responses, occur before 500-600ms)
- The slow route (explicit responses, occur after 500/600 ms)
These two routes to decision making have different characteristics and are often referred to as System 1 (fast, subconscious) and System 2 (slow, conscious).
System 1 is very fast to respond (in milliseconds after an event), reacts automatically and stores information associatively (i.e. concepts that are like each other are shared closely together)
System 2 refers to conscious mental operations (e.g. rational evaluations, long-term goal setting etc) and is much slower, controlled and rule following.
An analogy can be made with learning to ride bike (System 2) and becoming a proficient cyclist. Once you have learned to cycle, System 1 takes over and the skill is automatised.
Importantly, once a task or preference becomes automatic (and controlled by System 1) it is often difficult to recover the rules or reasons by which we learned how to perform or prefer that task (or choice). This is why it is often difficult for proficient drivers to teach others how to drive, or for consumers to recall why they became Coke rather than Pepsi drinkers (or vice versa) in the first place, particularly if the habitual behaviour or preference was learned a long time ago.
For more details about System 1 (implicit/subconscious) and System 2 (explicit/conscious) you may like to read Daniel Kahneman’s book entitled Thinking Fast and Slow.
How fast is an implicit response?
So how fast is fast? Let’s take a quick look at a simple cognitive timeline.
The moment that a stimulus occurs (e.g. time = 0ms) the brain automatically starts to process it. It takes somewhere between 100 – 200ms for humans to detect a stimulus and respond to it by, for example, pressing a button. At this point, the brain has not had sufficient time to necessarily recognise what the stimulus is, simply that there is one.
By 400ms the brain is able to discriminate a word (e.g. “stripe”) from a non-word (e.g. “prtise”), again without requiring conscious processing, and 150-300ms later, we can effect a response demonstrating that the stimulus has been correctly identified. Of course, we have to allow for some margin of human error (pressing the wrong button by mistake), which is why it is important to collect multiple trials of the same type and obtain an average.
Importantly, all of this occurs BEFORE the stimulus has been processed to any great extent by conscious brain mechanisms and therefore any response obtained after 200ms (see below) but before ~600-700ms is considered uncontaminated by conscious thought processes.
On the other hand, responses made before 200ms after the onset of the stimulus are too fast and should be removed.
How do you clean and trim the data?
There are several factors that should be considered when designing or analyzing implicit data. Different vendors or consultants may have different methods for controlling for handedness, individual differences in response times and inaccurate or repetitive responding. Before the data are analysed, these and other confounds need to be taken into consideration and the data adjusted, trimmed or otherwise cleaned.
As a general rule, all vendors will remove outliers from the data – responses which are too fast or too slow to be a bona fide implicit response.
A basic guideline on removing misleading trials (data trimming)
Responses made before 200ms are usually considered too fast
Responses made after 650ms-1s are usually considered too slow **
** But note, there is no exact time window that differentiates implicit from explicit responses (because people respond faster on some trials than others and some words take longer for the brain to read subconsciously than others) but those shown above provide a pretty good guide.
The best advice is to look closely at the data. If you look at the averaged reaction times for each trial type, obtained across a group of individuals, you should observe what’s referred to as a normal distribution (most responses are huddled around the same time window).
Most responses fall under the grey area and therefore represent the average reaction time at the top of the curve. Responses at either end of the curve are too fast (green) or too slow (red) and only very few people respond in these timeframes – so these are what we call “outliers” and you may need to remove them before analyzing the data.
How do IRTs reveal subconscious thoughts or feelings?
We have now established that implicit responses have to be captured very quickly, in considerably less than a second. But what does that response tell you? It is not sufficient for respondents to simply detect that they have seen a brand logo or packaging design “flashed up” for a few milliseconds on the computer screen.
Marketers need to understand what each particular image/logo/pack design/campaign automatically triggers in consumers’ brains. For example, the memories that are linked to that brand concept, the implicit biases or preferences that are stored about that concept and even the type of emotions it involuntarily evokes. Psychologists refer to this as “priming”.
So how is it that the perception of a concept can trigger or “prime” related concepts and how can IRTs reveal what these are?
Our brains contain a vast semantic network – an interconnected web of knowledge. Each concept stored in this network is represented as a “node” (a collection of brain cells that code and store that concept).
Importantly, each concept shares connections with other concepts that are related or similar in some way. In the image below, networks of related concepts are coded in different colours.
So your concept of the brand Apple, for example, is likely to be connected to concepts of computers, Steve Jobs, fruit and emotional attributes such as how the brand makes you feel or what it’s benefits are –in the case of the illustration below, perhaps “pretty cool”.
In a typical IRT test to identify which concepts are most closely associated with either of two competing brands (see the example experiment below), when the respondent is exposed to a stimulus such as the Apple logo, that exposure activates that concept in their semantic network and triggers what is known as spreading activation – the automatic or involuntary activation of other related concepts. In other words, the Apple logo has “primed” the activation of the related concepts stored in your brain. The result of this is that we are faster to react to the appearance of a stimulus that is related to Apple than one that is unrelated.
IMPORTANT:
The speed with which other related concepts are activated by the primed concept (in this case, Apple) is directly related to the strength of their association with that brand. And it works the other way around too. In the image below, if the word “style” is the prime, it too will activate other related concepts, one of which should be Apple.
Therefore it is the strength of association between brands/pack designs/adverts/products and a range of related concepts (e.g. brand benefits, preferences, emotional associations, intent to purchase concepts etc) that IRTs used in market research aim to measure.
“IRT’s are used to find out what people store in their heads about a brand and which concepts are influencing their behaviour to a greater or lesser extent.”
What does an implicit test look like?
So what does an implicit test actually look like? What does a respondent see or do when they are asked to take part in an implicit survey?
The first point to make is that there is no one implicit task. Rather, there is a family of IRTs. But most vendors tend to converge on two or three key ones in order to measure the strength of association between marketing stimuli (e.g. brand, logo, pack design, ad) and a range of brand related concepts; whereas there are other tests which simply measure which stimuli are most attention grabbing.
You may have heard of the Implicit Association Test or “IAT”. This is test used predominantly by academics to uncover individual biases or hidden attitudes (e.g. racial or gender prejudices). The test is well-suited for this purpose but requires multiple trials of each condition (e.g. pairing of positive or negative words with black/white faces) and therefore the number of words or attributes that can tested in any one IAT test is rather limited.
A more commonly used test is the semantic priming paradigm – a test that has been used by psychologists for considerably longer than the IAT and which has undergone considerably more validation and reliability testing. Semantic priming can be used, for example, to measure the strength of association between a brand (or brands) and a range of brand attributes (brand positioning) or to uncover which brand assets are most closely associated with a brand.
The figures below demonstrate a simple semantic priming task designed to measure the strength of association between a range of brand attributes (or “primes”) and 2 competing brands (Apple and Samsung).
After respondents have logged in online to the research study, and completed some standard demographic questions, they are informed that they will be asked to do a task which requires them to respond on the computer keyboard as quickly and as accurately as possible.
Step 1:
Respondents are first trained to respond as quickly and as accurately as possible to the two targets (Apple logo or Samsung logo). Only one logo appears at a time and for a very brief period. The correct response involves pressing the “E” button on the keyboard if Apple appears, and the “I” button on the keyboard if the Samsung logo appears. Most respondents can learn to do this within 30s – 1 minute.
During this training phase, respondents are receive an automatic feedback alert (e.g. a tone or visual cue) if they are too slow to respond or if they press an incorrect button.
After the respondent is able to respond quickly and accurately enough (i.e. within the implicit time window), they are informed that the test proper will start.
Step 2:
Respondents are reminded that the task will be the same but that before each discrimination trial, a word or image may be briefly flashed on the screen. In some cases they are told to ignore the words and to continue to concentrate on correctly discriminating between the two logos.
* Note that the precise instructions will vary from vendor from vendor.
Below you will see that in one trial, that a brand attribute (in this case “stylish”) will be flashed up for ~ 250ms just before either the Apple or Samsung logo. The respondent is still required to press the “E” key corresponding to the Samsung logo if it appears.
The number of different brand attributes (or visual brand assets) that are flashed up can vary. But in almost all implicit tests, each prime word/attribute will be presented before each target (Apple or Samsung) in randomized order and multiple times.
Let’s look at this from another perspective. In Figure 1 below, the word “progressive” is flashed up before Apple (top) and flashed up before Samsung (below). If “progressive” is a concept implicit associated in your mind with Apple, then you will be faster to accurately detect and respond to it, than if Samsung appears (Figure 2).
Figure 1
Figure 2
After multiple trials in which each prime is presented multiple times before each logo and the results averaged across several hundred respondents, a computer algorithm tests whether the reaction times for each brand attribute are statistically different when they are presented before each brand.
What do the results look like?
This chart shows how the attributes can then be rank ordered, statistically. Bars above the 0 line correspond to attributes more closely associated with Brand A, bars below the line show those that are more closely associated with Brand B.
Attributes that are either not associated with either brand and/or are associated with both equally, are shown in grey (ns = not statistically significantly different) and will fall close to 0 along the x axis. There is a way of resolving which of these two explanations is correct (both not associated or both strongly associated) but it is beyond the scope of this article.
This is just one way to represent the data. You could similarly use spider plots or any graphical method to illustrate which attributes belong to which brand and to what extent.
Whichever method you choose, providing the data are analysed using appropriate statistical tests, interpretation of the results should be reasonably straightforward. Be warned, however, when cutting the results by multiple datacuts (e.g. age, gender, brand usage etc), this approach will generate a large number of output charts that often require a skilled implicit researcher to pull together into a coherent report.
A family of IRTs
I have just shown you one implicit reaction time task, but as I mentioned above, there is in fact, a whole family of them. Each test is designed to address different types of questions. Semantic or affective priming (above) is widely used in the context of brand and pack evaluation and for the pre and post assessment of creative executions.
Neither is it necessary to employ an A vs B design in which only 2 brands are tested. Monadic approaches allow clients to test a single brand against a neutral baseline and repeat this for multiple brands, whereby all brands involved in a study can be compared against a neutral baseline.
Below are a few other implicit reaction time paradigms that you may encounter:
Word identification (e.g. target/non-target)
Go/No Go Association Task
Affective or semantic priming
Implicit Association Test (IAT)
Visual search
You can find demos of some of these tests at Project Implicit which was launched to allow academics to study biases and attitudes using the Inquisit platform.
Reliability and validity
An important point to remember is that psychologists have been using these tests for nearly 40 years, and the published academic literature has shown that across many different sectors or fields, when implicit tests are pitted against the explicit responses subjects provided at the time of testing, time and time again, implicit tests have proved themselves to much more accurate predictors of subsequent behaviour. This has been shown across many contexts including:
- Consumer choice (Carney & Banaji, 2012)
- Change of job (Hippel et al 2009)
- Personal skills (Filipkowski, 2011)
- Pilot’s risk-taking behaviour (Molesworth & Chang, 2009)
- Aggressive sporting behaviour (Teubel et al, 2011)
- Stress response (Quirin et al, 2009)
- Future depressive episodes (Franck et al 2007)
Commercial applications of implicit tests
IRTs are now being used by companies worldwide to find out what their customers really think and feel about their brands, products and services. Here are just a few of the very wide range of questions that can be tackled using IRTs:
- Which packaging design stands out most on the shelf?
- Which packaging design implicitly communicates my brand benefits?
- How does the packaging influence perception of the product?
- Which brand attributes are most strongly associated with my brand?
- How do these associations change over time (brand tracking)
- What is the likely acceptance of the new brand extension?
- Which new product prototype will consumers accept?
- Which multisensory features of my product drive preference?
- What needstates are evoked by different flavours or fragrances?
- Which media platform is most effective for my creative execution?
- Which facets of my website are most engaging?
This list is almost endless.
Because IRTs allow you to test either a fixed set of attributes (e.g. many vendors may use a psychological model of rewards) or a bespoke set selected to address a very specific marketing question (e.g. category-specific words, words unique to a particular brand or product, sensory words, words that define a particular target audience etc.) the approach lends itself to almost any marketing (or indeed, other sector) question.
TAKE HOMES
- IRTs are fast becoming part of mainstream market research and a highly valuable tool for capturing consumers’ subconscious responses.
- Because they are predominantly web-based, they offer fast turnaround times using online panels, are highly scalable and cost-effective.
- IRTs can be seamlessly combined with qualitative data – the latter for example can be used to derive the attributes that are subsequently fed into an IRT, or used to aid interpretation of the implicit output.
- IRTs are a vital new tool that any company can, and should, leverage for competitive advantage through deeper consumer insight.
If you would like to know about IRTs and how you can include them in your market research mix, please do not hesitate to get in touch with me via gemma@gemmacalvert.com.