Assessing Performance Outcomes in Marketing
Katsikeas et al (2016)
The measures of performance used across (and sometimes within) studies are often incommensurate, making
synthesis across studies and cumulative knowledge development difficult, if not impossible. This article
provides a critical assessment of performance outcomes used in marketing literature, by providing an
evaluative framework which can be used to analyze the measures of marketing performance outcomes in
empirical studies.
Purpose article: provide a critical assessment of performance outcomes used in marketing literature to
improve marketing performance studies.
Operational performance refers to the fulfillment of goals within different value-chain activity areas of the firm
that may subsequently lead to organizational performance, that is 'the economic outcomes resulting from the
interplay among an organization's attributes, actions and environment.
The chain of marketing performance outcomes: Katsikeas et al build a conceptual model to identify the
fundamental steps in the creation of operational and organizational performance outcomes. The six elements
in the pinkish blocks are related to marketing performance:
Drivers of performance:
1. Firm's marketing resources, strategy and actions = e.g. a realized product. Provides base for realized
marketing programs.
2. Realized marketing programs = resources, strategy and actions that have become manifest in the
market, that people can actually see. Actions that people have been noticing. 1 and 2 are very close.
6 categories of performance measures:
1. Operational performance = degree to which you achieve your goals
, a. Customer mindset = soft measures, what is going on in the head of the customer?
b. Customer behavior = post-purchase behaviors > results. Not subscribing for a newsletter or
something (that is realized marketing programs), these are customer behavior related
performance measures, not all customer behavior.
c. Product-market performance = measurable on product-level. Focus on firm performance
d. Customer level performance = measurable on customer-level > with which customers are we
doing well / not so well? Increasingly being done. ‘Share of wallet’ is a method to calculate how
much a customer buys from your company in the same product range; e.g. you could be buying
90% of all your groceries at Albert Heijn, but sometimes you might go to Jumbo for groceries too.
2. Organizational performance = more economic measures:
a. Accounting performance = financial measures, sales revenue, profit, cash flow, cost etc.
b. Financial market performance = market we get our financing from > stock market, banks. They are
interested in investor returns, equity risk, credit rating, cost of capital.
This figure is dynamic in two main respects:
• Firms reinvest their financial resources to build and maintain their marketing related resources and
capabilities (red-dotted line 'reinvestment)
• Firms learn by going through the stages of marketing performance outcome chain in ways that lead to
adjustments to their selection and management of future marketing resources and marketing program
actions (red-dotted line 'learning')
A framework for evaluating marketing performance outcomes should consider five critical issues (evaluative
framework):
1. Theoretical rationale
2. Conceptual approach
3. Aspects of performance
4. Referents
5. Time horizon
,Additional notes:
• Theoretic rationale: do firms use a theoretic rationale why they choose a particular performance
measure?
• Latent construct are indicators that together represent a latent construct
• Aggregate constructs is rarely done
• Accounting performance is measured a lot (easily measured, databases available)
• Customer level performance, not used a lot in academic literature
• Inputs: sales relative to how much money we had to spent
• Temporal: sales relative to time
Methodology
Literature review > inclusion of 998 articles out of the 15 most influential journals.
Results
, Theoretical rationale
Less than 10% (9.7) explicitly provide a clear definition and theoretical justification for the adopted
conceptualization of marketing performance
Conceptual approach
The majority of studies employ latent construct conceptualisation (55.4%), 44,5% adopt a separate constructs
approach.
Frequency of use of marketing performance measures in published studies (table 3)
The accounting and product market aspects of performance have dominated research on performance
outcomes of marketing. Financial-market measures are growing most rapidly > rapid rise in interest in the
financial performance outcomes of marketing, with particular emphasis on shareholders. Also noteworthy: yet
growth is a key goal in most firms, very small number of growth measures are used > only sales.
• Product market measures: the most widely used performance indicator at the product-market level is
market share.
• Accounting measures: measures related to profit and sales revenue are the two most widely used
accounting measures. Only a small number of studies employs cash-flow measures > surprising given the
advantages of measuring cash-flow
• Financial-market measures: rapid recent rise in the use of stock-market related measures of
performance > renewed emphasis on shareholder value and the emergence of the marketing finance
interface research stream. Researches also begun examining risk-related performance outcomes.
Referents of assessment (table 2)
Measures that are relative to inputs and absolute measures dominate in published studies of marketing
performance outcomes. Only a small number of studies adopted goal-related referents, which might be due to
the difficulty in operationalizing such referents using secondary data.
Time horizon of assessment (table 2)
Current-period performance is the dominant time horizon > may be due to difficulty in capturing future time
frames for researchers using primary data.
Additional data collection and design characteristics
• Both B2C and B2B
• Majority of studies had one-off perspective (assessed performance at one point in time)
• More than 1/4 adopted a long term focus
• Greater use of subjective than objective performance measures
Discussion
• Our results show a rapid rise in interest in assessing marketing's performance outcomes.
• 38% of the studies we examined rely on a single measure of performance, and an additional 40% treat
performance as an latent variable but do not report correlations between indicators. Yet: the limited
literature on performance assessment in marketing clearly indicate that performance is a complex,
multidimensional construct.
• Relatively few of the studies that use more than one performance measure report the correlations, but
those that do suggests that different performance indicators are generally only relatively weak
correlated
• With no common conceptualization and operationalization of the different aspects of marketing
performance we have no way of knowing whether the differences in results across empirical studies are
due to the effect of differences in:
o The study context
o The independent variables examined
o The selection and operationalization of dependent variables
Given the current state of performance conceptualisation and operationalisation, and the problems it poses for
knowledge development, Katsikeas et al. (2016) state that immediate action is required. As a discipline,
marketing needs a core set of agreed-on, common priority conceptualisations and operationalisations for
Katsikeas et al (2016)
The measures of performance used across (and sometimes within) studies are often incommensurate, making
synthesis across studies and cumulative knowledge development difficult, if not impossible. This article
provides a critical assessment of performance outcomes used in marketing literature, by providing an
evaluative framework which can be used to analyze the measures of marketing performance outcomes in
empirical studies.
Purpose article: provide a critical assessment of performance outcomes used in marketing literature to
improve marketing performance studies.
Operational performance refers to the fulfillment of goals within different value-chain activity areas of the firm
that may subsequently lead to organizational performance, that is 'the economic outcomes resulting from the
interplay among an organization's attributes, actions and environment.
The chain of marketing performance outcomes: Katsikeas et al build a conceptual model to identify the
fundamental steps in the creation of operational and organizational performance outcomes. The six elements
in the pinkish blocks are related to marketing performance:
Drivers of performance:
1. Firm's marketing resources, strategy and actions = e.g. a realized product. Provides base for realized
marketing programs.
2. Realized marketing programs = resources, strategy and actions that have become manifest in the
market, that people can actually see. Actions that people have been noticing. 1 and 2 are very close.
6 categories of performance measures:
1. Operational performance = degree to which you achieve your goals
, a. Customer mindset = soft measures, what is going on in the head of the customer?
b. Customer behavior = post-purchase behaviors > results. Not subscribing for a newsletter or
something (that is realized marketing programs), these are customer behavior related
performance measures, not all customer behavior.
c. Product-market performance = measurable on product-level. Focus on firm performance
d. Customer level performance = measurable on customer-level > with which customers are we
doing well / not so well? Increasingly being done. ‘Share of wallet’ is a method to calculate how
much a customer buys from your company in the same product range; e.g. you could be buying
90% of all your groceries at Albert Heijn, but sometimes you might go to Jumbo for groceries too.
2. Organizational performance = more economic measures:
a. Accounting performance = financial measures, sales revenue, profit, cash flow, cost etc.
b. Financial market performance = market we get our financing from > stock market, banks. They are
interested in investor returns, equity risk, credit rating, cost of capital.
This figure is dynamic in two main respects:
• Firms reinvest their financial resources to build and maintain their marketing related resources and
capabilities (red-dotted line 'reinvestment)
• Firms learn by going through the stages of marketing performance outcome chain in ways that lead to
adjustments to their selection and management of future marketing resources and marketing program
actions (red-dotted line 'learning')
A framework for evaluating marketing performance outcomes should consider five critical issues (evaluative
framework):
1. Theoretical rationale
2. Conceptual approach
3. Aspects of performance
4. Referents
5. Time horizon
,Additional notes:
• Theoretic rationale: do firms use a theoretic rationale why they choose a particular performance
measure?
• Latent construct are indicators that together represent a latent construct
• Aggregate constructs is rarely done
• Accounting performance is measured a lot (easily measured, databases available)
• Customer level performance, not used a lot in academic literature
• Inputs: sales relative to how much money we had to spent
• Temporal: sales relative to time
Methodology
Literature review > inclusion of 998 articles out of the 15 most influential journals.
Results
, Theoretical rationale
Less than 10% (9.7) explicitly provide a clear definition and theoretical justification for the adopted
conceptualization of marketing performance
Conceptual approach
The majority of studies employ latent construct conceptualisation (55.4%), 44,5% adopt a separate constructs
approach.
Frequency of use of marketing performance measures in published studies (table 3)
The accounting and product market aspects of performance have dominated research on performance
outcomes of marketing. Financial-market measures are growing most rapidly > rapid rise in interest in the
financial performance outcomes of marketing, with particular emphasis on shareholders. Also noteworthy: yet
growth is a key goal in most firms, very small number of growth measures are used > only sales.
• Product market measures: the most widely used performance indicator at the product-market level is
market share.
• Accounting measures: measures related to profit and sales revenue are the two most widely used
accounting measures. Only a small number of studies employs cash-flow measures > surprising given the
advantages of measuring cash-flow
• Financial-market measures: rapid recent rise in the use of stock-market related measures of
performance > renewed emphasis on shareholder value and the emergence of the marketing finance
interface research stream. Researches also begun examining risk-related performance outcomes.
Referents of assessment (table 2)
Measures that are relative to inputs and absolute measures dominate in published studies of marketing
performance outcomes. Only a small number of studies adopted goal-related referents, which might be due to
the difficulty in operationalizing such referents using secondary data.
Time horizon of assessment (table 2)
Current-period performance is the dominant time horizon > may be due to difficulty in capturing future time
frames for researchers using primary data.
Additional data collection and design characteristics
• Both B2C and B2B
• Majority of studies had one-off perspective (assessed performance at one point in time)
• More than 1/4 adopted a long term focus
• Greater use of subjective than objective performance measures
Discussion
• Our results show a rapid rise in interest in assessing marketing's performance outcomes.
• 38% of the studies we examined rely on a single measure of performance, and an additional 40% treat
performance as an latent variable but do not report correlations between indicators. Yet: the limited
literature on performance assessment in marketing clearly indicate that performance is a complex,
multidimensional construct.
• Relatively few of the studies that use more than one performance measure report the correlations, but
those that do suggests that different performance indicators are generally only relatively weak
correlated
• With no common conceptualization and operationalization of the different aspects of marketing
performance we have no way of knowing whether the differences in results across empirical studies are
due to the effect of differences in:
o The study context
o The independent variables examined
o The selection and operationalization of dependent variables
Given the current state of performance conceptualisation and operationalisation, and the problems it poses for
knowledge development, Katsikeas et al. (2016) state that immediate action is required. As a discipline,
marketing needs a core set of agreed-on, common priority conceptualisations and operationalisations for
