Morphing Theory and Applications

Abstract

As electronic commerce often matches or exceeds traditional bricks-and-mortar commerce, firms seek to optimize their online marketing efforts. When feasible, these firms customize marketing efforts to the needs and desires of individual consumers, thereby increasing click-through-rates (CTR) and conversion (sales). When done well, such customization enhances consumer relationships and builds trust. In this chapter we review almost 10 years of morphing experience, including various proofs-of-concept. We start with an overview of the morphing concept and an illustrative example. We then describe how morphing and multi-armed bandits can change the way firms design and run online experiments. We then discuss the analytics of morphing, based on three published papers. We conclude with pratical recommendations for morphing applications, including key decisions, priors and convergence, data, roadmap, do’s and don’ts, open questions and relevant challenges.

Appendix: The Equations of Morphing

Let $n$ index consumers, $r$ index segments, $m$ index morphs, and $t$ index clicks for each consumer. Capital letters, $R$, $M$, and $T_{n}$ denote totals. Let $c_{tn}$ denote the $t$th click by $n$th consumer and $\vec{c}_{tn} = \{ c_{1n} ,c_{2n} , \ldots , c_{tn} \}$ denote the vector of clicks up to an including the $t$th click. At each click choice, the consumer faces $J_{tn}$ click alternatives as denoted by $c_{tnj}$ where $j$ indexes click alternatives. Let $c_{tnj} = 1$ if consumer $n$ clicks the $j$th click alternative on the $t$th click; $c_{tnj} = 0$ otherwise. Let $\vec{x}_{tnj}$ denote the characteristics for click-alternative $j$ faced by consumer $n$ on the $t$th click. Let $\vec{X}_{tn}$ be the set of $\vec{x}_{tnj}$’s up to an including the $t$th click for all $j = 1 \,\,to\,\, J_{tn}$. Let $\tilde{u}_{tnj}$ be the utility that consumer $n$ obtains from clicking on the $j$th click alternative on the $t$th click. Let $\vec{\omega }_{r}$ be a vector of click-alternative-characteristic preferences for the $r$th consumer segment and $\tilde{\epsilon }_{tnj}$ be an extreme value error such that $\tilde{u}_{tnj} = \vec{x}_{tnh}^{\prime} \vec{\omega }_{r} + \tilde{\epsilon}_{tnj}$. Let $\Omega$ be the matrix of the $\vec{\omega }_{r}$’s. Let $\delta_{mn} = 1$ if the $n$th consumer makes a purchase after seeing morph $m$; $\delta_{mn} = 0$ otherwise.

The likelihood that the $n$th respondent chooses clicks $\vec{c}_{{T_{n} n}}$ given the consumer belongs to segment $r$ is given by:

$$\Pr \left( {\vec{c}_{{T_{n} n}} | r_{n} = r,\Omega , \vec{X}_{tn} } \right) = \Pr \left( {\vec{c}_{{T_{n} n}} |r_{n} = r} \right) = \mathop \prod \limits_{t = 1}^{{T_{n} }} \mathop \prod \limits_{j = 1}^{{J_{tn} }} \left( {\frac{{\exp \left[ {\vec{x}_{tnj}^{'} \vec{\omega }_{r} } \right]}}{{\mathop \sum \nolimits_{\ell = 1}^{{{\text{J}}_{\text{t}} }} \exp \left[ {\vec{x}_{tn\ell }^{'} \vec{\omega }_{r} } \right]}}} \right)^{{c_{ntj} }}$$

(18.1)

We estimate $\Omega$ from the calibration study by forming the likelihood over all respondents and by using standard maximum-likelihood methods or Bayesian methods. Denote these estimates by ${\widehat{\Omega }}$.

In Morphing 1.0, we observe the consumer’s clickstream up to the $\tau_{o}^{{}}$th click. The unconditional prior probabilities, $\Pr_{\text{o}} (r_{n} = r)$ are observed in the calibration study or from website experience. Bayes Theorem provides:

$$q_{{rn\tau_{o} }} (\vec{c}_{{\tau_{o} n}} , {\widehat{\Omega }}, \vec{X}_{{\tau_{o} n}} ) \equiv { \Pr }\left( {r_{n} = r|\vec{c}_{{\tau_{o} n}} , {\widehat{\Omega }}, \vec{X}_{{\tau_{o} n}} } \right) = \frac{{\Pr \left\{ {\vec{c}_{{\tau_{o} n}} |r_{n} = r,{\widehat{\Omega }}, \vec{X}_{{\tau_{o} n}} ) {\text{Pr}}_{0} (r_{n} = r)} \right.}}{{\mathop \sum \nolimits_{s = 1}^{R} \Pr \left\{ {\vec{c}_{{\tau_{o} n}} |r_{n} = s, {\widehat{\Omega }}, \vec{X}_{{\tau_{o} n}} ) {\text{Pr}}_{0} (r_{n} = s)} \right.}}$$

(18.2)

For ease of exposition, we temporarily add the $r$ subscript to $\delta_{rmn}$ to indicate a situation in which the segment, $r$, is known. Let $p_{rmn}$ be the probability that consumer $n$ in segment $r$, who experienced morph $m$, will make a purchase (or other success criterion). This probability is distributed: $f_{n} (p_{rmn} |\alpha_{rmn} , \beta_{rmn} ) \sim p_{rmn}^{{\alpha_{rmn} - 1}} \left( {1 - p_{rmn} } \right)^{{\beta_{rmn} - 1}}$ where $\alpha_{rmn}$ and $\beta_{rmn}$ are parameters of the beta distribution. Updating implies $\alpha_{rm,n + 1} = \alpha_{rmn} + \delta_{rmn}$ and $\beta_{rm,n + 1} = \beta_{rmn} + (1 - \delta_{rmn} )$. Normalizing the value of a purchase to 1.0, the expected immediate reward is $E[p_{rmn} |\alpha_{rmn} ,\beta_{rmn} ] = \alpha_{rmn} /(\alpha_{rmn} + \beta_{rmn} )$.

Let $G_{rmn}$ be the Gittins’ index for the $m$th morph for consumers in segment $r$, let $a \le 1$ be the discount rate from one consumer to the next, and let $V_{Gittins} (\alpha_{rmn} ,\beta_{rmn} , a)$ be the value of continuing with parameters $a$, $\alpha_{rmn}$, and $\beta_{rmn}$. We table $G_{rmn}$ by iteratively solving the Bellman equation.

$$V_{Gittins} \left( {\alpha_{rmn} ,\beta_{rmn} ,a} \right) = \hbox{max} \left\{ {\begin{array}{*{20}c} {\frac{{G_{rmn} }}{1 - a}, \frac{{\alpha_{rmn} }}{{\alpha_{rmn} + \beta_{rmn} }}\left[ {1 + aV_{Gittins} (\alpha_{rmn} + 1,\beta_{rmn} ,a} \right]} \\ { + \frac{{\beta_{rmn} }}{{\alpha_{rmn} + \beta_{rmn} }}aV_{Gittins} \left( {\alpha_{rmn} ,\beta_{rmn} + 1,a} \right)} \\ \end{array} } \right\}$$

(18.3)

When consumer segments are latent, we replace the Gittins’ index with the expected Gittins’ index, $EGI_{mn}$.

$$EGI_{mn} = \mathop \sum \limits_{r = 1}^{R} q_{{rn\tau_{o} }} (\vec{c}_{{\tau_{o} n}} , {\widehat{\Omega }}, \vec{X}_{{\tau_{o} n}} )G_{rmn} (\alpha_{rmn} ,\beta_{rmn} , a)$$

(18.4)

For latent segments, the updating equations are based on “fractional observations.” Details are available in Hauser et al. (2014).

$$\begin{aligned} \alpha_{rm,n + 1} = & \alpha_{rmn} + q_{{rnT_{n} }} (\vec{c}_{{T_{n} n}} , {\widehat{\Omega }}, \vec{X}_{{T_{n} n}} )\delta_{mn} \\ \beta_{rm,n + 1} = & \beta_{rmn} + q_{{rnT_{n} }} (\vec{c}_{{T_{n} n}} , {\widehat{\Omega }}, \vec{X}_{{T_{n} n}} )(1 - \delta_{mn} ) \\ \end{aligned}$$

(18.5)

For the Morphing 2.0 extension, let $w_{t}$ be the weight for observation period $t$ and let $\gamma$ be the multiplicative switching cost. We add a $t$ subscript to morphs such that $m_{tn}$ indicates the morph seen by consumer $n$ in the $t$th observation period. To keep track of morph changes, we define an indicator variable such that $\Delta _{{m_{tn}^{'} tn}} = 1$ if we change to morph $m_{tn} '$ for consumer $n$ in period $t$; $\Delta _{{m_{tn}^{'} tn}} = 0$ otherwise. Because the consumer may see many morphs, we drop the $m$ subscript from $\delta_{mn}$ such that $\delta_{n} = 1$ if the consumer makes a purchase; $\delta_{n} = 0$ otherwise.

To determine when to morph, we solve a Bellman equation by backward recursion for each consumer. The immediate reward is the $\gamma$-discounted, weighted expected Gittins’ index. The expectation uses $q_{rn} (\vec{c}_{t - 1,n} , {\widehat{\Omega }}, \vec{X}_{t - 1,n} )$ because this inferred probability represents our expectations over all future clicks. The segment-conditional continuation reward is $V_{\tau } \left( {m_{\tau n}^{*} , m_{\tau - 1,n} , \vec{c}_{\tau - 1,n} , {\widehat{\Omega }}, \vec{X}_{\tau - 1,n} |r_{true,n} = s} \right)$. It is computed by keeping track of morph changes for $\tau \ge t$. We take the expectation with respect to the probability of observing each consumer segment to obtain the unconditional reward. Let $\psi_{t}$ be the probability of exit after the $t$th observation period and let ${\bar{\varPsi }}\left( {S |t - 1} \right) = E_{n} [\mathop \prod \nolimits_{s = t}^{S} (1 - \psi_{s} )]$, Then the Bellman equation is:

$$\begin{aligned} & V_{t} (m_{tn}^{*} , m_{t - 1,n} , \vec{c}_{t - 1,n} , {\widehat{\Omega }}, \vec{X}_{t - 1,n} )\\ & \quad = \max_{{m_{tn} }} \left\{ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {\gamma^{{\Delta _{{m_{tn} }} }} w_{t} \mathop \sum \limits_{r} q_{rn} \left( {\vec{c}_{t - 1,n} , {\widehat{\Omega }}, \vec{X}_{t - 1,n} } \right)G_{{rm_{tn} n}}\Psi \left( {t |t - 1} \right) + } \\ {\mathop \sum \limits_{s} \left[ {q_{sn} \left( {\vec{c}_{t - 1,n} , {\widehat{\Omega }}, \vec{X}_{t - 1,n} } \right)V_{t + 1} \left( {m_{t + 1,n}^{*} ,m_{tn} , \vec{c}_{t - 1,n} , {\widehat{\Omega }}, \vec{X}_{t - 1,n} , r.e. |s} \right)} \right]\Psi (t + 1|t)} \\ \end{array} } \\ \\ \end{array} } \right\} \end{aligned}$$

(18.6)

We let $\eta_{mnt} = 1$ if consumer $n$ saw morph $m$ during the $t$th observation period; $\eta_{mnt} = 0$ otherwise. Generalized fractional-observation updating becomes:

$$\alpha_{rm,n + 1} = \alpha_{rmn} + q_{r} \left( {\vec{c}_{{T_{n} n}} , {\widehat{\Omega }},\vec{X}_{{T_{n} n}} } \right)\gamma^{{N_{{T_{n} }} }} \left( {\mathop \sum \limits_{t = 1}^{{T_{n} }} \eta_{mnt} w_{t} } \right)\delta_{n}$$

(18.7)

$$\beta_{rm,n + 1} = \beta_{rmn} + q_{r} \left( {\vec{c}_{{T_{n} n}} , {\widehat{\Omega }},\vec{X}_{{T_{n} n}} } \right)\gamma^{{N_{{T_{n} }} }} \left( {\mathop \sum \limits_{t = 1}^{{T_{n} }} \eta_{mnt} w_{t} } \right)(1 - \delta_{n} )$$