Improving Email Marketing Response: Case Study

¶ … Cause-And-Effect Relationships

Required: Use the data presented in Table 1 below to conduct a design of experiment (DOE) in order to test cause-and-effect relationships in business process for the company

Improving Email Response Rate

Run

Heading

Email Open

Replicate

Response Rate

Generic

Detailed

Generic

Detailed

Generic

HTML

Detailed

HTML

Generic

HTML

Detailed

HTML

Generic

Detailed

Generic

Detailed

Generic

HTML

Detailed

HTML

Generic

HTML

Detailed

HTML

Taking averages of the response rate in the repeat experiments, we end up with table 1.1 as shown below:

Run

Email Heading

Email Open

Email Body

Response Rate

Generic

No

Text

(46+38)/2

Detailed

No

Text

(34+38)/2

Generic

Yes

Text

(56+59)/2

Detailed

Yes

Text

74(68+80)/2

Generic

No

HTML

26(25+27)/2

Detailed

No

HTML

27(22+32)/2

Generic

Yes

HTML

22(21+23)/2

Detailed

Yes

HTML

26(19+33)/2

Coding the data in table 1.1 in the form of 1 and -1 for all the three key factors, where i) for the Email Heading factor (abbreviated as EH) 1 represents 'Generic' and -1 represents 'Detailed'; ii) for the Email Opening factor (abbreviated as EO) 1 represents 'No' and -1 represents 'Yes'; and iii) for the Email Body factor (abbreviated as EB) 1 represents 'Text' and -1 represents 'HTML', we end up with table 1.2:

Run

EH

EO

EB

Response Rate

1

1

1

1

42

2

-1

1

1

36

3

1

-1

1

57.5

4

-1

-1

1

74

5

1

1

-1

26

6

-1

1

-1

27

7

1

-1

-1

22

8

-1

-1

-1

26

Calculating the main effect of each of the three factors on the response rate:

Main effect of email heading (EH):

Total response rate for EH when EH is equal to 1 = 163

Total response rate for EH when EH is equal to -1 = 189.5

Slope or Average (163-189.5)/2 = (13.25)

Main Effect of Email Opening (EO)

Total response rate for EO when EO is equal to 1 = 131

Total response rate for EO when EO is equal to -1 = 179.5

Slope or Average (131-179.5)/2 = (24.25)

Main Effect of Email Body (EB)

Total response rate for EB when EB is equal to 1 = 209.5

Total response rate for EH when EH is equal to -1 = 101

Slope or Average (209.5-101)/2 = 54.25

Calculating the interaction effect of a combination of factors on the response rate to determine whether or not there is an interaction:

Interaction Effect of Email Heading (EH) and Email Opening (EO):

Total response rate for EH*EO when EH*EO is equal to 1 = 168

Total response rate for EH*EO when EH*EO is equal to -1 = 142.5

Slope or Average (168-142.5)/2 = 12.75

Interaction Effect of Email Heading (EH) and Email Body (EB):

Total response rate for EH*EB when EH*EB is equal to 1 = 152.5

Total response rate for EH*EB when EH*EB is equal to -1 = 158

Slope or Average (152.5-158)/2 = (2.75)

Interaction Effect of Email Opening (EO) and Email Body (EB):

Total response rate for EO*EB when EO*EB is equal to 1 = 126

Total response rate for EO*EB when EO*EB is equal to -1 = 184.5

Slope or Average (126-184.5)/2 = (29.25)

Interaction Effect of all the three Factors (EO*EB*EH):

Total response rate for EH*EO*EB when EH*EO*EB is equal to 1 = 165

Total response rate for EH*EO*EB when EH*EO*EB is equal to -1 = 145.5

Slope or Average (165-145.5)/2 = 9.75

The fact that response rates vary with different levels of the three factors indicates that there is a main effect in the population (Morris, 2010). This implies that we could use the main effects scatter plot to determine the pareto levels of operation for the three factors (Morris, 2010). However, the interaction results show that there is some significant interaction among factors. Towards this end, rather than use the main effects plot, we could use the interactions effect plot, which has the advantage of showing the extent to which one factor depends on the levels of the other factors under investigation (Roy, 2001). In this case, the interactions effects chart would have the advantage of showing just how much factors depend on the levels of other factors to influence the response rate (Roy, 2001). Our interactions chart would be as shown in fig 1 below.

Fig 1: Interactions Effect Chart for EH, EB, and EO on Response Rate

RR

Factor 2 (EO)

Factor 3 (EB)

Factor 1

(EH)

-1 1

Recommended Action: fig 1 above shows that the EB factor strongly depends on the levels of the EO and EH factors to influence the response rates, whereas the EO factor depends on the levels of the EH factor only very weakly. All the same, there is evidence of an interaction, implying that changes in the response rates cannot be explained by the independent effects of the factors being tested (Manly, 1992). In other words, the response rate is maximized if the three factors are used in combination, although the effect of the EB on the EO and the EH is greater than that of the EO on the EH. Having established that, we need to determine the optimal factor combination (that which maximizes the response rate). The chart indicates that the highest response rate is achieved when the EH and EO factors are at -1, and the EB factor is at 1. This implies that in order to maximize the response rate of its email advertising, the company needs to make use of i) a detailed email heading; ii) a 'YES' Email Open, and iii) Text as opposed to HTML in the email body.