Why Gains Today Can Fade Away

Show code 
import pandas as pd
import numpy as np
import os
import sys
sys.path.append("../python")
import general
import visualizations
import erosion
Show code 
from IPython.display import HTML

HTML('''<script>
code_show=true; 
function code_toggle() {
 if (code_show){
 $('div.input').hide();
 } else {
 $('div.input').show();
 }
 code_show = !code_show
} 
$( document ).ready(code_toggle);
</script>
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')
Show code 
params = {'OUTPUT' : {'path' : os.path.join('output_html', 'erosion'),
                      'name' : 'erosion_20201012'},
         'SPLIT_STATS' : {'beta_conversion': 0.10,
                          'control_conversion' : 0.07,
                          'natural_conversion' : 0.01,
                          'beta_n' : 1000,
                          'control_n' : 2000},
          'TIME' : {'num_periods' : 12}}
Show code 
convert_df = erosion.generateConversionDF(params)
Show code 
# convert_df = pd.DataFrame(convert_stats)
erosion.calculateRatesAndSignificance(convert_df, params)

Imagine, one morning, a beta created
Experiments conjured, control groups instated
Comparing the outcomes, the beta’s inflated
Signficance tests will affirm…

But measuring monthly, the margin decreases
Advantage persists, but significance ceases
The upside envisioned is falling to pieces
As we reach the end of our term…

Show code 
visualizations.erosionLines(convert_df, params)

Remember that even without interaction
Conversions occur of their own benefaction
Our efforts may aid, but despite our inaction
Some wins might accrue on their own!

And thus, when the beta has reached its conclusion
Those left unconverted create an illusion
Their counts in control groups create a profusion
And thus we diminish what’s grown!

Remember, for betas, the greatest successes
Leave fewer “unconverts” time’s passage addresses
Control groups, in contrast, have massive excesses
Of those who remain to be turned

Though those left behind will convert in proportion
The stragglers occur as uneven distortion
So when the new converts we aim to apportion
Our prior advantage is burned!

Show code 
if not os.path.exists(params['OUTPUT']['path']): os.makedirs(params['OUTPUT']['path'])
general.publish('erosion', params['OUTPUT']['path'], params['OUTPUT']['name'])