# spotify/confidence

• Stars
238
• Rank 166,369 (Top 4 %)
• Language
• Created over 3 years ago
• Updated 9 months ago

### Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

# Spotify Confidence

Python library for AB test analysis.

## Why use Spotify Confidence?

Spotify Confidence provides convinience wrappers around statsmodel's various functions for computing p-values and confidence intervalls. With Spotify Confidence it's easy to compute several p-values and confidence bounds in one go, e.g. one for each country or for each date. Each function comes in two versions:

• one that return a pandas dataframe,
• one that returns a Chartify chart.

Spotify Confidence has support calculating p-values and confidence intervals using Z-statistics, Student's T-statistics (or more exactly Welch's T-test), as well as Chi-squared statistics. It also supports a variance reduction technique based on using pre-exposure data to fit a linear model.

There is also a Bayesian alternative in the BetaBinomial class.

## Basic Example

``````import spotify_confidence as confidence
import pandas as pd

data = pd.DataFrame(
{'variation_name': ['treatment1', 'control', 'treatment2', 'treatment3'],
'success': [50, 40, 10, 20],
'total': [100, 100, 50, 60]
}
)

test = confidence.ZTest(
data,
numerator_column='success',
numerator_sum_squares_column=None,
denominator_column='total',
categorical_group_columns='variation_name',
correction_method='bonferroni')

test.summary()
test.difference(level_1='control', level_2='treatment1')
test.multiple_difference(level='control', level_as_reference=True)

test.summary_plot().show()
test.difference_plot(level_1='control', level_2='treatment1').show()
test.multiple_difference_plot(level='control', level_as_reference=True).show()
``````

There's a lot more you can do:

• Segment results by one or more dimensions
• Use non-inferiority margins
• Group sequential tests
• Sample size and power calculations
• etc

See jupyter notebooks in `examples` folder for more complete examples.

## Installation

Spotify Confidence can be installed via pip:

`pip install spotify-confidence`

Find the latest release version here

### Code of Conduct

This project adheres to the Open Code of Conduct By participating, you are expected to honor this code.