# Bootstrap resampling

## Bootstrap Resampling

Bootstrap resampling is a statistical technique to measure the error in a given statistic that has been computed from a sample population. It is a simple yet powerful methord that relies heavily on computational power. The basic premise is that instead of using a theoretical or mathematical model for the parent distribution from which our observed samples were drawn from, we can use the distribution of the observed samples as an approximation for the parent distribution.

### The Algorithm

Let’s say we observe ${\displaystyle N}$ data samples, denoted as ${\displaystyle {\vec {x}}=(x_{1},x_{2},x_{3},...,x_{N})}$, and we want to compute a statistic ${\displaystyle {\hat {\theta }}=s({\vec {x}})}$. This statistic could be the mean or median of our samples, but could also be something much more complex. In measuring ${\displaystyle {\hat {\theta }}}$ from our data, we want to know how close our estimator is to the true value of ${\displaystyle \theta }$, so we need to compute an error estimate for ${\displaystyle {\hat {\theta }}}$. This can be done using the following bootstrap resampling algorithm:

1. Make a bootstrap sample ${\displaystyle x^{\star }}$ by sampling with replacement from the original data samples. This bootstrap sample should also be of length ${\displaystyle N}$ and may contain repetitions of the same data sample (since we sampled with replacement).
2. Repeat this process and create ${\displaystyle B}$ bootstrap samples. Generally, ${\displaystyle B=1000-10000}$, in order to reduce the amount of random scatter in the measurement of the bootstrap error.
3. Compute the same desired statistic for each of the bootstrap samples, ${\displaystyle {\hat {\theta }}^{\star ,b}=s(x^{\star ,b})}$, where ${\displaystyle b}$ ranges from 1 to ${\displaystyle B}$. We will call the quantities ${\displaystyle {\hat {\theta }}^{\star ,b}}$ our bootstrap replications.
4. From the ${\displaystyle B}$ bootstrap replications, compute the standard deviation as
${\displaystyle \sigma _{\mathrm {boot} }^{2}=\sum _{b=1}^{B}\left[{\hat {\theta }}^{\star ,b}-\langle {\hat {\theta }}^{\star }\rangle \right]^{2}/(B-1),\,\!}$

where the mean of the bootstrap replications is given by

${\displaystyle \langle {\hat {\theta }}^{\star }\rangle =\sum _{b=1}^{B}{\hat {\theta }}^{\star ,b}/B.\,\!}$