$\text{The law of large numbers is usually covered in statistical courses,}\\ \text{but primarily from a theoretical perspective.}$
$\text{We will look at the law of large numbers from a practical and philosophical standpoint.}$
$\text{What is the law of large numbers?}$
$\text{Roughly speaking, when the sample size becomes large,}\\ \text{the sample mean converges to the population mean (expected value).}$
$\text{Mathematically,} \\ \bar{x} \to \operatorname{E}(X) \ \text{as } n \to \infty \\ \text{of course, given that}\ \operatorname{E}(X)\ \text{does exists}\\ \text{Where, } \ \bar{x} \ \text{is the sample mean,}\\ \operatorname{E}(X) \ \text{is the population mean and,}\\ n \ \text{is the sample size} $
$\text{But the question is how big the sample size should be.}$
set.seed(1045)
n= seq(100,10^4,length=10)
mu= 8.7 # Why 8.7? You'll see later!
var=100 # High Variance
s=c()
count=1
for(i in n)
{
samp=rnorm(i,mean=mu,sd=sqrt(var))
s[count]= mean(samp)
count=count+1
}
plot(n,s,type="l",col="red",main="Law of Large Numbers for Normal Distribution",
xlab="number of samples", ylab='sample mean',lwd=2,ylim=c(5,13))
abline(h=mu,col="green",lwd=2)
legend("top",legend=c("Sample Mean", "True Mean"), col=c("red","green"),
lwd=2,cex=1.4,bty="n")
$ \text{For a normal distribution, even with a large variance of about 100,}\\ \text{we observe convergence before a sample size of 10000.}$
$ \text{But in reality, things are hardly normal. In fact, extreme events drive reality.}$
$ \text{Let's consider a Pareto 80/20 distribution.}$
$ \text{Why a Pareto distribution ?}$
$ \text{Pareto distribution can be helpful whenever things exhibit extreme behavior.}$
$ \text{A Pareto 80/20 distribution tells us that roughly 80 percent of the consequences come from 20 percent}\\ \text{of the events, and 20 percent of the consequences come from 80 percent of the events.}$
$ \text{For example, the richest 20 percent hold 80 percent of the world's income,}\\ \text{and the rest 80 percent of the people have the rest 20 percent of the income.}$
# The expected value (mu) for the normal distribution corresponds
# to the same expected value for the Pareto
library(VGAM)
set.seed(1045)
n= seq(100,10^8,length=10)
alpha=1.13 # for pareto 80/20
xmin=1
s=c()
count=1
for(i in n)
{
samp=rpareto(n = i,scale=xmin,shape=alpha)
s[count]= mean(samp)
count=count+1
}
expected_val= (alpha*xmin)/(alpha-1)
plot(n,s,type="l",col="red",main="Law of Large Numbers for Pareto Distribution",
xlab="number of samples", ylab='sample mean',lwd=2)
abline(h=expected_val,col="green",lwd=2)
legend("top",legend=c("Sample Mean", "True Mean"), col=c("red","green"),
lwd=2,cex=1.4,bty="n")
Loading required package: stats4 Loading required package: splines
$\text{Now, even after observing a sample size of the order of magnitude}\ 10^8\ \text{still, no convergence is obtained.}$
$\text{Philosophically, for a real-world process, even after observing a large amount of data,} \\ \text{you can still end up making wrong estimations.}$
$\text{In fact, for something like Pareto 80/20, even with a sample size of order}\ 10^6, \\ \text{in almost 90% of the cases, your sample mean will end up underestimating the expected value} \\ \text{or population mean.}$
set.seed(1045)
n=1000000
sim=10000
k=c()
for(i in 1:sim)
{
samp=rpareto(n = n,scale=xmin,shape=alpha)
k[i]= mean(samp)
}
s=length(which(k<=expected_val))/sim
# In how many percent of the simulations you
#end up underestimating the expected value?
paste("In ",s*100," % of the simulations you will end up
underestimating the expected value")