Monte Carlo Hypothesis Testing (Critical Value)

In stead of getting the critical value from a given distribution and given $\alpha$, we might draw samples directly out of the population and calculate the test statistics of them in order to find the critical value.

This is the same as saying that we find the critical value from the simulated distribution of the test statistics.

For example, suppose we are looking at the mcdata and testing the hypotheses
H0: $\mu = 454$
H1: $\mu < 454$

Therefore, this is a lower tail test.
Given that the population sigma is 7.8.
We first check whether the population come from a normal distribution using normplot().

>> load mcdata;

>> normplot(mcdata);

 Now let's do some Monte Carlo

n=length(mcdata);
alpha=0.05;
M=1000; % number of Monte Carlo trials
sigma=7.8;
sigxbar = sigma/sqrt(n);
 
% start simulation to get a critical value
Tm = zeros(1,M);
for i=1:M
    %Generate a random sample under H0
    xs = sigma*randn(1,n)+454;
    Tm(i)=(mean(xs)-454)/sigxbar;
end
cv = quantile(Tm,alpha)
 
%get the observed test statistics
tobs = (mean(mcdata)-454)/sigxbar

>> mc_hypotest
cv =
   -1.6439

tobs =
   -2.5641
Since the observed value of test statistic is less than the critical value, we reject the H0.

Notice that the only difference between the Monte Carlo method and the regular Hypothesis testing is how to get the critical value. Usually, for Monte Carlo, we do not know what type of population distribution. However, in this example, we choose the normal distribution and try drawing samples out of it.