Markov Chain Monte Carlo
MCMC vs Monte Carlo Simulation vs Monte Carlo Sampling
These three terms sound similar but represent nested concepts.
Monte Carlo Simulation
The broadest umbrella. Any method that uses random sampling to study a system or estimate a quantity — simulating neural noise, pricing financial options, estimating by throwing random darts. The goal is to mimic a stochastic process.
Example
Simulating the “Forest Ranger”
Signal Detection Theory (SDT)is widely applied in psychophysics, where we model an observer’s ability to distinguish signal from noise.
1. Setting the model
Human perception is noisy and prone to internal or external interference, so we define:
- Noise (): when there is no smoke, factors like water vapor or shifting shadows create sensory noise. The noise distribution can be modeled as .
- Signal + Noise (): when actual smoke is present, the distribution of signal + noise is also a normal distribution, . Here, .
- Decision criterion (): Forest ranger's internal threshold. If the perceived smoke intensity exceeds , the ranger sounds the alarm.
2. Monte Carlo iteration
We use a computer to act as the “virtual ranger” and conduct 10,000 simulated trials:
- Random sampling: in each trial, randomly draw a value from either the or distribution. This represents the instantaneous sensory strength perceived by the ranger.
- Decision making: compare the sampled value against to determine the response.
3. Statistical output (results analysis)
Aggregating 10,000 trials gives the probabilities of the four possible outcomes:
| Outcome | Truth | Response |
|---|---|---|
| Hit | Smoke present | Alarm |
| Miss | Smoke present | No alarm |
| False alarm | No smoke | Alarm |
| Correct rejection | No smoke | No alarm |
Interactive visualization
Drag the sliders to change the distribution means and the criterion. Shaded regions show the four outcomes. Click the button to run 10,000 actual samples and compare empirical to theoretical rates.
sensitivity = 2.00
Monte Carlo Sampling
A narrower sub-category. Draws samples from a known probability distribution to approximate expectations:
Requires that be sampleable directly (or via rejection / importance sampling). Samples are i.i.d.
Markov Chain Monte Carlo (MCMC)
A technique for doing Monte Carlo sampling when you cannot sample from directly. It constructs a Markov chain whose stationary distribution is ; each draw depends only on the previous one. Heavily used for Bayesian posteriors, where the normalizing constant is unknown:
Samples are correlated, not i.i.d. — analysis requires burn-in and convergence diagnostics (, ESS). Common algorithms: Metropolis–Hastings, Gibbs, HMC, NUTS.
MCMC ⊂ Monte Carlo Sampling ⊂ Monte Carlo Simulation