Metropolis-Hastings In the Real World
Metropolis-Hastings In the Real World
Last time, I introduced the Metropolis-Hastings algorithm for approximating probability distributions with an unknown normalizing factor, and I alluded to applications of this algorithm in Bayesian inference. I also mentioned how I chose a particularly easy example and that the real world poses more serious challenges.
Today, the training wheels are coming off! I'll be applying Metropolis-Hastings to do Bayesian inference in a problem like the ones you might encounter in the real world doing machine learning. We won't know the true distribution we're approximating, and we'll have to deal with the kinds of restrictions that come up in practice.
When we're trying to fit a probabilistic model to the real world, we aren't usually interested in the minutia of the code, and we want the feedback loop between proposing a model and getting results to be as quick and painless as possible. As such, I'll pay extra attention to how Metropolis-Hastings can be automated into an "inference button" that, given a set of prior beliefs and evidence, gives us our new beliefs with as little hand tuning as possible. Let's get started!
Metropolis-Hastings In the Real World
Last time, I introduced the Metropolis-Hastings algorithm for approximating probability distributions with an unknown normalizing factor, and I alluded to applications of this algorithm in Bayesian inference. I also mentioned how I chose a particularly easy example and that the real world poses more serious challenges.
Today, the training wheels are coming off! I'll be applying Metropolis-Hastings to do Bayesian inference in a problem like the ones you might encounter in the real world doing machine learning. We won't know the true distribution we're approximating, and we'll have to deal with the kinds of restrictions that come up in practice.
When we're trying to fit a probabilistic model to the real world, we aren't usually interested in the minutia of the code, and we want the feedback loop between proposing a model and getting results to be as quick and painless as possible. As such, I'll pay extra attention to how Metropolis-Hastings can be automated into an "inference button" that, given a set of prior beliefs and evidence, gives us our new beliefs with as little hand tuning as possible. Let's get started!