K-Means Clustering as Gaussian Mixture Modeling

Let's say we have some data we want to split into clusters. A common algorithm for doing this defines a cluster using a cluster center. Each point then gets assigned to the cluster center that it's closest to. We can fit this model using a technique called Lloyd's algorithm:

  • Start with kk cluster centers, initialized however we like.1
  • Repeat the following two steps until the cluster centers no longer change:
    • Assign each point to the cluster of the center it's closest to.
    • Move each cluster center to the centroid (average) of all of the points of its cluster.
  1. How you do this matters a fair bit—this is one of the weaknesses of k-means. We're not going to worry about it here, but if you're using this in the real world you should look at k-means++ and similar initialization strategies.↩

K-Means Clustering as Gaussian Mixture Modeling

Let's say we have some data we want to split into clusters. A common algorithm for doing this defines a cluster using a cluster center. Each point then gets assigned to the cluster center that it's closest to. We can fit this model using a technique called Lloyd's algorithm:

  • Start with kk cluster centers, initialized however we like.1
  • Repeat the following two steps until the cluster centers no longer change:
    • Assign each point to the cluster of the center it's closest to.
    • Move each cluster center to the centroid (average) of all of the points of its cluster.
  1. How you do this matters a fair bit—this is one of the weaknesses of k-means. We're not going to worry about it here, but if you're using this in the real world you should look at k-means++ and similar initialization strategies.↩