Generative AI models have made significant advancements by drawing inspiration from the principles of physics. Two prominent examples are Pixel Generative Models (PGMs) and Diffusion Models.

Pixel Generative Models (PGMs): PGMs treat data points as electrons and exploit the electric field generated by these "charges" to map the complicated data distribution to a simpler, circular distribution. By learning the electric field approximator, PGMs can generate new data samples by sampling from the simple distribution and traveling backward along the electric field lines.

Diffusion Models: Diffusion models draw inspiration from the concept of thermodynamics and the random motion of atoms. They view pixels in an image as atoms and simulate their diffusion process. By learning how the atoms (pixels) diffuse, diffusion models can generate new images by starting with Gaussian noise and reversing the diffusion process to obtain novel samples from the data distribution.

Both PGMs and Diffusion Models leverage the principles of physics, such as electrostatics and thermodynamics, to overcome the challenge of directly learning and sampling from the complex data distributions. By mapping the complicated distributions to simpler ones, these models can effectively generate new data samples that capture the underlying patterns in the training data.

PGMs treat data points as electrons and exploit the electric field that these data points generate. Consider a two-dimensional data distribution, such as the height and weight of humans. Imagine this data distribution as a charge distribution, where points with higher probability have more electric charge.

The electric field of this charge distribution would be complicated and have high curvature around the distribution itself. However, as we zoom out, the electric field becomes more regular. At very far distances, the charge distribution would look like a point charge, and the electric field would be simple, pointing radially outward in every direction.

The key insight is that the complicated electric field around the charge distribution must connect smoothly to this radial distribution at far distances. This provides a mapping from the complicated data distribution to a simple, circular distribution.

To generate data, we can simply generate simple, spherical data and then travel backward along the electric field lines to yield new data points from the original data distribution. In practice, we learn an approximate electric field by using a U-Net that takes an input vector for a point in space and returns the electric field vector at that point.

This approach, known as PGMs, was introduced at the end of last year, and a successor, PGM++, was published more recently. The authors argue that PGMs offer benefits over diffusion models, which power stable diffusion and Dolly.

Diffusion models, which power models like Stable Diffusion, draw inspiration from the principles of thermodynamics. The key insight is that the random motion of atoms, as described by thermodynamics, can be mapped to the random diffusion of pixel values in an image.

Thermodynamics views atoms as coins, where the macroscopic behavior of a large ensemble of coins (atoms) can be very different from the microscopic behavior of individual coins. For example, the probability of all coins landing heads up is much lower than the probability of 50% of the coins landing heads up, even though each coin has a 50% chance individually.

Similarly, in diffusion models, pixel values in an image are treated as atoms undergoing random walks. Just as the random motion of food coloring in water leads to a uniform distribution, the random motion of pixels leads to Gaussian noise, which can be thought of as the image equivalent of uniform color.

By learning how this diffusion process works for a particular dataset of images, diffusion models can then reverse the process. They can start with Gaussian noise and gradually "undo" the diffusion to generate novel, realistic-looking images. This is analogous to taking a randomly colored image and tracing back the diffusion process to recover the original image.

The mathematical details of how this works can be explored further in the introduction to diffusion models on the blog. But the key takeaway is that the principles of thermodynamics and random walks provide a powerful framework for building state-of-the-art generative AI models.

The distinct fields of physics and AI have often cross-pollinated, with important concepts from mathematics and physics driving progress in AI. In this video, we explored how AI has drawn inspiration from the fields of electrostatics and thermodynamics to create state-of-the-art generative AI models.

Generative AI models work by sampling from a data distribution, which can be a complex task for high-dimensional data like images. To overcome this challenge, AI researchers have turned to physical principles to map the complicated data distribution to a simpler one.

In the case of Plug-and-Play Generative Models (PGMs), the electric field generated by data points, treated as charged particles, provides a mapping from the complex data distribution to a simpler, circular distribution. By learning this electric field, PGMs can generate new data by sampling from the simple distribution and traveling along the electric field lines.

Similarly, diffusion models, which power models like Stable Diffusion, exploit the concept of diffusion from thermodynamics. Just as the random motion of atoms leads to a Gaussian distribution, diffusion models view pixels in an image as "atoms" undergoing random walks, allowing them to generate new images by starting with Gaussian noise and reversing the diffusion process.

These examples demonstrate how the cross-pollination of physics and AI can lead to powerful and innovative generative models. By understanding and leveraging the principles of electrostatics and thermodynamics, researchers have found new ways to tackle the challenges of high-dimensional data generation, paving the way for further advancements in the field of AI.