Stanford CS153: Frontier Systems · Lecture recording

Latent diffusion, multimodality, and the Flux flywheel

Speaker: Andreas Blattmann — Co-founder @ Black Forest Labs Date: Spring 2026 · Duration: ~61m

TL;DR

Andreas Blattmann tells the story of how frontier visual systems moved from unimodal image generation to a broader quest for multimodal visual intelligence. The lecture’s main thesis is that models become truly useful not when they merely generate pretty pictures, but when they learn over natural representations, close the user-feedback loop, and unify understanding across image, video, audio, and text.

Key Concepts

From a small Heidelberg lab to latent diffusion

Andy’s origin story matters because it establishes the Black Forest Labs worldview: frontier progress does not always start with the most compute. In 2019, his lab was a small computer-vision group competing intellectually with much larger research teams. Their constraint was simple: image generation was computationally expensive, and they did not have hyperscaler-scale resources.

That pressure led them toward a key idea: do not model pixels directly if the pixel space is wasteful. Instead, first learn a compressed, perceptually meaningful latent representation and then train the generative model in that space. This was the basis of latent diffusion.

The practical result was enormous leverage: with far less compute, they could train models that were competitive with or better than better-funded peers.

Stable Diffusion as the legibility moment for visual AI

Andy and Anjney both frame Stable Diffusion as a threshold event. Generative modeling had existed before, but Stable Diffusion made the progress visible to a broader audience. It crossed a line where non-specialists could immediately see the capability jump.

That legibility mattered socially as much as technically. It changed visual AI from a niche research topic into something the wider developer and consumer ecosystem could recognize, experiment with, and build around.

The lecture presents Stable Diffusion not only as a model release, but as the moment when visual systems became an obvious part of the frontier stack.

Natural vs artificial representations

One of the deepest ideas in the lecture is Andy’s distinction between:

  • natural representations: image, video, audio
  • artificial / human-made representations: text

His claim is not that text is unimportant. It is that text is already a highly compressed, low-redundancy abstraction layer created by humans for efficient communication. Natural data, by contrast, is high-bandwidth, redundant, and closer to how the world is actually experienced.

That leads to a strong learning thesis:

  • humans first learn from seeing, hearing, and interacting
  • higher intelligence is built on those natural signals
  • models that start from language alone are starting from the wrong abstraction layer

This is a direct challenge to the idea that language is the sole interface to intelligence.

Why multimodality matters for “real” visual intelligence

The lecture argues that unimodal text-to-image systems were an important but limited phase. Stable Diffusion-class models were primarily content-creation engines. They could produce artistic images, style transfers, and marketing assets, but they were not yet general visual intelligence systems.

Andy’s view is that the frontier now lies in combining natural modalities:

  • image
  • video
  • audio
  • text

The reason is causal understanding. If a model sees two objects collide and also hears the corresponding sound, it can learn a richer world model than from vision alone. Multimodal correlations help the system move from surface-level generation toward something closer to grounded understanding.

That shift broadens the use case space from “creative image generation” to:

  • physical AI
  • robotics
  • computer use
  • world modeling and simulation
  • still, of course, content creation

The Flux flywheel: user demand shaped the model roadmap

A particularly strong systems lesson in the lecture is how Black Forest Labs used deployment data to decide what to build next.

Flux 1 began as a text-image model. But once users started pushing it in the real world, the team learned that many people did not merely want prompt-based generation. They wanted controlled editing, especially character consistency.

This is a powerful example of the feedback loop:

  1. release a strong base model
  2. watch what users attempt to do with it
  3. identify the highest-friction repeated demand
  4. post-train or extend the model toward that use case

That process produced Flux 1 Context, which the lecture presents as a major advance for scalable image editing and consistent identity preservation.

Character consistency as a frontier problem

The lecture spends meaningful time on why image editing and character consistency mattered. Earlier image models could generate beautiful content, but they were weak at precise control. A prompt like “give this exact person a hat” would often produce someone else entirely.

From Andy’s perspective, this was not evidence that the problem was impossible. It was evidence that the next important capability gap had become legible. Once the team saw enough real user behavior, the path toward solving it became clearer.

This section also doubles as a startup lesson: a frontier company has to resist panic when a larger competitor releases something flashy. Instead, it has to read the data carefully, find the unsolved problem still left on the table, and attack that problem with conviction.

Open-weight distribution as a learning advantage

The talk implicitly makes a case for open models as a learning system. Because users can take an open-weight model into many settings, the company gets richer signal about edge cases, preferences, workflows, and failure modes.

Anjney argues that the open-vs-closed debate is often overstated philosophically. The more important question is where each distribution strategy fits the market:

  • open can be especially valuable where preferences are heterogeneous and long-tailed
  • closed can make more sense where user needs are narrower and more standardized

In Black Forest Labs’ case, open distribution helped surface the contexts that mattered most for post-training.

Self-flow and multimodal reasoning

The lecture closes its technical arc with self-flow. Andy presents it as a way to align the internal representations of visual generative models with richer semantic structure, especially once the goal is to go multimodal.

Historically, similar alignment work focused on a single modality, often by bringing generative models closer to pretrained representation-learning models such as DINO. But that becomes limiting once the real target is a shared multimodal representation space.

Self-flow is introduced as a mechanism for making that leap: not just generating coherent pixels, but moving toward models that understand what is happening semantically across multiple natural data streams.

Guardrails, standards, and culture

The final part of the lecture reframes visual AI companies as infrastructure providers. Andy’s position is that guardrails should apply consistently across customers, even powerful ones. That means:

  • strong content filters
  • compliance with EU requirements
  • deletion pathways for user data
  • no special exemptions for large partners who want weaker protections

This ties back to Lecture 1’s idea that infrastructure eventually requires trusted standards and institutions. In Black Forest Labs’ case, the lab itself is trying to behave like one of those institutions.

Industry Insights

  • Black Forest Labs grew out of a compute-constrained research culture that treated efficiency as a strategic advantage, not a handicap.
  • Stable Diffusion showed that open distribution can be a force multiplier for adoption and feedback.
  • Flux 1 Context emerged from observing what users repeatedly tried to do, not from a purely abstract research roadmap.
  • The company’s trajectory suggests that visual-model companies can become infrastructure businesses, not just demo labs.
  • Anjney cites Black Forest Labs as going from zero to several hundred million in revenue while remaining an unusually small and focused team.
  • Open vs closed is framed as a commercial design choice rather than a pure ideology.

Memorable Moments

  • The lecture opens with “Bella Napoli” and the CS153 Spotify playlist, keeping the “AI Coachella” mood alive.
  • Anjney frames the class as taking “field trips” into frontier factories, with Black Forest Labs as the visual-intelligence version.
  • Andy describes studying mechanical engineering as the default “classic German education” path before discovering AI.
  • The lecture repeatedly returns to calmness under pressure: do not panic after a competitor launch; map the frontier and keep shipping.

Notable Quotes

“We had to come up with more efficient algorithms.”

“We are absolutely convinced that this will be the fundament of all the higher intelligence that these systems will eventually have.”

“You should start with first principles, how we humans do it.”

“People want to actually do image editing.”

“Being a standard and being infrastructure that people can rely on means you don’t treat different people differently.”

Reading Materials

Official Readings

No explicit formal reading is assigned in the lecture itself, but one paper is effectively promoted to required background.

Unofficial / Supplementary

  1. Latent Diffusion / Stable Diffusion
    • Central to the origin story of Black Forest Labs’ approach to efficient image generation.
  2. DINO and representation-learning work for images
    • Mentioned as part of the background for aligning generative models with semantic structure.
  3. Self-flow
    • Presented as a key mechanism for multimodal alignment and a recommended paper for anyone tracking the visual frontier.

Connections