The success of “next token prediction” in language models sparked the AI revolution, but extending this paradigm to images has proven challenging. Early attempts like DALL-E showed promise by discretizing images into sequential tokens, but suffered from low visual fidelity, distorted outputs, and failure to adhere to complex instructions when rendering intricate details.
These shortcomings likely stem from cumulative errors during autoregressive inference and information loss during the discretization process. The field swiftly shifted toward diffusion models, but this created architectural and modeling heterogeneity that presents challenges for integrating robust semantic capabilities into image generation.
