Reasoning with Intelligence: A New Blueprint for Controllable Generative AI
Generative AI has already dazzled the world with its capabilities. Whether it’s crafting photorealistic images, composing music, generating dialogue, or designing product prototypes, these models have reached a level of fluency and creativity that borders on uncanny. But as impressive as these feats are, they highlight an even more pressing problem – one that sits squarely at the center of real-world deployment: What happens when we need these models not just to generate, but to reason?
That’s the central question driving the research of NUS Presidential Young Professor (PYP) Anji Liu, one of NUS Computing’s newest faculty members. In a landscape saturated with increasingly capable AI systems, Dr. Liu’s work stands out by tackling a critical gap: how to design generative AI models that can not only produce compelling outputs, but also reason about and adhere to constraints, logical, probabilistic, or ethical, with the rigor of classical AI and the flexibility of modern deep learning.
His research marks a pivotal shift. It’s about making generative AI not just expressive, but dependable. And that shift, toward AI that can reason with precision, could change how we design with AI, interact with it, and trust it.
The Problem: When Intelligence Isn’t Enough
Most generative models today, from diffusion-based image generators to large language models (LLMs), rely heavily on statistical correlations learned from massive datasets. They’re incredibly good at predicting what comes next or producing plausible variations. But what they often lack is intrinsic reasoning – an internal mechanism to ensure that their outputs respect constraints, obey rules, or satisfy domain-specific properties.
This limitation isn’t just theoretical. It’s practical and consequential. Imagine a generative model that designs a new molecule but produces something chemically invalid, or a language model that produces an answer that violates ethical norms. In contexts like healthcare, engineering, law, or scientific discovery, these failures can’t be waved away as quirks. They become showstoppers.
To address this, Assistant Professor Liu’s research proposes a radical rethink of how generative models can be built and used. At its core is a simple but ambitious idea: can we fuse the exactness of symbolic reasoning with the scalability of modern AI?
The Core Innovation: Probabilistic Circuits
At the heart of this work are probabilistic circuits (PCs) – a class of generative models structured as directed acyclic graphs that allow for efficient and exact probabilistic reasoning. Unlike traditional deep learning models that often require expensive sampling or approximation methods, PCs are designed to support tractable computations for a wide range of probabilistic queries.
Think of a probabilistic circuit as a carefully constructed logic machine. Each node performs specific probabilistic operations – multiplying probabilities at product nodes or combining outcomes at sum nodes. By connecting these sum and product nodes in specific ways, PCs can compute complex queries like entropy, divergence, or conditional probabilities exactly in polynomial (often linear) time with respect to the model size.
The major contribution here is building what he calls a compositional atlas of operations: an automated pipeline to design efficient inference algorithms for complex queries such as Kullback-Leibler divergence or alpha divergence – tasks foundational to machine learning, decision-making, and model evaluation.
Bridging the Gap: Making PCs Practical
Historically, PCs have been limited in adoption because they’ve lagged behind neural networks in tasks like density estimation and scalability. Dr. Liu’s research systematically dismantles those barriers.
He introduces a series of algorithmic innovations to improve PC training and performance, some key examples are:
- Pruning and Growing: This approach removes unnecessary or low-contributing components of a PC (pruning) and selectively expands useful parts (growing), resulting in models that are both compact and more accurate.
- Latent Variable Distillation (LVD): Here, PCs are trained with supervision from large pre-trained models (like BERT or Masked Autoencoders). This helps overcome the performance plateau observed in larger PCs, enabling them to make better use of their increased capacity.
On the system side, Dr. Liu developed PyJuice, a GPU-optimized compiler and execution framework for PCs. PyJuice translates the computation graph of a PC into a highly parallel block structure, making it amenable to fast training on modern hardware. With it, Liu was able to train 500-million-parameter PCs in under 10 hours – cutting training time by more than 10 times compared to previous systems.
Real-World Impact: Reasoning in the Wild
With these breakthroughs, Dr. Liu’s work doesn’t just push theory; it enables real-world applications across multiple domains. Here are four standout examples:
- Controllable Image Generation with Tiramisu
Image inpainting – filling in missing regions of an image – typically relies on diffusion models, which are great at realism but struggle with strict constraints. Dr. Liu’s system, Tiramisu (Tractable Image Inpainting via Steering Diffusion Models), fuses a probabilistic circuit with a diffusion model to steer generation toward semantically consistent and constraint-satisfying outcomes. It consistently outperforms prior state-of-the-art models, especially on difficult tasks like blending features from multiple images or working with large occlusions.
- Fast, Reliable Data Compression
Lossless data compression demands precise probability modeling – something most expressive generative models can’t deliver efficiently. Dr. Liu’s PC-based compressor achieved near state-of-the-art compression rates on MNIST while being 5 to 40 times faster than leading neural compressors like BitSwap. By making high-performance compression both fast and theoretically principled, this could reshape everything from web delivery to storage infrastructure.
- Safe and Reliable Decision-Making with Trifle
In offline reinforcement learning (RL), agents need to learn from past data and make good decisions without direct environment interaction. Trifle (Tractable Inference for Offline RL), Dr. Liu’s RL framework, uses PCs to guarantee that agents can reliably choose high-reward actions. It achieved best-in-class performance on seven of nine benchmarks and showed robustness in uncertain environments. This has strong implications for safe AI deployment in robotics, healthcare, and automated planning.
Final Thoughts: A New Standard for Intelligence
Assistant Professor Anji Liu’s research is much more than a technical achievement. It’s a vision statement; a roadmap for how we can build generative AI that’s not only powerful and creative, but also controlled, explainable, and trustworthy.
Here’s why this matters. As AI systems increasingly take on roles in sensitive domains (e.g., designing drugs, automating infrastructure, managing data, advising people), the demand for correctness under constraint becomes non-negotiable. You don’t want a system that only works most of the time. You want one that works correctly every time, according to the rules.
That’s what this research unlocks. The probabilistic circuits framework, combined with scalable training algorithms and efficient system implementations, provides a way to encode principled reasoning into generative models. These aren’t just ad hoc fixes. They’re mathematically grounded, verifiably correct, and now computationally feasible.
And we’re only scratching the surface of where this can go.
- In medicine, AI models could design treatment plans or molecules that are guaranteed to meet safety and efficacy constraints, on the fly, under guidance from a clinician.
- In engineering, autonomous systems like drones or self-driving cars could be trained not just to react, but to reason about physical laws and ethical parameters; right in the decision loop.
- In education, AI tutors could deliver learning plans that balance difficulty, personalization, and curriculum goals, while adhering to privacy and safety requirements.
- In creative industries, imagine blending semantic elements from different sources (e.g., voices, images, writing styles) without breaking copyright or coherence. Semantic fusion will no longer be a gimmick. It’ll be a capability.
Perhaps most importantly, this work signals a direction for the field. One that values not just what AI can do, but how it does it. We’re entering an era where AI’s ability to reason, to justify, to align with real-world logic and constraints, becomes as important as its ability to generate or predict.
This is where Dr. Liu’s work stands as a foundational step. It sets a new bar for what it means to be intelligent. Not just fast. Not just fluent. But precise. Principled. Controllable. And in that lies the future of trustworthy, transformative AI.
