LLMs | Web IR NLP Group @ NUS

AnalysisBank: An Expert Analysis Pattern Library for Financial Report Generation

Tue, 01 Sep 2026 00:00:00 +0000

REPA: Reproducibility Evaluation via an Autonomous Pipeline Architecture

Tue, 07 Jul 2026 00:00:00 +0000

Yuxi Xie Defends Her Ph.D. Thesis on Closed-Loop Scaling

Fri, 08 May 2026 00:00:00 +0000

Congratulations to Yuxi Xie on successfully defending her Ph.D. thesis, “Closed Loop Scaling: Autonomous Improvement of LLM and VLM Reasoning”, from WING!

We wish her every success as she moves into her next chapter.

Many thanks to her committee members, Zhiyong Huang, Anthony K. H. Tung, and See-Kiong Ng, and to her many collaborators, including Michael Shieh, Junxian He, and Kenji Kawaguchi.

Special thanks as well to Prof. William Wang for hosting Yuxi at the University of California, Santa Barbara.

Lyle Ungar Visits WING to Give Invited NAII Seminar on Evaluating LLMs for Cross-Cultural Coaching and Therapy

Thu, 30 Apr 2026 11:00:00 +0800

Did you miss Lyle Ungar’s NUS AI Institute seminar on evaluating large language models for cross-cultural coaching and therapy?

Watch the recording to learn how frontier LLMs still struggle with culturally appropriate conversation and what can be done about it.

Lyle Ungar (University of Pennsylvania) explores how to evaluate LLMs on subjective tasks like coaching and therapy across cultures. The talk highlights:

An interpretable framework for comparing linguistic politeness cues across languages
Evidence that LLMs can homogenize non-Western styles toward stylistic neutrality, a failure missed by standard metrics
RASTA, a retrieval-based method that preserves culturally appropriate style without hurting content
The Culturally-Aware Conversations benchmark

Lyle Ungar is Professor of Computer and Information Science at the University of Pennsylvania, with cross-appointments in Psychology, Bioengineering, Genomics & Computational Biology, and Operations, Information & Decisions. His group builds explainable NLP and socio-emotionally sensitive AI for psychological and medical research.

This NUS AI Institute seminar was recorded and rebroadcast with explicit permission from Lyle Ungar.

Recording: Watch the seminar

KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narration

Wed, 05 Nov 2025 00:00:00 +0000

TART – An Open-Source Tool-Augmented Framework for Explainable Table-Based Reasoning

Wed, 01 Jan 2025 00:00:00 +0000

TART is designed to overcome the limitations of LLMs in table reasoning by combining structured table processing, precise tool execution, and explainability. The key components of TART are:

Table Formatter: Prepares tables by formatting, aligning, and cleaning the data for structured input.
Tool Maker: Dynamically generates tools (Python functions) tailored for complex table operations.
Explanation Generator: Integrates structured reasoning steps and tool outputs into human-readable explanations.

TART is evaluated on nine table-based reasoning benchmarks, including SCITAB, TabFact, FinQA, and HybridQA. The framework consistently outperforms Chain-of-Thought (CoT) prompting by integrating structured tool-assisted reasoning.

TART has received the Best Paper Runner-Up Award at the 3rd Table Representation Learning Workshop at NeurIPS 2024 and has been accepted at NAACL 2025. Its code and dataset are publicly available for further research. Researchers working on LLM explainability, table reasoning, and AI-assisted fact-checking are encouraged to contribute.

Analytical Report Generation with LLMs

Tue, 12 Nov 2024 00:00:00 +0000

While LLMs excel in language understanding and can pass professional financial examinations, they struggle to generate expert-level analytical reasoning from real data. In this project, we explore analytical report generation using LLMs, turning complex market data and corporate disclosures into actionable, decision-relevant narratives. We first develop DataTales (EMNLP 2024), a benchmark pairing 4.9k financial market data tables with human reports to diagnose the exact gap between LLM generation and human creation. Secondly, we propose KAHAN (Findings of EMNLP 2025), a knowledge-augmented hierarchical analysis framework that coordinates analysis from entity metrics to market-wide patterns using executable code. Thirdly, we introduce AnalysisBank (EMNLP 2026), a distilled library of over 5,300 expert analysis patterns that conditions reasoning directly on observed data signals to generate novel, data-specific insights.

Yajing, a PhD student at WING and a Senior Data Scientist at Rio Tinto, started her research driven by the real-world business need for timely daily market reports. While working alongside market professionals, she observed how human commentary takes hours to draft while existing automation defaults to shallow summaries. Advised by Prof. Min-Yen Kan, she formalized this industry pain point into a systematic study of execution failures in generative models. Together with their collaborators, they developed structured methodologies to decouple calculation from composition and condition reasoning moves directly on observed data patterns. Their work demonstrates that grounded analytical depth can be systematically achieved across financial modalities and transferred to domains like medical diagnostics and scientific writing.

SCITAB – A Benchmark for Scientific Table-Based Fact-Checking

Wed, 06 Dec 2023 00:00:00 +0000

SCITAB was developed to fill the gap in scientific fact-checking benchmarks by incorporating claims derived from actual scientific papers rather than crowd-sourced ones. These claims require multi-step reasoning to verify against evidence in tables. The dataset was created using a human–model collaboration approach, where real-world claims were filtered, counterclaims were generated using InstructGPT, and all claims were manually verified by expert annotators.

The dataset has been extensively evaluated with state-of-the-art models, including TAPAS, TAPEX, Vicuna, and GPT-4, revealing that existing table-based models struggle with SCITAB. Only GPT-4 shows notable improvements, highlighting the dataset’s challenge and potential for advancing research in table-based reasoning.

SCITAB was presented at EMNLP 2023, and the dataset is publicly available for further research. Researchers interested in compositional reasoning, table fact-checking, and model robustness are encouraged to explore SCITAB and contribute to its future developments.