Liangming Pan | Web IR NLP Group @ NUS

Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems

Fri, 30 Jan 2026 00:00:00 +0000

MuSLR: Multimodal Symbolic Logical Reasoning

Mon, 01 Dec 2025 00:00:00 +0000

M^3Check: A Demonstration System for Multilingual Multi-clue Multi-Hop Fact-Checking

Wed, 01 Jan 2025 00:00:00 +0000

Imagine that, while we cannot simply Google a complex claim, we can still find some relevant clues to help us. The question is, how can we use these clues to verify the input claim? M^3Check is a RAG model that follows three major steps: 1) break down the claim into clues, 2) verify each clue based on articles, and 3) conclude, check, and reason the claim based on a structured, supervised, and verified chain of clues. This project leverages findings from multiple projects, offering a good example of how research outcomes can be applied to address a real-world problem.

TART – An Open-Source Tool-Augmented Framework for Explainable Table-Based Reasoning

Wed, 01 Jan 2025 00:00:00 +0000

TART is designed to overcome the limitations of LLMs in table reasoning by combining structured table processing, precise tool execution, and explainability. The key components of TART are:

Table Formatter: Prepares tables by formatting, aligning, and cleaning the data for structured input.
Tool Maker: Dynamically generates tools (Python functions) tailored for complex table operations.
Explanation Generator: Integrates structured reasoning steps and tool outputs into human-readable explanations.

TART is evaluated on nine table-based reasoning benchmarks, including SCITAB, TabFact, FinQA, and HybridQA. The framework consistently outperforms Chain-of-Thought (CoT) prompting by integrating structured tool-assisted reasoning.

TART has received the Best Paper Runner-Up Award at the 3rd Table Representation Learning Workshop at NeurIPS 2024 and has been accepted at NAACL 2025. Its code and dataset are publicly available for further research. Researchers working on LLM explainability, table reasoning, and AI-assisted fact-checking are encouraged to contribute.

SCITAB – A Benchmark for Scientific Table-Based Fact-Checking

Wed, 06 Dec 2023 00:00:00 +0000

SCITAB was developed to fill the gap in scientific fact-checking benchmarks by incorporating claims derived from actual scientific papers rather than crowd-sourced ones. These claims require multi-step reasoning to verify against evidence in tables. The dataset was created using a human–model collaboration approach, where real-world claims were filtered, counterclaims were generated using InstructGPT, and all claims were manually verified by expert annotators.

The dataset has been extensively evaluated with state-of-the-art models, including TAPAS, TAPEX, Vicuna, and GPT-4, revealing that existing table-based models struggle with SCITAB. Only GPT-4 shows notable improvements, highlighting the dataset’s challenge and potential for advancing research in table-based reasoning.

SCITAB was presented at EMNLP 2023, and the dataset is publicly available for further research. Researchers interested in compositional reasoning, table fact-checking, and model robustness are encouraged to explore SCITAB and contribute to its future developments.

Doolittle: Benchmarks and Corpora for Academic Writing Formalization

Fri, 01 Dec 2023 00:00:00 +0000

On the Risk of Misinformation Pollution with Large Language Models

Fri, 01 Dec 2023 00:00:00 +0000

QACheck: A Demonstration System for Question-Guided Multi-Hop Fact-Checking

Fri, 01 Dec 2023 00:00:00 +0000

SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables

Fri, 01 Dec 2023 00:00:00 +0000

Attacking Open-domain Question Answering by Injecting Misinformation

Wed, 01 Nov 2023 00:00:00 +0000

FollowupQG: Towards information-seeking follow-up question generation

Wed, 01 Nov 2023 00:00:00 +0000

Investigating Zero- and Few-shot Generalization in Fact Verification

Wed, 01 Nov 2023 00:00:00 +0000

Fact-Checking Complex Claims with Program-Guided Reasoning

Sat, 01 Jul 2023 00:00:00 +0000

Interpreting the Robustness of Neural NLP Models to Textual Perturbations

Sun, 01 May 2022 00:00:00 +0000

Zero-shot Fact Verification by Claim Generation

Sun, 01 Aug 2021 00:00:00 +0000

Unsupervised Multi-hop Question Answering by Question Generation

Tue, 01 Jun 2021 00:00:00 +0000

Exploring Question-Specific Rewards for Generating Deep Questions

Tue, 01 Dec 2020 00:00:00 +0000

Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen

Wed, 01 Jul 2020 00:00:00 +0000

Semantic Graphs for Generating Deep Questions

Wed, 01 Jul 2020 00:00:00 +0000

Resource Mention Extraction for MOOC Discussion Forums

Tue, 01 Jan 2019 00:00:00 +0000