Nancy F. Chen | Web IR NLP Group @ NUS

Bridging Talk and Thought: Understanding Dialogue Dynamics Across Collaborative Problem-Solving Contexts

Sat, 01 Aug 2026 00:00:00 +0000

CIVIC-AI: Stanford's SALT Lab and WING.NUS Collaborate on Social Intelligence and the Future of AI and Work

Mon, 22 Jun 2026 00:00:00 +0000

The inaugural CIVIC-AI workshop (13–14 July 2026) kicks off a major collaboration between Stanford’s SALT Lab and WING.NUS, bringing together researchers, policymakers, and funding agencies to examine the Social Intelligence of Foundation Models and the Future of AI and Work.

The CIVIC-AI Workshop is an upcoming two-day event in Singapore that will explore the profound societal impacts of AI adoption. The workshop is generously funded by the Ministry of Digital Development and Information of the Republic of Singapore under its AI Visiting Professorship grant, and is also supported by the Ministry of Manpower.

The event centers on three core questions:

Cross-cultural Dimensions: How do cultural context, economic development, and policy environments shape AI adoption and the lived experience of automation across different societies?
Future of Work & Automation: Which professions and tasks are being automated away versus augmented, and who bears the costs of displacement?
Human Advantage: What constitutes irreducible human value when a large fraction of knowledge work can be automated, and how do we measure and support it?

The first day (13 July) focuses on the theme of Social Intelligence, featuring invited talks by researchers such as Yohan Jo, Nancy F. Chen, and Lizi Liao, alongside lightning talks and a panel discussion. The highlight of the day will be a keynote by Diyi Yang from Stanford’s SALT Lab on “Optimizing Human-AI Collaboration,” which will introduce General User Models (GUMs) and Next Action Prediction (NAP) to foster proactive and meaningful human-AI interaction.

The second day (14 July) will center on the Future of Work With AI Agents in Singapore, featuring presentations by Diyi Yang and Min-Yen Kan, followed by extensive discussion sessions to finalize directions and next steps for the collaboration.

Registration for the workshop closes on 6 July 2026. For more details and to register, visit the CIVIC-AI workshop website.

The workshop is made possible by the Singapore Global AI Visiting Professorship (AIVP), an initiative by the Ministry of Digital Development and Information to strengthen Singapore’s AI research capabilities by attracting world-class expertise. Min-Yen Kan (WING.NUS) and Diyi Yang (SALT Lab, Stanford University) are among six NUS Computing faculty teams awarded AIVP grants, for their project “Evaluating and Building Socially Intelligent Foundation Models” — the research collaboration that underpins CIVIC-AI.

Self-Improving and Self-Adapting Agents

Mon, 01 Dec 2025 00:00:00 +0000

🎯 Overview: We study test-time scaling (TTS) methods for self-improving and self-adapting agents, advancing a new paradigm of artificial intelligence in which autonomous systems do not merely act, but evolve reliably through learning from experience, refining behavior at test time, and autonomously modifying their own learning mechanisms.

🧠 Abstract: Although modern foundation models (FMs) like ChatGPT are extraordinarily capable, they remain largely fragile: small variations in input (aka ‘‘prompts’’), such as subtle phrasing changes or unintended noise, can lead to contradictory or ungrounded reasoning. This limits their broader deployment in high-stakes domains like healthcare, law, and science, where precision, reliability, and interpretability are essential.

In this project, we aim to build the next generation of FM systems that can continually assess and correct their behavior at test time, making them more trustworthy and capable. Our studies span three main aspects:

(1) Understand Fragility of FMs: How can we interpretably understand model behavior through the lens of prompts to improve FMs reliably? (NLPromptEval @ ACL'25, FormatBiasEval @ NAACL'25)
(2) Study TTS Methods: With the above understanding, how can we develop TTS to enhance model effectively and reliably? (Multi-Expert Prompting @ EMNLP'24, adv-ICL @ ACL'24, Chain-of-Opinion @ COLING'25)
(3) Self-Improving Agents: How can we build powerful TTS-powered agentic systems that critique and refine their own behavior, ultimately enabling autonomous self-improvement? (LongGuide (Self-Adapting Agent) @ ACL'25, VISTA (Self-Improving Agent))

Our findings so far have offered a foundation for building foundation model agents that reliably improve through interaction, feedback, and structure-aware test-time methods—without reliance on additional gradient-based training.

🔥 News: We are expanding our studies towards multiple directions. Please review our work above. If you have a strong interest in the topics listed and a solid background in mathematics and programming, and optionally in research, come to join us!

Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines

Fri, 01 Aug 2025 00:00:00 +0000

What Makes a Good Natural Language Prompt?

Fri, 01 Aug 2025 00:00:00 +0000

LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs

Mon, 05 May 2025 00:00:00 +0000

Aligning Large Language Models with Human Opinions through Persona Selection and Value-Belief-Norm Reasoning

Wed, 01 Jan 2025 00:00:00 +0000

DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation

Wed, 01 Jan 2025 00:00:00 +0000

Multi-expert Prompting Improves Reliability, Safety and Usefulness of Large Language Models

Thu, 12 Dec 2024 00:00:00 +0000

Yisong Miao receives the Senior Area Chair Award at ACL 2024!

Thu, 15 Aug 2024 00:00:00 +0000

Congratulations to WING members, Yisong Miao, Min-Yen Kan, and their collaborators for winning the Senior Area Chair Award at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) for their paper, “Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations.”

Our group is thrilled to announce that we have received a Senior Area Chair Award (Discourse and Pragmatics track) at ACL 2024 for our paper, “Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations.” This work, led by our group members Yisong Miao and A/P Min-Yen Kan, in collaboration with Hongfu Liu, Dr. Wenqiang Lei, and Dr. Nancy Chen, introduces a novel scoring framework that leverages LLMs in a Socratic style, testing models on targeted questions, counterfactuals, and converse questions.

Learn more about this paper.

Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations

Sun, 11 Aug 2024 00:00:00 +0000

Prompt Optimization via Adversarial In-Context Learning

Thu, 08 Aug 2024 00:00:00 +0000

Congratulations, Dr. Taha Aksu!

Tue, 02 Apr 2024 00:00:00 +0000

Congratulations to Dr. Taha Aksu on successfully defending his PhD thesis titled, “Task-Oriented Dialogue Understanding and Evaluation using Minimal Data.”

We are thrilled to announce that Dr. Taha Aksu has successfully completed his PhD defense! His thesis focused on advancing task-oriented dialogue systems through efficient understanding and evaluation with minimal data. Dr. Aksu now joins Salesforce AI in Singapore, and we look forward to seeing his future achievements.

A special thanks to his examiners Prof. Tat-Seng Chua (first from the left), A.P. Brian Lim (second from the left), A/P Ye Wang (top right), and I2R co-supervisor Dr. Nancy Chen (third from the left) along with all of his supporters along the way!

Learn more about Dr. Taha Aksu Homepage on his work.

CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation

Fri, 01 Dec 2023 00:00:00 +0000

Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation

Sat, 01 Jul 2023 00:00:00 +0000

N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking

Sun, 01 May 2022 00:00:00 +0000

Velocidapter: Task-oriented Dialogue Comprehension Modeling Pairing Synthetic Text Generation with Domain Adaptation

Thu, 01 Jul 2021 00:00:00 +0000