NLP | Web IR NLP Group @ NUS

Lyle Ungar Visits WING to Give Invited NAII Seminar on Evaluating LLMs for Cross-Cultural Coaching and Therapy

Thu, 30 Apr 2026 11:00:00 +0800

Did you miss Lyle Ungar’s NUS AI Institute seminar on evaluating large language models for cross-cultural coaching and therapy?

Watch the recording to learn how frontier LLMs still struggle with culturally appropriate conversation and what can be done about it.

Lyle Ungar (University of Pennsylvania) explores how to evaluate LLMs on subjective tasks like coaching and therapy across cultures. The talk highlights:

An interpretable framework for comparing linguistic politeness cues across languages
Evidence that LLMs can homogenize non-Western styles toward stylistic neutrality, a failure missed by standard metrics
RASTA, a retrieval-based method that preserves culturally appropriate style without hurting content
The Culturally-Aware Conversations benchmark

Lyle Ungar is Professor of Computer and Information Science at the University of Pennsylvania, with cross-appointments in Psychology, Bioengineering, Genomics & Computational Biology, and Operations, Information & Decisions. His group builds explainable NLP and socio-emotionally sensitive AI for psychological and medical research.

This NUS AI Institute seminar was recorded and rebroadcast with explicit permission from Lyle Ungar.

Recording: Watch the seminar

SSNLP 2026: WING Members Present Three Posters

Wed, 21 Jan 2026 00:00:00 +0000

The Singapore Symposium on Natural Language Processing (SSNLP) returns on Wednesday, January 21, 2026, at SUTD as a full-day, in-person event. Since 2018, SSNLP has been the annual gathering of Singapore’s NLP community, bringing together students, faculty, and industry researchers to share ongoing work and spark collaboration. Held in conjunction with AAAI 2026, this year’s symposium will feature invited keynotes from leading researchers in the field, a panel discussion, and an immersive poster-focused program showcasing recent work from Singapore-based researchers published in top venues such as ACL, EMNLP, NeurIPS, ICLR, and AAAI this year (this short intro is propagated from SSNLP 2026 website).

WING members are delighted to attend the seminar, enjoying keynote talks from guest researchers who travel all the way from AoE to Singapore. We also appreciate the panel discussions led by PC Chairs (from *ACL, ICLR, and NeurIPS conference), who share valuable insights on the peer review process, particularly from the students’ perspective.

We also engage in the immersive poster program, and our group is excited to present three posters:

Discursive Circuits: How Do Language Models Understand Discourse Relations? by Yisong Miao and Min-Yen Kan (EMNLP 2025)
SkyLadder: Better and Faster Pretraining via Context Window Scheduling by Tongyao Zhu, Qian Liu, Haonan Wang, Shiqi Chen, Xiangming Gu, Tianyu Pang, and Min-Yen Kan (NeurIPS 2025)
Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation Detection by Fanxiao Li, Jiaying Wu, Tingchao Fu, Yunyun Dong, Bingbing Song and Wei Zhou (AAAI 2026).

We thank the symposium organizers for the opportunity to present our work, as well as the enthusiastic audience who engaged with us during the poster sessions!

Below is a gallery of the poster presentations, we thank Prof Min for taking these photos!

Yisong presenting his work on “Discursive Circuits: How Do Language Models Understand Discourse Relations?”:

Tongyao presenting his work on “SkyLadder: Better and Faster Pretraining via Context Window Scheduling”:

Fanxiao presenting his work on “Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation Detection”:

WING Members Present at NUS AI Seminar at the Computer Science Department

Wed, 19 Nov 2025 00:00:00 +0000

NUS AI Seminar at the Computer Science Department is a new initiative to bring together AI researchers to share their latest research findings and insights. There are eight speakers in the Fall 2025 semester, usually hosted on Wednesday afternoons at Seminar Room 21. WING members are excited to present their research alongside other presenters from NUS Computing, MIT, and UCLA. We appreciate Prof. Lee Wee Sun and PhD student Guoji Fu for hosting the seminar.

Yisong Miao gave a seminar presentation titled “Faithfulness and Interpretability for Discourse Understanding” on November 19th, 2025. This talk combines his work on Discursive Socratic Questioning (ACL 2024) and Discursive Circuits (EMNLP 2025). He explained how our team tackles the technical challenge of evaluating a model’s discourse faithfulness and formalizing a unique setup for transformer circuit discovery. The seminar ends with an audience discussion on the conceptual difference between truthfulness and faithfulness, as well as designing new methods using sparse circuits in future work.

Yajing Yang presented “From Data to Insights: LLMs for Financial Narration”, also on November 19th, 2025, highlighting her research on financial data narration. Drawing from DataTales: A Benchmark for Real-World Intelligent Data Narration (EMNLP 2024) and KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narration (EMNLP 2025), she described our team’s efforts to build a realistic benchmark and improve generation quality through knowledge integration and hierarchical analysis. The talk sparked discussion on remaining challenges in financial report generation and how LLMs could be used to produce more actionable and informative insights.

On December 3rd, 2025, Xuan Long Do delivered a seminar titled “Toward Autonomous Self-Improving Foundation Models”. The presentation spanned multiple projects, including “What Makes a Good Natural Language Prompt?” (ACL 2025), Multi-Expert Prompting (EMNLP 2024), and VISTA, his 2025 Google internship project on a self-improving video generation agent. Beginning with an analysis of prompt and model fragility, he showed how multi-agent reasoning can help address these limitations before moving toward the broader vision of autonomous self-improving foundation models. The seminar concluded with an audience discussion on future research directions in this area.

Below is a gallery of the seminar photos, some of which are enhanced and regenerated via Google’s Nano Banana

Yisong Miao presenting his work on “Faithfulness and Interpretability for Discourse Understanding” at the AI Seminar on November 19th, 2025:

Yajing Yang presenting her work on “From Data to Insights: LLMs for Financial Narration” at the AI Seminar on November 19th, 2025:

Xuan Long Do presenting his work on “Toward Autonomous Self-Improving Foundation Models” at the AI Seminar on December 3rd, 2025:

WING Members Present at NUS Computing's Inaugural AI-X: Industry Day

Tue, 18 Nov 2025 00:00:00 +0000

A gallery of additional photos from the event is included below, with thanks to the official event photographer.

Discourse and LLMs

Tue, 04 Nov 2025 00:00:00 +0000

Large language models (LLMs) can now generate coherent text, but can they truly understand language? In this project, we explore discourse understanding, a foundational objective in natural language processing (NLP). In the first part, we refine the notion of understanding as faithful behavior under a series of Socratic-style questions (appeared at ACL 2024). In the second part, we take a causal approach to mechanistically interpret the key components of transformer models in discourse processing (appeared at EMNLP 2025).

Yisong, a PhD student at WING, began his research journey by tackling the fundamental question: Do language models truly understand text? Over time, he refined this broad inquiry into the specific challenge of discourse understanding. In his recent work, advised by Prof. Min, they introduced a more precise framework for measuring discourse comprehension from three discourse-specific perspectives, while also contributing insights into broader LLM challenges such as logical consistency. Currently, Yisong and Min are delving deeper into the mechanisms behind how models process and understand discourse, and they discovered that sparse computational graphs, termed as “discursive circuits”, control how models process discourse relations.

Summarizing Scientific Content with Social Media Language

Fri, 01 Aug 2025 00:00:00 +0000

As social media dominates public information consumption, effectively communicating scientific knowledge through these platforms is crucial. Despite advances in document summarization, complex scientific jargon remains a barrier for general audiences, limiting access to important scientific information. This project develops methodologies to simplify scientific content while preserving accuracy, translating it into language that engages social media users. Our goal is to increase accessibility of scientific knowledge and enhance its influence on informed societal decision-making.

Navigating Ethical Challenges in NLP: Hands-on strategies for students and researchers (Tutorial at ACL '25)

Sun, 27 Jul 2025 00:00:00 +0000

Min's invited talk on prompting at the the IT:U Austria Summer School

Fri, 25 Jul 2025 00:00:00 +0000

Check out Min’s talk that makes an analogy to the boy who harnessed the wind with LLM prompting:

Pictures of the NLP Summer School

(Courtesy the IT:U Austria staff. Pictures cleared by the participants with informed consent.)

A Comparative Analysis of Representation Flow in State-Space and Transformer Architectures

Wed, 01 Jan 2025 00:00:00 +0000

State-space models (SSMs) have emerged as promising alternatives to Transformers, particularly for long-context tasks, due to their efficiency in modeling long-range dependencies through structured state transitions. While prior work has focused on interpreting final-layer outputs, this study investigates the feature flow across layers in SSMs. By comparing this behavior to Transformers, we show fundamental differences in how contextual information is encoded and propagated. Our analysis reveals trade-offs in efficiency and expressivity, offering a deeper understanding of learning dynamics in both architectures. This work not only advances our understanding of SSMs but also lays the foundation for designing hybrid models that combine the strengths of both paradigms.

ChatCRS: Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

Wed, 01 Jan 2025 00:00:00 +0000

This project is inspired by the idea of applying LLMs for zero-shot conversational recommendation. Advanced LLMs (e.g., ChatGPT) are limited in domain-specific CRS tasks for 1. generating grounded responses with recommendation- oriented knowledge, or 2. proactively leading the conversations through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute sig- nificantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into several sub-tasks through the implementation of 1. a knowledge retrieval agent using a tool-augmented approach to reason over external knowledge bases and 2. a goal-planning agent for dialogue goal prediction. Incorporating those external inputs, LLMs proactively plan interactions and generate outputs with rich information. Experiments on two multi-goal CRS datasets reveal that ChatCRS sets new state-of-the-art benchmarks, improving language quality of informativeness by 17% and proactivity by 27%, with tenfold recommendation accuracy enhancement

Assessing the AI Scientist

Mon, 26 Aug 2024 00:00:00 +0000

Our collaboration with Joeran Beel and Moritz Baumgart of ISG Siegen originated from Joeran’s sabbatical visit with WING in Oct 2024. Moritz, a talented undergraduate student at University of Siegen, joined us as a research assistant, working remotely which coordinated with his advisor here in NUS.

Over the course of Joeran’s visit, we were able to install and test the AI Scientist system advertised as an automated robo scientist by Sakana.AI. We find positive use cases, but also weaknesses that need much handholding from us.

Haomeng, Rui Feng and Nura join our team as students working towards assessing the growing variety of LLM based research agents going into 2025.

Benchmarking and Improving OCR Systems for Southeast Asian Languages

Thu, 01 Aug 2024 00:00:00 +0000

While Optical Character Recognition (OCR) has been widely studied for high-resource languages such as English and Chinese, its performance on Southeast Asian (SEA) languages remains largely unexplored. This study addresses this gap by evaluating three OCR tools — EasyOCR, Tesseract, and the transformer-based General OCR Theory (GOT) — on English, Indonesian, Vietnamese, and Thai. We introduce a reusable pipeline for collecting textual data from Wikipedia and benchmarking OCR tools. Contrary to popular belief, our results show that OCR tools perform well on complex scripts like Vietnamese and Thai, with most errors arising from misclassifying characters outside the target language. Additionally, we demonstrate the effectiveness of fine-tuning GOT with limited training data, yielding notable improvements on Vietnamese and Thai.

Linguistics Informed Natural Language Processing with Graph Networks

Thu, 01 Aug 2024 00:00:00 +0000

This project tackles the challenge of extractive question answering (QA), where the goal is to find a precise span of text from a passage that answers a given question. Traditional models often rely on deep language representations but may overlook the underlying syntactic and semantic structure of the context. By addressing this gap, the project aims to create more interpretable and linguistically aware QA systems that can better understand how information is organized and related within text.

To achieve this, we first parse each context passage into a constituency tree using a stack Bi-LSTM, forming the backbone of a graph representation. We then enrich this graph with semantic relationships predicted by a knowledge graph link prediction model. The combined structure is processed by a heterogeneous graph transformer integrating both syntactic and semantic cues for answer extraction. This approach allows for more explainable reasoning and provides a structured way to incorporate external knowledge into the QA process.

Emoji Composition

Mon, 20 May 2024 00:00:00 +0000

The project began when Yisong, intrigued by the Emoji Mashup Bot, explored the question: Can emojis be composed to convey meanings like English phrases? With the support of his advisor, Prof. Min, he recruited an undergraduate student, Zi Yun Yang, to collaborate on the project. Ms. Yang built an annotation platform to collect emoji compositions and their corresponding English phrases from her peers. After her graduation, Yisong and the team added another layer of annotation on composition types and formalized an emoji-based textual entailment task (EmoTE).

The first iteration of this work was published at LREC-COLING 2024. Prof. Min was not listed as an author due to his role as a conference chair, but this project cannot be finished without his guidance. The project continues to gain interest, particularly among undergraduates, with many students from our local NLP course using the emoji dataset for their final projects.

Perspectives on crowdsourcing annotations for natural language processing

Fri, 01 Mar 2013 00:00:00 +0000

Re-engineering the OPAC User Interface

Fri, 04 Jun 2004 00:00:00 +0000

When library patrons utilize an online public access catalog (OPAC), they tend to type very few query words. Although it has been observed that patrons often have specific information needs, current search engine usability encourages users to underspecify their queries. With an OPAC’s fast response times and the difficulty of using more advanced query operators, users are pushed towards a probe-and-browse mode of information seeking. Additionally, patrons have adapted (or forced to adapt) to OPACs and give keywords as their queries, rather than more precise queries. As a consequence, the search results often only approximate the patron’s information need, missing crucial resources that may have been phrased differently or offering search results that may be phrased exactly as wanted but which only address the patron’s information need tangentially.

One solution is to teach the library patron to use advanced query syntax and to formulate more precise queries. However, this solution is both labor-intensive for library staff and time-intensive for patrons. Furthermore, different OPACs support different levels of advanced capabilities and often represent these operators with different syntax. An alternative solution that we propose is the use of advanced query analysis and query expansion. Rather than change the behavior of the patron, a system can analyze their keyword queries to infer more precise queries that uses advanced operators when appropriate. To make these inferences, the system will not only rely on logic but also will dynamically access both a) historical query logs and b) the library catalog to assess the feasibility of its suggestions. The proposed research will target three different types of query rewriting/expansion: 1) correction of misspellings, 2) inferring the relationships between a query’s noun and noun phrases, and 3) inferring intended advanced query syntax. The realization of these techniques will allow patrons to continue using OPACs by issuing simple keyword searchers while benefiting from more precise querying and alternative search suggestions that would originate from the implemented system.

In our current work we have examined how to re-engineer and design the User Interface to better supoort the actual information seeking methods used by library patrons. Jesse has re-engineered the work and has incorporated our own NUS OPAC as well as Colorado State University’s OPAC results into his tabbed, overview+details framework. If you have a library catalog with MARC21 results that can be exported we can wrap our UI around your database. Contact us for more information.

In collaboration with Prof. Danny C. C. Poo.

Light Verb Constructions

Sat, 01 May 2004 00:00:00 +0000

Project Duration: May 2004 - July 2005.

A light verb construction (LVC) is a verb–complement pair in which the verb has little lexical meaning and much of the semantic content of the construction is obtained from the complement. Examples of LVCs are “make a decision” and “give a presentation”, and these pose challenges for natural language processing and understanding. In this project, we investigate methods to identify LVCs from a corpus, as well as recognizing linguistic features of LVCs.

Definition Question Answering

Fri, 01 Aug 2003 00:00:00 +0000

Joint work with PRIS group led by Prof. Tat-Seng Chua

We explore advanced techniques in definition question answering: soft pattern matching, and boosting of IR recall and precision of extended definition sentences using external web resources and historical query logs. We also explore the construction of fluent definitions using sentence understanding and re-synthesis.

Project Deliverables (excluding publications)

DefSearch: definition searcher demonstration, based on work in TREC 2003 and a paper in WWW 2004.
JavaRAP: Lappin and Leass' pronominal anaphora resolution program, implemented as a Java program. Freely downloadable and installable. Published in LREC 2004.
Processed Gigaword and AQUAINT corpora. Fully POS tagged and parsed versions of two commonly-used corpora used in TREC research.
Processed DUC corpus. Canonicalized versions of the 2001-2004 corpus for internal research.