Assessing the AI Scientist

Mon, 26 Aug 2024 00:00:00 +0000

Our collaboration with Joeran Beel and Moritz Baumgart of ISG Siegen originated from Joeran’s sabbatical visit with WING in Oct 2024. Moritz, a talented undergraduate student at University of Siegen, joined us as a research assistant, working remotely which coordinated with his advisor here in NUS.

Over the course of Joeran’s visit, we were able to install and test the AI Scientist system advertised as an automated robo scientist by Sakana.AI. We find positive use cases, but also weaknesses that need much handholding from us.

Haomeng, Rui Feng and Nura join our team as students working towards assessing the growing variety of LLM based research agents going into 2025.

Definition Question Answering

Fri, 01 Aug 2003 00:00:00 +0000

Joint work with PRIS group led by Prof. Tat-Seng Chua

We explore advanced techniques in definition question answering: soft pattern matching, and boosting of IR recall and precision of extended definition sentences using external web resources and historical query logs. We also explore the construction of fluent definitions using sentence understanding and re-synthesis.

Project Deliverables (excluding publications)

DefSearch: definition searcher demonstration, based on work in TREC 2003 and a paper in WWW 2004.
JavaRAP: Lappin and Leass' pronominal anaphora resolution program, implemented as a Java program. Freely downloadable and installable. Published in LREC 2004.
Processed Gigaword and AQUAINT corpora. Fully POS tagged and parsed versions of two commonly-used corpora used in TREC research.
Processed DUC corpus. Canonicalized versions of the 2001-2004 corpus for internal research.

IR | Web IR NLP Group @ NUS

Assessing the AI Scientist

Definition Question Answering

Project Deliverables (excluding publications)