## Research

__Overview__

__Overview__

My research interests lie at the intersection of

**machine learning**,

**information theory**, and

**high-dimensional statistics**, with ongoing areas of interest including the following:

- Information-theoretic understanding of statistical inference and learning problems
- Adaptive decision-making under uncertainty (e.g., Bayesian optimization, bandits)
- Scalable algorithms for large-scale inference and learning (e.g., group testing, graph learning)
- Robustness considerations in machine learning

If you have been admitted to the NUS PhD program and are looking for a supervisor, feel free to email me to arrange a meeting. Other prospective PhD applicants are also welcome to get in touch, but I apologize that I may not reply to most enquiries. Admission to NUS can be done through the Department of Computer Science, the Department of Mathematics, or the Institute of Data Science.

If you would like to apply for a post-doc or research assistant position, please send me your CV and an outline of your research interests. Applicants should have a strong track record in an area related to my research interests, such as machine learning, information theory, statistics, statistical signal processing, or theoretical computer science.

__Research Group__

__Research Group__

- Lan Truong (post-doc)
- Zhaoqiang Liu (post-doc)
- Eric Han (PhD student)
- Anamay Chaturvedi (research assistant)
- Avtansh Tiwari (intern)
- Zexin Wang (intern)

__Research Funding__

__Research Funding__

- (May 2019 - May 2024)
*Robust Statistical Model Under Model Uncertainty*, Singapore National Research Foundation (NRF) Fellowship ($2.29M) - (Nov. 2018 - Oct. 2021)
*Information-Theoretic Methods in Data Science*, NUS Early Career Research Award ($500k) - (Jan. 2018 - Jan. 2021)
*Theoretical and Algorithmic Advances in Noisy Adaptive Group Testing*, NUS Startup Grant ($180k)

__Ongoing Research Projects__

__Ongoing Research Projects__

Some potential research projects that students and post-docs could pursue with me are listed below; this list is far from exhaustive.

### 1) Information-theoretic limits of statistical inference and learning problems

The field of information theory was introduced as a means for understanding the fundamental limits of data compression and transmission, and has shaped the design of practical communication systems for decades. This project will pursue the emerging perspective that information theory is not only a theory of communication, but a far-reaching theory of data benefiting diverse inference and learning problems such as estimation, prediction, and optimization. This perspective leads to principled mathematical approaches to certifying the near-optimality of practical algorithms, and steering practical research towards where the greatest improvements are possible.

**Selected relevant publications:**

- An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation
- Limits on Support Recovery with Probabilistic Models: An Information-Theoretic Framework
- Tight Regret Bounds for Bayesian Optimization in One Dimension

### 2) High-dimensional Bayesian optimization

Bayesian optimization (BO) has recently emerged as a versatile tool for optimizing "black-box" functions, with particular success in automating machine learning algorithms by learning a good set of hyperparameters, as well as other applications such as robotics and materials design. The main modeling assumption of BO is that the function is well-approximated by a Gaussian process, whose smoothness properties are dictated by a kernel function.

While BO has had considerable success in optimizing functions with a small number of inputs (e.g., 10 or less), the "high-dimensional" setting (i.e., a large number of input variables) generally remains one of the most significant unsolved challenges in the field. This project will seek to develop novel algorithms for solving high-dimensional BO in a principled and rigorous manner, with an emphasis on establishing powerful kernels with low-dimensional structure permitting tractable optimization.

**Selected relevant publications:**

- High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
- Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation
- Time-Varying Gaussian Process Bandit Optimization

### 3) Robustness considerations in machine learning

Robustness requirements pose many of the most important unsolved challenges in modern machine learning, arising from sources of uncertainty such as mismatched modeling assumptions, corrupted data, and the presence of adversaries. For instance, large distributed learning systems must be able to deal with individual node failures, robotics tasks learned in a simulated environment should be designed to degrade as little as possible when transferred to a real environment, and robustness against adversarial attacks remains a considerable unsolved challenge in deep learning, just to name a few examples. This project will seek to better understand some of the most practically pertinent sources of uncertainty and develop new algorithms that are robust in the face of this uncertainty, with rigorous guarantees.

**Selected relevant publications:**

- Adversarially Robust Optimization with Gaussian Processes
- Robust Submodular Maximization: A Non-Uniform Partitioning Approach

### 4) Algorithms and theory for large-scale graph learning

This project will explore large-scale graph learning problems relevant to modern statistics and machine learning, including (i) learning conditional independence relations among large collections of random variables based on random samples, and (ii) direct large-scale graph learning based on a limited collection of queries (e.g., "Does this group of nodes contain an edge?"). We will aim to develop a detailed understanding of how the difficulty of these problems is impacted under practically-motivated settings, such as the ability to sample adaptively based on past observations, limitations imposed on the computation and storage, and less stringent performance criteria based on approximate recovery. We will develop such an understanding via both fundamental performance characterizations and rigorous practical algorithms.

**Selected relevant publications:**

- Learning Erdős-Rényi Random Graphs via Edge Detecting Queries
- Lower Bounds on Active Learning for Graphical Model Selection
- On the Difficulty of Selecting Ising Models with Approximate Recovery

### 5) Adaptive algorithms for group testing and pooled data

Group testing is a classical sparse estimation problem that seeks to identify "defective" items by testing groups of items in pools, with recent applications including database systems, pattern matching, and DNA testing. One of the defining features of this problem is the distinction between the non-adaptive and adaptive settings: In the non-adaptive case, all tests must be designed in advance, whereas in the adaptive case, each test can be designed based on the previous outcomes.

While highly effective adaptive algorithms are known for noiseless group testing, surprisingly little is known in noisy settings and in more general pooled data problems. This project will seek to close this gap by developing rigorous adaptive testing strategies, including (i) techniques building on the ubiquitous Multi-Armed Bandit (MAB) framework for selecting actions that have a high "reward", and (ii) techniques inspired by state-of-the-art feedback codes in communication.

**Selected relevant publications:**

- Group Testing: An Information Theory Perspective
- Noisy Adaptive Group Testing: Bounds and Algorithms
- Phase Transitions in Group Testing

### 6) Deep learning methods in signal processing and communications

Decades of research have led to an extensive range of powerful techniques for signal processing and communication problems. Following the tremendous success of deep learning in applications such as image recognition and language processing, it has been proposed that instead of designing explicit models and algorithms, better performance can often be achieved by using deep learning methods to learn from data. For instance, neural networks can perform a central part of the encoder and/or decoder in communication, and can provide powerful generative models for modeling structure in high-dimensional signals. This project will explore this exciting new research avenue from both a theoretical and practical perspective.

**Selected relevant publications:**