avatar

I study how data, optimization, and model architecture jointly shape what models memorize and learn. My research interests lie in developing better learnable memory both during training and at test time, in terms of efficiency, privacy and utility. Here is my CV (updated May 28, 2026).

Earlier in my PhD at National University of Singapore, I focused on measuring and controlling train-time memorization. I have worked on

  1. empirical memorization measurments based on inference attacks [ACM CCS 2022];
  2. analytical memorization estimates based on modelling neural network training dynamics, e.g., via Gaussian process [ICLR 2024], neural tangent kernels [NeurIPS 2023] and stochastic differential equations [NeurIPS 2021, NeurIPS 2022];
  3. controlling memorization via instance-optimal differentially private learning [NeurIPS 2025] and boosting fact memorization via training data selection [ICML 2026].

Recently, I’m also interested in understanding the benefits of parameter-separated storage of facts versus reasoning rules.

For my works in privacy & security, I’m a recipient of the 2024 Apple Scholars in AI/ML PhD Fellowship and the 2023-2024 Google PhD Fellowship in security and privacy. Before my PhD, I obtained my B.S. in Computational Mathematics from the University of Science and Technology of China.

Selected Publications

(* indicates equal contributions)

  1. Cram Less to Fit More: Training Data Pruning Improves Fact Memorization [Paper]
    Jiayuan Ye, Vitaly Feldman, Kunal Talwar
    ICML 2026
    Also received Best Paper Award at DATA-FM workshop @ ICLR 2026

  2. Leave-One-Out Distinguishability in Machine Learning [Paper] [Code]
    Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri
    ICLR 2024

  3. Instance-Optimality for Private KL Distribution Estimation [Paper]
    Jiayuan Ye, Vitaly Feldman, Kunal Talwar
    NeurIPS 2025
    Also received Spotlight (Top 3%)

  4. How Much of My Dataset Did You Use? Quantitative Data Usage Inference in ML [Paper] [Code]
    Yao Tong*, Jiayuan Ye*, Sajjad Zarifzadeh, Reza Shokri
    ICLR 2025
    Also received Oral (Top 2%)

  5. Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks [Paper]
    Jiayuan Ye, Zhenyu Zhu, Fanghui Liu, Reza Shokri, Volkan Cevher
    NeurIPS 2023

  6. Differentially Private Learning Needs Hidden State (Or Much Faster Convergence) [Paper]
    Jiayuan Ye, Reza Shokri
    NeurIPS 2022

  7. Enhanced Membership Inference Attacks Against Machine Learning Models [Paper] [Slides] [Code]
    Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, Reza Shokri
    CCS 2022
    Among top 10 most cited papers published in security conferences in 2022. [Link]

  8. Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent [Paper]
    Rishav Chourasia*, Jiayuan Ye*, Reza Shokri
    NeurIPS 2021
    Also received Spotlight (Top 3%)

Other Publications Contributed To

  1. Optimal Splitting of Language Models from Mixtures to Specialized Domains [Paper]
    Skyler Seto, Pierre Ablin, Anastasiia Filippova, Jiayuan Ye, Louis Béthune, Angelos Katharopoulos, David Grangier
    ICML 2026

  2. Generalization in LLM Problem Solving: The Case of the Shortest Path [Paper]
    Yao Tong, Jiayuan Ye, Anastasia Borovykh, Reza Shokri
    ICLR 2026

  3. Unified Enhancement of Privacy Bounds for Mixture Mechanisms via -Differential Privacy [Paper]
    Chendi Wang*, Buxin Su*, Jiayuan Ye, Reza Shokri, Weijie J Su
    NeurIPS 2023

  4. Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning [Paper]
    Zebang Shen, Jiayuan Ye, Anmin Kang, Hamed Hassani, Reza Shokri
    ICLR 2023