Binghui Li (李柄辉)

Binghui Li (李柄辉)

A Ph.D. student at PKU

About me

My name is Binghui Li. I am a fourth-year Ph.D. student in The Center for Machine Learning Research (CMLR), Peking University, where I am extremely fortunate to be advised by Prof. Liwei Wang and Prof. Lei Wu. I am also pleased to have worked closely with Prof. Yuanzhi Li, Prof. Ruoyu Sun, Prof. Jian Li, Prof. Kaifeng Lyu, and Prof. Pinyan Lu. Before joining CMLR, I received my Bachelor’s degree in Intelligence Science and Technology (Honor Track) from Turing Class, Peking University.

I currently focus on the theoretical foundations of large language model (LLM) pre-training, particularly scaling laws and optimization dynamics. Our recent work explores optimizers [14, 17, 18], learning-rate schedules [10, 15], batch-size schedules [12], and data reuse [11]. A major thread of my research is the development of functional scaling laws (FSL) [10, 12, 15, 17]. In this line of work, we develop a framework to characterize learning dynamics and understand how optimization choices shape scaling behavior. Through these efforts, we aim to provide principled guidance for efficient LLM pre-training.

If you are interested, please feel free to contact me (libinghui@pku.edu.cn or WeChat)!

Education

  • Ph.D. in Data Science (Computer Science and Technology), Peking University, 2023-2028 (Expected).

  • B.S. in Intelligence Science and Technology (Summa Cum Laude), Peking University, 2019-2023.
    Overall GPA: 3.82 / 4.00, Average Score: 90.2 / 100, Rank: 2 / 81

Research Experience

  • (Research Intern) ByteDance Seed, Edge Group, 2026.2-now
    Research on scaling laws and optimization dynamics for large language model pre-training.

Research Highlights

LLM Pre-training: Scaling Laws & Optimization

I study scaling laws and optimization dynamics to understand LLM pre-training and improve its efficiency.

  • Functional scaling laws and training schedules. We develop a framework of functional scaling laws (FSL) to characterize learning dynamics and the effects of learning-rate schedules [10]. Building on this framework, we study optimal learning-rate schedules [15], batch-size scheduling [12], and differences in scaling behavior between SignSGD and SGD [17].
  • Optimization dynamics and parameterization. We investigate Muon’s training dynamics and scaling laws in associative memory learning [14], as well as when and how it acts as a preconditioner in the presence of structured gradient noise [18]. We also study the role of scale vectors in LLMs—parameters that are small in number but significant in their effects [16].
  • Data reuse and multi-epoch scaling. We study how repeated training on finite datasets affects scaling behavior, characterizing how the benefits of data reuse depend on dataset size and distribution [11].

AI Safety: Foundations of Adversarial Robustness

My work on AI safety focuses on adversarial robustness: understanding why neural networks are vulnerable to small input perturbations and when robust learning is possible.

  • Limits of robust generalization. We investigate the expressive power required for robust learning, identifying fundamental barriers that help explain why robust generalization can be substantially harder than standard generalization [3].
  • Feature learning and adversarial training. We study how adversarial training can improve robustness through feature learning [4], and how the implicit bias of gradient descent can lead to non-robust representations [5].
  • Robust overfitting and training dynamics. We examine generalization and robust overfitting in adversarial training through the complementary perspectives of representation complexity and training dynamics [7].

My other research interests include multimodal retrieval-augmented generation [6, 9] and mathematical discovery, including sphere-packing constructions [8].

Publications & Preprints

* indicates equal contribution, † means project lead.

Invited Talks

  • 2026.10, At the Twelfth Beijing Computational Mathematics Postgraduate Forum, Peking University, Understanding LLM Pre-Training Dynamics via Functional Scaling Laws

  • 2026.05, At the Fifth Turing Student Research Forum, Peking University, A Functional Perspective on Scaling Laws in LLM Pre-Training [Event]

  • 2026.05, At the FAI Seminar (International Seminar on Foundational Artificial Intelligence), Understanding LLM Pre-Training Dynamics via Functional Scaling Laws [Video]

  • 2025.11, At the Eleventh Beijing Computational Mathematics Postgraduate Forum, Peking University, Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules

  • 2025.11, At the Yao Class Seminar, Tsinghua University, Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules

  • 2025.06, At the FAI Seminar (International Seminar on Foundational Artificial Intelligence), Theoretical Understanding of Adversarial Examples in Deep Learning: Expressive Power and Training Dynamics [Video]

  • 2025.04, At the Applied Mathematics Youth Seminar, Peking University, Theoretical Understanding of Adversarial Examples: Expressive Power and Training Dynamics [Slides]

  • 2024.11, At the Turing Student CS Frontier Tutorial, Peking University, Theoretical Understanding of Adversarial Examples: Expressive Power and Training Dynamics [Slides]

  • 2024.11, At the Tenth Beijing Computational Mathematics Postgraduate Forum, Peking University, Theoretical Understanding of Adversarial Examples: Expressive Power and Training Dynamics [Slides]

  • 2024.05, At the Turing Student Research Forum, Peking University, Understanding Adversarial Examples and Adversarial Training in Deep Learning: A Feature Learning View [Slides]

  • 2023.08, At International Joint Conference on Theoretical Computer Science – Frontier of Algorithmic Wisdom (IJTCS-FAW 2023), Robust Generalization Requires Exponentially Large Models [Slides]

  • 2023.05, At the Turing Student Research Forum, Peking University, Robust Generalization Requires Exponentially Large Models [Video]

  • 2022.05, At the Fourth BAAI Conference, Robust Generalization Requires Exponentially Large Models [Slides] [Video]

Selected Awards

Teaching

  • 2025 Fall: CS 34100 Mathematical Foundations for the Information Age, PKU (TA, with Prof. Yuqing Kong)

  • 2025 Fall: CS 33400 Discrete Mathematics and Structures, PKU (TA, with Prof. Tianren Liu)

  • 2025 Spring: CS 34010 Randomness and Computation, PKU (TA, with Prof. Yuqing Kong)

  • 2024 Fall: CS 33400 Discrete Mathematics and Structures, PKU (TA, with Prof. Tianren Liu)

  • 2023 Fall: CS 33400 Discrete Mathematics and Structures, PKU (TA, with Prof. Tianren Liu)

  • 2022 Fall: CS 33400 Discrete Mathematics and Structures, PKU (TA, with Prof. Xiaotie Deng)

  • 2021 Fall: CS 33400 Discrete Mathematics and Structures, PKU (TA, with Prof. Xiaotie Deng)

Professional Service

  • Conference reviewer: ICLR (2023, 2025–2027), NeurIPS (2024–2026), AISTATS (2025–2027), ICML (2025–2026), AAAI (2026–2027), CVPR (2026), ECCV (2026).
  • Journal reviewer: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
  • Committee service: Member of the Turing Class Research Committee (2020–present).

Selected Courses

  • Machine Learning: 93/100
  • Numerical Methods (B): 95/100
  • High-Dimensional Probability: 90/100
  • Study and Practice on Topics of Frontier Computing (II): 96/100
  • Introduction to the Theory of Computation: 92/100
  • Mathematical Analysis (I): 97.5/100
  • Discrete Mathematics and Structures (I): 97/100
  • Probability Theory and Statistics (A): 96/100
  • Python Programming and Application: 96/100
  • Advanced Algebra (II): 99/100
  • Practice of Programming in C&C++: 93/100
  • Algorithm Design and Analysis (Honor Track): 91.5/100
  • Information Theory: 90/100
  • Mathematical Foundations for the Information Age: 92/100
  • The Brain and Cognitive Science: 91/100