Fei Lu

Associate Professor, Department of Mathematics, Johns Hopkins University

My current research focuses on learning dynamics from data. One topic is nonparametric learning of the interaction laws in systems of interacting particles/agents, and another is data-driven model reduction for complex systems in computation such as fluid dynamics and molecular dynamics simulation. I view dynamical systems as a description of stochastic processes and take an inference approach to learn the dynamics from data, so I am also interested in closely related topics such as data assimilation, sequential Monte Carlo methods, deterministic and stochastic dynamical systems and PDEs, ergodicity theory, and learning theory.

Learning interacting particle systems Data-driven stochastic model reduction Data Adaptive RKHS Regularization

Data assimilation and inference Malliavan calculus and SPDEs

---

Data Adaptive RKHS Regularization for linear inverse problems

Regularization is crucial for ill-posed machine learning and inverse problems that aim to construct robust generalizable models. The classical Tikhonov regularization $$ \| Ax-b \|^2 + \lambda \| x \|_*^2 $$ has two components a regularization norm setting the hypothesis space and the hyper-parameter setting the strength. There is a large literature on selecting the hyper-parameter. However, there is a relatively small amount of research on selecting the regularization norm, not to mention making the norm adaptive.

DARTR works on selecting the regularization norm adaptive to the problem and/or the data. It utilizes the norm of a data-adaptive RKHS ensuring that the solution is inside the function space of identifiability. When combined with the L-curve method, DARTR leads to accurate convergent estimators robust to numerical error and noise. It has been shown to outperform the L2-regularizer.

DARTR for learning kernels in operators.

⭐ ⭐ ⭐ F. Lu, Q.Lang and Q. An. Data adaptive RKHS Tikhonov regularization for learning kernels in operators.

MATLAB code
This project focuses on a vector estimator that views the kernel as a vector on the grid points (equivalent to using piecewise constant basis functions on grid points). In this setting, there is no additional regularization from the basis functions. We illustrate its performance in examples including integral operators, nonlinear operators, and nonlocal operators with discrete synthetic data. Numerical results show that DARTR leads to an accurate estimator robust to both numerical errors due to discrete data and noise in data, and the estimator converges at a consistent rate as the data mesh refines under different levels of noises, outperforming two improved baseline regularizers using $l^2$ and $L^2$ norms.

F. Lu,Q.An and Y. Yu Nonparametric learning of kernels in nonlocal operators.
This project focuses on kernels in nonlocal operators. It uses B-spline basis functions. We systematically study the method with synthetic data, showing the convergence of estimators. Furthermore, the method successfully learns a homogenized model for stress wave propagation in a heterogeneous solid, revealing the unknown governing laws from real-world data at the microscale. Our DARTR outperforms baseline methods in robustness, generalizability, and accuracy in synthetic and real-world datasets.

⭐ ⭐ ⭐ Quanjun Lang and F.Lu. Small noise analysis for Tikhonov and RKHS regularizations. arXiv2305   PDF  

F.Lu and Miao-Jung Yvonne Ou. An adaptive RKHS regularization for Fredholm integral equations. arXiv2303   PDF  

Iterative methods

⭐ ⭐ ⭐ Haibo Li, Jinchao Feng, and F.Lu. Scalable iterative data-adaptive RKHS regularization. arXiv2401   PDF   MATLAB code

Data adaptive RKHS priors for Bayesian inverse problems

Neil K. Chada, Quanjun Lang, F.Lu, and Xiong Wang. A data-adaptive prior for Bayesian learning of kernels in operators. arXiv2212   PDF