Symbolic Regression and Equation Discovery

Search space and discovery targets

• Expression discovery - Symbolic regression searches mathematical expressions as well as coefficients. SRBench separates predictive accuracy, expression complexity and exact recovery; a compact accurate formula need not be the generating equation. La Cava et al. 2021.
• Grammar and complexity - Operators, nesting constraints and expression costs define which equations are searchable. PySR exposes these choices explicitly; changing the grammar changes the problem, not just the optimizer. code.

Evolutionary and structured search

• Genetic programming / Operon - Evolves expression trees and optimizes numerical constants. Operon provides a C++ implementation; combining structural search with coefficient fitting differs from selecting terms in one fixed linear dictionary. code.
• AI Feynman - Combines neural fitting with recursive tests for symmetry, separability and other simplifying structure. Udrescu and Tegmark's search exploits properties of physics formulas rather than treating every expression equally. Udrescu and Tegmark 2020.
  • AI Feynman 2.0 - Uses gradients of a fitted neural model to discover generalized symmetries and graph modularity. Returns accuracy-complexity tradeoffs and extends discovery to probability densities through normalizing flows. Udrescu et al. 2020.
• PySR / SymbolicRegression.jl - Uses multiple evolving populations with an evolve-simplify-optimize loop. Searches expression structure while fitting constants and retaining accuracy-complexity alternatives; Python wraps a Julia backend. Cranmer 2023.

Probabilistic, exhaustive and shared-structure search

• Bayesian machine scientist - Samples equations with MCMC using an approximate model posterior and a prior learned from a corpus of formulas. Retaining alternatives represents uncertainty over expression structure rather than selecting only one best-fitting equation. Guimera et al. 2020.
• Exhaustive symbolic regression - Enumerates expressions within an operator set and size bound, then ranks them by minimum description length. Exhaustive structure search does not ensure a global optimum unless numerical constants are also optimized correctly. Bartlett et al. 2024.
• Multi-level symbolic regression - Learns a shared expression structure with group-specific coefficients. Parallel searches are merged across groups; MSRBench tests this intermediate setting between one pooled equation and unrelated fits for every group. Fong and Motani 2024.

Sparse dynamical-system identification

• SINDy - Selects sparse combinations of candidate functions to model dynamics from measurements. Brunton et al. rely on a suitable function library and state representation; sparsity alone cannot recover terms absent from that library. Brunton et al. 2016.
  • PDE-FIND - Rudy et al. select nonlinear and spatial-derivative terms by sparse regression. Multiple trajectories can distinguish equations that fit one trajectory; derivative estimation and library choice affect recovery. Rudy et al. 2017.
  • SINDy-PI - Searches implicit equations and rational dynamics through parallel sparse regressions. It avoids the noise-sensitive null-space approach of earlier implicit SINDy and extends recovery to studied ODEs, PDEs and conservation laws. Kaheman et al. 2020.
  • Weak SINDy - Uses integrated weak-form equations instead of pointwise derivative estimates. Messenger and Bortz recover dynamics more robustly under white measurement noise in tested systems; test functions and the candidate library remain part of identification. Messenger and Bortz 2021.
• PySINDy - Python implementation of sparse dynamical-system identification. Treat differentiation, candidate-library construction and sparse fitting as parts of the identification procedure, not interchangeable preprocessing details. de Silva et al. 2020.
• Universal differential equations - Combines known dynamics with trainable unknown terms. Rackauckas et al. fit hybrid systems and use symbolic regression to extract missing mechanisms; the equation is constrained by supplied physics and observed trajectories. Rackauckas et al. 2020.

Neural search and pretrained equation models

• Equation learning networks - Sahoo et al. use a shallow network with mathematical operations, including division, to identify concise equations. Extrapolation succeeds in studied analytical systems; it must be tested beyond the fitting range. Sahoo et al. 2018.
• Deep symbolic regression - An RNN samples expressions; risk-seeking policy gradients emphasize high-reward candidates. Petersen et al. optimize the expression search distribution, rather than using the neural network as the final predictor. Petersen et al. 2021.
• Neural-guided genetic programming - A neural generator seeds repeated genetic-programming searches and learns from their results. Mundhenk et al. find loosely coupled seeding effective in their benchmarks; the method combines search mechanisms. Mundhenk et al. 2021.
• Neural Symbolic Regression that Scales - Pretrains a transformer on generated equations and sampled input/output pairs, then uses its predictions to guide equation search. The synthetic equation distribution supplies the learned search prior. Biggio et al. 2021.
• End-to-end symbolic regression - A transformer predicts expressions including numerical constants, optionally followed by constant refinement. Kamienny et al. contrast this with predicting only an expression skeleton. Kamienny et al. 2022.
• Φ-SO: dimensional constraints - Generates expressions with consistent physical units during neural-guided search. Tenachi et al. reduce the candidate space through dimensional analysis; unit consistency alone does not establish a correct physical law. Tenachi et al. 2023.
• LLM-SR - Uses a language model to propose equation programs, fits their constants to data and evolves the candidates using fit feedback. Scientific descriptions provide prior information beyond numeric samples; the evaluation must distinguish discovery from recall. Shojaee et al. 2025.

Comparative evidence and evaluation

• SRBench findings - Across 252 problems, strong real-data methods combine genetic search with parameter fitting or semantic search. Neural and genetic approaches perform similarly on noisy exact-recovery tasks in this study. La Cava et al. 2021.
  • Expanded SRBench comparison - Adds methods and measures accuracy, expression size and energy. Some accuracy plots use the best test score across 30 runs: these describe an optimistic search envelope, not performance of a model selected without test labels. Aldeia et al. 2025.
• SRSD - Rebuilds physics-formula datasets with realistic variable ranges and adds irrelevant inputs. Matsubara et al. show that sampling and evaluation choices change what a symbolic-regression benchmark measures. Matsubara et al. 2022.
• Equation equivalence - Evaluate prediction, complexity and symbolic recovery separately. Equivalent algebraic forms need not have identical expression trees; SRSD's edit-distance metric measures structural similarity, not a general equivalence proof. Matsubara et al. 2022.
• LLM-SRBench - Uses transformed and synthetic equations to probe discovery beyond recall. It contains 239 problems in four domains; the best tested system reaches 31.5% symbolic accuracy on the transformed-equation track. Shojaee et al. 2025.

Related topics

• Feature selection
• Neurosymbolic learning
• Model predictive control
• AutoML
• Interpretability
• Evaluation