Symbolic Regression, explained

Symbolic Regression finds a formula that explains a target from its inputs, and it finds the form of the formula too: which inputs appear, which operations join them, and the constants inside. Other kinds of regression start from a form someone wrote down and learn only its numbers. Symbolic Regression searches for the form.

This page explains what that means, how such a search works, where it helps and where it does not, and how Diafunc uses it. The example is real: a public analysis of 308 measurements of yacht hulls.

A linear regression with two inputs is a form chosen in advance:

The data decides only a, b and c. If the true relation curves, multiplies two inputs or levels off, the straight form cannot follow it, however well its three numbers are chosen. A nonlinear regression or a curve fitter widens the choice of forms, but the form is still yours to name before the data is read.

Symbolic Regression leaves the form open. Given a target, the inputs, a set of operations and room for constants, it searches the space of formulas for one that predicts the target well and stays small. What comes back is the formula itself, written in the names of your columns:

Nobody wrote down a hyperbolic tangent, or the ratio between the Froude number and the position of the center of buoyancy, before the search began. Both were found. So was the decision to leave three of the six inputs out.

The building blocks. A formula is assembled from the inputs, constants, and operations such as addition, multiplication, division, powers, logarithms, exponentials and trigonometric functions. Each formula is a small tree: operations at the branches, inputs and constants at the leaves.

The search. The number of possible formulas grows explosively with their size, so no search can try them all. The classic approach is evolutionary: start with many random small formulas, let the better ones produce variations (swap a branch, change an operation, combine two candidates), and repeat over many generations. Candidates rise and fall; the form of the model emerges from the competition.

The constants. A formula's shape and its numbers are judged together. Once a candidate has a shape, its constants are tuned to the data, so that a good shape is not lost to a bad first guess of its numbers.

Two goals at once. Every candidate is scored on how well it predicts and on how small it is. A formula that adds a term must earn it with better predictions. Without the second goal a search would grow formulas until they memorised the data, and the result would be neither readable nor reliable.

Rows held back. How well a formula predicts is only worth knowing on rows it never saw. A sound analysis sets part of the table aside before the search starts and measures the final model there. In the Yacht analysis, 62 of the 308 rows were held out.

Ufinq is the Symbolic Regression technology Diafunc runs on, and evolutionary search is its foundation, not all of it. Every candidate is tuned and simplified before it competes. Decompositional learning splits hard problems apart and recomposes them. The search runs at scale, across generations of competing programs, and one model comes back, as simple as the data allows. The research, papers and benchmarks are public at ufinq.com.

The model above comes from a public analysis of the Yacht Hydrodynamics table: 308 towing-tank measurements, six inputs describing the hull and its speed, and the residuary resistance as the target. On the 62 rows the search never saw, the model explains 99.3 % of the variation in the target (R² 0.9935). A straight line through all six inputs, measured on the same rows, explains 67 % (R² 0.667).

Reading it tells you more than the figure does. Resistance rises ever more steeply with the Froude number, a dimensionless measure of the hull's speed. It falls with the prismatic coefficient and rises with the position of the center of buoyancy. The length-displacement ratio, the beam-draught ratio and the length-beam ratio do not appear: within this table they add nothing the three others do not already say. An engineer can check each of those statements against what they know about hulls, which is the point.

The report does not stop at praise. It warns that the model under-predicts by 0.282 on average on the held-out rows, and that the Froude number almost determines the target on its own. Both are true of the data, and both are worth knowing before the model is used. The full walk-through is in How to find an equation from data.

None of these is the better method in general. A linear model is the right answer when the relation is straight, and the report shows when it is: it scores the discovered model against a straight line on the same unseen rows. For the difference from neural networks in more depth, see Symbolic AI and deep learning.

Tables with named columns and a target. Measurements, records, experiments, operational data: anything where each row is a case and each column a quantity.

A relation a compact formula can carry. Physical and engineering relations, dose and response, prices and quantities, rates and saturation. Many relations in the world are of this kind, which is why laws are written as formulas.

Several inputs, or a form nobody can name. With one input and a known curve, a curve fitter is quicker and enough. Symbolic Regression earns its time when the form is unknown or the inputs interact.

When someone will ask why. A formula can be checked against what you know, explained to a colleague, and moved into a spreadsheet or a few lines of code.

Images, sound and free text. These have no named columns to build a formula from. A neural network is the established tool there.

A target that depends on what is not in the table. No search can find a cause that was never recorded. A weak model is then an honest answer: the inputs do not carry the target.

No guarantee of the true formula. The search finds a formula that explains the data well and stays small. When two inputs move together, it may use one and leave the other out; a formula describes the data, it does not prove cause. And it is no safer than any other model far outside the range of the data it saw, though you can at least read what it will do there.

It takes time. A search for a formula is far more work than tuning a fixed form. On Diafunc an analysis runs for the time you give it, from one hour; the chapters about the data arrive first.

Small tables. With few rows there is little to learn from and little to hold back. From about 150 rows a Diafunc report grades its figures; below about 60 it says the model itself is not to be relied on.

The idea of evolving formulas came with genetic programming, which John Koza described in his 1992 book Genetic Programming: programs, formulas among them, bred and selected like organisms. In 2009 Michael Schmidt and Hod Lipson showed in Science that such a search could recover physical laws from motion data, which brought the method wide attention. Since then many libraries and products have appeared, and in 2021 the field gained a shared benchmark, SRBench, on which methods are compared on the same datasets. An overview of the tools that exist today is in Symbolic Regression software.

Diafunc is a Symbolic AI Platform. You upload a table, choose the target, and optionally ask questions; an automated analysis discovers one readable model per target and writes the report around it: quality on held-out rows, the comparison with simple baselines, what each term does, and the concerns the analysis has about its own model. The technology underneath is Ufinq. It is benchmark-tested: evaluated on SRBench, the standard Symbolic Regression benchmark, with the results public at ufinq.com/benchmarks.