Models

Diafunc supports automatic modeling through Ufinq, our Symbolic Regression technology. The platform analyzes your data automatically and discovers a model for each target. A model shows the relationships in the data and can also be used for predictions.

Symbolic regression is a model-discovery method that searches the space of mathematical expressions for a function that describes a given dataset. The resulting model is a mathematical expression which approximately describes the relationship between the targets and the inputs. Diafunc says target for the column a model predicts and input for a column it reads; other tools and the literature use other names for the two. Also called: labels or dependent variables (targets), features or independent variables (inputs).

It aims at simple, explainable models that generalize beyond the observations (with known target values) they were found on. The result is a white-box model that a person can read, where black-box methods such as deep learning return parameters that cannot be read.

Ufinq, the automatic modeling and symbolic regression technology behind Diafunc, is a long-term research effort. Symbolic regression serves a core aim of data science: extracting understanding from data.

Ufinq

You can build a model from scratch by providing all observations yourself.

Manual Modeling

diafunc
    .model()
    .observe(2, 2)
    .observe(8, 8)
    .observe(50, 50)
    .observe(250, 250);

Modeling usually runs on selections, because they work on every kind of data source (tables, element attributes, ...) and read more fluently.

Modeling based on Selections

selections
    .model()
    .input("x1Key")
    .input("x2Key")
    .target("yKey");

input declares an input and target declares a target: the FDK uses the same words as the rest of Diafunc.

A table becomes a data source by selecting its rows.

Modeling based on Tables

table
    .selectRows()
    .model()
    .input("x1Column")
    .input("x2Column")
    .target("yColumn");

While the search runs, the notebook shows its progress. Afterwards the model's own page reports the figures measured during the search: quality, complexity and error on the rows the search worked with, the time it took and the share of the credit budget it used. A report leaves these figures out of its model chapter, because it leads with the same measures on rows the search never saw.

When modeling completes, the report leads with the model as a formula, says in plain words what it does, and shows how good it is: quality, complexity and error measured on rows the search never saw, and how the formula compares with a straight line through the same data.

Modeling deterministically holds out the first rows of the dataset so the resulting model can be evaluated on data it has never seen. Applying the model to the held-out rows produces actual-vs-predicted pairs that show how well it generalizes, which the training error alone cannot show.

Automatic analyses use the held-out rows to render a scatter chart of actual vs. predicted values inside the per-target Prediction sub-chapter.

Each model exposes importance metrics for each input that explain which inputs the output relies on:

Metric	Description
Permutation importance	How much the model's accuracy drops when the values of a single input are shuffled.
Sensitivity	How sharply the model's output changes as a single input changes.
Occurrences	How many times an input appears inside the symbolic expression.
Residual correlation	Correlation between an input and the model's residual error: a non-zero value hints that the input could still be better exploited.

In automatic analyses, the top five inputs of each model are surfaced in the Findings narrative; the full set is available from the FDK on the model entity.

Models can also be restated in plain English for a non-technical audience. In automatic analyses this restatement is produced by the platform's language model and leads with the top inputs, so a reader who skips the formula still learns which inputs the model relies on most.