Analyses
An analysis is a guided run powered by the Ufinq symbolic regression technology. The resulting report is a notebook inside a Diafunc project: editable, rerunnable, and shareable.
Starting an analysis
1. Project and data
CSV and Excel work best, and every other format Diafunc imports is accepted too: machine-learning, statistics, scientific and measurement files upload as they are. The data should be in a simple tabular format, with the column names in the first row or stored in the file itself.
Supported file formats
.data with .names), columnar (Parquet, Arrow, ORC, Avro), semi-structured (JSON, JSONL, YAML, XML, HTML, PDF), databases and archives (SQLite, DuckDB, Access, SQL dumps, zip, tar), scientific (HDF5, NetCDF, NumPy, MATLAB, LAS well logs, TDMS, MDF, FITS, NASA CDF, ROOT), statistical software (SAS, SPSS, Stata, R, gretl), official statistics (SDMX, JSON-stat, PC-Axis), map data (GeoJSON, shapefiles, dBase) and Python pickles, which are read without running any code. Gzipped / bzipped / xz-compressed variants are unpacked automatically. The format is detected from a combination of magic bytes, MIME type, and file extension; you can usually drag the file straight into the upload area without configuring anything. See the full supported file formats reference for the per-format matrix, the inner-table picker behavior for multi-table sources, and the known limitations of every importer. For dialect quirks (encodings, decimal separators, missing-value conventions, flattening nested JSON / YAML / XML) see preparing your data.
Several files
Stacking:
Files with the same columns are stacked into one table. The order of the columns, upper and lower case, and surrounding spaces do not matter. A column that one file lacks is named, and its cells are empty on that file's rows.A file that adds columns:
A file with other columns that shares a key with the data, such as a columnbatchwhose values occur in both, adds its columns to the rows that match. The key is found from the column names and their values, and the plan states it with the share of rows that found a match. No row is added or lost: a row without a match keeps empty cells in the added columns. Keys are compared as they are written; comparing without regard to case, leading zeros and number format is an option the plan offers when it would match more rows. You can change the key or leave the file out.Several rows per key:
When the added file has several rows for one row of the data, such as sensor readings per batch, it is first reduced to one row per key: a column of numbers becomes its mean, a column of labels its most frequent value, a timestamp its first and last value, and one more column counts the rows each key had. Each column has a selector to change this or leave the column out.Which file carries the rows:
Batches and readings per batch can be read as one row per batch or as one row per reading. Diafunc proposes one reading from the names and statistics of the columns, never from their values, and one button switches to the other. Choose the file in which one row is one observation of what you want to understand.What is left out:
A file that shares no key with the data, a key that matches too few rows, a key that repeats on both sides, and a file too large to add columns are left out, and the plan says which and why.The file column:
The table gets a column namedfilethat records which file each row came from. It is often real information (the machine, the site, the month) and can be used as an input or as the segment. You can switch it off.Rows in more than one file:
When exports overlap, a row that occurs value for value in more than one file is imported once, and the number of such rows is stated. Rows repeated inside one file are always kept. You can switch this off.What can be combined:
Every supported format can be combined with every other, and the sheets of a workbook or the tables of a database count as tables of their own. A zip or tar archive counts as its files, so an archive of part-files is stacked the same way; folders namedkey=valuebecome columns. Only a format read by a user-defined importer has to be uploaded on its own.
A file in another shape
One column per year, month or numbered step:
A table with the columnssite, 2019, 2020, 2021holds one yield per site and year, spread over columns. It is read with one row per site and year, in the columnssite,yearandyield.A header over several rows:
When the first rows of the data still hold names, units or groups, they become part of the column names and leave the data.Several numbers in one cell:
A column such asblood_pressureholding120/80becomes two columns of numbers.
The reshape is a function in your project. When the data is built, the file is imported as uploaded into one table, and a function named "Reshape" and the file's name writes the reshaped table. You can open that function in the Lab and change it; building the data again runs it as you left it. The report states that the file was read reshaped, with its size before and after.
A file that is fetched. A file does not have to be uploaded. Add from a data source in the upload step fetches it from a Google Sheet, an HTTPS or S3 address, or a database query, and keeps it up to date. It is stacked, matched by a key or reshaped like any other file, and every refresh builds the data again the same way; a refresh whose data no longer fits fails and leaves the data as it is. See
CSV files
File extension:
Use the file extension ".csv".Headers:
Put a short, clear and consistent header for each column in the first row.Encoding:
Save the file as UTF-8, which works across systems and supports every character.Avoid special characters:
Avoid special characters: they can break the import and make the data harder to read.
CSV Example File
Excel files
File extension:
Use the file extension ".xlsx".Headers:
Put a short, clear and consistent header for each column in the first row.No formatting:
Avoid cell formatting such as colors, borders or conditional formatting.Single sheet:
One sheet per file keeps it simple; a workbook with several sheets asks which one to import.No macros or pivot tables:
Do not use macros, pivot tables, charts or other advanced Excel features; keep the data raw.Data types:
Keep one type of data per column (all numbers or all text), and use only the text and number types. Numbers stored as text are fine.
Excel Example File
2. Targets and questions
Targets: pick one or more columns you want to explain or predict. Most analyses focus on a single target; two or three is the realistic upper end. The last column of your data is preselected by convention.
Questions: optionally add natural-language questions about your data (for example "Which columns are most related to customer churn?"). The assistant answers each question in a dedicated Questions & Answers chapter of the report, grounded in what the analysis found.
Columns and segments: closed by default, because most analyses need nothing from it. It holds three choices: columns to leave out of the model, columns you confirm as known before the target, and a column to segment by. While closed it says what is set, including any column Diafunc left out by itself, such as an identifier. It opens by itself when something in it wants your answer: a column Diafunc suggests leaving out, or a concern under Before you launch about one of your columns.
Names the data no longer has: a target, a column left out, a confirmed column and the segment are stored by column name. When the data of an analysis you have not launched yet is replaced outside the wizard, for example by a new upload from the command line or a column renamed in the Lab, a stored name may no longer be a column. The wizard marks each such name where its setting is, with Not in the data, and names them all next to the launch button, which stays disabled until you have removed them or chosen columns the data has. Nothing is removed for you. A name has to match its column exactly, including upper and lower case and spaces.
3. Depth
| Depth | Credits | Longest time | Best for |
|---|---|---|---|
| Express | 500 | 1 hour | A quick first look at a small dataset |
| Compact | 1,000 | 1 hour | |
| Quick | 2,000 | 2 hours | First-pass exploration |
| Standard | 4,000 | 4 hours | Most analyses of everyday datasets |
| Thorough | 8,000 | 8 hours | Datasets with many columns or subtle relations |
| Extended | 16,000 | 16 hours | |
| Deep | 32,000 | 1.3 days | Problems where quality matters most |
| Profound | 64,000 | 2.7 days | |
| Extensive | 128,000 | 5.3 days | |
| Comprehensive | 256,000 | 10.7 days | |
| Vast | 500,000 | 3 weeks | |
| Grand | 1,000,000 | 6 weeks |
Express is selected for an account's first analysis, Standard afterwards. Any depth can be chosen. The scale marks the steps your balance does not cover, and they stay selectable. From Profound on an analysis runs for days and shares the server with other analyses, so it may use less than its budget in that time; if the run is interrupted, its search starts again from the beginning and the interrupted attempt is not charged. The same scale comes back when an analysis is run again or refined, so a depth is chosen the same way everywhere.
Before you launch
While it runs
| Phase | What happens |
|---|---|
| Waiting for a slot | Queued. Diafunc limits how many analyses run at once; nothing is needed from you. The card says which of two waits it is: behind your own running analyses, where it starts when one of them finishes, or behind a busy platform, where it starts on its own as soon as a slot frees up. |
| Preparing | Diafunc sets up the project, reads your data and lays out the report notebook. |
| Profiling your data | Column descriptions, correlations and the first charts. This is where the earliest chapters appear in the notebook. |
| Discovering formulas | The symbolic search itself, one search per target column. Usually the longest phase: the searches share what the analysis's hours leave after preparing and profiling your data, less a short reserve for the charts and report that follow. |
| Building charts and diagnostics | Input importance, predictions and held-out results for each discovered model. |
| Writing the report | The headline, the stakeholder summary and the narrative chapters. Several of these are language-model steps, so this takes minutes rather than seconds. |
The wizard shows these phases as a rail above the progress bar, with the phases already passed, the one the run is in, and the ones still to come, so you can see where the run is without reading a percentage. Above the rail, a row of figures keeps the run's facts in view: how long it has been running, at most how long it has left, the credits spent against the budget, and during the search the current generation and the quality reached so far.
Several analyses at once
The report
| Chapter | Description |
|---|---|
| Evidence (Analysis) | Everything the analysis found, layered from data to conclusions. |
| Headline | The strongest findings at a glance, and the first thing a reader sees. |
| Concerns | Every issue the built-in detectors raised about the data or the models, collected in one place. Read this before trusting anything else. |
| Discovered Relations | The headline formulas Diafunc found for your targets. |
| Questions & Answers | An answer to each of the questions you asked in the wizard. Only present if you asked any. |
| Data | What is in your table. Read this to judge how far to trust the models. |
| Variable | A chapter per analyzed variable with Description, Correlation, and Visualization sub-chapters, Class Balance for non-numeric variables, and an Over Time view when the data has a date column. |
| Column Profiles | A compact profile of every column in one overview. |
| Column · name | Per-column detail. Free-text columns add token statistics and an embedding view; coordinate pairs add a Geospatial · lat × lng map. |
| Duplicate Rows | Duplicated rows in the dataset and how much of it they make up. |
| Leakage Suspicion | Columns that track a target almost completely, with the question whether each is known before the target, and the columns you have confirmed as such. |
| General | Dataset-wide description: Missing Values, Outliers, Correlation with Potential Data Leaks, and Clusters. A Segments chapter (Segment Distribution, Segment Profile, Target by Segment) is added when you picked a segment-by column. |
| Model | One chapter per target, plus cross-target chapters when you picked several. |
| Target | The discovered model: the formula, its metric gauges, a Prediction chapter that applies the model to held-out rows with a scatter chart of actual vs. predicted values, and the diagnostic chapters described below. |
| Composite Model | Several targets only: the per-target models assembled into one model of all targets, with the per-target chapters nested under it. |
| Leaderboard | Several targets only: all targets side by side against simple baselines. |
| All targets together | Several targets only: the quality across the targets and how the targets' errors relate. |
| Implications | What the results mean and what to do next. |
| Stakeholder Summary | A plain-language summary for readers who won't open the technical chapters. |
| Recommended Next Data | Which data would improve the models most if you collected it next. |
| Quality by Input Range | Model quality broken down by automatically chosen ranges of each input, to show where the model is weaker. It holds one Target by Input Range chapter for each target. |
| Per-Segment Predictions | Only if you picked a segment-by column: how each model predicts within each segment. |
| Fairness | Per target, how the model behaves across different groups of rows. |
| Rows unlike the ones the model learned from | How to recognize future rows that lie outside what the model was trained on. |
| Causal Honesty | What the formulas do and do not say about cause and effect. |
| Deployment Readiness | A checklist of what has (and hasn't) been verified before you rely on the model. |
| Provenance & Audit | What produced this report: the data, configuration, and models behind every finding. |
| Foundation | The data and the function that underlie the analysis. |
| Raw Data | The dataset exactly as you uploaded it. |
| Analysis Function | The function that produced this report, rerunnable on new data. |
| Processed Data | The cleaned, typed, and normalized dataset used for modeling. |
| Correlation Matrix | One row per pair of numeric columns with its correlation measures; every correlation in the report is read from it. |
| Held-out Rows of Target | Per target, the held-out rows the predictions were made for: Input Extraction and the Input Table hold the inputs of those rows, next to the function that predicts them. |
Target kinds
| Kind | The target looks like |
|---|---|
| Regression | A continuous number: the classical case. |
| Count | Non-negative whole-number counts. |
| Ordinal | A small set of ordered levels, such as ratings from 1 to 5. |
| Binary classification | A yes/no target. |
| Multiclass classification | A category with several possible values. |
| Time series | A number paired with a date column, judged in temporal order. |
| Several targets | Several targets modeled together as one composite model. |
| Structured target | A tuple-shaped target with meaningful internal structure. |
| Dense target | A matrix or tensor target. |
Several targets, one composite model
Formula explanations
The formulas come from Ufinq, our symbolic regression technology, which also runs standalone modeling in Diafunc projects.
Each model also has gauges summarizing its key metrics. On a model's own page, a model that came from an analysis leads with Quality, Complexity and Error measured on rows the analysis kept out of the search, the figures the report shows; the search's own figures follow under During the search, with Time and Credit budget. Under the gauges a line states the rows behind them, for every table: "Found on 30 rows · measured on 10 rows the search never saw". Hover a gauge, focus it with the keyboard, or tap it to see what it means and how to read it.
| Metric | Description |
|---|---|
| Quality | How well the formula predicts, marked down for the size of the formula. For a continuous target it sits at or just below R², and the gap between the two is what the formula's size costs: small for a formula fitted on many rows, visible for a long formula fitted on few. 0% means no better than always predicting the average, 100% means exact. Higher is better. |
| Complexity | How long the formula is, as a share of the complexity limit. Lower means a simpler formula. |
| Error | How far the predictions lie from the held-out values, as a share of the miss of always predicting the average: 0% is exact, 100% is no better than the average. For a count target the comparison is always predicting the average count, and for a yes/no or category target it is always predicting the most common class, so a formula that only repeats the most common class reads 100%. The gauge's tooltip says which comparison a figure uses. |
| Time | The search time used of the time allowed, and why the search stopped. |
| Credit budget | The credits used of the credits allowed for the search. Only the credits used are charged. |
Model diagnostics
| Chapter | Description |
|---|---|
| Which inputs matter most | The inputs the model relies on most, ranked (input importance). |
| Baselines | The model against the simple yardsticks of its kind of target (for a number: the mean, the median and a linear regression; for a yes or no: the most common class and a logistic regression): a formula is only interesting if it beats them. |
| Generalization | The figures on the training rows against those on the held-out rows: does the formula hold on rows it never saw? Below 30 held-out rows the chapter gives no verdict, however high the figure: it reads as not yet enough evidence. |
| How each term responds to its inputs | One curve per term of the formula over its inputs (per-term partial dependence). |
| Contributions per term | For a single row, how the terms add up to the prediction. |
| What each term is worth | How much quality each term contributes: what would be lost by dropping it. |
| What-If | Move the inputs and watch the prediction respond. |
| Sensitivity | How strongly the prediction reacts to each input. |
| How two inputs act together | Pairs of inputs whose combined contribution to the prediction differs from their separate contributions. |
| How the formula behaves outside the data | What the formula does as each input leaves the range of the training rows: where it grows without bound, levels off or is undefined. |
| Worst Predictions | The held-out rows the model gets most wrong, usually worth inspecting by hand. |
| Search Progress | Quality on the training rows over the generations of the search: did it converge, or did the time limit or the credit budget run out first? |
| Model Constraints | Mathematical-hygiene checks on the held-out predictions: finite outputs, within-domain values, no runaway predictions. |
Reading the data chapters
Missing Values: shows each column's row count against the dataset total. If a column is missing more than about 30% of its values, treat any formula that relies on it with caution: the model learned from whatever rows happened to have a value, which may not be representative.
Outliers: for every numeric column, counts the values that lie far outside the middle half of the column: more than 1.5 times the interquartile range below the first quartile or above the third. The table gives the two limits, how many values lie beyond each, their share of the column and the values that lie furthest out. The box plots draw the same values as points beyond their whiskers. A few extreme rows can pull formulas around. Consider whether those rows are data errors, rare-but-real events, or a signal worth modeling separately. A notebook created before October 2026 has an Outlier Ranges chapter in its place, which lists each column's quartiles beside its full range.
Class Balance (for non-numeric targets): shows the row count per target class. If one class holds more than about 90% of the rows, a model can look accurate only by always predicting the majority class. The analysis flags this so you can judge quality correctly.
Potential Data Leaks: when a column tracks a target almost completely (the stronger of Pearson's r and Spearman's rho at 0.95 or more), the report asks whether the column is known before the target. The coefficients cannot tell: a strong input whose value exists before the target's reads the same as a column computed from the target or recorded after it. A leaked column makes the discovered formula look right on these rows and fail on new data, because the column is not available when a prediction is needed. You answer the question on the concern with "known before the target", under Columns and segments in the Analyze wizard, or on the command line; a confirmed column raises no leakage concern. A column at 0.999 or more reproduces the target, which is what a copy looks like, and is critical when the model reads it.
Segments (only if you selected a segment-by column): the Segment Profile shows descriptive statistics grouped by segment, and the per-target Target by Segment chapters show how target values differ between groups. Look for variables whose mean, spread, or missing-value rate changes substantially between segments; those shifts are often more actionable than the aggregate formulas.
A chart of any two columns: the report draws each column's own charts, its relation to every target, and the strongest column pairs. Any other pair is drawn when you ask for it. Each column's chapter closes with Chart against another column, where that column is already chosen and you pick the other; Chart two columns, in the Data part before Further Charts and with its own entry in the chapter list, takes any two. The column list can be searched and states each column's type and role. The chart is the one the report would draw for the pair: a scatter for two numbers, box plots for a number by the groups of a category, a heat map of counts for two categories; where more than one fits you can switch between them. Above the chart the report says how many rows have a value in both columns, and when a pair cannot be charted it says why. The pair is part of the page address, so the link you copy opens the same chart. Every reader of the report can do this, signed out on a public project included, and it costs no credits. As an editor of the project you can add a drawn chart to the report with Keep in the report: it is written under Further Charts at the end of the Data part, for every reader and in the PDF and EPUB, and is drawn again from the new data when the analysis runs again. A row of the Correlation table opens its pair with Chart.
Reading the prediction chart
Points on the diagonal are accurate predictions. Vertical offsets reveal bias (the model systematically over- or under-predicts), vertical spread reveals noise (how much individual predictions vary), and points far from the diagonal are cases the model handles poorly, often the rows worth inspecting by hand.
After it runs
Sharing a report is not the same as making the project public. A public project is readable in full by anyone, its data and code included. It appears in Explore, and its notebook can be found by search engines. That is a separate choice in the project's settings, and a public project needs a license that says what other people may do with it.
Ask opens an Assistant session scoped to the analysis. The assistant has the findings in context, so follow-up questions are answered from the results of your run.
Run again or refine
Run again starts a new run of the same analysis on its data as it is now, with the same targets, columns and settings. It opens with the wizard's depth scale, set to the depth the analysis has, so a run can be given more time or fewer credits; the credits remaining on your account are shown beneath them. A depth you change is kept for the analysis, and its scheduled runs use it too. The current report stays readable while the run works and is replaced only once the new one is finished.
Refine starts a new analysis from this one, on a copy of its data with your changes applied: other targets, columns dropped, a segment, a focus on one value of a column, and a depth chosen on the same scale. The analysis you started from and its report stay exactly as they were, and the two can be compared. For a change a run cannot make, Run again points you to Refine instead.