Skip to content

Fit

Interactive curve fitting for Linux, Windows and macOS

User guide

Fitting

Fitting a model

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

A fit adjusts the parameters of the model's curves until the calculated profile matches the measured one as closely as it can.

A fit needs a profile to fit and a curve type to fit it with. Everything else - where the curves start, which stretches of the profile are scored, the background - can be prepared by hand or left to the program, which fills in whatever is missing before it starts.

There are three ways to start a fit, all in the Fit menu:

While a fit runs, the chart area shows its progress and every command that would change the model is greyed. Fit > Stop, or the Stop button of the Tools tab, ends it.

The fit itself runs in the compute server the window is connected to (Fit > Compute Server), not in the window. What it makes as small as possible is chosen under Fit > Loss Function, and the algorithm that does it under Fit > Minimizer.

Limitations

See also: Automatic decomposition · Minimize Difference · Why a fit command is greyed · The difference (R-factor) · The Tools tab · Fit progress

Automatic decomposition

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Fit > Automatically decides how many curves the profile needs: it starts with far more than any profile needs and removes every one the fit can do without.

Fit > Automatically - also the last button of the toolbar above the chart - runs five steps in the compute server:

Every curve is of the curve type selected on the Tools tab. When the run ends, the chart, the tables and the Model panel show the curves that remain. The removed curves are gone together with the positions they were placed from, so a later edit or fit does not bring them back.

The setting that decides the answer is that rule. By BIC, the default, the data decide; by Max Acceptable Difference, the limit you set does - ask for less accuracy and you get fewer, broader curves.

It is offered as soon as a profile is open, including before the background has been dealt with - which is exactly what its first step does.

A curve type placed from its own markup is not decomposed as peaks. With nothing marked, its pack finds and places the instances in the data as measured, one fit interval each, and they are fitted; then, as for peaks, the instances that no longer pay for themselves on the fitted model are removed by the Bayesian information criterion, smallest first, and the rest fitted again - though the pack decides which of its instances may go. With a markup already there, the run only fits it, so what you corrected or deleted by hand stays as you left it. No background curve is added to a model whose type takes none. The pack's guide says how it finds them.

Limitations

See also: How many curves · Max Acceptable Difference · Minimize Number of Curves · Fit progress

How many curves

Convention — Common practice or a guideline in the field, not a rule its sources require.

Fit > Remove Curves By chooses how the program decides how many curves a profile needs: by BIC, the default, or within a maximum difference you set.

With enough curves any profile can be fitted perfectly, noise included, so a least-squares fit alone cannot say how many curves there are. Fit starts with more curves than any profile needs and removes them one at a time; the rule decides which removals are kept.

The choice is saved in the project file. A project saved before the choice existed reopens with Max Acceptable Difference, the rule it was made with, so its results do not change.

Each fit interval is reduced on its own. Under either rule a curve whose amplitude has fallen below a thousandth of the largest in the interval is dropped at once, and candidates are taken in two passes: first the curve standing where the measured profile is steepest - on a flank it is often doing the work of a shoulder its neighbours can take over - then the curve with the smallest amplitude. A pass also ends when one curve is left, and a final fit polishes what survived.

By Max Acceptable Difference, a removal that breaks the limit is re-fitted, and if the fit still cannot meet it the curve goes back and the pass ends. Each re-fit stops as soon as the limit is met again, which keeps the method quick.

By BIC, a pass works in sweeps. Each candidate is judged as the model stands, its amplitude handed to the nearest curve; every removal that lowers BIC is kept, and what is left is re-fitted once before the next sweep. When a sweep removes nothing, the five candidates that came closest are each tried with a full re-fit, and the pass ends when none of them lowers BIC. Re-fitting after every candidate, as the limit does, would take hours on a crowded start.

Limitations

Sources

See also: Automatic decomposition · Max Acceptable Difference · Weighting · The difference (R-factor)

Minimize Number of Curves

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Removes every curve the fit can do without, starting from the curves and fit intervals you already have.

Fit > Minimize Number of Curves (F4) runs steps 4 and 5 of Fit > Automatically - the removal passes and the final fit - on the model as it stands. It does not touch the background, and it keeps the positions and fit intervals you placed.

What it needs is filled in only where it is missing: with fewer than two interval bounds it marks a fit interval around each peak, and with no curve positions it places a curve on every point of every peak, as the automatic run does.

Use it when you have placed more curves than you think are needed and want the data to decide which of them stay, by the rule chosen under Fit > Remove Curves By.

Curves it removes are gone together with their positions.

Limitations

See also: How many curves · Automatic decomposition · Why a fit command is greyed

Minimize Difference

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Fits the model as it stands: every curve is kept and only their parameters change.

Fit > Minimize Difference (F5), and the Fit button in the Fit group of the Tools tab, adjust the parameters of every curve until the selected loss function can be made no smaller. No curve is added or removed, and the maximum acceptable difference plays no part.

Anything missing is filled in first, as for the other fit commands: a fit interval around each peak when fewer than two interval bounds are marked, and a curve on every point of every peak when there are no curve positions.

Each fit interval is fitted as a problem of its own. The difference reported afterwards pools all of them into one figure.

Use it after placing curves by hand, after changing a setting such as the loss function or the engine, or to fit again from where the last fit left off.

Limitations

See also: Fitting a model · Loss Function · Minimizer · When a fit ends

Stop

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Ends the fit or other long computation that is running.

Fit > Stop, and the Stop button in the Fit group of the Tools tab, are available only while a computation runs; the rest of the time they are greyed.

The engine is told to end, and does so at its next step. The window then shows what the fit had reached, exactly as it does when a fit finishes by itself: the curves, the tables and the statistics in the status bar are refreshed.

Stopping an automatic decomposition ends its passes where they are: curves already removed stay removed.

Closing the window while a computation runs stops it the same way. The status bar says so, and the window closes once the computation has ended - after asking, as it always does, whether to save the project, which now holds what the computation reached.

Limitations

See also: When a fit ends · Fit progress

Why a fit command is greyed

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

The fit commands are offered only when there is something to fit, the background has been dealt with, and nothing else is running.

The window decides which commands it offers from the state of the compute server. The rules, in the order they apply:

Within the settings, single entries can be greyed with a reason in their hint, shown in the status bar:

Fit > Enable Curve Scaling and Fit > Compute Server are never greyed: they are settings, and a changed setting applies to the next fit.

Limitations

See also: Fitting a model · Automatic decomposition · What the fit will actually do

The difference (R-factor)

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

The number Fit reports for how well the model matches the data is an R-factor: the squared differences divided by the square of the summed data.

Throughout the program "difference" and "R-factor" name the same number. Over the points inside the fit intervals it is

R = sum( (s * calc - obs)^2 ) / ( sum obs )^2

where obs is a measured value, calc the model's value at the same point, and s the curve-scaling factor of the interval (1 when Fit > Enable Curve Scaling is off). With several fit intervals, the sums are pooled over all of them before dividing, so the figure means the same however the profile is divided up.

Dividing by the data makes it a relative measure: the same sample measured for ten times as long gives the same R-factor. A perfect fit gives 0. It is shown as a plain number, such as 7.7E-6 - not in percent - while Fit > Set Max Acceptable Difference asks for its ceiling in percent.

It is the default loss function, so it is usually also what the fit minimises. Whatever loss function is selected, the number reported and the number compared with the maximum acceptable difference is this R-factor.

While a fit runs, it is shown in the status bar and above the progress chart. When the fit ends, the status bar shows the reduced chi-squared and R-squared of the result in its place.

In diffraction the name R-factor usually means the profile R-factors of Rietveld refinement: Rp, the summed absolute differences over the summed data, and Rwp, the square root of the weighted squared differences over the weighted squared data. Fit's number shares their intent - a difference relative to the data - but is neither: it is unweighted, squared, and has no square root, so it is not directly comparable with an Rp or Rwp quoted elsewhere.

Limitations

Sources

See also: Max Acceptable Difference · Loss Function · Curve scaling · Samples no curve covers

Max Acceptable Difference

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

The ceiling on the R-factor that decides how many curves are kept while Fit > Remove Curves By > Max Acceptable Difference is chosen.

Fit > Set Max Acceptable Difference opens the Max Acceptable Difference dialog, which has one field: "Max acceptable difference (percent):". Type the ceiling in percent, with a full stop as the decimal separator, and press OK. The default is 0.01, meaning an R-factor of 0.0001.

Anything that is not a number is refused with "Please enter a valid number (for example, 1.5)." and the dialog stays open. Cancel leaves the ceiling as it was.

The ceiling is used by Fit > Automatically and Fit > Minimize Number of Curves while curves are removed by it: a curve is removed only if the fit interval it belongs to still meets the ceiling without it. Under Fit > Remove Curves By > BIC nothing reads it, and the entry is greyed. Fit > Minimize Difference never uses it.

Choose it near the level the noise of your data allows. Too low, and noise is fitted with extra curves; too high, and real components are merged away.

It is saved in the project file, so a project reopens with the ceiling it was fitted with.

Limitations

See also: How many curves · The difference (R-factor) · Automatic decomposition

Curve scaling

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

With curve scaling on, the model is multiplied by one factor per fit interval so that its area matches the data before the two are compared.

Fit > Enable Curve Scaling is a tick, and it is on by default. While it applies, the built-in engine compares s times the model with the data, where s is the sum of the measured values in the fit interval divided by the sum of the model's. The curves then only have to get the shape right, and the factor takes care of the overall height, which helps the fit converge.

It does not apply in two cases, whatever the tick says:

The setting is saved in the project file.

Limitations

See also: The difference (R-factor) · Loss Function · What the fit will actually do

Minimizer

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Fit > Minimizer chooses the algorithm that searches for the best parameters; the first one, the default, needs nothing extra.

The entries of Fit > Minimizer are the engines this build offers, in order. The first is the default, and is also what a saved choice falls back to when it names an engine this build does not have. The choice is remembered between sessions.

Downhill Simplex (native) is the Nelder-Mead simplex method. It needs only the value of the loss function, not its derivatives, so it fits any curve type, including those that compute their points without a formula. It gives no uncertainties for the parameters.

Levenberg-Marquardt (Python/lmfit) uses lmfit's least_squares method - despite the name, scipy's Trust Region Reflective solver. It reports an uncertainty for each fitted parameter, shown after the value in the Curve Attributes table as "value ± error".

Downhill Simplex, curve by curve (native) runs the same simplex over one curve's parameters at a time: each curve that shapes only its own stretch of the data is a block of its own, and everything that shapes the whole interval - curves that are never zero, such as a Gaussian, and any parameter the curves share - is one block together. The widest block goes first, and the blocks are fitted again until a pass improves the loss by less than one percent; then one simplex over every parameter finishes from there. A model whose curves all shape the whole interval is one block, and gets exactly what Downhill Simplex (native) gives it.

Downhill Simplex, restarted (native) runs the simplex over every parameter, then builds a fresh simplex around the best point it reached and runs again, until a run improves the loss by less than one percent. Its first run is exactly Downhill Simplex (native), so it never ends above it. A simplex stops when its corners agree, which can be on a ridge rather than at the bottom; starting again looks around once more. Each later run steps each parameter forward or backward at random, so every parameter is tried from both sides - the simplex's own restarts only ever turn the first two. The random choices come from a fixed seed, so fitting the same model again gives the same result.

An engine that fits by evaluating a formula is greyed while the selected curve type has none, and if it was selected, the default engine is selected instead. An engine that can only minimise a sum of squares leaves a loss function that is not one to the built-in engine; a dialog and the status bar say so.

Without Python: an installed copy carries the Python engine with it and needs nothing more. A copy built from source needs Python and three libraries set up on the machine that runs the compute server (Help > Compute Backends says how). If they are missing, a fit with that engine does not start: the server answers that the Python backend is not available. Nothing switches engines behind your back; choose Downhill Simplex (native), which needs no Python, to fit.

Limitations

Sources

See also: Compute backends · Weighting · What the fit will actually do · The Curve Attributes table

Loss Function

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Fit > Loss Function chooses the number a fit makes as small as possible; the default, R-factor, suits almost every fit.

Fit > Loss Function is the last entry of the Fit menu. Its entries are the objectives this build offers, the default first; the choice is remembered between sessions. Each entry's hint says what it measures:

R-factor and Sum of squares differ only by a constant factor, so they find the same parameters; only the number differs. Relative deviation sums absolute rather than squared differences, so a few large misfits count for less. R-factor (legacy) divides by the model instead of the data: with curve scaling on, a fit can lower it by inflating the model without matching the data any better, which is why it is greyed for a curve type whose amplitude is free, and replaced by R-factor if it was selected.

Whatever is chosen, the difference the program reports, and compares with the maximum acceptable difference, is the R-factor. The statistics shown after a fit do not depend on the choice either; only the fitted parameters do.

Limitations

See also: The difference (R-factor) · What the fit will actually do · Curve scaling

Weighting

Convention — Common practice or a guideline in the field, not a rule its sources require.

Weighting decides how much each point counts in a fit by the Python engine; counting data are conventionally weighted by one over the square root of the count.

The Fit > Weighting submenu is shown only while an engine that can be weighted is selected under Fit > Minimizer - the Python one. The built-in engine always fits unweighted, so the menu is hidden under it. It has two entries:

The choice is remembered between sessions. The reduced chi-squared shown in the status bar after any fit uses the Poisson weights, whichever engine ran.

Limitations

Sources

See also: Minimizer · Compute backends

What the fit will actually do

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

When a choice cannot be honoured, the fit changes it and says so: always in the status bar, and in a dialog when you make the choice.

The right-hand end of the status bar always says what the next fit will do - for example "Minimising R-factor." or "Fitting with the built-in engine, minimising Relative deviation." When something was changed, hovering over the status bar shows why.

Three things can be changed. The compute server decides with the same rule the status bar reports, so the two cannot disagree:

When you change the engine, the loss function or the curve type and the result would be changed, a dialog titled "About this fit" explains it. It is not repeated while you stay with that selection, and not shown at start-up for a setting chosen earlier. Curve scaling being switched off never opens the dialog; hovering over the status bar explains it.

See also: Loss Function · Minimizer · Curve scaling · The status bar

When a fit ends

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

The chart, the tables and the Model panel are refreshed from the result, and the status bar shows how long the fit took and how good it is.

While a fit runs, the R-factor tab of the chart area draws its progress - and, with View > Animation Mode ticked, the Model tab the model moving - and the tables keep the last finished result. The R-factor curve is kept with the model the run reached, saved in the project, and drawn again whenever that model is opened.

When it ends, whether it finished or was stopped, the window:

The elapsed time counts only the time the computer was awake. A fit left running while the machine slept or hibernated does not count the sleep, and neither does the time axis of the progress chart.

The result is part of the project from then on, and is kept on disk when the project is saved. It is also recorded in the History tab, beside every model an earlier fit reached, so it can be made current again after later changes.

A model put back by value - a project reopened, or a model made current from the History tab - is measured as it comes back, so its R-factor is the one it was recorded with. A model that was never fitted reads "Not calculated", as it did while it was being edited.

Limitations

See also: Fit progress · Animation Mode · The status bar · The difference (R-factor) · Samples no curve covers

Samples no curve covers

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

A point inside a fit interval that no curve of the model reaches is modelled as 0, still counts in the R-factor, and is named where the result is shown.

Some curves exist only over a stretch of the data and are exactly zero outside it - a wave pattern, for instance, runs from its first point to its last. A Gaussian or a Lorentzian is never exactly zero, so it always reaches every point of its fit interval.

Where no curve reaches a point inside a fit interval, the model is 0 there, and the point still counts in the R-factor and in every other figure. On data that sits far from zero - a price series near 1000, say - that one point can outweigh every other difference together: one point left at the start of a series once made a model score seventy times worse over the whole profile than over two intervals that began one point later.

Fit does not drop such points from the figures. A number that changed with how the model happens to be placed would no longer compare one model with another. It says instead where they are. When a fit ends, a dialog gives how many there are, where, and how much of the squared difference between the data and the model they make; the status bar says how many, and the History entry of the model keeps the whole of it.

Before a fit starts - Fit > Minimize Difference or Fit > Minimize Number of Curves - the same is asked first, while it can still be corrected: Yes fits the model as it is, No leaves it untouched so the intervals or the curves can be changed. Fit > Automatically is not asked: it replaces the intervals with its own, and says when it ends what that left out.

The question and the dialog appear once for the same points between them. A later fit that leaves the same points uncovered says so only in the status bar and in History; points elsewhere, or the same ones coming back after they were covered, bring the dialog back.

Two ways forward. To measure only what the model describes, narrow the fit interval to the stretch the curves cover (Model > Fit Intervals > Start Manual Selection, or the Fit Intervals table). To describe the points as well, extend the model over them - move or add a curve so that one reaches them.

Limitations

See also: The difference (R-factor) · When a fit ends · The status bar

Is the difference only noise?

Canonical — Stated by the field's authoritative sources, which the explanation quotes and cites.

A model that explains the data leaves a difference that is noise; after every fit the Ljung-Box test asks whether it is, and the wavelet transform names where it is not.

The R-factor says how large the difference between the data and the model is, not what it is made of. A model can score well and still leave a wave or a peak it does not describe; what is left then follows itself - each point of the difference says something about the next. Noise does not.

After every fit the difference over each fit interval is tested whole. Each point is first divided by the noise expected where it is - the same estimate the automatic reduction weighs curves with - so a quiet stretch and a loud one count alike. The test is Ljung and Box's: the difference's first autocorrelations, up to ten of them or one for every five points, squared and summed. The intervals' tests are combined by Fisher's method into one p.

WHERE it is not noise is found point by point, by the wavelet transform (Torrence and Compo): a Fourier transform over a window that grows with the rhythm looked at, from four points up. At each point it asks how much rhythm of each length the difference holds there, against what noise would hold, and the chance noise holds that much is known exactly. Every point of every fit interval is judged together, held to a false discovery rate of 5 % (Benjamini and Hochberg): tested one by one at 5 %, a thousand points of pure noise would name fifty.

The status bar says "Difference: random (p = ...)" or how many stretches are not, the chart shades those stretches, and the model's History entry keeps the verdict.

Under the model's chart, the Difference structure strip draws that judgement point by point: how much rhythm the difference holds at each point of the fit intervals, against what noise would hold there - 1 for noise. Where the curve reaches the dashed line the point is named, and those are the shaded stretches above it; a spike is a place the model has something left to describe. The difference itself is drawn in the strip too, against its own axis on the right, so a spike of the curve sits over the stretch of the difference that made it. The strip moves and zooms with the chart above it.

A module included in the build may draw the same transform as a chart of its own, and its chapter says how to read it.

A shaded stretch is where the model has something left to describe: a curve to add, one to move, or a fit interval that cuts a feature in two.

Q = n(n + 2) sum_(k=1..h) r_k^2 / (n - k), which for white noise follows a chi-square of h degrees of freedom.

Limitations

Sources

See also: The difference (R-factor) · When a fit ends · The status bar

Compute Server

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

The window does no fitting itself: every computation runs in a compute server, fit_server, and Fit > Compute Server says where it is.

Fit > Compute Server opens a box titled Compute Server that asks for the "Server URL (fit_server must be running there):". It offers the address in use, http://127.0.0.1:8787 - a server on this computer - unless another has been set. Leaving it empty also means that default.

On Windows, the launcher the installed shortcut starts makes sure a server is answering at the default address before it opens the window.

Enter the address of a fit_server running elsewhere - a machine with more cores, say - to fit there. After OK the window checks at once whether anything answers. If nothing does, it says "No server answered at" the address, and "Start fit_server there - the application cannot fit without it." If a server answers and a profile is open, the profile is sent to it.

The address is remembered between sessions. A running fit_server describes its own interface: open the address followed by /docs in a web browser.

Limitations

See also: Compute backends · Minimizer

Compute backends

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

Help > Compute Backends summarises the engines a fit can run on and how to set up the optional Python one.

Fitting runs in the compute server the window is connected to (Fit > Compute Server). Behind it there are two engines, chosen under Fit > Minimizer:

fit_server looks for that environment in a folder named sidecar under %LOCALAPPDATA%\Fit\py on Windows and ~/.local/share/fit/py elsewhere ($XDG_DATA_HOME replaces ~/.local/share when it is set), or under the folder named by the environment variable FIT_PY_HOME when that is set. The libraries and their versions are listed in Worker/py/requirements.txt.

You never start the Python engine yourself: fit_server starts it the first time a fit asks for it and stops it when it exits. The window only ever talks to fit_server.

A third arrangement needs no engine choice at all: the same fit_server on another machine, reached through Fit > Compute Server.

Limitations

See also: Setting up the Python engine · Minimizer · Compute Server · Weighting · Where Fit keeps its files

Setting up the Python engine

This software's choice — A decision made by this software rather than by the field's sources; the explanation says why it was made.

An installed copy needs no setup: the installer puts the Python engine beside the compute server, with its own Python. A copy built from source needs a one-time setup: a private Python environment with numpy, scipy and lmfit at the tested versions.

WHAT AN INSTALLER CARRIES: the engine frozen into one folder, sidecar, beside fit_server - Python, numpy, scipy, lmfit and the parts any module of that product adds - with the licence texts of each in sidecar/licenses. The compute server looks there first, so nothing on the machine is used or changed, and nothing below is needed.

THE FIRST START IS SLOW. The engine starts when you choose it, so it is usually ready before the first fit. The first time it starts after installing, the operating system checks each of its libraries once, and that can take a minute or two; every later start takes seconds. A fit that has to wait for it says so above the chart - "Starting the Python engine" - with the time elapsed. Stop ends the wait the way it ends a fit: the model is kept as it was, nothing is fitted, and the engine goes on starting, so the next fit finds it further along.

From source, you need Python 3.10 to 3.13 and the list of pinned libraries, Worker/py/requirements.txt, from the source repository (https://github.com/dvmorozov/fit). The libraries go into a separate virtual environment at exactly those versions, so your own Python is not touched and fitted numbers reproduce.

On Linux and macOS, from the folder holding the repository:

On Windows, in PowerShell:

--only-binary=:all: stops pip from compiling numpy or scipy from source when your Python is newer than the pinned versions have ready-made packages for; it fails instead, and a Python in the range above is the fix.

The environment belongs to the machine, not to a copy of the program: every installation and every checkout on the machine shares it. Set the environment variable FIT_PY_HOME to keep it elsewhere; it then goes in the sidecar folder under that path.

To check it, start the Python engine by hand with sidecar/bin/python Worker/py/fit_backend.py --port 8788 (on Windows, sidecar\Scripts\python.exe) and open http://127.0.0.1:8788/health in a browser: it answers {"ok": true, ...}. Stop it with Ctrl+C - fit_server starts its own when the engine is chosen or a fit needs it.

Limitations

See also: Compute backends · Minimizer