Skip to contents

The chain

Points and rasters to a patch store, the store to a tuned network, the chosen configuration to a final model under several seeds, the model to maps – and what the model learned.

dsm_prepare()
Build a patch store from a point table and a folder of aligned rasters.
dsm_load()
Load everything a run needs, and check that it fits together.
dsm_train()
Fit and tune a model over a resampling plan.
dsm_final()
Refit the configuration a tuning run selects, under N seeds.
dsm_predict()
Predict a map from a final model.
dsm_importance()
Variable importance of a fitted model, by the method you choose.
dsm_report_final()
The declaration of a final model that already exists.

Example data

A small synthetic landscape, and a run fitted on it once a session.

example_landscape()
A small synthetic landscape, for the examples and the tests.
example_run()
A small fitted run on the example landscape, made once a session.

Who trains and who scores

Resampling designs, the fold plans they cut – with a test set and an optional calibration set carved by the same criterion – and the checks of a plan.

spatial_cv()
Spatially blocked k-fold: whole blocks of ground go to one fold.
knndm_cv()
Folds matched to where the map will be predicted (kNNDM).
random_cv()
Random k-fold. Ignores geography by construction.
holdout_cv()
A single train/validation/test split.
region_cv()
Leave-region-out, on a grouping that already exists (biome, catchment, ...).
resolve_resampling()
Turn a spec into a fold plan against real points.
spatial_folds()
Spatially blocked folds of a point table.
knndm_folds()
Fold plan from k-fold Nearest Neighbour Distance Matching.
random_folds()
Random k-fold over the train+validation pool.
holdout()
A single train/validation/test split of a point table.
region_folds()
Fold by a grouping column: no group is split across folds.
prediction_sample()
A regular sample of the prediction area, as kNNDM's predpoints.
check_fold_plan()
Check a plan's folds are well formed, and describe them.
fold_leakage_report()
How much does each fold still leak?

Models, grids and devices

list_models()
Every registered model, as a table.
get_model()
Fetch a registered model.
model_spec()
Describe a model the framework can fit.
register_model()
Add a model to the registry.
caret_spec()
Turn a caret method into a model_spec.
caret_available()
Every caret method that could be borrowed here.
make_tune_grid()
Generate a random hyperparameter grid (like caret's tuneLength).
make_manual_tune_grid()
Build a manual grid from explicit lists of values per parameter.
target_transform_spec()
The forward and inverse functions for a named target transform.
setup_torch_device()
Configure torch threads and select compute device.
set_torch_threads()
Set torch's thread pools.

Choosing a configuration

Repetitions summarised, the noise floor of the seeds, the one-standard-error rule, and a test set scored only after the choice is frozen.

summarise_resamples()
Aggregate a unit-level comparison table into one row per config.
seed_noise_floor()
How much does a metric move when ONLY the seed changes?
print_noise_floor()
Print a noise-floor report in the terms it should be read in.
one_se()
Pick the simplest config within one standard error of the best.
print_one_se()
Say what one_se() did, including when it did nothing.
paired_family_test()
Paired comparison between two model families.
freeze_selection()
Record which config was chosen, and when.
score_test_grid()
Score every trained unit of a tuning run on the held-out test set.
selected_config_id()
The configuration a final run deployed.
latest_run_dir()
The most recent run under base, by time, and only if it finished.

Intervals

Conformal intervals calibrated on cross-validated residuals, on a calibration set (split conformal) or with every fold’s model (CV+), at a constant width or one that follows the level and the dissimilarity index, with points or whole groups weighing alike – and their coverage.

conformal_calibrate()
Calibrate a conformal interval from held-out residuals.
conformal_interval()
Turn predictions into intervals.
conformal_scale_fit()
Fit the scale of an interval: |residual| on covariates that say how wrong.
conformal_scaled_calibrate()
Calibrate an interval whose width is a fitted scale.
conformal_scaled_interval()
Turn predictions into intervals with a fitted scale.
cv_plus_calibrate()
Calibrate a CV+ interval from the folds of a cross-validation.
cv_plus_interval()
Turn the fold models' predictions into CV+ intervals.
conformal_cv()
Calibrate on the folds of a resampling run, then report coverage.
cv_residuals()
Cross-validated residuals from a tuning run, for calibration.
cv_residuals_for_config()
Cross-validated residuals of a configuration, from any tuning run.
picp_report()
Coverage, and whether it holds where it is needed.

The mean surface

The conditional mean of a log-scale model, by Duan’s smearing.

smear()
The conditional MEAN surface, from transform-space predictions.
smear_map()
The mean surface of a median map already made.
smearing_factor()
The smearing factor, from held-out residuals in transform space.
smearing_from_run()
The smearing factor from a tuning run's out-of-fold predictions.
smearing_check()
The smearing factors on held-out points: which one unbiases the mean surface.

Where the map may be believed

The dissimilarity index and the area of applicability (Meyer and Pebesma 2021).

aoa_reference()
The dissimilarity reference of a fitted model.
aoa_di()
The DI of new rows of RAW predictor values, QC'd and scaled as the reference was.
aoa_threshold()
The DI threshold that separates the area of applicability.
inside_aoa()
Is each row inside the area of applicability?
print_aoa()
Report what an AOA covers.
di_reference()
Summarise a training set for dissimilarity-index computation.
dissimilarity_index()
The dissimilarity index of new data.

What the model learned

Importance by permutation, SHAP, SAGE, refits and the patch’s context; accumulated local effects; SHAP over the map.

permutation_importance()
Permutation importance: how much a fitted model relies on each variable.
shap_importance()
SHAP importance: how each variable pushes each prediction, up or down.
sage_importance()
SAGE: how much of the model's skill each variable carries, shared fairly.
refit_importance()
Refit importance: what the model loses when trained without a variable.
context_importance()
Context importance: how far from the point, and at which scale, the model reads.
ale_effect()
ALE: how the prediction changes along each variable's range.
compare_importance()
Several importances, side by side.
importance_groups()
Which channels the importance treats as one variable.
importance_points()
Points of the map to explain: a regular grid of the model's raster cells.
importance_map()
Maps of SHAP values, from an importance computed at points of the map.
importance_weights()
One weight per channel, for the area of applicability, from an importance.
occlusion_report()
What each part of the patch is worth to one trained unit of a tuning run.
plot(<dsm_importance>)
Plot a variable importance.
plot(<importance_comparison>)
Plot importances side by side.
plot(<importance_map>)
Plot maps of SHAP values.

Errors on clustered points

Intervals for a difference in error, drawn from whole blocks of equal area.

block_bootstrap()
A metric, or two models' difference in it, with an interval from whole blocks.
block_bootstrap_by_size()
The interval as the blocks grow.
equal_area_blocks()
Square blocks of equal area, in kilometres.
spatial_correlogram()
Moran's I by distance class: how far apart two points stop resembling each other.
plot(<block_bootstrap_by_size>)
Plot the interval as the blocks grow.
plot(<spatial_correlogram>)
Plot a spatial correlogram.

Metrics

calc_metrics()
Compute all regression metrics for one obs/pred pair.
ccc()
Lin's Concordance Correlation Coefficient.