Skip to content

Preprocessing Cheat Sheet

This page summarizes how dataset defaults and model metadata combine in the current pipeline implementation.

Dataset-side defaults

Dataset type Default variance_threshold Default svd Notes
descriptor disabled disabled Descriptor cleanup focuses on string parsing, dropping all-NaN columns, and median imputation.
fingerprint enabled at 0.01 enabled at 10 components Fingerprints are coerced to numeric and cast to float32.

Model-side switches

Model metadata comes from the selected hyperparameter YAML:

  • requires_scaler: controls whether the configured scaler is inserted
  • use_smote: controls whether the configured sampler is inserted

Current quick-grid models:

  • adaboostclassifier
  • gradientboostingclassifier
  • bernoullinb
  • kneighborsclassifier
  • logisticregression
  • randomforestclassifier
  • extratreesclassifier
  • svc
  • sgdclassifier
  • xgbclassifier
  • lgbmclassifier

Actual order reminder

The pipeline order is not the same as older docs in this repo. The current order is:

  1. polynomial features
  2. non-variance pre-model steps
  3. variance-threshold steps
  4. scaler when requires_scaler is true
  5. dimensionality-reduction pre-model steps such as SVD
  6. PCA
  7. sampler when enabled and use_smote is true
  8. model

Practical examples

  • Descriptor dataset + logisticregression: scaler usually appears, sampler appears if enabled, SVD only appears if the dataset config turns it on.
  • Fingerprint dataset + randomforestclassifier: variance threshold and SVD appear by default, scaler is skipped, sampler usually appears.
  • Any dataset + extratreesclassifier: sampler is skipped by default because use_smote is false in the quick hyperparameter grid.

Common config combinations

Use these levers when you need to change preprocessing:

  • set training.scaler.enabled: false to disable all scalers even for models that request one
  • set variance_threshold: false on a fingerprint dataset to disable the default filter
  • set svd: false on a fingerprint dataset to disable the default dimensionality reduction
  • use training_overrides on a dataset entry when only one dataset should behave differently
  • set training.pca.enabled: true when you want an additional PCA stage after any dataset-defined SVD

Sanity checks

Before a long run:

  • confirm metadata columns like Known, ID, SMILES, and kekule_smiles are dropped
  • confirm descriptor CSV paths use current *_descriptors.csv naming
  • confirm training.grid_search.scoring matches the metric you want to optimize
  • confirm the selected models actually exist in the referenced hyperparameter file