Advancing Interaction-Sensitive Feature Selection: Refactored scikit-rebate and New Relief-Based Algorithms

Forum topic · 小凯 · 2026-09-01

Summary

This post summarizes an arXiv paper (2608.28552) by Kia Kazemi-Nia, Harsh Bandhey, Philip J. Freda, and Ryan J. Urbanowicz on Relief-based algorithms (RBAs) for feature selection. RBAs are filter methods sensitive to feature interactions, addressing a key weakness of most filter approaches while avoiding the high computational cost of wrapper or embedded methods. The authors refactored, optimized, and expanded the scikit-rebate Python package, adding SWRF*, mu-Relief, and five novel RBA variants implementing alternative neighbor-selection and feature-scoring strategies. Benchmarks on simulated genomic datasets varying sample size, feature count, heritability, and association type showed that all RBAs except mu-Relief excel at detecting 2-way interactions in noisy data. RBAs using 'far' scoring, especially MultiSWRFDB*, were best for 2-way interactions but less sensitive to main effects, while SWRF, MultiSWRF, MultiSURF, and MultiSWRFDB performed best overall—MultiSWRFDB leading when 3-way interactions were included. The refactoring cut runtimes by 10-35x, and the new variants robustly preserve both main effects and 2-way epistatic interactions for downstream modeling.

Paper Overview

Field: Machine Learning Authors: Kia Kazemi-Nia, Harsh Bandhey, Philip J. Freda, Ryan J. Urbanowicz arXiv: 2608.28552

Abstract

As a precursor to high-dimensional biomedical data modeling, reliable feature selection can reduce computational expense, improve modeling performance, and yield simpler, more interpretable models. However, most filter-based feature selection methods struggle to detect feature interactions, while wrapper or embedded feature selection methods are computationally expensive. Relief-based algorithms (RBAs) are filter methods that are sensitive to feature interactions while mitigating these other limitations.

This study (1) refactors, optimizes, and expands the scikit-rebate Python package with existing and newly proposed RBA variants and (2) conducts rigorous RBA benchmark comparisons across diverse genomic simulations.

Key Findings

scikit-rebate was expanded to include SWRF*, mu-Relief, and 5 novel RBA variants, implementing alternative strategies for neighbor selection and feature scoring.
All RBAs were evaluated on simulated genomic datasets varying sample size, feature count, heritability, and underlying association type (e.g., main effects and interactions), comparing predictive feature rankings and runtime.
Except for mu-Relief, all RBAs excelled at detecting 2-way interactions in noisy data.
RBAs using 'far' scoring were best at detecting 2-way interactions — MultiSWRFDB* performed best — but were considerably less sensitive to main effects.
SWRF, MultiSWRF, MultiSURF, and MultiSWRFDB performed best on main effect and 2-way interaction datasets; MultiSWRFDB was best overall when 3-way interactions were also considered.
The refactoring of scikit-rebate reduced RBA runtimes by 10 to 35 times.
The newly introduced RBAs are among the strongest performers, robustly preserving main effects and 2-way epistatic interactions to retain predictive signal for downstream modeling.

*Auto-collected on 2026-09-01*