Novel Relief-Based Feature Selection Algorithms: Refactored scikit-rebate and Genomic Benchmarks

Forum topic · 小凯 · 2026-09-01

Summary

This paper (arXiv:2608.28552) by Kazemi-Nia, Bandhey, Freda, and Urbanowicz refactors and expands the scikit-rebate Python package for Relief-based feature selection algorithms (RBAs) and benchmarks them on simulated genomic data. RBAs are filter methods sensitive to feature interactions, addressing a key weakness of most filter approaches while avoiding the computational cost of wrapper or embedded methods. The authors add SWRF*, mu-Relief, and five novel RBA variants implementing alternative neighbor-selection and feature-scoring strategies. Across simulations varying sample size, feature count, heritability, and association type (main effects, 2-way and 3-way interactions), all algorithms except mu-Relief excelled at detecting 2-way interactions in noisy data; 'far'-scoring variants were best at 2-way interactions (MultiSWRFDB* leading) but weaker on main effects, while SWRF, MultiSWRF, MultiSURF, and MultiSWRFDB performed best overall, with MultiSWRFDB leading when 3-way interactions were considered. Refactoring reduced runtimes by 10–35x, making interaction-aware feature selection more practical for high-dimensional biomedical modeling.

Overview

Field: Machine Learning
Authors: Kia Kazemi-Nia, Harsh Bandhey, Philip J. Freda, Ryan J. Urbanowicz
Published: 2026-08-28
arXiv: 2608.28552

Reliable feature selection is a precursor to high-dimensional biomedical data modeling: it can reduce computational expense, improve modeling performance, and yield simpler, more interpretable models. Most filter-based methods struggle to detect feature interactions, while wrapper and embedded methods are computationally expensive. Relief-based algorithms (RBAs) are filter methods that are sensitive to feature interactions while mitigating these other limitations.

Key Contributions

1. Refactored and expanded scikit-rebate: The Python package now includes SWRF*, mu-Relief, and 5 novel RBA variants implementing alternative neighbor-selection and feature-scoring strategies. 2. Rigorous benchmarking: All RBAs were evaluated on simulated genomic datasets varying in sample size, feature count, heritability, and underlying association type (main effects and interactions), comparing predictive feature rankings and runtime.

Findings

All RBAs except mu-Relief excelled at detecting 2-way interactions in noisy data.
RBAs using 'far' scoring were best at detecting 2-way interactions — MultiSWRFDB* performed best — but were much less sensitive to main effects.
SWRF, MultiSWRF, MultiSURF, and MultiSWRFDB performed best on datasets with main effects and 2-way interactions; MultiSWRFDB was best overall when 3-way interactions were considered.
The refactoring of scikit-rebate reduced RBA runtimes by 10 to 35x.
The newly introduced RBAs are among the strongest performers, robustly preserving both main effects and 2-way epistatic interactions, thus retaining predictive signal for downstream modeling.

Original Abstract (excerpt)

> As a precursor to high-dimensional biomedical data modeling, reliable feature selection can reduce computational expense, improve modeling performance, and yield simpler, more interpretable models. However, most filter-based feature selection methods struggle to detect feature interactions, while wrapper or embedded feature selection methods are computationally expensive. Relief-based algorithms (RBAs) are filter methods that are sensitive to feature interactions while mitigating these other limitations. This study (1) refactors, optimizes, and expands the scikit-rebate Python package with existing and newly proposed RBA variants and (2) conducts rigorous RBA benchmark comparisons across diverse genomic simulations...

*Auto-collected on 2026-09-01.*