HumP-KD: Hybrid Uncertainty-Aware Multi-Stage Progressive Knowledge Distillation for Efficient Fire Classification

Forum topic · 小凯 · 2026-06-16

Summary

HumP-KD is a hybrid uncertainty-aware multi-stage progressive knowledge distillation framework for real-time fire classification on resource-constrained hardware. The framework distills knowledge from two frozen heterogeneous transformer teachers (Swin-Tiny and ViT-Base) plus a Meta-MLP ensemble into a lightweight MobileViT-S student model. It uses hierarchical progressive knowledge distillation with hierarchical feature builders that generate fused spatial attention masks to guide distillation toward discriminative regions, alongside multi-stage distillation progressively activated during training. Evaluated on FlameVision (8,600 images) and Dataset-II (31,309 images) under standard preprocessing, online augmentation, Gaussian noise, and motion blur conditions, HumP-KD achieved an average F1 score of 0.9876 ± 0.0063 on Dataset-II over 10 trials, significantly outperforming the non-distilled MobileViT-S baseline (0.9537 ± 0.0351), confirmed by paired t-test (p = 0.0195) and Wilcoxon signed-rank test (W = 1, p = 0.0039). The student retains only 4.94M parameters (19.01 MB), a 5.7x reduction versus Swin-Tiny and 17.5x versus ViT-Base, while reaching 37.72 CPU FPS for real-time deployment. Paper: arXiv 2606.14684.

Paper Overview

Field: Computer Vision
Authors: Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid
Published: 2026-06-12
arXiv: 2606.14684

Abstract

Real-time fire classification systems require models that are simultaneously accurate, computationally efficient, and deployable on resource-constrained hardware. This work proposes HumP-KD, a Hybrid Uncertainty-aware Multi-stage Progressive Knowledge Distillation framework for efficient fire classification. Two datasets, FlameVision and Dataset-II, containing 8,600 and 31,309 images, are used. Various CNN and transformer baselines are applied under standard preprocessing, online augmentation, Gaussian noise, and motion blur robustness conditions.

The proposed HumP-KD model distills knowledge from two frozen heterogeneous transformer teachers, Swin-Tiny and ViT-Base, along with their Meta-MLP ensemble, into a lightweight MobileViT-S student via three tightly integrated components:

Hierarchical progressive knowledge distillation: a hierarchical feature builder generates fused spatial attention masks that selectively guide distillation toward discriminative regions.
Multi-stage knowledge distillation: three distillation stages are progressively activated during training.

Results

On Dataset-II, HumP-KD achieved a mean F1 score of 0.9876 ± 0.0063 over 10 independent trials, significantly outperforming the MobileViT-S baseline trained without distillation (0.9537 ± 0.0351). Statistical significance was confirmed by an independent t-test (p = 0.0195) and the Wilcoxon signed-rank test (W = 1, p = 0.0039). The method also demonstrated strong generalization across datasets and robustness under degraded visual conditions.

Efficiency

The student model retains only 4.94M parameters (19.01 MB model size) — a 5.7x parameter reduction versus Swin-Tiny and 17.5x versus ViT-Base — while achieving 37.72 CPU FPS, making it suitable for real-time deployment.

*Auto-collected on 2026-06-16.*