Geng Academic Fraud Review Report: "Is Mamba effective for time series forecasting?"

Academic fraud report · Geng Detector

Summary

This report assesses the Neurocomputing paper (DOI: 10.1016/j.neucom.2024.129178) by Wang et al., titled "Is Mamba effective for time series forecasting?" Overall verdict: questionable (yellow), not outright fraud. The authors compare a proposed S-Mamba model against nine baselines on thirteen datasets. Two issues stand out: (1) Table 1 reports reversed timestep counts for the ETT datasets—ETTh1/ETTh2 (hourly) are listed with 69,680 steps while ETTm1/ETTm2 (15-minute) are listed with 17,420 steps, contradicting well-known dataset statistics from the original Informer paper where the values should be swapped; (2) all main results tables (Tables 2–5) report only single-point MSE/MAE values without standard deviation, standard error, or confidence intervals, despite razor-thin margins (e.g., 0.333 vs 0.334) between S-Mamba and iTransformer on ETTm1. Mitigating factors include a genuine, named GitHub repository linked to the first author and a self-consistent publication timeline. Image-level duplication checks were not feasible because raw figures were unavailable.

Verdict

Questionable (Yellow). The paper contains credible signs of carelessness rather than fabrication. Two substantive concerns are documented below, balanced by two positive indicators of transparency.

Key findings

Reversed ETT timestep counts in Table 1: The paper swaps the reported timesteps between ETTm1/ETTm2 (15-minute granularity) and ETTh1/ETTh2 (hourly granularity). Standard dataset statistics from the original Informer release indicate the hourly datasets contain 17,420 records and the 15-minute datasets contain 69,680 records over roughly two years—i.e., the inverse of what this paper states. This is consistent with a copy-paste or drafting error rather than fabricated experiments, but it is a hard factual error in the data description.
Absence of variance reporting across all main result tables (Tables 2–5): Only single MSE/MAE values are reported across thirteen datasets and nine baselines. No standard deviation, standard error, or confidence intervals are provided. Several claimed wins occur at the third decimal place (e.g., ETTm1: S-Mamba 0.333 vs iTransformer 0.334), which are not statistically meaningful without repeated runs.
Positive: open-source code with identifiable author account: The abstract links to https://github.com/wzhwzhwzh0921/S-D-Mamba, an account that aligns with first author Zihan Wang. Genuine, persistent-code repositories substantially increase the likelihood that the underlying experiments were actually executed.
Positive: internally consistent timeline: Submission on 22 June 2024, acceptance 7 December 2024; the Mamba architecture (arXiv:2312.00752, Dec 2023) and iTransformer (arXiv:2310.06625, Oct 2023) both precede submission.

Evidence highlights

Table 1 (Page 5, Section 5.1) shows Timesteps for ETTm1/ETTm2 = 17,420 and ETTh1/ETTh2 = 69,680; the canonical values from the Informer paper are the reverse.
Tables 2, 3, 4, 5 (Sections 5.2, 5.4) present single MSE/MAE numbers only; for example, Table 3 reports S-Mamba 0.333 and iTransformer 0.334 on ETTm1.
DOI 10.1016/j.neucom.2024.129178; published by Neurocomputing, available online 14 December 2024.
Open code: https://github.com/wzhwzhwzh0921/S-D-Mamba

Notes

Image-based duplication and splicing checks (Geng methods 1 and 3) could not be performed because raw figure pixel data was not provided to the system; only text content was available.
The Table 1 error is severe enough to require a corrigendum regardless of intent.
The lack of variance statistics is a methodological weakness rather than evidence of fraud, but it materially weakens the strength of any claimed SOTA result.
Suggested follow-up actions from the original reviewer include posting a structured query on PubPeer targeting Table 1 and the missing variance statistics; contacting authors for raw run logs is recommended before escalating further.