A Reproducibility Audit of Deep Learning Papers in Structural Health Monitoring
Sixty-four papers, eleven runnable artefacts
Rafiq Islam
University of Dhaka
Abstract
We attempted to reproduce the headline result of every deep-learning paper published in four structural health monitoring venues between 2020 and 2024 (n = 64). Eleven provided a runnable artefact; of those, four reproduced within one standard deviation of the reported metric. We report the taxonomy of failures — missing preprocessing, undisclosed hyperparameters, proprietary sensor data — and argue that the field's reported accuracy gains are not currently distinguishable from evaluation drift. We publish our audit harness and per-paper notes.
Keywords:
reproducibilitystructural health monitoringdeep learningresearch integritynegative results
References
- Farrar, C. R., & Worden, K. (2012). Structural Health Monitoring: A Machine Learning Perspective. Wiley.
- Pineau, J., et al. (2021). Improving reproducibility in machine learning research. JMLR, 22(164), 1-20.
- Gundersen, O. E., & Kjensmo, S. (2018). State of the art: Reproducibility in artificial intelligence. AAAI.