A Dataset-Centric Analysis of Evaluation Protocols in Face Morph Attack Detection

Authors

• Sheerko R. HmaSalah, Department of Software Engineering, Faculty of Engineering, Koya University, Danielle Mitterrand Boulevard, Koya KOY45, Kurdistan Region, IraqFollow
• Abdulbasit K. Al-Talabani, Department of Software Engineering, Faculty of Engineering, Koya University, Danielle Mitterrand Boulevard, Koya KOY45, Kurdistan Region, Iraq
• Aras T. Asaad, Department of Mathematics and Natural Sciences, American University of Iraq, Kirkuk Main Road, Raparin, Sulaimani 46001, Kurdistan Region, Iraq

How to Cite This Article

HmaSalah, Sheerko R.; Al-Talabani, Abdulbasit K.; and Asaad, Aras T. (2026) "A Dataset-Centric Analysis of Evaluation Protocols in Face Morph Attack Detection," Polytechnic Journal: Vol. 16: Iss. 2, Article 5.
DOI: https://doi.org/10.59341/2707-7799.1885

Document Type

Original Article

Abstract

Face morph attack detection (MAD) is often evaluated using datasets collected from diverse sources, created for different purposes, and acquired under varying conditions. This heterogeneity raises questions about how evaluation protocols and dataset characteristics influence reported performance. This work adopts a dataset-centric perspective to analyze the effects of evaluation protocol and dataset composition, including visual quality and morph-to-genuine imbalance, on MAD performance. Experiments are conducted on eight datasets and four morphing techniques using a fixed pipeline evaluated across six backbones, including Vision Transformer (ViT) and five CNN architectures, and ten evaluation scenarios covering homogeneous and heterogeneous cross-morph generalization. Performance is analyzed using threshold-independent and threshold-dependent measures, including Equal Error Rate (EER), Bona Fide Presentation Classification Error Rate (BPCER) at fixed Attack Presentation Classification Error Rate (APCER) operating points of 1% and 5%, and Detection Error Trade-off (DET) curves. Across datasets and architectures, heterogeneous testing yields higher error rates, revealing a generalization gap when models encounter unseen morphing techniques. Moreover, heterogeneous evaluation expands inter-architecture variability, exposing backbone-dependent robustness differences compressed under homogeneous evaluations. Several datasets achieve very low error under homogeneous evaluations yet degrade substantially under heterogeneous evaluations, indicating that homogeneous evaluation can mask sensitivity to protocol shift. The heterogeneity ratio shows that relative degradation is not monotonic with dataset scale, suggesting additional dataset-specific effects. Spearman correlation analysis further indicates a weak-to-moderate, non-monotonic association between morph-to-genuine imbalance and average error across backbones. Overall, reliable MAD evaluation requires heterogeneous evaluations, threshold-aware analysis, and explicit consideration of dataset properties beyond scale alone

Receive Date

04 May 2026

Revise Date

18 Aug 2026

Accept Date

04 Sep 2026

Publication Date

2026