Galaxy mergers are crucial for understanding galaxy evolution, and with large upcoming datasets, automated methods, such as convolutional neural networks (CNNs), are essential for efficient detection. It is understood that CNNs classify mergers by identifying deviations from the regular, expected shapes of galaxies, in particular faint features, which are indicative of a merger event. In this work, we present a novel investigation of the relative importance of different morphological components: faint residual features, position, and spatial structure, in CNN-based binary classification of galaxies into merger and non-merger classes. Using mock images from the IllustrisTNG simulations processed to mimic Hyper Suprime-Cam observations, we fit Sérsic profiles to each galaxy and generate three datasets: original images, model images containing only smooth Sérsic profiles, and residual images highlighting faint features after model subtraction. We train three identical CNNs on these datasets: CNN1 on original images, CNN2 on model images, and CNN3 on residual images. CNN1, trained on full images, achieves the highest accuracy of 74%. CNN2, using only shape information including source position, achieves 70%, while CNN3, using only faint residual features, achieves 68%. We find that galaxy merger classification is possible using either faint features or the position and Sérsic profile information present in residual and model images, respectively. Our results demonstrate that not only faint features but also source position information play complementary roles in merger classification. This has important implications for the design and interpretation of machine learning methods for galaxy morphology, in particular in regimes where specific image components may be enhanced or suppressed.
