Context. Mergers are an important aspect of galaxy formation and evolution. With large upcoming surveys, such as Euclid and LSST, accurate techniques that are fast and efficient are needed to identify galaxy mergers for further study. Aims. We aim to test whether deep learning techniques can be used to reproduce visual classification of observations, physical classification of simulations and highlight any differences between these two classifications. As one of the main difficulties of merger studies is the lack of a truth sample, we can use our method to test biases in visually identified merger catalogues. Methods. We developed a convolutional neural network architecture and trained it in two ways: one with observations from SDSS and one with simulated galaxies from EAGLE, processed to mimic the SDSS observations. The SDSS images were also classified by the simulation trained network and the EAGLE images classified by the observation trained network. Results. The observationally trained network achieves an accuracy of 91.5% while the simulation trained network achieves 65.2% on the visually classified SDSS and physically classified EAGLE images respectively. Classifying the SDSS images with the simulation trained network was less successful, only achieving an accuracy of 64.6%, while classifying the EAGLE images with the observation network was very poor, achieving an accuracy of only 53.0% with preferential assignment to the non-merger classification. This suggests that most of the simulated mergers do not have conspicuous merger features and visually identified merger catalogues from observations are incomplete and biased towards certain merger types. Conclusions. The networks trained and tested with the same data perform the best, with observations performing better than simulations, a result of the observational sample being biased towards conspicuous mergers. Classifying SDSS observations with the simulation trained network has proven to work, providing tantalising prospects for using simulation trained networks for galaxy identification in large surveys.