AbstractThe process of determining the number and characteristics of sources in astronomical images is so fundamental to a large range of astronomical problems that it is perhaps surprising that no standard procedure has ever been defined that has well-understood properties with a high degree of statistical rigour on completeness and reliability. The Evolutionary Map of the Universe (EMU) survey with the Australian Square Kilometre Array Pathfinder (ASKAP), a continuum survey of the Southern Hemisphere up to declination +30°, aims to utilise an automated source identification and measurement approach that is demonstrably optimal, to maximise the reliability, utility and robustness of the resulting radio source catalogues. A key stage in source extraction methods is the background estimation (background level and noise level) and the choice of a threshold high enough to reject false sources, yet not so high that the catalogues are significantly incomplete. In this analysis, we present results from testing the SExtractor, Selavy (Duchamp), and SFIND source extraction tools on simulated data. In particular, the effects of background estimation, threshold and false-discovery rate settings are explored. For parameters that give similar completeness, we find the false-discovery rate method employed by SFIND results in a more reliable catalogue compared to the peak threshold methods of SExtractor and Selavy.