Context. Kernel phase is a data analysis method based on a generalization of the notion of closure phase, which was invented in the context of interferometry, but it applies to well corrected diffraction dominated images produced by an arbitrary aperture. The linear model upon which it relies theoretically leads to the formation of observable quantities robust against residual aberrations. Aims. In practice, the detection limits that have been reported thus far seem to be dominated by systematic errors induced by calibration biases that were not sufficiently filtered out by the kernel projection operator. This paper focuses on the impact the initial modeling of the aperture has on these errors and introduces a strategy to mitigate them, using a more accurate aperture transmission model. Methods. The paper first uses idealized monochromatic simulations of a nontrivial aperture to illustrate the impact modeling choices have on calibration errors. It then applies the outlined prescription to two distinct data sets of images whose analysis has previously been published. Results. The use of a transmission model to describe the aperture results is a significant improvement over the previous type of analysis. The thus reprocessed data sets generally lead to more accurate results, which are less affected by systematic errors. Conclusions. As kernel-phase observing programs are becoming more ambitious, accuracy in the aperture description is becoming paramount to avoid situations where contrast detection limits are dominated by systematic errors. The prescriptions outlined in this paper will benefit from any attempt at exploiting kernel phase for high-contrast detection.