Massive stars in binaries can give rise to extreme phenomena such as X-ray binaries and gravitational wave sources after one or both stars end their lives as core-collapse supernovae. Stars in close orbit around a stellar or compact companion are expected to explode as “stripped-envelope supernovae”, showing no (Type Ib/c) or little (Type IIb) signs of hydrogen in the spectra, because hydrogen-rich progenitors are too large to fit. The physical processes responsible for the stripping process and the fate of the companion are still very poorly understood. Aiming to find new clues, we investigate Cas A, which is a very young (∼340 yr) and near (∼3.4 kpc) remnant of a core-collapse supernova. Cas A has been subject to several searches for possible companions, all unsuccessfully. We present new measurements of the proper motions and photometry of stars in the vicinity based on deep HST ACS/WFC and WFC3-IR data. We identify stellar sources that are close enough in projection but using their proper motions we show that none are compatible with being at the location of center at the time of explosion, in agreement with earlier findings. Our photometric measurements allow us to place much deeper (order-of-magnitude) upper limits on the brightness of possible undetected companions. We systematically compare them with model predictions for a wide variety of scenarios. We can confidently rule out the presence of any stellar companion of any reasonable mass and age (main sequence, pre main sequence or stripped) ruling out what many considered to be likely evolutionary scenarios for Type IIb supernova (SN IIb). More exotic scenarios that predict the presence of a compact companion (white dwarf, neutron star or black hole) are still possible as well as scenarios where the progenitor of Cas A was single at the moment of explosion (either because it was truly single, or resulted from a binary that was disrupted, or from a binary merger). The presence of a compact companion would imply that Cas A is of interest to study exotic outcomes of binary evolution. The single-at-death solution would still require fine-tuning of the process that removed most of the envelope through a mass-loss mechanism yet to be identified. We discuss how future constraints from Gaia and even deeper photometric studies may help to place further constraints.