Context. Fast radio bursts (FRBs) are millisecond-long bursts uniquely detected at radio frequencies. FRB 131104 is the only case for which a γ-ray transient positionally and temporally consistent was claimed. This high-energy transient had a duration of ∼400 s and a 15–150 keV fluence S_γ ∼ 4 × 10⁻⁶ erg cm⁻². However, the association with the FRB is still debated. Aims. We aim at testing the systematic presence of an associated transient high-energy counterpart throughout a sample of the FRB population. Methods. We used an approach like that used in machine learning methodologies to accurately model the highly-variable Fermi/GBM instrumental background on a time interval comparable to the duration of the proposed γ-ray counterpart of FRB 131104. A possible γ-ray signal is then constrained considering sample average lightcurves. Results. We constrain the fluence of the possible γ-ray signal in the 8–1000 keV band down to 6.4 × 10⁻⁷ (7.1 × 10⁻⁸) erg cm⁻² for a 200-s (1-s) integration time. Furthermore, we found the radio-to-gamma fluence ratio to be η > 10⁸ Jy ms erg⁻¹ cm². Conclusions. Our fluence limits exclude ∼94% of Fermi/GBM detected long gamma-ray bursts and ∼96% of Fermi/GBM detected short gamma-ray bursts. In addition, our limits on the radio-to-gamma fluence ratio point to a different emission mechanism from that of magnetar giant flares. Finally, we exclude a γ-ray counterpart as fluent as the one possibly associated with FRB 131104 to be a common feature of FRBs.