Context. A group of trans-Neptunian objects (TNOs) are dynamically related to the dwarf planet 136108 Haumea. Ten of them show strong indications of water ice on their surfaces, are assumed to have resulted from a collision, and are accepted as the only known TNO collisional family. Nineteen other dynamically similar objects lack water ice absorptions and are hypothesized to be dynamical interlopers. Aims. We have made observations to determine sizes and geometric albedos of six of the accepted Haumea family members and one dynamical interloper. Ten other dynamical interlopers have been measured by previous works. We compare the individual and statistical properties of the family members and interlopers, examining the size and albedo distributions of both groups. We also examine implications for the total mass of the family and their ejection velocities. Methods. We use far-infrared space-based telescopes to observe the target TNOs near their thermal peak and combine these data with optical magnitudes to derive sizes and albedos using radiometric techniques. Using measured and inferred sizes together with ejection velocities, we determine the power-law slope of ejection velocity as a function of effective diameter. Results. The detected Haumea family members have a diversity of geometric albedos ~0.3–0.8, which are higher than geometric albedos of dynamically similar objects without water ice. The median geometric albedo for accepted family members is p_V = 0.48_−0.18^+0.28, compared to 0.08_−0.05^+0.07 for the dynamical interlopers. In the size range D = 175−300 km, the slope of the cumulative size distribution is q = 3.2_−0.4^+0.7 for accepted family members, steeper than the q = 2.0 ± 0.6 slope for the dynamical interlopers with D < 500 km. The total mass of Haumea’s moons and family members is 2.4% of Haumea’s mass. The ejection velocities required to emplace them on their current orbits show a dependence on diameter, with a power-law slope of 0.21–0.50.