ABSTRACT How and when in the star formation sequence do dust grains start to grow into pebbles is a cornerstone question to both star and planet formation. We compute the polarized radiative transfer from a model solar-type protostellar core, using the polaris code, aligning the dust grains with the local magnetic field, following the radiative torques (RATs) theory. We test the dependency of the resulting dust polarized emission with the maximum grain size of the dust size distribution at the envelope scale, from $a_\mathrm{max}=1\,$ to $50 \,\hbox{$\mu $m}$. Our work shows that, in the framework of RAT alignment, large dust grains are required to produce polarized dust emission at levels similar to those currently observed in solar-type protostellar envelopes at millimetre wavelengths. Considering the current theoretical difficulties to align a large fraction of small ISM-like grains in the conditions typical of protostellar envelopes, our results suggest that grain growth (typically $\gt 10 \,\hbox{$\mu $m}$) might have already significantly progressed at scales 100–1000 au in the youngest objects, observed less than 105 yr after the onset of collapse. Observations of dust polarized emission might open a new avenue to explore dust pristine properties and describe, for example, the initial conditions for the formation of planetesimals.