We present the second data release (DR2) of the X-shooter Spectral Library (XSL), which contains all the spectra obtained over the six semesters of that program. This release supersedes our first data release from Chen et al. (2014, A&A, 565, A117), with a larger number of spectra (813 observations of 666 stars) and with a more extended wavelength coverage as the data from the near-infrared arm of the X-shooter spectrograph are now included. The DR2 spectra then consist of three segments that were observed simultaneously and, if combined, cover the range between ∼300 nm and ∼2.45 μm at a spectral resolving power close to R = 10 000. The spectra were corrected for instrument transmission and telluric absorption, and they were also corrected for wavelength-dependent flux-losses in 85% of the cases. On average, synthesized broad-band colors agree with those of the MILES library and of the combined IRTF and Extended IRTF libraries to within ∼1%. The scatter in these comparisons indicates typical errors on individual colors in the XSL of 2−4%. The comparison with 2MASS point source photometry shows systematics of up to 5% in some colors, which we attribute mostly to zero-point or transmission curve errors and a scatter that is consistent with the above uncertainty estimates. The final spectra were corrected for radial velocity and are provided in the rest-frame (with wavelengths in air). The spectra cover a large range of spectral types and chemical compositions (with an emphasis on the red giant branch), which makes this library an asset when creating stellar population synthesis models or for the validation of near-ultraviolet to near-infrared theoretical stellar spectra across the Hertzsprung-Russell diagram.