Context. The SPHERE “planet finder” is an extreme adaptive optics (AO) instrument for high resolution and high contrast observations at the Very Large Telescope (VLT). We describe the Zurich Imaging Polarimeter (ZIMPOL), the visual focal plane subsystem of SPHERE, which pushes the limits of current AO systems to shorter wavelengths, higher spatial resolution, and much improved polarimetric performance. Aims. We present a detailed characterization of SPHERE/ZIMPOL which should be useful for an optimal planning of observations and for improving the data reduction and calibration. We aim to provide new benchmarks for the performance of high contrast instruments, in particular for polarimetric differential imaging. Methods. We have analyzed SPHERE/ZIMPOL point spread functions (PSFs) and measure the normalized peak surface brightness, the encircled energy, and the full width half maximum (FWHM) for different wavelengths, atmospheric conditions, star brightness, and instrument modes. Coronagraphic images are described and the peak flux attenuation and the off-axis flux transmission are determined. Simultaneous images of the coronagraphic focal plane and the pupil plane are analyzed and the suppression of the diffraction rings by the pupil stop is investigated. We compared the performance at small separation for different coronagraphs with tests for the binary α Hyi with a separation of 92 mas and a contrast of Δm ≈ 6^m. For the polarimetric mode we made the instrument calibrations using zero polarization and high polarization standard stars and here we give a recipe for the absolute calibration of polarimetric data. The data show small (< 1 mas) but disturbing differential polarimetric beam shifts, which can be explained as Goos-Hähnchen shifts from the inclined mirrors, and we discuss how to correct this effect. The polarimetric sensitivity is investigated with non-coronagraphic and deep, coronagraphic observations of the dust scattering around the symbiotic Mira variable R Aqr. Results. SPHERE/ZIMPOL reaches routinely an angular resolution (FWHM) of 22−28 mas, and a normalized peak surface brightness of SB₀ − m_star ≈ −6.5^m arcsec⁻² for the V-, R- and I-band. The AO performance is worse for mediocre ≳1.0″ seeing conditions, faint stars m_R ≳ 9^m, or in the presence of the “low wind” effect (telescope seeing). The coronagraphs are effective in attenuating the PSF peak by factors of > 100, and the suppression of the diffracted light improves the contrast performance by a factor of approximately two in the separation range 0.06″−0.20″. The polarimetric sensitivity is Δp < 0.01% and the polarization zero point can be calibrated to better than Δp ≈ 0.1%. The contrast limits for differential polarimetric imaging for the 400 s I-band data of R Aqr at a separation of ρ = 0.86″ are for the surface brightness contrast SB_pol( ρ)−m_star ≈ 8^m arcsec⁻² and for the point source contrast m_pol( ρ)−m_star ≈ 15^m and much lower limits are achievable with deeper observations. Conclusions. SPHERE/ZIMPOL achieves imaging performances in the visual range with unprecedented characteristics, in particular very high spatial resolution and very high polarimetric contrast. This instrument opens up many new research opportunities for the detailed investigation of circumstellar dust, in scattered and therefore polarized light, for the investigation of faint companions, and for the mapping of circumstellar Hα emission.