Absorbers seen in the spectrum of background quasars are a unique tool to select HI-rich galaxies at all redshifts. These allow us to determine the cosmological evolution of H I gas, $\Omega_{\rm HI+HeII}$, a possible indicator of gas consumption as star formation proceeds. The damped Lyman-$\alpha$ systems (DLAs with NHI$\ge$ 10$^{20.3}$ cm$^{-2}$) in particular are believed to contain a large fraction of the H I gas, but there are also indications that lower column density systems, namely “sub-damped Lyman-$\alpha$” systems play a role at high-redshift. Here we present the discovery of high-redshift sub-DLAs based on 17 $z>4$ quasar spectra observed with the Ultraviolet-Visual Echelle Spectrograph (UVES) on VLT. This sample is composed of 21 new sub-DLAs which, together with another 10 systems from previous ESO archive studies, make up a homogeneous sample. The redshift evolution of the number density of several classes of absorbers is derived and shows that all systems seem to be evolving in the redshift range from $z=5$ to $z\sim3$. The redshift evolution of the column density distribution, $f(N,z)$, down to NHI=10$^{19}$ cm$^{-2}$ is also presented. A departure from a power law due to a flattening of $f(N,z)$ in the sub-DLA regime is present in the data. $f(N,z)$ is further used to determine the H I gas mass contained in sub-DLAs at $z>2$. The complete sample shows that sub-DLAs are important at all redshifts from $z=5$ to $z=2$.