Continuous valuations have been proposed by several authors as a way of modelling probabilistic non-determinism in programming language semantics. Let $(X, {\cal O})$ be a topological space. A quasi-simple valuation on $X$ is the sup of a directed family of simple valuations. We show that quasi-simple valuations are exactly those valuations that extend to continuous valuations to the Alexandroff topology on the specialisation preordering of the topology $\cal O$. A number of applications are presented. In particular, we recover Jones' result that every continuous valuation is quasi-simple if $X$ is a continuous dcpo – in this case there is a least extension to the Alexandroff topology. We show that this can be refined if $X$ is algebraic, where every continuous valuation is the sup of a directed family of simple valuations based on finite elements. We exhibit another class of spaces in which every continuous valuation is quasi-simple, the so-called finitarily coherent spaces – in this case there is a largest extension to the Alexandroff topology. In general, the extension to the Alexandroff topology is not unique, unless, for example, the original valuation is bicontinuous. We also show that other natural spaces of valuations, namely those of discrete valuations and point-continuous valuations, can be characterised by similar extension theorems.