Bulges are a major galaxy component in the nearby universe, and are one of the primary features that differentiates and defines galaxies. The origin of bulges can be directly probed in part by examining distant galaxies to search for high redshift bulges, and to study the properties of bulges in formation. We review the evidence for bulges at high redshift in this article, and how by studying bulges through a variety of approaches, including through morphological, colour, and stellar mass selection, we can determine when and how these systems assembled. We argue that the majority of the most massive ‘classical’ bulges are in place by z ~ 1.5 − 2, and likely formed very early through major mergers. Other, likely lower mass, bulges form through a secular process along with their disks. Direct observations suggest that these two formation processes are occurring, as spheroids are commonly seen at z < 1, as are disks and spiral galaxies in the form of luminous diffuse objects, clump-clusters, and chain galaxies. However, bulge+disk systems are relatively rare until z ~ 1, suggesting that this structural assembly occurred relatively late.