The Sombrero galaxy (M104) is an interesting object for a dust energy balance study due to its very symmetric dust lane, its proximity and its (nearly edge-on) inclination of 84°. From a panchromatic radiative transfer analysis, including scattering, absorption and thermal dust re-emission, we construct a standard model for M104 accounting for observations in the optical wave bands (stellar SED, images and extinction profiles in the V and RC band). This standard model underestimates the observed dust emission at infrared wavelengths by a factor of ~ 3, similar to the discrepancy found in other energy balance studies of edge-on spirals. Supplementing this standard model with a young stellar component of low star formation activity in both the inner disk (SFR ~ 0.21 M⊙ yr−1) and dust ring (SFR ~ 0.05 M⊙ yr−1), we are capable of solving the discrepancy in the dust energy budget of the Sombrero galaxy at wavelengths shortwards of 100 μm. To account for the remaining discrepancy at longer wavelengths, we propose a secondary dust component distributed in quiescent clumps. This model with a clumpy dust structure predicts three-quarters of the total dust content to reside in compact dust clouds with no associated embedded sources.