The eccentric orbits of the known extrasolar giant planets provide evidence that most planet-forming environments undergo violent dynamical instabilities. Here, we numerically simulate the impact of giant planet instabilities on planetary systems as a whole. We find that populations of inner rocky and outer icy bodies are both shaped by the giant planet dynamics and are naturally correlated. Strong instabilities – those with very eccentric surviving giant planets – completely clear out their inner and outer regions. In contrast, systems with stable or low-mass giant planets form terrestrial planets in their inner regions and outer icy bodies produce dust that is observable as debris disks at mid-infrared wavelengths. Fifteen to twenty percent of old stars are observed to have bright debris disks (at λ ~ 70μm) and we predict that these signpost dynamically calm environments that should contain terrestrial planets.