Considerable research has shown that, in the cross-section, segregation is associated with detrimental neighborhood outcomes for blacks and improved neighborhood outcomes for whites. However, it is unclear whether early-life experiences of segregation shape later-life neighborhood outcomes, whether this association persists for those who migrate out of the metropolitan areas in which they grew up, and how these relationships differ for blacks and whites. Using the Panel Study of Income Dynamics from 1979 to 2013, we find that the level of segregation experienced during adolescence is associated with significantly worse neighborhood outcomes in adulthood for blacks. However, migrating out of the metropolitan area an individual grew up in substantially moderates these relationships. In contrast, adolescent segregation is associated with improved, or not significantly different, neighborhood outcomes in adulthood for whites. These findings have important implications for theorizing about the mechanisms linking segregation and neighborhood outcomes and for considering potential means of assuaging racial disparities in harmful neighborhood exposures.