Semiclassical gravity is believed to describe the semiclassical regime of quantum gravity and be relevant in situations sufficiently far from the Planck scale where quantum effects and gravity are both important, such as in black holes or cosmology. However, its mathematical and conceptual structure has not been fully understood. In this talk, I describe some advances made in this direction, as well as the challenges that lie ahead and the opportunities that may come from a rigorous understanding of semiclassical gravity, especially in important open questions like the black hole information loss puzzle.