The Manual of the Planes
Ideally, in Dungeons & Dragons your players will be leveling up and able to beat the living shit out of anything on God's green Earth... or Oerth, or Krynn, whatever. You've been using stuff like evil wizards summoning demons, or fire elementals, or djinni, and maybe your players are gonna get the idea that they're gonna close the Hellmouth from the other side.
The Manual of the Planes, as its name suggests, is a sourcebook specifically examining the finer workings of the multiverse; the names and natures of various planes of existence, descriptions of extraplanar races, historical events of multiversial importance, and other such matters. All of the goodies needed to at least point a DM in the right direction of sending their players out into the wider multiverse - or at least giving conjurers some more bang for their buck.
There have been four versions of the Manual throughout D&D's many editions so far.
Advanced Dungeons & Dragons 1st edition saw the first instance. Jeff Grubb wrote up chapters of fluff, settings info, survival skills and random encounters for the alternate dimension in the default "Great Wheel" cosmology. Id Software later would plagiarise its cover art for the Cacodemon sprite in the original Doom.
You Know Who didn't want a Manual of the Planes for 2nd Edition. But as her tenure waned, TSR instead published an entire campaign setting, Planescape: which got a dozen different books focused on specific planes to flesh out the Great Wheel like they never had been before.
D&D 3rd edition produced the second version of the Manual, which broke traditions slightly: rather than just focus on the Great Wheel, it also suggested some ways of having alternate cosmologies, and had some player-focused content, in the form of planar-focused prestige classes. It still spent most of its text giving details on what the geography and afterlife is like in each of the Outer Planes, and gave monster manual entries (finally!) for denizens you would find only when traveling beyond your home plane, so it wasn't that different to its predecessor. Contains a vague reference to D20 Modern (explicitly acknowledged in the FAQ for that system) by stating that "the Plane of Shadow might lead to alternate Material Planes and other planes of existence. This is a perilous way to travel, because the way to other planes plunges through parts of the Plane of Shadow that are not coexistent with any known plane". When 3.5e happened, WoC realized that a lot of material in the MotP was out of date, so they created a reboot/sequel named the Planar Handbook.
D&D 4th edition gave us what is technically the third edition of the Manual. Though 4e's switchover from the Great Wheel to the World Axis was controversial, the 4e MotP stuck to its formula; examining all of the planes in as much detail as space allowed, and bolstering it with new planar monsters, magic items, Paragon Paths, vessels and rituals. Because of how new the planes were, further planar sourcebooks were ultimately released, most obviously The Planes Above & Below, which focused on the Astral Sea and Elemental Chaos respectively.
The 4e MotP also got a little meta on us, featuring a Wondrous Item called "The Manual of the Planes", a mystical grimoire which allowed its bearer to more quickly, cheaply and accurately perform the Analyze Portal ritual, and which could be used to change the destination of an open portal 1/day. Whatever else this was, it was also a nice little nod to a more classic D&D artifact, the Codex of Infinite Planes, which would see its own 4e debut in Mordenkainen's Magnificent Emporium.