Vancian Casting, also known as Vancian Magic, is the style of magic utilized in Dungeons & Dragons for most of its lifespan. It was inspired by the "Dying Earth" novels, a series of Science Fantasy novels written by author Jack Vance and set on a post-apocalyptic earth where the most powerful spells would literally burn themselves out of a wizards brain when cast. Gygax loved these novels, and decided that they made the most sense for how to handle magic in his developing RPG.
This style of magic was D&D's most distinguishing magical feature; it required wizards to spend a significant amount of down time in order to memorize spells from their spellbook, with a mechanic called "spell slots" dictating how many spells a wizard could memorize at once. When a spell was cast, it wiped itself from the caster's brain; the only way to cast a given spell multiple times per day was to memorize it multiple times. Once a wizard had cast all of their memorized spells, they were completely out of magic.
A rough analogy would be comparing the wizard's mind to a quiver and each individual spell to an arrow. Once they're all fired, it's useless.
Vancian Casting began to lose its grip in 3rd edition; the Sorcerer was introduced as (and the bard reworked into) an alternative to this mechanic, where instead they had a much smaller pool of spells known, but could cast any spell they knew as often as they liked, with spell slots serving as a kind of tally-mark-based mana system. This introduced the "spontaneous caster" archetype to D&D, and ultimately to Pathfinder.
When 4th edition came out, that was the end of Vancian Casting. The wizard's spellbook became a way to increase its spells known and to swap its layout of available spells, whilst spells themselves now could be cast either at will, once per encounter, or once per day.
5th edition dialed back from 4th, but maintained the end of Vancian Casting; now, effectively caster functioned akin to the spontaneous casters of 3e, with the addendum of cantrip spells which could be cast at-will. The only semblance of Vancian Casting that remains is that the wizard maintains its 4e Spellbook feature, which allows it to change out the spells in its "spell pool" as part of an extended rest. This gives them a greater amount of versatility that almost any other spellcasting class in the edition.
This has been a skubby development, with grognards hating on it and modern audiences loving it. Although even fans of the 5e style will admit that it has made the sorcerer into a poor man's wizard due to bad design choices - namely, the fact the sorcerer, previously designed as the "flexible arcanist", had no game mechanics to allow for quick and easy spell-slot recovery. Oh, it can "convert" spell points into spell-slots, but it has a very small pool of spell-points and it only regains those after an 8-hour "long rest". In comparison, the wizard can automatically regain a certain amount of low-level spell slots on a hour-long short rest.