Challenge Rating was a mechanic introduced in Dungeons & Dragons 3e to try and make it easier for DMs to judge what kind of monsters were suitable to throw against their party and give them a reasonable chance of succeeding. It... didn't exactly work as planned. On paper, a monster with a particular Challenge Rating is a challenge for four player characters whose level matches the number. The number is based on an average of Hit Dice, Armor Class, Spell Resistance, Damage Reduction, damage output and all other abilities it might have.
While the concept seems simple enough, in practice it means that the times you run into low-level monsters with very high HD or AC, high damage output, or have abilities that either inflict a continuous effect that a party of that level cannot easily deal with or outright kills you, it's not necessarily just the DM being a dick. Worse yet is that certain immunities are much harder to deal with for lower level characters, with wererats in particular being low in CR but high in ability to mulch first level parties with no easy way to bypass their damage reduction. Swarms and incorporeal creatures are notorious for being low CR but impossible for low level parties to damage. A lot of monsters simply didn't have correct CR values in the first place: Dragons were intentionally under CRed to make them boss monsters (ignoring how boss monsters are supposed to be over the party's normal CR anyways) and a House Cat is deadlier than some things twice its CR, while an Ogre Mage is totally ineffective against level appropriate foes.
Because of this correctly utilizing CR to create balanced encounters that don't stomp your players is more of an art than a science, which ironically is the same problem that WotC tried to solve with the concept of the Challenge Rating.
5e solved the issue somewhat with the encounter building tables of the 10/10/2016 Unearthed Arcana article. The following table is plucked from that article, and lists the ratio of the number of adventurers needed for a monster of equal level. For example, a 6th level character can handle two monsters with CR1, while a 7th level character can handle three. Do this for every character you have in your party and presto, you're done. This still requires some fidgeting however: it would take four 3rd level characters to take down a CR2 monster, an example of which would be a skeletal minotaur. That thing can hit for 4d8+4 damage (22 average) on the charge, which means that with some luck it can one-shot a third level character.
The Monster Manual itself also points out that CR isn't a substitute for brains, specifically calling out the rakshasa as an example of a monster whose immunities let it punch well above its weight class and ruin a party's day if they're not properly equipped.
As such, always keep an eye out for such outliers when building encounters.