Attilan Rough Riders
"I would say that was the cavalry, but... I've never seen a line of horses crash into the battlefield from outer space before."
- – Agent Washington, Red vs. Blue: Revelation
The Attilan Roughriders are Imperial Guard horse cavalry formations originating from the planet of Attila. They are, essentially, Hun warriors in SPESS. They are fierce fighters, and it is said that they are born in the saddle (and more than likely will die in it). While they can be deployed en masse, they usually doled out in small units to other regiments that can make use of their swift flanking abilities. They are also used in a variety of asymmetric roles such as message running, scouting, insurgency suppression, raiding, and just generally being an annoying long-term hindrance from within the enemy's rear support areas. In battle they employ foul tempered horses built like tanks (and sometimes incorporating parts from them) and use explosive-tipped lances to fuck up your shit. They also employ the same variety of small arms, special weaponry, and close combat weapons that you would expect from any Imperial Guard unit.
The Only War supplement Hammer of the Emperor gives a slightly unnerving fact about these guys: Guardsmen need food to eat, but a horse can survive on vegetation alone. Therefore, if Roughriders ever find themselves in a situation where they can't find edible food or water they can drink the blood of their steeds as a stopgap measure. By taking just a few swallows that the steed can easily replace, the Rough Rider can sustain himself for a considerable time until he can resupply. Indeed, the fact that the Attilan uses a horse, not a vehicle, is the main reason why he is still relevant on the many varied battlefields of the 40k universe.
Any Imperial Guard cavalry are generally called "Rough Riders." Although this is obviously a bit of a misnomer when it concerns cavalry formations drawn from other worlds, the Attilans themselves can thus be considered the vanilla cavalry of the Imperial Guard. Unfortunately, as of 8th/9th Edition Attilan Rough Riders are long out of production and are considered a Legends unit. If you want to field models of your own, you'll either have to convert some or scour the dark recesses of Ebay.
That being said, if you're willing to go a non-GW route then Victoria Miniatures makes Rough Rider Legs that meld really nicely with the Cadian Shock Troopers torsos and the various torsos that are available elsewhere on their site. They also make arms with lances.