Warband can refer to either of two things in /tg/ terminology.
Firstly, warband is a genre of tabletop game that sits at an interim point between a board-based wargame and a roleplaying game. Warband games have much the same structure as war-games - revolving around a conflict between two or more opposing forces, depicted as miniature units, but condense the numbers. Instead of the vast army of the typical wargame, a warband game focuses on a smaller force, typically averaging between 6 to two dozen individual soldiers. Whilst the game still uses the fairly simplistic unit confrontation mechanics of a wargame, campaign rules allow for each player's individual troops to grow and progress as multiple battles are fought, experiencing positive and negative development such as learning new skills, gaining increased stats, acquiring - or losing - gear, or suffering lingering injuries. This allows for greater personalization of each trooper and provides a sense of individuality.
Examples of warband games include:
Secondly, "warband" is one of many names for a band of warriors who have gathered together for a common cause. The term has connotations of illegal behavior, ill-discipline, and general rowdyness, and is usually associated with more "savage" races such as orcs. This is the original version of the term, and inspired the name for the aforementioned small-scale warband/RPG hybrids.