In any case, most of the prisoners are probably not from the West but from the East, either from distant kingdoms that Sauron has yet to subdue or insurrectionists from those territories he or his allies have conquered.  There are presumably both stray Avari and other dwarves in the little-known eastern regions of Middle Earth.*  But I agree that the bulk of the prisoners would be men.  Perhaps a few rebel Uruk-hai as well?

Of western prisoners, I would expect a smattering of Gondorians, Rohirrim, Woodmen, Beornings, Wood-elves, Bardings, and dwarves of Erebor.  Some of these would likely have been transferred from Dol Goldur when Sauron returned home.  No ents, probably.  Entwives?

*And there may be dwarves in the East who serve Sauron.  Though we don't hear of this in the War of the Ring, it did happen in the Last Alliance.