One way, instead of a hard and east limit, might be to allow any number of lancemates for a character with high enough charisma - but if a character can have more than 4 lancemates, then (max_lancemates - 4) people with zero skills jump into your lance and refuse to leave. These can be levelled up (at, say, half the rate of a normal lancemate) but, if killed, will be replaced by another zero-skill lancemate having a charisma meltdown the moment you find a crowd (i.e. anywhere safe enough to use robotics).
How does that sound?