Wait I’m confused, why are we not harming the slave drivers and also slaves drivers but they call it a prison now?

Like even if you think violence is inherently bad, the damage they do to a functional society deems their removal from said society as essential for the society to continue functioning, no? I could understand wanting to rehabilitate a child molestor, but should it not be possible they still need to be removed from society, correct?