Peer punishment can help groups to establish collectively beneficial public goods. However, when humans condition punishment on other factors than poor contribution, punishment can become ineffective and group cooperation deteriorates. Here we show that this happens in pluriform groups where members have different socio-demographic characteristics. In our public good provision experiment, participants were confronted with a public good from which all group members benefitted equally, and in-between rounds they could punish each other. Groups were uniform (members shared the same academic background) or pluriform (half the members shared the same academic background, and the other half shared another background). We show that punishment effectively enforced cooperation in uniform groups where punishment was conditioned on poor contribution. In pluriform groups, punishment was conditioned on poor contribution too, but also partially on others’ social-demographic characteristics – dissimilar others were punished more than similar others regardless of their contribution. As a result, punishment lost its effectiveness in deterring free-riding and maintaining public good provision. Follow-up experiments indicated that such discriminatory punishment was used to demarcate and reinforce subgroup boundaries. This work reveals that peer punishment fails to enforce cooperation in groups with a pluriform structure, which is rule rather than exception in contemporary societies.