RareVillages makes villages much rarer, not by decreasing the chance of them spawning, but by increasing the space between them. You can use Jacobsjo's Minecraft Datapack Map to view the difference between Vanilla Minecraft and RareVillages. Terralith is supported in this version. RareVillages also groups vanilla Villages and Fortified Villages together. This means that you wont find a Fortified Village right next to a vanilla Village, unlike what you could find in Terralith alone.
If you don't want to use Terralith, or use a Spigot fork, you can find the vanilla version of RareVillages here.