So i understood you just want some local storage system with some fault tolerance.
ZFS will do that. Nothing fancy, just volumes as either blockdevice or ZFS filesystem.
If you want something more fancy, maybe even distributed, check out storage cluster systems with erasure coding, less storage wasted than with pure replication, though comes at reconstruction cost if something goes wrong.
MinIO comes to mind, tough i never used it… my requirements seem to be so rare, these tools only get close :/
afaik you can add more disks and nodes more or less dynamically with it.
Yeah, it needs those rules for e.g. port-forwarding into the containers.
But it doesnt really ‘nuke’ existing ones.
I have simply placed my rules at higher priority than normal. Very simple in nftables and good to not have rules mixed between nftables and iptables in unexpected ways.
You should filter as early as possible anyways to reduce ressource usage on e.g. connection tracking.