Yote.zip

Yeah I wouldn’t bother. It intends for you to have a duplicate copy on every device, which is probably not what you want. Syncthing is really good for things like synchronizing notes, calendars, password databases, music, etc to your devices. Things that you want to access in both places, but that are usually disconnected from each other from time to time.

Conduit is also licensed under Apache 2.0, so it could also be taken closed source at any point in time. The reason this wouldn’t impact Conduit as much is that there’re other contributors, whilst Synapse and Dendrite are almost exclusively developed by Element.

Right. The current perspective is based on the idea that if Synapse/Dendrite go closed-source right now, an open source version would be good as dead. Element is responsible for 95% of Synapse/Dendrite and I’m sure a community fork would have to play a lot of catch-up to figure out how to keep it going. If the community was more involved in Synapse/Dendrite implementation (and if Element let them) there would be less cause for alarm, as closing the source would just mean an immediate community fork and putting Element on ignore. Also to reiterate, The Matrix Foundation is not going along with Element on this move, and even if Element pulled something shady the Matrix Core Spec etc. would still remain open and under the Foundation’s control, so the max we have to lose is Synapse/Dendrite and all of Element’s developers.

As for the rest I agree and I do actually trust that Element is simply playing their only card here. These maneuvers are all required in order for Element to survive as a company at all, but they also unfortunately leave this backdoor open as a consequence. Matthew has pinky-promised over and over that they are only acting in good faith and that they would never use the backdoor, but it’s understandable that the presence of the backdoor is putting everyone at unease. Best case scenario we take this as a warning sign that if Element drops dead tomorrow then Matrix is also dead. If people want Matrix to not be practically owned by Element then we should diversify and prepare escape plans.

It depends on what your workflow/usecase for putting documents on the drive currently is. Syncthing is usually intended to be put on two separate devices, and then a folder on each device gets synchronized - meaning you have a folder of your documents on each device. Is there any reason not to just mount the network drive’s folder and drag the documents in that way?

This is actually quite a controversial change mainly because of their switch to a CLA. This indirectly gives them the opportunity to switch the license to closed source whenever they feel like it in the future. Semi-controversially, they are also primarly making this AGPL change in order to begin selling dual-licensing to companies. The Matrix Foundation itself does not support this change from Element, though Element is within its rights to do so.

You can read some more thoughts on this from the pessimistic folks at HackerNews. My main takeaway is that I don’t trust Element because I don’t trust anyone. I’m sure they’re doing this in good faith but I don’t like the power they have at the moment. I hope this is what’s needed to begin focusing efforts on alternative homeserver implementations like Conduit.

Syncthing - No introduction needed. Couldn’t live without it.

Healthchecks.io (you can self host this) - Dead man’s switch monitoring for all my automation. Most of my automated scripts hit up a Healthchecks endpoint when they run, and if they fail to hit the endpoint on a regular schedule I get notified. Mandatory for my anxiety.

I prefer recertified ones if they’re significantly cheaper, but that’s up to you. Recertified will likely fail faster but when they’re close to ~60% of the cost it makes sense to gamble.

As for which RAID that is up to you and how you’re setting up your array. If you’re running ZFS then mirrored pairs are somewhat flexible since you can add a pair whenever you want of any size disks, but they will cost you 50% of your disk space in redundancy. For RAID5/6 you want the disk sizes to match and for ZFS you won’t be able to add any disks to a RAID5/6 array for about a year - the code that adds that feature is coming in the next release which will take about a year.

I’m not sure what’s around for Germany, sorry. You may be able to use eBay to find local sellers cross-posting from their normal website?

It was probably me. I use these two places + eBay primarily but I’m sure there are other good ones out there.

• https://serverpartdeals.com/collections/hard-drives

• https://diskprices.com

Edit: also for posterity this is a cool site but shucking drives hasn’t been viable for a long time as far as I’ve seen: https://shucks.top/

FYI: RAIDZ expansion just got merged: https://github.com/openzfs/zfs/pull/15022

Estimated timeline is about a year from now for OpenZFS 2.3 which will include it.

You can also use MergerFS+SnapRAID over individual BTRFS disks which will give you a pseudo-RAID5/6 that is safe. You dedicate one or more disks to hold parity, and the rest will hold data. At a specified time interval, parity will be calculated by SnapRAID and stored on the parity disk (not realtime). MergerFS will scatter your files across the data disks without using striping, and present them under one mount point. Speed will be limited to the disk that has the file. Unmitigated failure of a disk will only lose the files that were assigned to that disk, due to lack of striping. Disks can be pulled and plugged in elsewhere to access the files they are responsible for.

It’s a bit of a weird-feeling solution if you’re used to traditional RAID but it’s very flexible because you can add and remove disks and they can be any size, as long as your parity disks are the largest.

Mirrored vdevs allow growth by adding a pair at a time, yes. Healing works with mirrors, because each of the two disks in a mirror are supposed to have the same data as each other. When a read or scrub happens, if there’s any checksum failures it will replace the failed block on Disk1 with Disk2’s copy of that block.

Many ZFS’ers swear by mirrored vdevs because they give you the best performance, they’re more flexible, and resilvering from a failed mirror disk is an order of magnitude faster than resilvering from a failed RAIDZ - leaving less time for a second disk failure. The big downside is that they eat 50% of your disk capacity. I personally run mirrored vdevs because it’s more flexible for a small home NAS, and I make up for some of the disk inefficiency by being able to buy any-size disks on sale and throw them in whenever I see a good price.

Yeah ECC RAM is great in general but there’s nothing about ZFS that likes ECC more than any other thing you do on your computer. You are not totally safe from bit flips unless every machine in the transaction has ECC RAM. Your workstation could flip a bit on a file as it’s sending it to your ZFS pool, and your ECC’d ZFS pool will hold that bit flip as gospel.

The main problem with self-healing is that ZFS needs to have access to two copies of data, usually solved by having 2+ disks. When you expose an mdadm device ZFS will only perceive one disk and one copy of data, so it won’t try to store 2 copies of data anywhere. Underneath, mdadm will be storing the two copies of data, so any healing would need to be handled by mdadm directly instead. ZFS normally auto-heals when it reads data and when it scrubs, but in this setup mdadm would need to start the healing process through whatever measures it has (probably just scrubbing?)

ZFS can grow if it has extra space on the disk. The obvious answer is that you should really be using RAIDZ2 instead if you are going with ZFS, but I assume you don’t like the inflexibility of RAIDZ resizing. RAIDZ expansion has been merged into OpenZFS, but it will probably take a year or so to actually land in the next release. RAIDZ2 could still be an option if you aren’t planning on growing before it lands. I don’t have much experience with mdadm, but my guess is that with mdadm+ZFS, features like self-healing won’t work because ZFS isn’t aware of the RAID at a low-level. I would expect it to be slightly janky in a lot of ways compared to RAIDZ, and if you still want to try it you may become the foremost expert on the combination.

ZFS without redundancy is not great in the sense that redundancy is ideal in all scenarios, but it’s still a modern filesystem with a lot of good features, just like BTRFS. The main problem will be that it can detect data corruption but not heal it automatically. Transparent compression, snapshotting, data checksums, copy-on-write (power loss resiliency), and reflinking are modern features of both ZFS/BTRFS, and BTRFS additionally offers offline-deduplication, meaning you can deduplicate any data block that exists twice in your pool without incurring the massive resources that ZFS deduplication requires. ZFS is the more mature of the two, and I would use that if you’ve already got ZFS tooling set up on your machine.

Note that the TrueNAS forums spread a lot of FUD about ZFS, but ZFS without redundancy is ok. I would take anything alarmist from there with a grain of salt. BTRFS and ZFS both store 2 copies of all metadata by default, so bitrot will be auto-healed on a filesystem level when it’s read or scrubbed.

Edit: As for write amplification, just use ashift=12 and don’t worry too much about it.

ZFS doesn’t eat your SSD endurance. If anything it is the best option since you can enable ZSTD compression for smaller reads/writes and reads will often come from the RAM-based ARC cache instead of your SSDs. ZFS is also practically allergic to rewriting data that already exists in the pool, so once something is written it should never cost a write again - especially if you’re using OpenZFS 2.2 or above which has reflinking.

My guess is you were reading about SLOG devices, which do need heavier endurance as they replicate every write coming into your HDD array (every synchronous write, anyway). SLOG devices are only useful in HDD pools, and even then they’re not a must-have.

IMO just throw in whatever is cheapest or has your desired performance. Modern SSD write endurance is way better than it used to be and even if you somehow use it all up after a decade, the money you save by buying a cheaper one will pay for the replacement.

I would also recommend using ZFS or BTRFS on the data drive, even without redundancy. These filesystems store checksums of all data so you know if anything has bitrot when you scrub it. XFS/Ext4/etc store your data but they have no idea if it’s still good or not.

I would prefer 1. Restoring a failed ZFS mirror is easy, and you can continue to operate while a new drive arrives.

2 will get you more space in theory but you’ll have downtime with any problem like you said, and you’ll also have slower speeds without the mirror.

3 is unnecessary unless you have a good reason.

I don’t see any disadvantages with Proxmox and VMs on the same disk, as Proxmox shouldn’t have much activity going on.

My suggestion is to set up Proxmox under a VM and give it some virtual disks to replicate these setups and then yank a disk and try to recover. Write down the steps it takes to get back to a normal system and see if that affects your decision.

IMO just get whatever the cheapest one is of those big manufacturers. You should be running some sort of redundancy for your disks anyway, and disk failures are always a gamble no matter what you do to pre-emptively stop them. Personally I buy cheap refurbished drives and throw them into my RAID with the foregone conclusion that I might need to replace them sooner than a new drive, but I’m also saving so much money by buying refurbished that replacement cost will be cheap. Check ebay or ServerPartDeals if you subscribe to this line of thinking.

Edit: This would be sort of similar to “cattle not pets”, where you strategize for failure instead of trying to prevent it from failing.

Oh I didn’t realize. I wonder why they would do that? Either way it’s not a huge deal - the main problem with TrueNAS Scale is that you actively cannot install Docker onto it because it will conflict with the normal TrueNAS Scale app system. There are technically ways to get Docker working on a TrueNAS Scale system but they’re unsupported and likely to break frequently on updates. Debian and OpenMediaVault should behave similarly in terms of getting Docker set up.

I would only recommend TrueNAS Scale if you want an easy way to manage a ZFS pool. I wouldn’t use it for anything related to containers or logic as it’s very inflexible and only supports specific usecases through its weird app system. Debian is my usual choice for basically everything, but it will be fully DIY. OpenMediaVault is a good turnkey option based on Debian that’s similar to TrueNAS Scale, except it allows you to run plain Docker and other custom usecases. I also like Proxmox (based on Debian) a lot but it’s a bit too advanced if you don’t need its hypervisor functionality.