I’m redoing my home server. I’m moving from arch to fedora server to force myself to learn that stuff.
I also want to use the bcachefs subvolume and compression features. I am presented with a couple issues I need smarter people to explain to me though.
subvolumes: My understanding is subvolumes in bcachefs dont get mounted separately like you would expect. they are just folders that you can isolate a snapshot to. so I dont need to mount a subvolume, just mount the filesystem it exists in already and it comes along for the ride. but what happens if I want to rollback? Lets say I want to rollback to a previous snapshot on the / but not mess with anything in the subvolume /home/thatuser. wouldn’t rolling back to the previous root also rollback the subvolumes contained within? perhaps I should create a separate partition just for root afterall.
mount: I can mount with
mount -t bcachefs /dev/device1:/dev/device2:/dev/device3
and everything works. However, if I mount with
mount -t bcachefs UUID=(externalUUIDfromshow-super)
not all devices in the pool will attach. some stay detached. so how am I supposed to mount them all at boot? I can’t just refer to them by device path because if I add more drives later, those paths may change.
compression: the documentation is kind of out-of-sync on different sites but what I was able to piece together is you can set different rules for compression recursively for different directories/subvolumes like this…
sudo bcachefs set-file-option /mnt/tempy/ --compression=zstd:3 --background_compression=zstd:12
then read back what was already set with…
getfattr -d -m 'bcachefs_effective\.' /mnt/tempy
so while the above makes sense for a root or user home folder, for VM disk images, I might want to set…
sudo bcachefs set-file-option /mnt/tempy/vmstorage/ --compression=none --background_compression=none --nocow
or even do something similar without the nocow for media folders within the compressed home user folder…
sudo bcachefs set-file-option /mnt/tempy/home/thatuser --compression=zstd:3 --background_compression=zstd:12
sudo bcachefs set-file-option /mnt/tempy/home/thatuser/media --compression=none --background_compression=none
mount at boot: okay, I know I need the DKMS and userspace tools installed as separate packages since Kent was naughty and got his project booted out of the kernel. but I’m not entirely certain how to convey to fedora it needs to bundle those pieces with the initiramfs. I know arch uses something different from fedora to do that, with fedora using dracut. I need to learn how to use dracut.
install: this I have no idea what to do. Fedora’s anaconda expects something very different and will not eat what I’m trying to feed it. so I need to find a way to bootstrap a fedora server install on to this system. Supposedly theres a way to do that with DNF.
Anyway, could someone explain the subvolume puzzle to me and maybe give pointers on bootstrapping fedora server and teaching it to mount all the devices in the pool at boot?
You must log in or # to comment.
The replies here make me so mad at Kent Overstreet.
I love bcachefs and was using it on quite a few systems. When it was in the mainline kernel, interest was building. I feel like we could have been just a few months from experimental coming off and adoption skyrocketing.
Then Kent got it pulled from the kernel (so not interested in the “fighting for users” misdirection). Now, as evidenced by the comments here, most users will not touch it.
I needed it in the kernel so I have been migrating away too but it breaks my heart.
I am sure somebody will use it, maybe even more than the small number that have historically. And Kent will probably tell himself that is ok.
It sucks.
Now, I did not write a COW filesystem. So I guess I am getting what I am owed (nothing). That does not dull the sting much though.
This always happens in the dev community when someone gets high up. Even Linus had numerous incidents like this. Remember when he went after the Intel engineer? Or all of NVIDIA? Or the more recent hits like the big endian RISCV stuff?
As I see it, Bcachefs is a good idea and it’s progressing well, there’s just a lot of bureaucracy Kent does NOT manage well. So there are two paths forward:
- Kent gets a middleman a few years from now to nerf his work with other devs.
- the projects forks off.
No argument.
I do not see much chance of a middle-man though and the alternative means much less adoption.
My issue is not with Kent’s strong technical opinions. I like those. Well, except that abusing other people as cover for his inability to follow the rules is not cool.
Linus can be a dick but he is typically making technical arguments at least (and usually quite good ones). Kent likes to play the “engineering” card but the drama is always about process, not technology, and he is the one being called out. So trying to pretend he is defending better engineering just makes the behaviour worse.
NVIDIA were breaking the rules (legally even). They have come around.
More big endiian in the kernel for no reason is a negative.
Not sure about the Intel engineer. Linus can be a jerk though so not assuming he was right if I do not know the situation.
If you can’t even wrap your head around it, why are you using it for your server? Other FS have all the features you need without all the problems.
Well once upon a time I did. I used arch back then and I didn’t use subvolumes. It was a simpler setup and worked fine. It also wasn’t exposed to WAN. Now I want to learn about subvolumes and I want to learn fedora.
I like picking projects just a bit out of reach for me to focus on and really dig into. It gives me something to obsess over other than, well, -gestures broadly-.
Arguing with this pile of micro center parts is basically therapy for me.
I hate to reply because I don’t have the answer to your question, just a remark which you may not care for: why bcachefs, especially on fedora server which has a rapidly advancing kernel? Bcachefs is out of the kernel tree. It is just going to be a constant maintenance burden on you to upkeep it with your server.
Btrfs will support subvolumes, compression, nodatacow directories, and everything else you might want while not being a thing you have to manually keep up with.
I would not expect the fedora installer to have any support for bcachefs, because fedora doesn’t have support for it generally.
Oh I understand the skepticism. Basically, I like the puzzle. I’ve been using bcachefs for a few years now on my little arm servers running arch and I like it. I like the tiered storage functionality. It’s a cool little thing I get to play with.
I’m collapsing my little boxes into one and fedora has better security defaults. It’s harder to do something stupid with fedora than arch. It also forces me to learn selinux rather than just bitch about it. The box has a VM running opnsense on it with a pci passthru of a NIC. Arch can do that but fedora has a certain structure to it that helps avoid some irresponsible configs so it’s kinda forcing me to learn how it wants to do things.
This little experiment is a terrible idea for a professional deployment, especially at my skill level, but for my home setup it’s the perfect puzzle for me to play with.
I was talking to a friend years ago about our homelabs. He was floored by my wild and crazy approach. He said “your homelab should be like a zen garden.” I countered “mine is like a trailer park after a tornado.”
I want to learn things and bcachefs is interesting, even if Kent pisses off literally every kernel developer.
As for btrfs, I don’t use it because I’m an adult. Years after abandonment it still stinks of Oracle. Kent is a move-fast-and-break things kind of guy that the old guard of kernel devs fucking hate and they aren’t wrong to feel that way. But he’s always been honest about what bcachefs actually is. Btrfs, on the other hand, is still oracle nonsense that should never be used.
If you need me to oversimplify things, I tend to back off anything Oracle wherever I can. Everything they have ever touched fucking sucks.
So why is btrfs bad? Is it just because oracle bad or is there an actual reason?
Afaik btrfs was developed by a person at oracle, but not for or by oracle.
They have performance issues as snapshots accumulate. Snapshots don’t fork predictably (filesystem devs expect one thing, sysadmins expect another.) RAID write holes still exist in configurations they dismiss as “exotic.” Use of oracle tech normalizes using their stuff in a community that knows better. Btrfs development has pretty much stagnated as there have been some optimizations but few long standing issues have actually been addressed. Btrfs has been pulled right out of distros that praised it before too. While that’s not as dramatic as Kent getting his ass yeeted from the kernel it’s still pretty high up there. Btrfs has made data vanish permanently in the past. Bcachefs has a reputation of hiding it from you until you coax it back out. It’s never really gone, it’s just inaccessible until a bug is fixed. And it always got fixed.
The problem isn’t the filesystem. It’s the developer. It’s not enough that he’s right. He has to cooperate with others who are also right to be in the kernel and he just won’t do it. So now he is outside the kernel. The project will carry on but it’s going to be niche.
