I might take over one of these one-year-free hosted Lemmy instances on my server infrastructure, but I read several times now that Lemmy’s image hosting system Pict-rs is using a lot of storage quickly.
The server I could run this on is limited to 32gb ssd storage with no easy way to expand it.
Is there some way to limit the image storage use and automatically prune old images that are not user or community icons or such?
pict-rs doesn’t keep track of how often it serves different images, so there’s not a good metric for pruning old images. That said, 0.4 will introduce functionality for cleaning up processed images (e.g. resizes/thumbnails), removing their files & metadata. If they are viewed again, they will be re-generated.
0.4 will also include the ability to scale down images on upload, rather than storing the original resolution. This is not yet implemented, but it’s on my roadmap.
All this said, it is already possible to use pict-rs with object storage (s3-compatible), rather than block storage. That’s a good option if your hosting provider offers it
Actually S3 compatible interface might be interesting to link Pict-rs to Garage…
I am aware of garage, but haven’t tested it yet with pict-rs. It’s a cool project for sure
That sounds promising. Any idea when 0.4 will be released?
Object-storage on large cloud providers is not an option for me for various reasons (privacy, legal etc.).
I can only say “when it’s ready.” I think most of what I want to include in 0.4 is there, but I don’t have a ton of time to work on it currently. I might see if I can get my last feature changes in this weekend, then it will be a matter of ensuring the 0.3 -> 0.4 upgrade is smooth, and that storage migration is solid
Update on this: I got the feature work done this weekend, so now I’ll be testing it a bunch for upgrades and storage migrations
Hello there! I am one of the administrators at Beehaw. If I’m reading and understanding your comment correctly, then this could solve our most pressing problem of running out of server disk space.
Is there a time-frame when you expect to have pict-rs 0.4 available?
Is deduplication supported by re-using images already in storage if newly upload images share the same hash with them?
Yes. It uses sha256 rather than perceptual hashing, but that’s Good Enough™️
Why not SHA-512 or SHA3?
I chose it at the start of the project 🤷
Maybe is it worthy to make a smooth change to this in the future? https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions