Today is an exciting day for the Lemmy project.
Almost one year after first enabling federation, we now federate with other projects for the first time! According to some people’s definition, this finally makes us part of the Fediverse.
It took a lot of work to make this possible, so big thanks to NLnet for funding our full time work on Lemmy, and to @lanodan and @asonix for helping to figure out how Pleroma and Mastodon federation works (it’s difficult because they have almost no documentation).
What is Lemmy?
Lemmy is similar to sites like Reddit, Lobste.rs, or Hacker News: you subscribe to communities you’re interested in, post links and discussions, then vote and comment on them. Lemmy isn’t just a reddit alternative; its a network of interconnected communities ran by different people and organizations, all combining to create a single, personalized front page of your favorite news, articles, and memes.
Major Changes
Federation code rewrite
The rewrite of the federation code started by @nutomic in August is now mostly complete. As a result, the code is much cleaner, and has tests to guarantee no breaking changes between Lemmy versions. As a side effect of this rewrite, it was now relatively easy to enable federation with other projects.
Mastodon and Pleroma users can:
- View Lemmy communities, user profiles, posts and comments
- Follow Lemmy communities to receive new posts and comments
- Replies (mentions) work in both directions, including notifications
In addition, Pleroma users can exchange private messages with Lemmy users.
Note that Pleroma and Mastodon rely on a compatibility mode in Lemmy, which means that they won’t receive events like Deletes or Votes. Other projects whose federation works similar to Pleroma/Mastodon will likely also federate.
Hardcoded slur filter removed
Lemmy finally has essential moderation tools (reporting, user/community blocking), so the hardcoded filter isn’t necessary anymore. If you want to keep using the slur filter, copy these lines to your config file when upgrading, and adjust to your liking.
Upgrade notes
If you installed Lemmy without Ansible, you need to edit the nginx config file to follow this example. Otherwise Mastodon won’t be able to fetch objects or actors from Lemmy. Ansible applies this change automatically.
No other changes are necessary for federation with Mastodon/Pleroma. Just make sure your allowlist/blocklist allows it. Then get started by searching the URL of a Lemmy user or Community in Mastodon.
Note that Mastodon and Pleroma are much, much bigger than Lemmy at this point, with a combined 3 milion users and 4500 instances, compared to 20.000 users and 35 instances for Lemmy (source). The existing mod tools in Lemmy might not be adequate to handle that at the moment.
Be aware that if you have federation enabled in the Lemmy config, Mastodon and Pleroma users can now fetch all posts and comments, to view them and share with their followers. The Lemmy blocklist/allowlist can not prevent this, it only prevents posts/comments from blocked instances to be shown on your own instance. The only solution to this problem is disabling federation, or waiting for signed fetch to be implemented.
If you want to use federation, but review new instances before federating with them, use the allowlist. You can switch from open federation to allowlist federation by pasting the output of the command below into federation.allowed_instances in the Lemmy config.
curl https://your-instance.com/api/v3/site | jq -c .federated_instances.linked
The lemmy.hjson additional_slurs field has changed its name to slur_filter.
Follow the Docker or Ansible upgrade instructions here.
Lemmy-Ansible
We’ve now separated our ansible install method (the preferred way to deploy Lemmy) into its own repo, lemmy-ansible. Let us know if you need help migrating existing installations over to it.
Changes
API
- There is now a
GetUnreadCountin the API to check the count of your unread messages, replies, and mentions. - A full list of the API changes can be seen on this diff of lemmy-js-client: 0.13.0 -> 0.14.0-rc.1 .
Lemmy Server
- More federation compat (#1894)
- Adding clippy:unwrap to husky. Fixes #1892 (#1893)
- Remove header guard for activitypub routes
- Add federation test cases for Smithereen and Mastodon
- Reduce stack memory usage in apub code
- Remove ActivityFields trait, deserialize into another struct instead
- Check if post or comment are deleted first. Fixes #1864 (#1867)
- Correctly use and document check_is_apub_id_valid() param use_strict_allowlist
- Convert note.content and chat_message.content to html (fixes #1871)
- Upgrade background_jobs to 0.9.1 #1820 (#1875)
- Fix husky fmt hook. (#1868)
- Renaming to slur_filter. Fixes #1773 (#1801)
- Three instance inbox bug (#1866)
- Remove ansible from this repo. (#1829)
- Rewrite collections to use new fetcher (#1861)
- Dont blank out post or community info. Fixes #1813 (#1841)
- Format config/defaults.hjson before committing (#1860)
- Breaking apub changes (#1859)
- Pleroma federation2 (#1855)
- Create a custom pre-commit hook, generates config/defaults.hjson (#1857)
- Add cargo metadata to all crates (#1853)
- Add both (De)Serialize to all models (#1851)
- Adding GetUnreadCount to the API. Fixes #1794 (#1842)
- Federate reports (#1830)
- Fix saved posts and hide read posts issue. Fixes #1839 (#1840)
- Dont allow posts to deleted / removed communities. Fixes #1827 (#1828)
- Dont swallow API errors (fixes #1834) (#1837)
- Fix federation of initial post/comment vote (fixes #1824) (#1835)
- Fix clippy warnings added in nightly (#1833)
- Admins can view all reports. Fixes #1810 (#1825)
- Adding a message_id to emails. Fixes #1807 (#1826)
- Generate config docs from code (#1786)
- Trying a background_jobs fix. #1820 (#1822)
- mark parent as read on reply (#1819)
- Move code to apub library (#1795)
- Adding honeypot to user and post creation. Fixes #1802 (#1803)
- Add database host back into config file (#1805)
@dessalines What’s wrong with publishing a root post a Note? Shouldn’t be too hard.
Or just any type, be it a Page, a Note, a peertube video, or a pixelfed photo?
Maybe in the future. But our funding milestone was federation with Pleroma, so thats done. We dont have any more work in that direction on the roadmap for now.
@nutomic how much funding do you need so that at least Mastodon/Pleroma/Misskey/etc. are able to create threads by simply mentioning the community’s actor in the post?
Its not a question of money, i just have other things to work on. Contributions welcome, although i’m not sure this is a good idea, cause mastodon doesnt even have separate input fields for link/title/body. Best open an issue to discuss it first.
Posts also have a lot of extra info, things like a link, title, body, community… comments have only a body.
There are a lot of things that aren’t going to match up with community-focused link aggregators and user-focused microblogging ones.
@dessalines you guys have a potential to be much more than just a link aggreagtor.
I maintain that anybody who makes a working, viable groups implementation that covers most of the Fediverse will define the direction of this network for the next five years.
You are the closest so far.
Lemmy has always been a link aggregator/forum, we dont have any plans to change it into something else. Federation with pleroma (and as a side effect, mastodon) was only one milestone of many. If there are problems, they can be solved on the mastodon/pleroma side just as well, maybe better.
@nutomic a problem of “umable to start a thread on Lemmy” can’t be solved on Pleroma.
Its not a problem in Lemmy, as other Lemmy instances can create threads over federation just fine. The problem is that Mastodon/Pleroma cant send the right data which Lemmy expects. And it doesnt make any sense for us to try and extract 4 fields (url, title, body, community) from the single text field that masto/pleroma send. So you should open issues in their respective repos instead. Below is the json format we expect, most of the fields are optional.
https://join-lemmy.org/docs/en/federation/lemmy_protocol.html#post
@nutomic Now, look. Implementing this feature on your side only requires effort on your part. Implementing it on everyone else’s side requires effort on the part of:
- Mastodon
- Pleroma
- Pixelfed
- Smithereen
- Misskey
- Friendica
- Hubzilla
May be even
- Peertube and Funkwhale
And probably others. This is at least 10x more effort. Imagine coordinating this mess.
So you are suggesting that Lemmy should add ugly and unintuitive compatibility hacks for 9 different projects. Thats not going to happen. Our federation docs are here, if anything is unclear I am happy to talk with the respective developers.