You know I'm a committed user of the fediverse, perhaps this post will surprise
you. Still, at some point the truth has to be told, before lying leads to a
catastrophe.
I think I've been present in the fediverse (sometimes hosting a pod of some
software and sometimes not) since
TBF that’s only because mastodon, with it’s majority userbase inherited from gnu social, insisted on doing things outside of the AP protocol, to be able to claim those gnu social users in fact. Thus dragging everyone back to the past unnecessarily, and creating confusion for people who expect to be able to simply implement the AP spec as written.
It’s more complicated than that. AP is a “compromise” protocol that was created to get some of the 50+ different decentralized federated social networks to maybe consider talking to one another.
For that reason, it only supported some of the simpler use-cases (public posts, DMs), but lacked explicit specification for more advanced use-cases (edits? groups? Markdown content? etc).
Different pieces of server software started implementing these features on top of AP. Today there are instances that support Markdown, and those that don’t (notably mastodon.social, for example), that support groups (in several different ways that this can be implemented) and not, etc.
This is an evolving protocol, this is the price for innovation happening (instead of protocol actually stagnating like SMTP). On the other hand, the danger is that it ends up being XMPP, with hundreds of XEPs implemented seemingly randomly across servers and clients, leading to users never being really sure if a given functionality is actually supported by the combination of server software and client software used by them and whoever they are talking to (potentially 4 different pieces of software, draw a matrix of supported XEPs for that!).
So, it’s all the more funny that the author uses both SMTP and XMPP as examples of (presumably) “better” protocols.
So i don’t agree with everything the original author said, but i don’t agree with all you said either. The thing is SMTP is stagnating because the mailboxes we use in practice is controlled by a cartel of 4-5 megacorporporations, but at least it’s all specified even though in practice you need to deal with implementation-specific edge-cases.
For XMPP the ecosystem is far from stagnating but lacks resources. There’s tons of specs and it’s hard to figure out the correct way to do things on your own, but fortunately there’s projects like modernxmpp.net to help you out, and if you ask in chatrooms people will point you in the right direction. Also, XMPP from the start is explicitly conceived to be extensible with a tree-like data format (XML) so it’s easy to build pretty much anything on top and people are doing it (realtime code collaboration, video conferencing, social networking etc).
Yes both protocols have downsides but they nailed something that ActivityPub didn’t: they have clearly-defined use-cases. ActivityPub/ActivityStreams is super generic and that’s really cool in the spirit of a generic server and specific clients using the C2S protocol, but now we have the worst of both worlds where each use-case employs different data types in a seemingly-random fashion (so people will focus on interop with a specific implementation rather than defining a sub-specification for a specific usecase), and client apps use custom APIs all over.
This situation is worse for users because the protocol is intended for abstract content but all implementations are very opinionated (yet unspecified), so as a user you need different accounts on Mastodon, Lemmy, Peertube, Mobilizon, Funkwhale, PixelFed, and soon (hopefully) Gitea. How do i follow another user across all those services? Sure we could use rel=me links but who does that on the fediverse? In email/XMPP world at least i have a consistent address no matter what client i use, and there is a clean separation of concerns between clients and servers: the server provides abstract features (storage/federation) following specifications (roster, vCard, MIX, PEP, MAM etc) and the client implements the application logic relevant to the use-case, which is also defined by a specification (eg. MUC, microblogging over XMPP…).
I think the ecosystem/situation could be vastly improved in XMPP land in some regards, but by all means the model is vastly better than what the fediverse is doing in practice so far (not necessarily on paper). To elaborate on areas which could be improved:
I personally have strong opinions beyond these points, but i believe any federated protocol would benefit from at least those few points. If you can’t test for client/server correctness, can’t understand how specifications interweave at a glance, and don’t have a community that’s focused on UX, i don’t think it’s gonna be very successful.
To be clear, i’m not dismissive of the fediverse ecosystem (or matrix ecosystem for that matter) there’s a lot of cool innovation happening in there. But i can’t help but notice that they tried to address some implementation issues with newer protocols, and came with new protocol/implementation flaws in the process. And now we have two more protocols to interop with :P
Plenty of people, including yours truly (note the green checkmarked link to my blog in my profile). I keep seeing such green-checkmarked links in profiles, so this is not just a couple of people. It’s a thing.
Oh that’s pretty cool, thanks for sharing! I was not aware this had caught on so please don’t take my initial comment as a sarcastic dismissal but rather as a lack of awareness :)
I just followed the link to your blog and i really like the design. Also i liked what i read, especially about contract-based dependency management. I myself had some thoughts (just raw notes with lots of typos/mistakes not a published articles yet) on the topic (about how to specify API/CLI contracts as part documents readable both by humans and by machines) and i’d be thrilled to discuss it further and maybe actually start implementing something (i’m part-time hacking on a SSG, a DIY CI/CD system, and a selfhosting distro and i could really use some specification/testing system that’s not overtly complex).
Hope to see you around on IRC/XMPP maybe?
<3
Aww, thank you!
Perhaps we need a Lemmy community for that kind of stuff? 🤔
Oh hah, I used to (I even ran several XMPP servers back in the day). Now I am on Matrix, as this is just the easiest for me. And fedi, of course.
I don’t think it’s exactly the right tool for the job, but it’d be better than nothing. I’m personally interested in tooling that would help bridging the gap between curated info (like a wiki), long-lived discussions (mailing-list/forum) and instant messaging (chatrooms), but i don’t know any tool that would help with that yet.
Just curious: how is Matrix easier as a sysadmin? I’ve personally heard quite opposite feedback. Do you have maybe a bifrost instance on hackerspace.pl that i could reach you through? Otherwise i’ll try through a public instance :)
Take care <3
Easy: I don’t run it. Simplicity for me comes not from how easy it is to sysadmin it, but how many communities I am a part of use it. IRC, as much as I hate to say it, is slowly dying away (FreeNode shitstorm did not help here), and XMPP is not great at creating community spaces. So, Matrix it is for me.
No clue what bifrost is.
That will probably work best.
Yeah i agree. I started talking/drafting about XMPP Spaces a few months back. Hopefully i can find time to implement that in a client/server sometime :)
I added you on Matrix via aria-net.org gateway. bifrost is a matrix<->XMPP gateway but unfortunately the official matrix.org instance is really bad bad bad but from what i hear the aria-net.org fork runs much better…i guess we’ll see!
The people that utilize rel=me for its use case.
Also what’s rel=me?
It’s just a semantic tag you add to a link on an IndieWeb profile (h-card) to indicate other addresses which describe the same person. So basically if lemmy implemented basic microformats2 (semantic classes for HTML, part of indieweb ecosystem), you could have an “alternative links” setting in your Lemmy profile that would link to your XMPP, email, mastodon or whatever other addresses. Ideally, we wouldn’t even need so many accounts/addresses for a single identity, but as an intermediate measure a multi-protocol client (or a s2s bridge/gateway like bridgy could enable you to subscribe to one person’s feed across all protocols.
The indieweb is based on a POSSE principle where your own website is the original source of truth for content but is socially-aware of both: what you republished elsewhere (eg. a twitter post) and replies coming in from elsewhere (eg. a reply on Lemmy). The rel=me link is one of the semantic foundations that enables this. That’s just one approach to identity which places the web front and center, but we could also mention other approaches:
The rel=me specification addresses the “discoverability” problem. Both DNS and PGP can store arbitrary data and could be used to advertise different identities however i don’t know of standard/specified ways to do it. Something else to consider about digital identities is that you sometimes want different inboxes (collections in ActivityPub parlance) under the same identity, like some people have
bank@mydomainandecommerce@mydomainemail addresses. I think that’s a rather compelling argument for domain name as identity but that’s a complex topic with a lot of nuance (and it’s not incompatible with backing your domain name with a cryptographic identifier like the GNU Name System does).I’m happy to elaborate on certain points, or to provide detailed links (which i did not do in this reply sorry, but any web search engine should be able to help you out) if you ask for it :)
This might help:
https://www.zylstra.org/blog/2018/10/mastodon-rel-me/
Having been in the decentralized social media and prior semantic social web game long before ActivityPub, I can say without a doubt that ActivityPub does not qualify as innovation but rather regression, and that’s even before mastodon fucked it up, which is basically what the author is complaining about without realizing it.
I didn’t say ActivityPub is innovation. I specifically said ActivityPub “only supported some of the simpler use-cases”. In that sense we agree.
But I wouldn’t call it a regression, either. It’s a lowest-common-denominator kind of thing, where yeah it does not support a lot of fancy features, but after 15 years of dozens of different projects each making their own precious incompatible protocol that had ~2k users each, now we have a protocol that brings a bunch of different projects together.
That’s a huge step forward. Decentralized social networks always suffered from the Network effect working against them. By agreeing on a protocol, as non-ideal as it is, this got turned around, somewhat.
Overall I get what you’re saying and sort of agree but…
I was in those socialcg meetings. What was agreed upon didn’t always make it into the protocol because mastodon devs had an outsized influence, and so even when the majority voted on certain things the chair went with what the Mastodon devs wanted.
And then to add insult to injury the Mastodon devs decided not follow the protocol anyway.
And then because MastoPub had the most users, many newbie devs thought that’s what they had to implement rather than AP.
And again, most of the things complained about by this person are due to that side of the story.
It’s a tragedy of history. And yeah, I guess you could call it a success story too but at what cost? What type of success story would we have had if it didn’t go down that way? I argue, much greater success, and much fewer people questioning whether fediverse is a good way forward.
Ah, thank you for that context. I was not on these meetings, but I did follow the e-mail conversations in the fedsocweb working group. And I vividly remember the protocol measuring contest that any suggestion of finding a common ground and choosing/designing a common protocol for the 50+ different, incompatible decentralized social media protocols devolved into very quickly.
At the same time, I was on Diaspora; and on StatusNet, which was all-but killed by Evan developing a yet another incompatible protocol PumpIO and just migrating the biggest StatusNet instance to it, thus tearing the heart out of broader StatusNet. The Network Effect worked against tiny, incompatible, decentralized social networks, and so we were all stuck in walled gardens.
Then comes ActivityPub and suddenly a dozen or so different decentralized social media projects talk the same language. The Network Effect starts working in our favour. That’s a big deal!
So that’s the lens I see ActivityPub through. Not saying AP is perfect. But it’s a large step in a good direction.
Done is better than perfect, I guess.
But do they really talk to each other? They share Note objects at best. That’s to say nothing about how most still don’t really support urls as actors, and instead fall back on webfinger, which is deliberately not part of AP proper. And even then masto only supports alphanumeric names, so most new software copies that limitation to remain compatible, along with many other masto limitations. You are right about the lowest common denominator though.
Compared to, say, 9 years ago? When Diaspora would not federate with StatusNet, and Friendica would try to implement both of their protocols to try to create some form of interoperability between the two? With pump.io, tent.io, and Red doing things differently, and all this fragmenting the decentralized federated social media scene to a point of complete irrelevance?
When to the question I had asked on the
public-fedsocwebmailing list about how maybe, you know, there should be some effort to get different projects to speak the same protocol, the answers ranged from “not going to happen” through suggesting Bitcoin integration and claiming all these networks are too different to mentioning Usenet and NNTP.Today, thanks to ActivityPub (as imperfect as it is), from my Mastodon account I can follow PeerTube channels, WriteAs blogs, Mobilizon events, participate in threads like this one here on Lemmy, follow photostories on Pixelfed, and talk to people who have accounts on Friendica, Pleroma, MissKey, and who knows what other type of instances.
So yes, very clearly they do talk to each other, and very clearly this is already making a difference. Frankly I am flabberghasted this needs to be spelled out explicitly.
Of course, Diaspora still cannot federate with anyone else. But this is their choice. We now finally have a single huge network of different yet compatible (on a basic but important level) instances, so when a user asks “where should I move off of Twitter or Facebook”, different decentralized social networks need not fight among themselves to convince them to move to them specifically.
Instead of competing, a large number of federated social networking projects are cooperating, and surfing the Network effect together.
This is makes a lot of difference.
I haven’t yet followed your links but there are some things that came to mind from the comment text.
Mostly just that one of the reasons I liked both friendica and Red (who share a common author), was precisely the agnosticism toward protocol. The platforms do what they can within the confines of each protocol, and simultaneously support as many as they can.
There was a dearth of development resources on friendica when the founder left but it didn’t take long for the community to pitch in and catch up.
At least, that’s the way I remember things.
ActivityPub is the standardization of the ActivityPump (aka PumpIO) protocol, so all this came from that massive fuck you Even threw at the community. Set us back years, but we’re starting to see progress again these days I think, a little.
Well, progress is often not linear, or even monotonic.
note: I am new to the history of the ActivityPub standard.
Do you think that ActivityPub is capable of reform?
Do you think it would be possible to successfully propose to W3C that Mastodon doesn’t care to follow the protocol and those SocialCG decisions in their favor should be revised? What were some of those decisions?
Do you think Mastodon is increasingly in a position where it is ‘too big to lose’, and that if things escalated it would abandon federation with non-Mastodon platforms over following the majority-decided protocol? Will we see a situation where platforms are again using multiple federation protocols?
Hard to say. The data model has certain limitations that make it difficult to be used for more advanced things and it is structured in a way that seems to encourage nosql style document databases and has a sort of viral influence on the development of new apps.†
However, it can get the job done with enough brute force – though if you are doing anything other than microblogging MastoPub apps likely won’t work with it even if you are AP compliant.
No, the W3C process is such that the committee is closed and only addon specs can be ratified by the Community Group. I don’t remember any specific decisions but I remember at least a few occasions where everyone voted and there was a clear majority but the masto devs threw a tantrum and threatened to take their ball and go home. The results were mixed. Sometimes it simply went unspecified to “keep the peace” and others the majority was just flat out ignored and the opposite was put in spec. Excuses were always made and had me feeling like the whole voting process was a sham.
Not increasingly. It has always been that way. They don’t federate with non-mastopub platforms and the platforms that aren’t microblogs have had to limit their features to be MastoPub compatible.
The ones that have continue to do so and I still believe it’s the best (only) way.
@rysiek@szmer.info says that such a strategy results in apps that can talk with friendica or hubzilla but not each other, but I say that’s the situation we still have, where advanced apps can talk to each other but not Mastodon.
There will always be fragmentation. If that fragmentation falls on the majority (Mastodon) being inconvenienced I think that is a better result than the freedom fighters and innovators being sabotaged and having no options at all.
– ** –
† Here I am referring to the problem when developing a new app where if you plan to support AP first and foremost, it is easiest to structure your whole app and data model around AP, and then you are kind of stuck in that mindset permanently. Whereas if you develop your app with an open mind, then go back and add AP support you won’t cripple yourself.
Edit: I just remembered a couple examples (sort of). One was there was talk of a discovery mechanism that didn’t depend on webfinger, but Mastodon rejected it because they wanted to maintain backwards compatibility with gnu social (so they could bootstrap their userbase that gave them their influence).
Another was they almost didn’t include sharedInbox in the spec because Mastodon didn’t want to use it and thought it was stupid or evil or something. Luckily it still made it in but the compromise was that it was optional, putting extra burden on all other servers to support sending separate copies to every single user’s inbox.
Thank you for the very detailed answer!
I’m not sure if it’s the same thing, but I recall a Gab developer justifying their removal of federation in 2019, one of the reasons being that malicious actors were spinning up fake instances with thousands of users to make a server send separate copies of a message to every single user’s inbox, slowing the site down. Would shared inboxes help to prevent this type of attack, or is it something else?
Indeed. Making sharedinbox a requirement would mean that a server could simply refuse to do it the other way and then be immune from that attack. But because it is optional, all servers must then be vulnerable to this attack.
It can be mitigated by batching, and delivering say only 5 copies to one server at a time, but that would have to be very carefully crafted to not cause queue backup for other messages.
The ultimate workaround is queueless delivery, but there will still always be some penalty of having to keep revisiting a particular server.
A malicious actor can also deliberately slowly respond to deliveries, forcing the sending server to keep many sockets open.
Thank you for all the context! Fully agreed on basically everything, esp. that there’s always going to be some fragmentation. Still happy that we were able to limit that substantially. 🙂
Some more context I just remembered (funny how things come in waves):
Notice that I always say Mastodon devs and don’t name particular people. Part of that is out of respect and to keep it from seeming personal, but another important thing is that there were several Mastodon devs involved in the committee.
So when I say that there was a clear majority that means several Mastodon devs had a vote and they still lost.
But what happens in committee is people are allowed to argue for or against motions. At times, there would only be one person willing to argue on one side while several Mastodon devs would argue on the other.
So even if there was a majority vote numerically, there was a larger perceived dissent that would prevent motions from passing.
One chair was more affected by this than the other but again out of respect I won’t say which.
This is important to understanding why standards committees sometimes have undesirable outcomes. It’s also one of the reasons why sometimes groups committee shop and prefer W3C or IEEE or any of the others.
Standards committees is actually one of
humanity’ssociety’s biggest unsolved problems. 🙃