I've had only one issue with ejabberd (didn't read the docs close enough and had to move from mnesia to sql when I hit the size limit) in over a decade of self-hosting it.
Knowing which XEPs you actually want is a bit of a pain, but has gotten easier. Now I usually just periodically look at the current prosody example[1] which lists extensions by "essential" "recommended" "nice to have" and "other" and it's easy enough to find the matching modules for ejabberd.
As someone who has looked into forking Matrix for a new type of chat service, I'm grateful to see a more in-depth look at running it behind the scenes. Thank you.
I’ve been running a Matrix server for about two years on a Proxmox host in a colo I rent for the purpose (plus some other hobby stuff, but mainly because I just think it’s cool). This playbook is awesome and it’s pretty easy to set up and keep running:
https://github.com/spantaleev/matrix-docker-ansible-deploy
Regarding the "Requires federation" section, that is not true. I've been running a small family-only homeserver for several years now, and had federation disabled on it from the very beginning, and there have been exactly zero issues related to (lack of) federation with it.
As a former user I felt these pain points trying to do nothing more than have a very active one-on-one chat with a good friend. Tens of messages an hour, maybe 2 years running. Using matrix.org and the pre-X clients. It's fine for group chat (IRC style) but that's not a high bar.
(a) the encryption between using a mobile and the webapp desyncs/breaks all the time, it just sucks. I mean you'll get "cannot decrypt" a lot, have to bounce back and forth and generally try and force it to re-sync properly again. Sometimes never worked at all. Lots of issues on GH over the years.
(b) as mentioned in this article, insane delays on new message notif and sending and receiving. Just logging in on the webapp every morning took minutes of some sort of mysterious sync process, often the mobile app had the same problems. The X stuff may fix this, we were pre-X.
(c) cleanup. There's no message retention set on matrix.org, when I wanted to extract and remove our past chats the process and experience was excruciatingly bad. It took tens of hours over several weekends of the webapp (mobile completely non-op in practice for this) polling and loading old content, just so I could select 100 at a time to delete and then it took an hour. Once I started culling back over a year or so, the loading got longer and longer and longer, until eventually it 100% stopped working at all to load old messages.
Signal and DeltaChat are far, far better experiences for one-on-one chats with friends & family. The Delta client is a bit UI/UX behind but not horrible; e.g. you can't correct a typo in a sent message in Delta, unlike Signal - because each msg is a unique gpg-encrypted "email" rather than a database object that can be re-manipulated.
I've tried off and on to actually use Matrix. I was a bit of a loud supporter in the early days. Unfortunately, it looks like it still hasn't grown past the fundamental issues I was having then. It might be time to try something else
When I started using Matrix, Riot.im did not get notifications in time, or was a battery hog. And Synapse took up a lot of memory, with occasional slowdowns. But then, circa 2022, Synapse improved and Element seems to have worked well and was consistent and reliable on all platforms.
That's exactly the sorts of issues I was having. That and it somehow kept losing my encryption information so I'd lose access to all of my messages or something similarly ridiculous
I've mentioned this here before, but it bears repeating. A couple years ago or so, I made the catastrophic choice to use Dendrite as my homeserver software. It seemed like a safe bet: it was supposedly lighter weight than Synapse, being written in Go instead of Python and with everything reengineered from the ground up. It didn't support everything, but nothing in the disclaimers made it seem like it was about to abruptly become defunded and essentially unmaintained. Alas, that's exactly what happened not too long after I made that choice. Despite showing no interest in maintaining it, New Vector still found it necessary to relicense their abandonware under the more restrictive AGPL license. Good priorities. Then, when a security patch was needed, a new release was rushed out that included not just a security fix, but also a bug that caused Dendrite to completely stop processing messages for minutes at a time multiple times a day. (This only got fixed months later by a volunteer.) Joining large federated rooms on Dendrite took so long that I thought it was just broken; it could take hours to days to complete the operation! There was even a brief period when Element actually didn't even support Dendrite, leaving everyone locked out. Dendrite has never supported Element X, and the old sliding sync proxy was never updated to support the new simplified sliding sync either, which means you're stuck with the old slow sync and no support for things that require it. Also, most appservers still don't work right on Dendrite either. I got Mautrix-Discord working, but only for DMs.
I legitimately could go on and I'm sure I've forgotten things. It's amazing how quickly my experience with Dendrite went from pretty good to nightmare.
I realized that nobody in charge at the Matrix Foundation or New Vector really cared enough about leaving people stranded on a completely broken server to actually do anything about it (and trust me, I'm not alone. In every single federated room I've ever been in, I've always seen hostnames with dendrite subdomains. I could see them pass by in the logs while joining servers was taking hours.) I honestly considered just leaving the Matrix ecosystem, but I wasn't alone on my home server, so I decided to do my best to fix the problem. I wrote a tool that attempts to migrate the data from Dendrite to Synapse. This is a complicated operation that really took a huge amount of effort to get working, but after a couple of months of failed attempts, I had a test where I was able to seamlessly perform the migration and have clients continue to work and stay in federated rooms. So after getting it "close enough", I went ahead and gave the migration a shot in production and of course, it didn't work very well. All of the user accounts were intact, but a lot of stuff was broken. People indeed stayed in federated rooms, but my room state migration was definitely not 100% correct. Despite this, though, after manually cleaning up the database a bit more, hackishly while live, it was mostly a success. I believe I am probably the first person to directly move from Dendrite to Synapse.
So now that I am on Synapse, have my thoughts on Matrix changed? Yes. It's significantly better using Synapse, without question. The ecosystem is still a mess, but everything about Synapse is less broken than Dendrite. There are so many features Dendrite just doesn't do, like URL previews.
Why not contribute to Dendrite? Honestly, I don't want to. Their CLA sucks and they're not going to change it for me, and I don't think they're really going to spend time reviewing PRs given the circumstances. If I'm going to contribute to a project without retaining my rights I'd prefer to be on payroll. That's not something you should get from a community member. Either change the CLA to guarantee the project must stay open, or don't expect any free contributions.
I've been using Matrix for several years as a user. It works great. The problems decrypting messages have gone. X is becoming a good client.
I'm deleting my whatsapp and télégram accounts in a few weeks after a painful week-long backup...
Edit : I wonder how easy it is to backup a Matrix accounts's data. Conversations and files.
> I wonder how easy it is to backup a Matrix accounts's data. Conversations and files.
Not as easy as you might hope. The Element client has an export feature, but you have to manually activate it on each room/chat, and the export has a size cap so it may not work if you have lots of files you want to save. It's also pretty slow if the room has a lot of history. You could also try using something like Matrix Commander (a command-line client), but I couldn't get that to work fully either.
>this also creates a situation where anything said across federation cannot be unsaid, which is an ironic situation for a protocol/system that often comes up when talking about privacy.
How is it ironic? No protocol in the world can force anyone to delete anything from their own device. Chat apps that implement this function are either proprietary (so you cannot control what they can do) or, if OSS, do it on a pinky-promise-basis.
To add another data point, I've been hosting a (tiny) matrix server for a few months. I'm pretty comfortable with self-hosting using docker, so I opted not to use the ansible scripts in the hope that it'd keep my setup simpler and more maintainable. Somehow I didn't find any mentions of ESS until Synapse was already up and running, but Kubernetes would have been a dealbreaker for similar reasons.
In this short time I've run a database migration (sqlite is the default, but MAS requires postgres), tried and failed to migrate to MAS (required to use Element X) and have lost a couple of days messing around with coturn and eturnal with nothing to show for it -- my calls still don't connect when NAT is involved. I have to tell new users to ignore the recommendations to install Element X until I get MAS working.
There's a lot of room for foundational improvements here, even updating docs to point would-be server admins to the recommended setup du jour would help.
> While technically, Synapse can work with a sqlite database (and which at first seems like an OK choice for having <10 users on the server), it WILL become corrupted
I want to hear more about this. Is this because Synapse’s SQLite support is half-baked? What sort of corruption are we taking about?
Same setup here since 2017. Since then, RAM usage decreased by 60%.
The admin panel is not something I'd need but it would be a nice-to-have.
Started with postgres as I wouldn't go for anything else if I wanna use it for decades. It has 2.5GBs for 10users and I don't mind if it takes 10 or 20, that's something I expected. Never did a cleanup of anything, I just dumped the db and moved to OVH recently onto a new VPS with NVMe SSD, it flies.
The fact that I cannot delete attachments that users delete is certainly my biggest irritation, 50GBs of stuff I am not sure if I can or cannot delete, but considering the size, I am just gonna bite the bullet, couple terabytes should not be a problem in 25 years. But this is def something I would love to see addressed sooner rather than later. It must be a pain even for the matrix.org server team.
After moving to a better server I do not have issues with slow notification unless the phone is sleeping for longer period of time which is an android optimization (I'd assume). It is more reliable than teams at this point. One of my friends had issues but removing 15 old devices fixed the issue.
As for element-x, I did call out "the another rewrite" issue especially with android and I do think it makes things worse. I still do not know how am I supposed to fix calling and video between old and new clients. For now I don't bother with new clients and everyone is using old ones, but it starts to become an issue as classic clients are in maintenance only mode
Ran synapse for a few months, figured out all the tui clients are either abandonware or broken (originally thought i could use bitlbee, and did the install before realizing it was unusable).
Looked at current tui offerings now some years later, situation seems to be unchanged, the only client that ran back then was gomuks, and that has received a rewrite that hasn't reached feature parity yet.
I am probably the type of person referred to in the last part of xkcd 1782.
OT: I have some very big groups in Telegram (7 years or more, with a lot of pictures). Can Matrix (Rocketchat or alternatives) have similar storage features (with some migration scripts)? Thanks
I have been hosting synapse for 2 years now and it's been a smooth sail. I don't recall having any major breaking changes, most updates are smooth. Element client itself is definitely PITA but it's getting better
TLDR Self-hosting isn’t dying because people stopped caring. It’s dying because the complexity has gotten out of hand.
This post highlights how something that used to be a fun, lightweight hobby has turned into a full-time maintenance burden. Systems like Matrix are powerful, but they’ve become so intricate that even skilled engineers struggle to run them reliably. The result is a slow drift back toward centralized platforms, not out of preference, but because convenience keeps winning over autonomy. It’s a reminder of the growing gap between the ideal of a user-owned internet and the realities of modern software.
I would say it became much easier in recent years. docker-compose became defacto server "app install", any linux supports it. That includes GUI options like Truenas/Unraid and very nice admins like Dockge exist.
The company behind matrix is aiming at huge scale servers but if you care about unfederated private instances you will find there are few much simpler "one binary" projects that can even use file based sqlite/rocksdb. Hosting those couldn't be simpler. You actually don't even need docker just systemd service and switch binary when updating.
Yeah, I think this is true and it's unfortunate. I think it's one of the areas where what seemed like the plan for Matrix hasn't totally worked out in practice. And it's even just the intricacy of managing it; some of the problems are deeper in the design. When I started using it several years ago, it seemed their vision was a lot of people running a lot of small servers. But the full-replication nature of the protocol, plus the resource demands of the server software, make that kind of impractical. Even tech-oriented people may shy away if they find out the database could grow to tens of gigabytes, and Synapse is not exactly light on RAM or CPU either.
Perhaps in the future if implementations improve some of this may get better, and it will become more feasible for small operators to run their own servers. But by that time it will be harder to build trust because too many people will have written it off as bloated or unstable. I think it would have been better to start lean and keep the system in more of a nerd niche until that process of evolution reached a later stage.
NixOS makes this rather easy too. There exist modules for synapse, livekit, etc (MAS module is still a Pull Request, though) and the setup is quite doable: https://wiki.nixos.org/wiki/Matrix
However, you still need to know what you are doing (the manual helps) and connect the pieces (in theory there could be a nixpkgs module that does this for you but apparently nobody did bother). Once its done you can lean back.
I've been running my homeserver happily for > 5 years and it was fairly straightforward.
Ran a homeserver for 5 years on a minimal VPS and it worked fine. Upsides - works everywhere, self hosted, feature complete. Client software in the ecosystem mostly felt bloated, with the exception of NeoChat. By 2022 the clients could no longer call each other. Decommissioned it this year in favor of traditional XMPP which works fine and it's nice that notifications are appropriately processed, finally.
Our team highly appreciates the work done in Matrix it's just unfortunate that the elephant in the room was never addressed at the start of the project, which is the need for a -simple- first-party administrative dashboard or tool to manage users, storage, and configuration. Without that core component, then you've got a layer of complexity between an admin and an audit which will increase likelihood of misconfiguration or resource management issues.
Thanks for mentioning XMPP. I never ran a server, but as a user I've been enjoying it. Also has its problems - for example, I picked some server to create my first account, and the other day the server disappeared for 6 hours. What do normal users (i.e. non-newbies like me) do in practice in these situations?
That said, I would love to hear your experience running an XMPP server. Do you still run it?
Trusting your chat activity and availability to any XMPP server on the internet that allows free public registration is not ideal. If you want reliability then pay for it or use what you perceive as a highly credible free server. I do not run any public facing XMPP services so I do not have any input on that and as with Matrix I wouldn't roll your own unless you know exactly what you're doing due to inherit risks associated with these platforms in certain states of configuration or exposure.
> The only thing that I don't really understand is the decision on data replication. If a user on server A joins a room on server B, recent room data is copied from server B to server A and then kept in sync on both servers.
The idea here is that rooms are abstracted from servers and sort-of exist ephemerally. This has the advantage/disadvantage of making it hard for the underlying infrastructure to exert control over the hosted communities, and seems to have become a distinguishing feature of federation.
My experience of Matrix as a possible replacement for Discord has led me to believe it's mostly a disadvantage since it leads to gross misalignments between the communities in top and the infrastructure providers underneath. I consider e.g. Discourse to be much healthier (although I would like to see an app for Discourse so that my Discourse communities behave more like Discord/Slack servers) and it's frustrating to me that there hasn't been a clear "Discourse for chat" emerge to replace Discord.
> The idea here is that rooms are abstracted from servers and sort-of exist ephemerally
No, that's not even remotely true. In fact the opposite is true. The domain name of the server used to create the room is perpetually and permanently embedded in the room name and can't be changed, ever.
> it leads to gross misalignments between the communities in top and the infrastructure providers underneath
Yeah, this is a great way to put it. We see this a lot with the mod experience. So much of the mod tooling is "run this bot on your server" or "configure synapse like this". But those kinds of things are inaccessible to people who aren't able to run any server of any kind. Of course someone else can run the mod bot for them, but then you still have the same problem where it introduces a layer of friction between the person who needs to perform mod actions and the person who controls the mechanism for doing so.
Until there's a robust and full-featured mod toolkit built in at the protocol/client level, it's a dicey situation for people who want to use Matrix to host a community. The insidious part is that everything may seem fine until suddenly your room is flooded with images of gore or worse.
(By the way, we still miss you in the Python room on Matrix! :-)
I set up Whatsapp and Telegram bridges with Tuwunel today. The config file for mautrix-whatsapp indicates Dendrite should work as well so it seems just about every server implementation supports bridges?
> While technically, Synapse can work with a sqlite database (and which at first seems like an OK choice for having <10 users on the server), it WILL become corrupted.
Does anyone have any more information on this? Running Postgres is not a big deal, but I would expect SQLite to be fine given how well it works in my experience.
I've been running unfederated Matrix instance with sqlite for 10 years for 10-20 people some very active. In WAL mode sqlite is fine i am not sure why it would get corrupted.
Few years ago we did a data/history wipe (i managed to migrate accounts) because we switched to conduit.rs with sqlite. I very much prefer conduit to synapse.
I never had issue with sqlite it was mostly Synapse. With recent changes i think they are completely dropping the ball on small instances and it feels like the ecosystem is splitting. Element X is just not very good. It seems to exist to kill classic element and since for element x you need some server features that not all alternative synapse servers support.
> No image captions
>
> This is silly, but while (official?) bridges support image captions, official Element app does not.
Element Web & X support captions. (Element Web doesn't currently support authoring them, but can display them - obviuosly this is on the todo list though).
Element Classic has basically not been touched in 2.5 years, unfortunately, and so yes - doesn't support them. But at this point the old app is just not being developed; we don't have bandwidth to do both.
> Slow notifications
This sounds like it might be overloaded server problems ftr.
> Element X is Slower
> Somehow, it is slower. Clicking on a conversation takes 0.5-1.0 seconds to load it, compared to almost instant load on Classic.
> Sorry, but you have to run an auth server (matrix-authentication-service) if you want Element X to work.
This is a bit outrageous IMO. Actually breaking and deprecating the classic auth and requiring a new server component to keep the only actively supported client (which still can't properly manage keys or sessions on its own, like classic Element can, even as non-verified sessions are being disabled) is a bit rich.
> if i've got any of the above wrong please just say, rather than downvote :D
Maybe it's not necessarily that any of those are wrong per se as that they just don't address several other important issues raised in the article, such as (copied from headings in the article):
* Needs constant cleanup
* Database grows out of control
* Element X is now recommended as the new and better client. It is not.
* Calls are not backward compatible
Also saying the info is "outdated" because a bug was fixed two weeks ago (and you're not even sure if it made it into a built yet!) seems pretty disingenuous. More generally, I think the timeline on which info becomes "outdated" is maybe different for you than for actual users. Actual users expect to install it and have everything work. If there are major bugs, protocol changes, and app-throwaway-and-rewrites happening on a timescale of 2-3 years, that's still too fast for most people to consider their info outdated. If everything were going swimmingly with none of these problems for like 5 years, then yeah, okay, but that's not the situation.
god if there was a comprehensive chat protocol without a loser ceo i'd move to it tomorrow. matrix org staff don't know how to interact with other human beings.
I've been developing open source code for over 25 years. i've deployed hugely complicated systems like hadoop. I've never seen anything as hard to run as this + the bridges.
Been selfhosting synapse for about 1.5 years in a docker compose setup using bunkerweb (formerly "bunkerized nginx", which better explains it premise) reverse proxy, eturnal for TURN and postgres, also recently added livekit and MAS for element call and element X compatibility. All that runs on a small 2vcore/4gb VPS, and it runs pretty good, I experience a server crash every half a month, but that may be caused by the fact that bunkerweb isn't the most lightweight solution (they actually require 8GB RAM minimum, so I'm already beneath the limit), and also because I run some other software (mailserver, ebook server, plex, etc..).
My experience as a administrator has been pretty good, perhaps it's because from the beginning I was optimistic, it suited my needs as I wanted a selfhosted, modern and fairly convenient communication platform. From what I recall, most problems during configuration were caused mostly by bunkerweb (or rather my inability to correctly set it up to proxy requests correctly and not hijack the 4xx and 5xx HTTP codes). Synapse itself has been a pleasure to maintain, but also bear in mind that I did not tinker with with it, I basically set it up and let it run for about a year and then added MAS and livekit.
Yeah, disk usage sucks, for about 5-10 active users and 1.5 year usage my postgres "schemas" folder clocks at 10Gibs. It doesn't include the media_store catalog where synapse keeps media (images, videos). The homeserver is federated and I joined a couple of big rooms in the past. Mechanics mentioned in the links below do help though:
Clientwise also sucks, but I think enough has already been said on this matter. But it's good enough to keep my nontechnical friends using it. They do hate it, but not enough to kick me in the arse. Would love to say that this proves that element clientside is usable, but I also have to admit that my friends are just hella good guys who would even write pigeon mail to me if I stopped using anything else for communication :) for me as a techie, element is obviously alright. Clunky, but works. I think clients simply need more time.
What irritates me is the Matrix authentication service (MAS), it's kind of a separate service for matrix homeservers that handles logins specifficaly. You can't use element X without it. However when it's enabled, you cannot log in from your client, instead a web browser opens and shows the login panel where you have to authorize, and then it should return to the client. Except in my case it simply doesn't :( I observed that for some reason chromium based browsers won't redirect back to the element app, and it doesn't know that the authorization has been granted. I managed to bypass it by copying the URL and opening it on firefox, but in one instance even that didn't work.
But other than that MAS problems everything has been fine from administration standpoint. I think it simply needs more time, as it already has traction, I see that a lot of new projects seem to include a matrix room in their social/communication channels, frequently it's the only option besides the bugtracker. And I'm willing to wait patiently :)
edit: added links for people who also struggle with disk space usage
49 comments
[ 0.21 ms ] story [ 88.2 ms ] threadKnowing which XEPs you actually want is a bit of a pain, but has gotten easier. Now I usually just periodically look at the current prosody example[1] which lists extensions by "essential" "recommended" "nice to have" and "other" and it's easy enough to find the matching modules for ejabberd.
1: https://prosody.im/doc/example_config
(a) the encryption between using a mobile and the webapp desyncs/breaks all the time, it just sucks. I mean you'll get "cannot decrypt" a lot, have to bounce back and forth and generally try and force it to re-sync properly again. Sometimes never worked at all. Lots of issues on GH over the years.
(b) as mentioned in this article, insane delays on new message notif and sending and receiving. Just logging in on the webapp every morning took minutes of some sort of mysterious sync process, often the mobile app had the same problems. The X stuff may fix this, we were pre-X.
(c) cleanup. There's no message retention set on matrix.org, when I wanted to extract and remove our past chats the process and experience was excruciatingly bad. It took tens of hours over several weekends of the webapp (mobile completely non-op in practice for this) polling and loading old content, just so I could select 100 at a time to delete and then it took an hour. Once I started culling back over a year or so, the loading got longer and longer and longer, until eventually it 100% stopped working at all to load old messages.
Signal and DeltaChat are far, far better experiences for one-on-one chats with friends & family. The Delta client is a bit UI/UX behind but not horrible; e.g. you can't correct a typo in a sent message in Delta, unlike Signal - because each msg is a unique gpg-encrypted "email" rather than a database object that can be re-manipulated.
When I started using Matrix, Riot.im did not get notifications in time, or was a battery hog. And Synapse took up a lot of memory, with occasional slowdowns. But then, circa 2022, Synapse improved and Element seems to have worked well and was consistent and reliable on all platforms.
I legitimately could go on and I'm sure I've forgotten things. It's amazing how quickly my experience with Dendrite went from pretty good to nightmare.
I realized that nobody in charge at the Matrix Foundation or New Vector really cared enough about leaving people stranded on a completely broken server to actually do anything about it (and trust me, I'm not alone. In every single federated room I've ever been in, I've always seen hostnames with dendrite subdomains. I could see them pass by in the logs while joining servers was taking hours.) I honestly considered just leaving the Matrix ecosystem, but I wasn't alone on my home server, so I decided to do my best to fix the problem. I wrote a tool that attempts to migrate the data from Dendrite to Synapse. This is a complicated operation that really took a huge amount of effort to get working, but after a couple of months of failed attempts, I had a test where I was able to seamlessly perform the migration and have clients continue to work and stay in federated rooms. So after getting it "close enough", I went ahead and gave the migration a shot in production and of course, it didn't work very well. All of the user accounts were intact, but a lot of stuff was broken. People indeed stayed in federated rooms, but my room state migration was definitely not 100% correct. Despite this, though, after manually cleaning up the database a bit more, hackishly while live, it was mostly a success. I believe I am probably the first person to directly move from Dendrite to Synapse.
So now that I am on Synapse, have my thoughts on Matrix changed? Yes. It's significantly better using Synapse, without question. The ecosystem is still a mess, but everything about Synapse is less broken than Dendrite. There are so many features Dendrite just doesn't do, like URL previews.
Why not contribute to Dendrite? Honestly, I don't want to. Their CLA sucks and they're not going to change it for me, and I don't think they're really going to spend time reviewing PRs given the circumstances. If I'm going to contribute to a project without retaining my rights I'd prefer to be on payroll. That's not something you should get from a community member. Either change the CLA to guarantee the project must stay open, or don't expect any free contributions.
Why not post my migration tool?...
Edit : I wonder how easy it is to backup a Matrix accounts's data. Conversations and files.
Not as easy as you might hope. The Element client has an export feature, but you have to manually activate it on each room/chat, and the export has a size cap so it may not work if you have lots of files you want to save. It's also pretty slow if the room has a lot of history. You could also try using something like Matrix Commander (a command-line client), but I couldn't get that to work fully either.
How is it ironic? No protocol in the world can force anyone to delete anything from their own device. Chat apps that implement this function are either proprietary (so you cannot control what they can do) or, if OSS, do it on a pinky-promise-basis.
In this short time I've run a database migration (sqlite is the default, but MAS requires postgres), tried and failed to migrate to MAS (required to use Element X) and have lost a couple of days messing around with coturn and eturnal with nothing to show for it -- my calls still don't connect when NAT is involved. I have to tell new users to ignore the recommendations to install Element X until I get MAS working.
There's a lot of room for foundational improvements here, even updating docs to point would-be server admins to the recommended setup du jour would help.
I want to hear more about this. Is this because Synapse’s SQLite support is half-baked? What sort of corruption are we taking about?
The fact that I cannot delete attachments that users delete is certainly my biggest irritation, 50GBs of stuff I am not sure if I can or cannot delete, but considering the size, I am just gonna bite the bullet, couple terabytes should not be a problem in 25 years. But this is def something I would love to see addressed sooner rather than later. It must be a pain even for the matrix.org server team.
After moving to a better server I do not have issues with slow notification unless the phone is sleeping for longer period of time which is an android optimization (I'd assume). It is more reliable than teams at this point. One of my friends had issues but removing 15 old devices fixed the issue.
As for element-x, I did call out "the another rewrite" issue especially with android and I do think it makes things worse. I still do not know how am I supposed to fix calling and video between old and new clients. For now I don't bother with new clients and everyone is using old ones, but it starts to become an issue as classic clients are in maintenance only mode
[1] https://conduit.rs/ and https://gitlab.com/famedly/conduit [2] https://gitlab.com/famedly/conduit/-/blob/next/docs/configur...
Looked at current tui offerings now some years later, situation seems to be unchanged, the only client that ran back then was gomuks, and that has received a rewrite that hasn't reached feature parity yet.
I am probably the type of person referred to in the last part of xkcd 1782.
This post highlights how something that used to be a fun, lightweight hobby has turned into a full-time maintenance burden. Systems like Matrix are powerful, but they’ve become so intricate that even skilled engineers struggle to run them reliably. The result is a slow drift back toward centralized platforms, not out of preference, but because convenience keeps winning over autonomy. It’s a reminder of the growing gap between the ideal of a user-owned internet and the realities of modern software.
The company behind matrix is aiming at huge scale servers but if you care about unfederated private instances you will find there are few much simpler "one binary" projects that can even use file based sqlite/rocksdb. Hosting those couldn't be simpler. You actually don't even need docker just systemd service and switch binary when updating.
Perhaps in the future if implementations improve some of this may get better, and it will become more feasible for small operators to run their own servers. But by that time it will be harder to build trust because too many people will have written it off as bloated or unstable. I think it would have been better to start lean and keep the system in more of a nerd niche until that process of evolution reached a later stage.
However, you still need to know what you are doing (the manual helps) and connect the pieces (in theory there could be a nixpkgs module that does this for you but apparently nobody did bother). Once its done you can lean back.
I've been running my homeserver happily for > 5 years and it was fairly straightforward.
Our team highly appreciates the work done in Matrix it's just unfortunate that the elephant in the room was never addressed at the start of the project, which is the need for a -simple- first-party administrative dashboard or tool to manage users, storage, and configuration. Without that core component, then you've got a layer of complexity between an admin and an audit which will increase likelihood of misconfiguration or resource management issues.
That said, I would love to hear your experience running an XMPP server. Do you still run it?
The idea here is that rooms are abstracted from servers and sort-of exist ephemerally. This has the advantage/disadvantage of making it hard for the underlying infrastructure to exert control over the hosted communities, and seems to have become a distinguishing feature of federation.
My experience of Matrix as a possible replacement for Discord has led me to believe it's mostly a disadvantage since it leads to gross misalignments between the communities in top and the infrastructure providers underneath. I consider e.g. Discourse to be much healthier (although I would like to see an app for Discourse so that my Discourse communities behave more like Discord/Slack servers) and it's frustrating to me that there hasn't been a clear "Discourse for chat" emerge to replace Discord.
No, that's not even remotely true. In fact the opposite is true. The domain name of the server used to create the room is perpetually and permanently embedded in the room name and can't be changed, ever.
Yeah, this is a great way to put it. We see this a lot with the mod experience. So much of the mod tooling is "run this bot on your server" or "configure synapse like this". But those kinds of things are inaccessible to people who aren't able to run any server of any kind. Of course someone else can run the mod bot for them, but then you still have the same problem where it introduces a layer of friction between the person who needs to perform mod actions and the person who controls the mechanism for doing so.
Until there's a robust and full-featured mod toolkit built in at the protocol/client level, it's a dicey situation for people who want to use Matrix to host a community. The insidious part is that everything may seem fine until suddenly your room is flooded with images of gore or worse.
(By the way, we still miss you in the Python room on Matrix! :-)
This is not true, at least today. Continuwuity, which is an alternative server implementation, and its predecessors support bridges very well.
Does anyone have any more information on this? Running Postgres is not a big deal, but I would expect SQLite to be fine given how well it works in my experience.
Few years ago we did a data/history wipe (i managed to migrate accounts) because we switched to conduit.rs with sqlite. I very much prefer conduit to synapse.
I never had issue with sqlite it was mostly Synapse. With recent changes i think they are completely dropping the ball on small instances and it feels like the ecosystem is splitting. Element X is just not very good. It seems to exist to kill classic element and since for element x you need some server features that not all alternative synapse servers support.
> Does not have an admin panel
The admin panel is at https://github.com/element-hq/element-admin (but it's relatively new, so many folks haven't noticed it exists)
> No image captions > > This is silly, but while (official?) bridges support image captions, official Element app does not.
Element Web & X support captions. (Element Web doesn't currently support authoring them, but can display them - obviuosly this is on the todo list though).
Element Classic has basically not been touched in 2.5 years, unfortunately, and so yes - doesn't support them. But at this point the old app is just not being developed; we don't have bandwidth to do both.
> Slow notifications
This sounds like it might be overloaded server problems ftr.
> Element X is Slower > Somehow, it is slower. Clicking on a conversation takes 0.5-1.0 seconds to load it, compared to almost instant load on Classic.
This was an Android specific bug which was fixed a few weeks ago; EXA should now be instant as it should be, at least on nightlies: it was stuff surrounding https://github.com/matrix-org/matrix-rust-sdk/pull/5841 and https://github.com/matrix-org/matrix-rust-sdk/pull/5854. Unsure if the fix has made it into a build yet.
EDIT: the fix shipped in stable Element X Android a few weeks ago.
> Conversations are sorted by... who knows. It is not recent nor alphabetical.
It should be by recency.
> Onboarding is bad
Sorry, but you have to run an auth server (matrix-authentication-service) if you want Element X to work.
EDIT: if i've got any of the above wrong please just say, rather than downvote :D
> Sorry, but you have to run an auth server (matrix-authentication-service) if you want Element X to work.
This is a bit outrageous IMO. Actually breaking and deprecating the classic auth and requiring a new server component to keep the only actively supported client (which still can't properly manage keys or sessions on its own, like classic Element can, even as non-verified sessions are being disabled) is a bit rich.
Maybe it's not necessarily that any of those are wrong per se as that they just don't address several other important issues raised in the article, such as (copied from headings in the article):
* Needs constant cleanup
* Database grows out of control
* Element X is now recommended as the new and better client. It is not.
* Calls are not backward compatible
Also saying the info is "outdated" because a bug was fixed two weeks ago (and you're not even sure if it made it into a built yet!) seems pretty disingenuous. More generally, I think the timeline on which info becomes "outdated" is maybe different for you than for actual users. Actual users expect to install it and have everything work. If there are major bugs, protocol changes, and app-throwaway-and-rewrites happening on a timescale of 2-3 years, that's still too fast for most people to consider their info outdated. If everything were going swimmingly with none of these problems for like 5 years, then yeah, okay, but that's not the situation.
My experience as a administrator has been pretty good, perhaps it's because from the beginning I was optimistic, it suited my needs as I wanted a selfhosted, modern and fairly convenient communication platform. From what I recall, most problems during configuration were caused mostly by bunkerweb (or rather my inability to correctly set it up to proxy requests correctly and not hijack the 4xx and 5xx HTTP codes). Synapse itself has been a pleasure to maintain, but also bear in mind that I did not tinker with with it, I basically set it up and let it run for about a year and then added MAS and livekit.
Yeah, disk usage sucks, for about 5-10 active users and 1.5 year usage my postgres "schemas" folder clocks at 10Gibs. It doesn't include the media_store catalog where synapse keeps media (images, videos). The homeserver is federated and I joined a couple of big rooms in the past. Mechanics mentioned in the links below do help though:
https://matrix-org.github.io/synapse/v1.40/admin_api/purge_h...
https://github.com/matrix-org/rust-synapse-compress-state
Clientwise also sucks, but I think enough has already been said on this matter. But it's good enough to keep my nontechnical friends using it. They do hate it, but not enough to kick me in the arse. Would love to say that this proves that element clientside is usable, but I also have to admit that my friends are just hella good guys who would even write pigeon mail to me if I stopped using anything else for communication :) for me as a techie, element is obviously alright. Clunky, but works. I think clients simply need more time.
What irritates me is the Matrix authentication service (MAS), it's kind of a separate service for matrix homeservers that handles logins specifficaly. You can't use element X without it. However when it's enabled, you cannot log in from your client, instead a web browser opens and shows the login panel where you have to authorize, and then it should return to the client. Except in my case it simply doesn't :( I observed that for some reason chromium based browsers won't redirect back to the element app, and it doesn't know that the authorization has been granted. I managed to bypass it by copying the URL and opening it on firefox, but in one instance even that didn't work.
But other than that MAS problems everything has been fine from administration standpoint. I think it simply needs more time, as it already has traction, I see that a lot of new projects seem to include a matrix room in their social/communication channels, frequently it's the only option besides the bugtracker. And I'm willing to wait patiently :)
edit: added links for people who also struggle with disk space usage
I don't... what? How can you... arrrgh!