One hell of a nothingburger. Of course you open yourself up to DoS/magnification attacks if you publicly expose a broker with broadcast/fanout functionality.
Yeah what an obvious article. Too bad brokers are all locked down by clientIds / passwords and certificates and if you start acting out of the ordinary you will get banned / blocked. Even 1 packet of a large size can cause being blocked in a good production environment.
Some places take security seriously. Banks do not fuck around on this front.
Some places security is where they shove all the diversity hires. It's not an afterthought it's a dark corner no one cares about.
Other places have security as theater. I know of quite a few companies who have security tools implemented just to have someone to blame when the fuck up happens. XXX was our provider, were sorry sue them, were going to!
The reality is that the decision makers heard what they should do, and made a choice, not understanding the risk. In that regard some of them are getting better (see theater), and making it worse.
So, I have a fair bit of insight into bank operations and their programming practices.
The vast majority of it is security theater. There is quite a bit of decent code near the core moving money portions, but once you start getting into the webapp side of things it quickly devolves into "Please do the basic minimum to ensure security of this application".
To be fair you'd be surprised how many Hassio hoppyist leave their MQTT setup default (or even without password). Was part of a community 2 years ago that was tinkering with IoT for home automation, and most of them were not that familiar with linux, let alone lua, and would leave all their setup default.
In a sense this is still more secure than your common proprietary solution: It is at least illegal for surveillance companies to connect to your open MQTT broker and record your house telemetry to try to get you to buy more crap!
Regarding MQTT in general - is my reading of the spec correct in that there are actually no message ordering guarantees across different topics ? This would imply that the common HA pattern of a single device publishing/receiving commands/status on multiple semantically-named topics (eg /dev001/onoff + /dev001/color) is actually a subtly incorrect use of the protocol? And the way to maintain ordering with the protocol is to have only a single topic for commands and a single topic for status updates? Because this is what I've concluded, but I'd love to be wrong!
That is unfortunate. The spec is pretty clear, but I keep hoping someone will point out that some brokers make that guarantee regardless of it being absent in the spec. Instead it just feels like everyone just ignores that much usage is subtly incorrect, and it's fine in practice.
I do understand the ability implied by the lack of guarantee, it just seems entirely unnecessary, especially for home automation. Sharding in general only seems useful for surveillance companies in the business of massively centralizing computing infrastructure to begin with. And furthermore with MQTT it seems similar scalability could be achieved by scaling out across clients rather than across topics. There is little need to disaggregate if you don't aggregate in the first place!
I've been surprised that a lot of developers have no idea how to use a queue effectively. I think you are right to be suspicious of any "established patterns" you come across, and even more of fancily named queue-based paradigms.
I haven't looked at the HA space, but I guess that explains this absolutely atrocious example they give of a Kafka schema in AsyncAPI.
Speaking only as a Home Assistant novice, not as someone who read the mqtt spec cover-to-cover...
I assume the reason was to keep topics to a single responsibility; if a trigger wants to subscribe to light.state= [on,off] notifications, it only wakes up if that single topic gets notified. It would not have to process light.color notifications or light.brightness notifications.
Sending the smallest payload to the "correct" topic was probably the goal, especially meaning that battery-operated devices only had to monitor the one topic. (Sending gets more complicated, but maybe it balances out in terms of reducing un-needed wakes due to events you do not care about.)
But I agree, that means you lose atomic state change events, unless you also had a topic that had the entire state.
The MQTT spec is actually quite straightforward and not at all tiring, so if you do end up having a reason to read it, don't hesitate to dig in! (I'm not saying you need to though)
I understand the rationale for multiple topics. I went down this road when I was trying to design the layout for my own MQTT-controlled device. I would have really liked taking advantage of topics as something closer to an in-system semantic model, rather than only using MQTT for a bare minimum stream and then stuffing more layers on top.
FWIW It's not merely losing atomic state changes that's the problem, but rather that updates can end up out of order. For example sending the commands [/light/brightness = 30, /light/state = on] can end up turning the light on with the previous (perhaps full) brightness before dimming it. And mitigating this by creating an additional topic with the entire state undoes most of the advantages of splitting it out in the first place, leaving more complexity and resource usage for little gain.
This seems similar to how wireless APs handle queues when devices are placed into 'power-saving' mode; and also the ability to fill the queue-stack with bogus frames
For those who read the article but still don't know, from wiki pedia:
Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name).[8] In the specification opened by IBM as version 3.1 the protocol was referred to as "MQ Telemetry Transport".[9][10] Subsequent versions released by OASIS strictly refers to the protocol as just "MQTT", although the technical committee itself is named "OASIS Message Queuing Telemetry Transport Technical Committee".[2] Since 2013, "MQTT" does not stand for anything.[11][8]
21 comments
[ 3.6 ms ] story [ 68.7 ms ] threadWhy has the security field degenerated to such a shitshow?
Welcome to the shitshow, it's always been this way in computing, security is always an afterthought.
Not everywhere.
Some places take security seriously. Banks do not fuck around on this front.
Some places security is where they shove all the diversity hires. It's not an afterthought it's a dark corner no one cares about.
Other places have security as theater. I know of quite a few companies who have security tools implemented just to have someone to blame when the fuck up happens. XXX was our provider, were sorry sue them, were going to!
The reality is that the decision makers heard what they should do, and made a choice, not understanding the risk. In that regard some of them are getting better (see theater), and making it worse.
So, I have a fair bit of insight into bank operations and their programming practices.
The vast majority of it is security theater. There is quite a bit of decent code near the core moving money portions, but once you start getting into the webapp side of things it quickly devolves into "Please do the basic minimum to ensure security of this application".
It would not be possible to shard topics across multiple nodes if there was an ordering requirement per client.
I do understand the ability implied by the lack of guarantee, it just seems entirely unnecessary, especially for home automation. Sharding in general only seems useful for surveillance companies in the business of massively centralizing computing infrastructure to begin with. And furthermore with MQTT it seems similar scalability could be achieved by scaling out across clients rather than across topics. There is little need to disaggregate if you don't aggregate in the first place!
I haven't looked at the HA space, but I guess that explains this absolutely atrocious example they give of a Kafka schema in AsyncAPI.
https://github.com/asyncapi/spec/blob/master/examples/street...
I assume the reason was to keep topics to a single responsibility; if a trigger wants to subscribe to light.state= [on,off] notifications, it only wakes up if that single topic gets notified. It would not have to process light.color notifications or light.brightness notifications.
Sending the smallest payload to the "correct" topic was probably the goal, especially meaning that battery-operated devices only had to monitor the one topic. (Sending gets more complicated, but maybe it balances out in terms of reducing un-needed wakes due to events you do not care about.)
But I agree, that means you lose atomic state change events, unless you also had a topic that had the entire state.
I understand the rationale for multiple topics. I went down this road when I was trying to design the layout for my own MQTT-controlled device. I would have really liked taking advantage of topics as something closer to an in-system semantic model, rather than only using MQTT for a bare minimum stream and then stuffing more layers on top.
FWIW It's not merely losing atomic state changes that's the problem, but rather that updates can end up out of order. For example sending the commands [/light/brightness = 30, /light/state = on] can end up turning the light on with the previous (perhaps full) brightness before dimming it. And mitigating this by creating an additional topic with the entire state undoes most of the advantages of splitting it out in the first place, leaving more complexity and resource usage for little gain.
Published: Jun 5, 2021
Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name).[8] In the specification opened by IBM as version 3.1 the protocol was referred to as "MQ Telemetry Transport".[9][10] Subsequent versions released by OASIS strictly refers to the protocol as just "MQTT", although the technical committee itself is named "OASIS Message Queuing Telemetry Transport Technical Committee".[2] Since 2013, "MQTT" does not stand for anything.[11][8]