Consider moving MQTT auth out of process

[amrc-benmorrow]

Currently the HiveMQ MQTT auth code runs in-process within the MQTT broker. This is the way HiveMQ expect auth plugins to run and is common practice generally in the Java world but has some disadvantages:

• The code is becoming quite complex and very F+-specific. A lot of additional code is being pulled into the MQTT broker, which is (in many ways) the most critical part of the whole F+ infrastructure.

• The threading facilities provided by the HiveMQ Community Edition plugin interface are inadequate for any real work, especially given that we are using GSSAPI libraries which cannot run asynchronously. The code works around this by building its own thread pool but this also indicates that we are trying to do too much inside the broker.

• Providing more dynamic ACL changes will require watching for change-notify packets from the services. Doing this from within the broker will mean using the packet-snooping APIs, which do not have the ability to filter the packets we watch. This is likely to both make it complicated to identify the packets we are interested in (we don’t have an MQTT client library to help) and run a high risk of slowing down all traffic unnecessarily.

We should consider whether it is sensible to instead implement a thinner plugin, which interfaces with a privileged client over MQTT. Client connections and ACL changes are not high-frequency events, so as long as the broker-side code has a ‘current ACL’ available for per-packet checking this should not impact performance.

If the interface is sensibly designed then longer-term it might be possible to implement the broker-side piece in Mosquitto. This would give us bridging and probably better performance. This would require core changes to Mosquitto; the current plugin interface is entirely synchronous (calls block the whole broker) and so completely inadequate for network-based auth.

[amrc-benmorrow]

Having looked further into the HiveMQ APIs it’s less clear this would be an improvement. Watching for packets from a privileged client will still require hooking into inbound PUBLISHes, and switching away from the core ACL support means every packet will have to be explicitly checked against our ACL.

HiveMQ has two sets of APIs that could be used to implement dynamic ACLs: the Authorizer APIs and the Interceptor APIs. As far as I can tell, they overlap substantially, the differences being:

• The Publish Authorizer API is called to authorise the Will in a CONNECT packet.
• It’s not clear whether the Connect Interceptor could accomplish this; the interaction with the authentication API is not specified explicitly.
• Only the Interceptor API would allow us to prevent delivery of packets once a client has been granted a subscription.

This suggests we might need to use the Authorizer API for publishes and the Interceptor for subscribes, which would be annoying.