Inconsistent implementation for MQTT 5 persistent sessions (clean_start)

[skobow]

Bug Description

I have noticed the python client uses clean_start=False for MQTT 5 connections on reconnecting if clean_start is not set by the user in the connect() method. This is caused by clean_start being optional and using MQTT_CLEAN_START_FIRST_ONLY as a default. At the same time the value for session expiry interval is not set or checked leading to possible 0 length interval. Even though this is a valid setting it is a contradictory combination as in this case no session is stored on the broker. The MQTT 5 specification defines the following for session expiry:

If the Session Expiry Interval is absent the value 0 is used. If it is set to 0, or is absent, the Session ends when the Network Connection is closed.

I don't understand the reason behind this implementation as from a user point of view it makes no sense to me. I either want persistent sessions (clean_start=False) or I don't (clean_start=True). The implementation can lead to unwanted / undefined behavior though.

This does also have implications for the broker as this potentially creates unwanted overhead. Creating persistent sessions can involve disk I/O operations as messages sent with QoS greater 0 need to get queued.

Reproduction

Connect to a broker using MQTT 5 w/0 setting clean_start in the connect method. Check connection parameters for client connection. clean_start is set to True. Do a server side disconnect and force the client (same instance) to reconnect. Check the client connect and see clean_start set to False.

Possible Solutions

From my POV there are these possible solutions ordered by favor:

1. Make clean_start non-optional and forcing the user to set a defined value
2. Set False or True to be the default value
3. Keep current logic but set sessions expiry interval to a value greater than zero

Environment

• Python version:
• Library version:
• Operating system (including version):
• MQTT server (name, version, configuration, hosting details): HiveMQ Enterprise 4.35

Logs

[rubenbaer]

Thank you for opening this issue. In my experience, many MQTT clients struggle with properly setting up MQTT sessions. However, this might not be solely a Paho MQTT issue. Proper session and QoS handling requires client session persistence, which is up the user. See also the known limitations.

That said, I don't believe what you describe is an inconsistency or an unintended behavior. It may be surprising in some cases, but it aligns with the specification you referenced and is well-defined.

I also want to highlight that the proposed solutions are not actual solutions. The issue at hand is that the session expiry interval is either unset or set to 0. Simply forcing a value on clean_start does not resolve anything if the session expiry interval remains 0. While setting it to UINT.MAX might seem like a workaround, doing so - especially with generated client IDs - will lead to issues on the broker. Arbitrarily choosing an expiry interval won't necessarily improve the situation either.

Furthermore, the choice isn't simply between:

a persistent session (clean_start=False) or not (clean_start=True)

The situation is a tad more subtle. (Note that you truly want to have a session expiry interval. This is one of the most significant improvements introduced since MQTT 3.1.1.)

I hope to convince you that MQTT_CLEAN_START_FIRST_ONLY is a reasonable option.

The specification states:

4.4 Message delivery retry
When a Client reconnects with Clean Start set to 0 and a session is present, both the Client and Server MUST resend any unacknowledged PUBLISH packets (where QoS > 0) and PUBREL packets using their original Packet Identifiers. This is the only circumstance where a Client or Server is REQUIRED to resend messages. Clients and Servers MUST NOT resend messages at any other time [MQTT-4.4.0-1].

So, the key question is: how can we ensure message delivery retry as specified?

Example: A publishing client with clean_start = False and session_expiry_interval > 0

Consider a scenario where a client publishes a QoS 2 message, but the connection is interrupted before the PUBREL packet is sent. If the client is restarted and a new MQTT client instance is created, it will attempt to reconnect to the existing session. However, since the original client state was lost, the required redelivery cannot happen.

This is why the ideal approach is not just setting clean_start = False or clean_start = True, but rather using MQTT_CLEAN_START_FIRST_ONLY with a positive session expiry interval.

Ultimately, redelivery can only work as long as the original MQTT client instance remains active, since its state is transient.

Question

What do you mean by "defined value" in your first proposed solution? Are you referring to simply True/False, or all three options?