Waiting for connection to be established before sending data to ThingsBoard!

[MannimMond86]

Title: tb_device_mqtt._publish_data triggers warning when publishing before is_connected() returns True

Description:
Has anyone experienced this issue with uv 0.8.8?
(Tested with both requires-python = "==3.11.*" and requires-python = ">=3.12".)

In tb_device_mqtt.py (line 1211), inside the _publish_data function, a warning is logged:

Waiting for connection to be established before sending data to ThingsBoard!

This occurs when the client attempts to publish data before is_connected() returns True.
If gw_send_telemetry or similar methods are called immediately after connect(), the connection handshake may not have completed yet, causing this warning.

---

Steps to Reproduce:

1. Create a minimal script:

import time
from tb_gateway_mqtt import TBGatewayMqttClient

gateway = TBGatewayMqttClient("thingsboard.teveo.de", username="B5TDJfR43CKEM3TSP0Vx")
gateway.connect()
gateway.gw_connect_device("Test Device A1")

gateway.gw_send_telemetry("Test Device A1", {"ts": int(round(time.time() * 1000)), "values": {"temperature": 42.2}})
gateway.gw_send_attributes("Test Device A1", {"firmwareVersion": "2.3.1"})

gateway.gw_disconnect_device("Test Device A1")
gateway.disconnect()

2. Run with:

uv run python main.py

3. Observe the warning message.

---

Environment:

Package             Version
------------------- --------
certifi             2025.8.3
charset-normalizer  3.4.3
idna                3.10
mmh3                5.2.0
orjson              3.11.1
pymmh3              0.0.5
requests            2.32.4
setuptools          80.9.0
tb-mqtt-client      1.13.9
tb-paho-mqtt-client 2.1.2

---

Expected Behavior:
Publishing methods should either:

• Block until the connection handshake is complete, or
• Provide a documented callback/event to signal when it is safe to publish.

Actual Behavior:
A warning is logged when attempting to publish immediately after calling connect().

Possible Fix:
Ensure connect() waits until is_connected() returns True before allowing gw_send_telemetry or similar publish operations to proceed.

tb_device_mqtt._publish_data triggers warning when publishing before is_connected() returns True

Technical Analysis of _publish_data Behavior

In tb_device_mqtt.py (lines 1198–1216), the _publish_data function operates as follows:

1. QoS Validation

   • If qos is not provided, the function uses the default self.quality_of_service.
   • Only values 0 and 1 are allowed; any other value raises a TBQoSException.

2. Wait Until Connected

   • While self.is_connected() returns False:
     • If self.stopped is True, the function returns immediately without publishing.
     • If more than 10 seconds have passed since the last log message, it logs:

       Waiting for connection to be established before sending data to ThingsBoard!
       

       and resets the timer.
     • Sleeps for 0.01 seconds before checking the connection state again.

3. Publish Data

   • Once the connection is established, _send_request is called to publish the data.

---

Reason for the Warning

This warning typically occurs when gw_send_telemetry or gw_send_attributes is called immediately after connect().
At that point, the MQTT client often has not yet completed its connection handshake, so self.is_connected() is still False. As a result, _publish_data enters its waiting loop and logs the warning message.

---

Proposed Improvement

To avoid logging this warning during normal startup:

Option A – Improve connect() Behavior

• Modify connect() to block until the MQTT connection handshake is complete and is_connected() returns True.
• This would ensure that any publish operation immediately after connect() is safe.

Option B – Adjust _publish_data Logging Logic

• Suppress the warning during the initial connection attempt.
• Only log the “Waiting for connection…” message if:
  • The client was previously connected and has now disconnected, or
  • The initial connection attempt has exceeded a configurable threshold (e.g., 5–10 seconds).

Option C – Provide an Explicit Ready Event or Callback

• Expose a mechanism (event, callback, or future) that signals when the client is fully connected, so users can hook into it before publishing.

---

These changes would prevent misleading warnings in typical usage scenarios and make connection handling more predictable for developers.

Proposal to Improve Connection Handling and Warning Behavior in tb_device_mqtt

This document proposes concrete changes to improve developer experience and reduce misleading warnings when publishing immediately after connect().

Context

Currently, _publish_data (around lines 1198–1216) actively waits until is_connected() returns True. During the initial connection phase, it logs the warning:

Waiting for connection to be established before sending data to ThingsBoard!

This can appear in normal startup scenarios when a user calls gw_send_telemetry or gw_send_attributes right after connect(), even though the system is behaving correctly and will soon be connected.

---

Goal

• Avoid misleading warnings during normal startup while preserving robust behavior.
• Provide a clear, documented point at which it is safe to publish messages.

---

Option A — Make connect() Block Until Ready

Idea: Ensure connect() returns only after the MQTT handshake is complete and is_connected() is True (or a timeout occurs). This aligns with typical user expectations: once connect() returns successfully, publishing is safe.

Sketch Implementation (illustrative):

# inside tb_device_mqtt.py
def connect(self, *args, **kwargs):
    # Keep existing connection logic
    timeout = kwargs.pop("timeout", 10)  # default; keep configurable
    super().connect(*args, **kwargs)

    # Actively wait for connectivity
    import time
    start = time.time()
    while not self.is_connected():
        if time.time() - start > timeout:
            raise TimeoutError("Connection to ThingsBoard not established in time")
        time.sleep(0.05)

Pros

• Simplifies application code; users can safely publish right after connect().
• Eliminates the initial “Waiting for connection…” warning in normal cases.

Cons

• Slightly changes the semantics of connect() (now it blocks until handshake). Consider documenting clearly and keeping a timeout parameter.

---

Option B — Defer/Suppress the Warning in _publish_data During Initial Connect

Idea: Keep the current waiting loop but suppress or defer the warning during the initial connection phase. Only log the warning if:

• The client was previously connected and is now disconnected, or
• The initial connect attempt surpasses a reasonable threshold (e.g., 5–10 seconds).

Sketch Implementation (illustrative):

# inside _publish_data
from time import monotonic, sleep

waiting_for_connection_message_time = 0.0
first_connect_start = monotonic()

while not self.is_connected():
    if self.stopped:
        return TBPublishInfo(paho.MQTTMessageInfo(None))

    elapsed_since_last_msg = monotonic() - waiting_for_connection_message_time
    total_elapsed = monotonic() - first_connect_start

    # Defer the first warning during initial connect
    if total_elapsed > 5.0 and elapsed_since_last_msg > 10.0:
        log.warning("Waiting for connection to be established before sending data to ThingsBoard!")
        waiting_for_connection_message_time = monotonic()

    sleep(0.01)

return self._send_request(...)

Pros

• Maintains current non-blocking connect() behavior.
• Reduces noise by avoiding warnings in normal startup sequences.

Cons

• Users still need to manage “ready-to-publish” timing themselves.

---

Option C — Provide an Explicit “Ready” Signal

Idea: Expose a callback/event/future that fires when the client becomes connected. For example:

# Pseudocode
client.on_ready = lambda: ...
# or
await client.ready()  # async future resolves on connection

Pros

• Offers a clean integration point for applications without changing connect() blocking semantics.
• Works well in both sync and async designs.

Cons

• Requires API surface changes and documentation.

[MannimMond86]

oh ok sry got longer then expected.... but maybe helpfull, best regards from Germany, Kiel Tjark ( from P.O.S. Experience )

[MannimMond86]

import time
import warnings
import track # aktiviert die Logging-Filter

#warnings.filterwarnings("ignore", category=UserWarning, module="pkg_resources")
#warnings.filterwarnings("ignore", category=UserWarning, module=r"^utils$")
warnings.filterwarnings("ignore", message=r"pkg_resources is deprecated as an API", category=UserWarning)

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger().setLevel(logging.DEBUG)

paho mqtt (falls verwendet)

try:
import paho.mqtt.client as mqtt
mqtt.logger.setLevel(logging.DEBUG)
except Exception:
pass

--- Debug-Hook: zeigt, wer die Zeile ausgibt ---

import builtins, inspect, sys

TARGET = "Waiting for connection to be established before sending data to ThingsBoard!"
orig_print = builtins.print

def traced_print(*args, **kwargs):
msg = " ".join(str(a) for a in args)
if TARGET in msg:
# wenige Frames reichen meist
stack = inspect.stack()
# 1 = Aufrufer von print(), 2 = dessen Aufrufer, …
culprit = stack[1]
file, line, func = culprit.filename, culprit.lineno, culprit.function
orig_print(f"[PRINT ORIGIN] {file}:{line} in {func}")
return orig_print(*args, **kwargs)

builtins.print = traced_print

--- Ende Debug-Hook ---

from tb_gateway_mqtt import TBGatewayMqttClient
gateway = TBGatewayMqttClient(
host="XXXXXXXXXXXXXXXXXX",
port=1883, # wichtig: unverschlüsselt
username="YYYYYYYYYYYYYYYYYYYYY", # Gateway-Device-Token
tls=False, # kein TLS
keep_alive=60
)
gateway.connect(timeout=10)
start = time.time()
while not gateway.is_connected():
if time.time() - start > 10:
raise TimeoutError("Verbindung zu ThingsBoard nicht aufgebaut")
time.sleep(0.05)

gateway.gw_connect_device("Test Device A1")

gateway.gw_send_telemetry("Test Device A1", {"ts": int(round(time.time() * 1000)), "values": {"temperature": 42.2}})
gateway.gw_send_attributes("Test Device A1", {"firmwareVersion": "2.3.1"})

gateway.gw_disconnect_device("Test Device A1")
gateway.disconnect()