golang paho mqtt dropping messages

Question

I've a mosquitto broker that is running in server behind a firewall. Ports needed are open and, from outside, I check that it's working with:

mosquitto_sub  -h [ip address] -t "#" -u [username] -P [password] -v


mosquitto_pub  -h [ip address] -t "topic" -u [username] -P [password] -m "hello"

and I can see the message published. I wanted to do the same with a small go program, the code is the following:

package main

import (
    "crypto/tls"
    "fmt"
    "time"
    "strconv"
    MQTT "github.com/eclipse/paho.mqtt.golang"
)

func messageHandler(c MQTT.Client, msg MQTT.Message) {
    fmt.Printf("TOPIC: %s\n", msg.Topic())
    fmt.Printf("MSG: %s\n", msg.Payload())
}

func connLostHandler(c MQTT.Client, err error) {
    fmt.Printf("Connection lost, reason: %v\n", err)
}

func main() {

    opts := MQTT.NewClientOptions()

    skipVerify := true
    opts.AddBroker("tcp://[ip]:1883")
    opts.SetTLSConfig(&tls.Config{InsecureSkipVerify: skipVerify})
    //opts.SetTLSConfig(&tls.Config{InsecureSkipVerify: skipVerify, ClientAuth: tls.NoClientCert})
    opts.SetClientID("myclientid")
    opts.SetAutoReconnect(true)
    //opts.SetCleanSession(true)
    opts.SetDefaultPublishHandler(messageHandler)
    opts.SetConnectionLostHandler(connLostHandler)

    opts.OnConnect = func(c MQTT.Client) {
        fmt.Printf("Client connected\n")
    }

    client := MQTT.NewClient(opts)
    token := client.Connect()
    token.Wait()
    fmt.Println("connected")
    if token.Error() != nil {
        fmt.Println("problems with connection")
        panic(token.Error())
    }
    for i := 0; i < 10; i++ {
        str := "hello: " + strconv.Itoa(i)
        token := client.Publish("topic/temperature", 0, false, str)
        token.Wait()
        if token.Error() != nil {
            fmt.Println("problems with pub")
        }
        fmt.Println("published")
        time.Sleep(1000 * time.Millisecond)
    }
    fmt.Println("finished")
    client.Disconnect(1)
}

The result I expected was to have every numbered hello published, but only few messages make it through. This is my broker configuration file and I think it has the bare-minimum configurations to make it work:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

log_dest file /var/log/mosquitto/mosquitto.log
log_type debug

allow_anonymous true

port 1883
listener 8883

I can tell I'm doing something wrong with the paho go library because I've done a similar script with python using the python paho mqtt library and it works perfectly:

import paho.mqtt.client as mqtt
import time

topic1 = "topic/temperature"

mqttClient = mqtt.Client()

mqttClient.connect([ip], 1883, 60)
mqttClient.loop_start()
for i in range(10000):
    mqttClient.publish(topic1, "hello - " + str(i), qos= 0)
    #time.sleep(1)
mqttClient.loop_stop()
mqttClient.disconnect()

I've also tried chaging QOS, but the results are similar. What am I missing?

what did your application log reveal, are there failure entries? — Wishwa Perera
Yes there are a bunch of Connection lost, reason: EOF Client connected I forgot to mention that both scripts are running on my raspberry Pi. If I run the same scripts on my computer it works better, there are not connections lost but it still drops some messages. — Jing
does it yield something useful when you enable logging ? github.com/eclipse/paho.mqtt.golang#logging — mh-cbon
Checking the Mosquitto logs would also be beneficial; one possible cause is another client using the same client id (MQTT requires that an existing connection with the same client ID as a new connection be dropped when the new connection is initiated). — Brits
The problem was that there was another machine publishing with the same clientid just like @Brits suggested. — Jing

Brits Brits · Accepted Answer · 2021-07-29T20:55:23

As per the comments - the issue was due to another client using the same client id. The easiest way to check for this is to read the broker logs (from the clients perspective the connection is just dropped without warning).

The MQTT spec states:

The Server MUST process a second CONNECT Packet sent from a Client as a protocol violation and disconnect the Client [MQTT-3.1.0-2]. See section 4.8 for information about handling errors.

This is a fairly common issue (and is the first thing mentioned in the common problems section of the project readme).

golang paho mqtt dropping messages

1 Answers