Fiware IoT Agent json is changing the service path of incoming messages if it is high load

[pratappulugoru]

IoT Agent JSON version the issue has been seen with

1.20

Bound or port used (API interaction)

Northbound (Provision API and NGSI Interactions)

NGSI version

NGSIv2

Are you running a container?

Yes, I am using a contaner (Docker, Kubernetes...)

Image type

normal

Expected behaviour you didn't see

We are currently using the VernemQ MQTT server in conjunction with a Fiware stack that comprises the IoT Agent and Orion Context Broker. When sending data to the IoT Agent, we utilize the message topic format api-key/device-id/attrs.

When the IoT Agent receives a message on this topic, it uses the provided 'api-key' to determine both the service path and service name. Once these values are determined, the payload is then forwarded to the Orion Context Broker using the v2/entities/ API.

Our setup works perfectly when the data rate is relatively low, around 40,000 records per hour. However, when the data rate increases to approximately 80,000 records per hour, we've noticed that the IoT Agent appears to change the service provider from, for example, 'serviceProvider1' to 'serviceProviderXy'.

We're currently at a loss as to why this change is occurring, and it is causing significant security concerns for our system. We would greatly appreciate any insights or assistance in understanding and resolving this issue.

We tried running multiple tests with higher data rate(>=80k records per hour) and observed in all the tests this phenomenon is common. When we run tests with lesser data rate, this is not occuring.

iotagent-node-lib - 2.18.0 version.

iotagent-json 1.20.0 version.

Orion - 3.4.0 version.

Unexpected behaviour you saw

No response

Steps to reproduce the problem

No response

Configs

environment:
    - "IOTA_CB_HOST=orion"
    - "IOTA_CB_PORT=1026"
    - "IOTA_NORTH_PORT=4041"
    - "IOTA_REGISTRY_TYPE=mongodb"
    - "IOTA_MONGO_HOST=mongodb"
    - "IOTA_MONGO_PORT=27017"
    - "IOTA_MONGO_DB=iotagent-json"
    - "IOTA_HTTP_PORT=7896"
    - "IOTA_PROVIDER_URL=http://iot-agent:4041"

Log output

No response

[AlvaroVega]

@pratappulugoru both iotagent and iota-node-lib are quite old, You should try with newer versions of iota where that issues where addressed.

[AlvaroVega]

@pratappulugoru since many versions ago (you can see full doc in https://github.com/telefonicaid/iotagent-json/blob/master/docs/usermanual.md#mqtt-binding) measures are sent on the /<protocol>/<api-key>/<device-id>/attrs topic, in case of use iotagent-json protocol is json

[pratappulugoru]

@AlvaroVega Thanks for the update. I will try with the latest version. Can you help me in understanding how Iot Agent can switch the service provider automatically. Is their any bug and fixed in latest versions. But i have not seen any thing like this in the release notes of latest versions.
Can you please help me in this.

[AlvaroVega]

I have no previous info about an iotagent can switch the service provider automatically, just some bugs related about logs (context domains) were printing service/subservice in a wrong way.

[mapedraza]

@pratappulugoru can you confirm you are receiving entities updates with a different fiware-service (I mean, if you have a context broker, there are entities created in the Context Broker, not only log outputs)?

[pratappulugoru]

Yes, we have recieved the entities with different service paths. Our probelm is if we are using multiple service path's then we are observing that only some requests are throwing Socket errors and those requests did not reach to Orion.- But all successfull messages reached to orion with different service path's also.

[AlvaroVega]

@pratappulugoru did you tried with latest iotagent version?

[pratappulugoru]

@AlvaroVega Not yet. We are planning to do in this week. I will let you know the results once test is done. I have recorded this issue in Orion.
telefonicaid/fiware-orion#4429
Please see the above link and i have provided all environmental variables of Orion also. Please look into. Iam suspecting on some of the environmental variables like "-multi service" and "notification mode". Iam currently using the default values . Please suggest if any changes required

[AlvaroVega]

It's very strange, neither old or newer versions we reproduce the problems your describe. Anyway, if you can provide any other detail, PoC, testCase or way for fully reproduce it we will appreciate it.

[pratappulugoru]

@AlvaroVega can you go thorough the below link
telefonicaid/iotagent-ul#383
I have provided all the details in the above link

[AlvaroVega]

@pratappulugoru telefonicaid/iotagent-ul#383 is about services/subservices showed in logs, not received in CB entities. IMHO is not the same.

[mapedraza]

Could you please share a step by step guide to reproduce it? This means:

1. A docker-compose.yml file containing all the services deployed or involved in your test case, with all your enviroment variables and config files related with those services
2. The script/service/tool to send the requests (135346 messages for 1 hour duration i.e 37 messages per second)
3. Curl request to provision to create the entities on the context broker and/or device group or devices provision for the IoT Agent

If you do not provide this, it would be impossible for us to reproduce the issue and give you support.

Thanks in advance

[pratapmai]

@AlvaroVega we have upgraded the iotagent and orion to the latest version. But still we are seeing the service provider mismatch in the logs. Its consistent issue. We are observing from last six months. Can you please help here?. For single service path, its works fine.

[AlvaroVega]

Hi @pratapmai, Where is the script that you use to ingest measures for different service path? Thank in advance