Procedure Name
How to profile event distribution in communication server.
Procedure Steps
Enable the following flag on the communication server that needs profiling in the acs.ini file in the existing [Service] section and restart communication server
DistributionPerformanceCounter=1
Two log files will be generated, distributionPerf.txt and lnlcomsrvr_eventsink_perf.
Lnlcomsrvr_eventsink_perf will show a breakdown of the events, the type (visible event or invisible hardware status event) and statistics for each panel they came from.
This type of profiling will help diagnosing the offending panel(s) in a situation where a large flood of hardware status events occurs in a short amount of time, or when trying to diagnose message bus pipe warnings for too many messages in the publisher queue in LnlLogError.log.
From the LnlLogError.log :
Description: Over 10001 messages in publisher queue. Slowing down new messages retrieval.
This situation can happen during a flood of events to a communication server.
From the extended log lnlcomsrvr_eventsink_perf :
DETAILS: DEBUG MSG: Event sink: Pipe size: 0, ew:366,
Statuses Stat: [Events: 98% Status: 2% ] |
Panel Stat: [(230: 99.7%)(32767: 0.3%)]
DETAILS: DEBUG MSG: Event distr: Pipe:5, ew:1486, 20211022 13:59:22, 7.2ms, Last message details: [Panel 230, Device 1, Aux 1, Message Type: EVENT, EventType: 4, EventID: 555, SN: 1635091659]
Pipe / pipe size : How many events exist in internal communication server event queue prior to distributing to log event thread, client event pipe, and message bus pipe.
Ew : How many items arrived from panels since the last report (one minute) and time / rate of distribution.
Panel Stat : PanelID and percentage of events incoming from each panel since last check.
To profile a single visible event the following query can be executed on the database by adding in the serial number and panelID in the where clause. Note that hardware status events are not visible in the database.
SELECT EVENTS.SERIALNUM,EVENT.EVDESCR as 'Event Description',EVENTS.EVENTID,EVENTS.DEVID as 'Device',EVENTS.MACHINE as 'PanelID',ACCESSPANE.NAME as 'Panel Name',EVENTS.INPUTDEVID as 'INPUT',EVENTS.EVENT_TIME_UTC FROM EVENTS
INNER JOIN EVENT ON EVENTS.EVENTTYPE=EVENT.EVTYPEID AND EVENTS.EVENTID=EVENT.EVID
INNER JOIN ACCESSPANE ON EVENTS.MACHINE=ACCESSPANE.PANELID
WHERE EVENTS.SERIALNUM = <serialnumber> AND PANELID = <panelid>
distributionPerf.txt profiling is useful when there is a suspected delay delivering events to thick clients. The log will show the serialnumber of the event in question and the time to deliver the event once received from the hardware.
[Lnlcommsrvr] SendMessageToAllClients 100 sn:1635075543 took 16 ms
Applies To
OnGuard 7.6 Update 2 and later
Additional Information
Copyright © 2022 Carrier. All rights reserved.
