JaaS statistics file
The JaaS analytics server (rtcstats-server) gathers various WebRTC and Jitsi Meet specific metrics.
Data points which are considered important in determining the quality of a meeting are aggregated and pushed to a centralized storage and then published to a dashboard that can be accessed from the JaaS Admin UI. More detailed information here
https://support.8x8.com/business-phone/meetings/meetings-analytics/Meeting_Analytics_Beta.
Aside from the aggregated statistics, the service generates a web hook containing a link to the complete raw data associated with a participant’s session. The Webhook is called when a participant leaves the meeting and the statistics server uploads the associated statistics file, more detailed information about the web hook structure can be found here:
https://developer.8x8.com/jaas/docs/webhooks-payload#rtcstats_uploaded
The actual data is stored in s3 as a compressed text file, with each individual line representing a statistics entry with various formats depending on the entry type.
Glossary of entry types that can be found in a JaaS statistics file
Entries are separated by newlines and are stored as JSON arrays with the following structure.
[
<TYPE>,
<PLACEHOLDER>,
<DATA>,
<EPOCH TIMESTAMP>,
<SEQUENCE NUMBER>
]
/**
* TYPE: A string that represents the type of the request, e.g. “identity”, “getstats”, “logs” etc.
* PLACEHOLDER: Is usually null, but in some cases like in the “getstats” can have values depending on the context.
* DATA: Depending on the entry type, data can have various data types like an JSON object, a string or a number.
* EPOCH TIMESTAMP: UTC epoch timestamp referring to when the event occurred on the participant’s application
* SEQUENCE NUMBER: Order in which the event was received.
**/
Entries fall into three logical categories:
- Meeting specific entries, these are generated by the Jitsi Meet application and are usually related to custom logic. Events such as
logs,dominantSpeaker,identityand others. - WebRTC event statistics are generated by the various WebRTC and GUM APIs used by audio/video flows e.g,
ontrack,onsignalingstatechange,createAnswer,getUserMedia. getStatsevents, these are generated by hooking into the RTCPeerConnection getStats method
connectionInfo entry
The connection info entry is more of a header that all dump files must have, it contains metadata about the client type and format of the statistics, which is needed when decoding the rest of the file. It is represented as a JSON object containing the following fields:
path: the paths of the meetingorigin: domain from which the connection originatedurl: complete meeting urluserAgent: browser informationclientProtocol: rtcstats protocol used to send statistics.statsFormat: various browsers have different WebRTC getStats formats.clientType: rtcstats-server can handle various client types from various backend components like JVB, JICOFO, JIGASI
For example:
[
"connectionInfo",
null,
{
"path": "/testmeeting?statsSessionId=e12448bd-8e32-4388-a345-ae4cbe0f50e7&isReconnect=false",
"origin": "https://8x8.vc",
"url": "https://8x8.vc//testmeeting?statsSessionId=e12448bd-8e32-4388-a345-ae4cbe0f50e7&isReconnect=false",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) 8x8Work/8.12.2 Chrome/120.0.6099.291 Electron/28.2.5 Safari/537.36",
"clientProtocol": "3.1_STANDARD",
"statsFormat": "chrome_standard",
"clientType": "RTCSTATS_CLIENT"
},
2312324213213
]
identity entry
Identity entries contain various information about the participant associated with the statistics file. It is represented as a JSON Object and contains a variety of metadata. Some notable fields are :
displayName: participant nameconfID: meeting nameconfName- complete meeting urldeploymentInfo- information about various service endpoints used by the application
For example:
[
"identity",
null,
{
"enableP2P": true,
"p2p": {
"enabled": true,
"mobileCodecPreferenceOrder": [
"VP8",
"H264",
"VP9"
],
"useStunTurn": true
},
"useStunTurn": true,
"useTurnUdp": true,
"channelLastN": 25,
"videoQuality": {
"vp9": {
"scalabilityModeEnabled": true,
"useSimulcast": false
},
"av1": {
"useSimulcast": false
}
},
"audioQuality": {
"stereo": false,
"enableOpusDtx": false
},
"transcription": {
"enabled": true
},
"recordings": {
"suggestRecording": false,
"showPrejoinWarning": true
},
"liveStreaming": {
"enabled": true
},
"enableForcedReload": false,
"feedbackPercentage": 0,
"screenshotCapture": {
"enabled": true,
"mode": "recording"
},
"deploymentInfo": {
"environment": "prod-8x8",
"envType": "prod",
"releaseNumber": "4953",
"shard": "prod-8x8-eu-frankfurt-1-s20",
"region": "eu-central-1",
"userRegion": "eu-central-1"
},
"e2eping": {
"enabled": false
},
"siteID": "8x8",
"confID": "8x8.vc/vpaas-magic-cookie-xxx-yyy-zzz/testmeeting",
"applicationName": "8x8 Work",
"transcriptionLanguage": "en-US",
"endpointId": "e29c49b6",
"confName": "testmeeting",
"displayName": "[email protected]",
"isBreakoutRoom": false
},
1713790871414,
1
]
In normal circumstances most configured fields from config.js will show up in the identity entries.
logs entry
Log entries contain application logs generated by Jitsi Meet. Using them one can reconstruct the exact browser logs which were recorded during the meeting on a per participant basis.
It is represented as a JSON Array of objects as can be observed in the following example.
For example:
[
"logs",
null,
[
{
"text": "2024-04-22T13:01:15.780Z [modules/xmpp/JingleSessionPC.js] <Object.callback>: JingleSessionPC[session=JVB,initiator=false,sid=7g95af65fus0f] onnegotiationneeded executed - OK",
"timestamp": "2024-04-22T13:01:15.780Z",
"count": 1
},
{
"text": "2024-04-22T13:01:15.840Z [modules/xmpp/SignalingLayerImpl.js] <Vh.setTrackMuteStatus>: Mute state of e29c49b6-a0 changed to muted=false",
"timestamp": "2024-04-22T13:01:15.840Z",
"count": 1
},
{
"text": "2024-04-22T13:01:15.840Z [modules/RTC/RTCUtils.js] onUserMediaSuccess",
"timestamp": "2024-04-22T13:01:15.840Z",
"count": 1
},
],
1713790905781,
59
]
dominantSpeaker entry
Contains information about the current and previous dominant speakers. It is represented as a JSON Object as can be observed in the example. The dominantSpeakerEndpoint field represents the endpoint ID of the meeting participant that just became a dominant speaker.
[
"dominantSpeaker",
null,
{
"dominantSpeakerEndpoint": "eca151a7",
"previousSpeakers": [
"256480c7",
"b0abcadd",
"e29c49b6"
]
},
1713790915855,
84
]
RTCPeerConnection related entries
These entries are generated by hooking into WebRTC's RTCPeerConnection object's methods and events.
The PLACEHOLDER field for these events will have an ID, identifying the RTCPeerConnection to which it belongs e.g. PC_0, PC_1 and so on.
We won’t describe each DATA field’s payload as they’re already properly documented here: https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection.
In order to get a sense of what each DATA entry contains simply follow the method/event name in the RTCPeerConnection specification. Technically, unless blacklisted, all methods and events that happen on a RTCPeerConnection will show up in the statistics file.
Method call entry examples:
["createAnswer","PC_0",{"offerToReceiveAudio":true,"offerToReceiveVideo":true},1713790871560,20]
["createAnswerOnSuccess","PC_0",{"type":"answer","sdp":"<SDP>"},1713790871586,21]
["setLocalDescription","PC_0",{"type":"answer","sdp":"<SDP>"},1713790875777,48]
["setLocalDescriptionOnSuccess","PC_0",null,1713790875779,50]
["setRemoteDescription","PC_0",{"type":"offer","sdp":"<SDP>"},1713790875777,48]
["setRemoteDescriptionOnSuccess","PC_0",null,1713790875779,50]
["constraints","PC_0",{"optional":[{"rtcStatsSFUP2P":false}]},1713790871485,4]
Event entry examples:
["onicegatheringstatechange","PC_0","gathering",1713790871597,25]
["oniceconnectionstatechange","PC_0","checking",1713790871598,26]
["ondtlsstatechange","PC_0","connecting",1713790871637,30]
["ontrack","PC_0","video:mixedlabelvideo0 stream:mixedmslabel",1713790871548,16]
["ontrack","PC_0","audio:b887152b-4a78-439d-b98c-0062993b60d5-1 stream:eca151a7-audio-0-1",1713790871539,14]
["onicecandidate","PC_0",null,1713790871640,33]
["onnegotiationneeded","PC_0",null,1713790875765,41]
getUserMedia entry
GUM entries are included in the statistics file. Similar to RTCPeerConnection events, we won't delve into the specifics of individual DATA payloads, as they are documented here:
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
["navigator.mediaDevices.getUserMedia",null,{"video":false,"audio":{"autoGainControl":true,"deviceId":"49a20912918fd02d56e0b2048a0713ca50b79a3370443eb84076749653184936","echoCancellation":true,"noiseSuppression":true}},1713790875774,46]
["navigator.mediaDevices.getUserMediaOnSuccess",null,{"id":"db512d54-5798-4486-a308-c80f760e29c7","tracks":[{"id":"36a19da2-2076-45a4-a89e-91010a28bfd9","kind":"audio","label":"Midnight (Bluetooth)","enabled":true,"muted":false,"readyState":"live"}]},1713790875750,50]
getstats entry
One significant aspect of the file is the 'getstats' entry, which contains the complete object retrieved by utilizing the getStats method on an RTCPeerConnection, as described in this resourcehttps://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/getStats.
To put things into context, the getStats object contains peer connection level QOS data such as:
RTT, Packet loss, Jitter, Bitrate estimations.
Individual audio/video track statistics such as: Bitrate, Frames sent, FIR count, NACK count, Concealed packets, Quality limitation changes.
The WebRTC stats specification goes into extensive detail about all of these and how they can be used to analyze call quality https://www.w3.org/TR/webrtc-stats/ . Like RTCPeerConnection events, if it’s in the specification and the browser/app from which the participant connects supports it, it will show up in the statistics file with the described payload
Note. GDPR sensitive data such as IPs is obfuscated before being sent.
For example:
[
"getstats",
"PC_0",
{
"CPUjjFb9rJ_dyZkkZI+": {
"timestamp": 0,
"type": "candidate-pair",
"availableOutgoingBitrate": 300000,
"bytesDiscardedOnSend": 0,
"bytesReceived": 1359,
"bytesSent": 859,
"consentRequestsSent": 2,
"currentRoundTripTime": 0.036,
"lastPacketReceivedTimestamp": 1713790871826,
"lastPacketSentTimestamp": 1713790871784,
"localCandidateId": "IUjjFb9rJ",
"nominated": true,
"packetsDiscardedOnSend": 0,
"packetsReceived": 12,
"packetsSent": 3,
"priority": 7277816997243403000,
"remoteCandidateId": "IdyZkkZI+",
"requestsReceived": 4,
"requestsSent": 3,
"responsesReceived": 3,
"responsesSent": 4,
"state": "succeeded",
"totalRoundTripTime": 0.114,
"transportId": "T01",
"writable": true
},
"IT01A383803203": {
"timestamp": 0,
"type": "inbound-rtp",
"kind": "audio",
"mediaType": "audio",
"ssrc": 383803203,
"transportId": "T01",
"jitter": 0,
"packetsLost": 0,
"packetsReceived": 0,
"bytesReceived": 0,
"concealedSamples": 0,
"concealmentEvents": 0,
"fecPacketsDiscarded": 0,
"fecPacketsReceived": 0,
"headerBytesReceived": 0,
"insertedSamplesForDeceleration": 0,
"jitterBufferDelay": 0,
"jitterBufferEmittedCount": 0,
"jitterBufferMinimumDelay": 0,
"jitterBufferTargetDelay": 0,
"mid": "0",
"packetsDiscarded": 0,
"playoutId": "AP",
"removedSamplesForAcceleration": 0,
"silentConcealedSamples": 0,
"totalAudioEnergy": 0,
"totalSamplesDuration": 0,
"totalSamplesReceived": 0,
"trackIdentifier": "mixedlabelaudio0"
},
"IT01A4213127780": {
"timestamp": 0,
"type": "inbound-rtp",
"kind": "audio",
"mediaType": "audio",
"ssrc": 4213127780,
"transportId": "T01",
"jitter": 0,
"packetsLost": 0,
"packetsReceived": 0,
"bytesReceived": 0,
"concealedSamples": 0,
"concealmentEvents": 0,
"fecPacketsDiscarded": 0,
"fecPacketsReceived": 0,
"headerBytesReceived": 0,
"insertedSamplesForDeceleration": 0,
"jitterBufferDelay": 0,
"jitterBufferEmittedCount": 0,
"jitterBufferMinimumDelay": 0,
"jitterBufferTargetDelay": 0,
"mid": "1",
"packetsDiscarded": 0,
"playoutId": "AP",
"removedSamplesForAcceleration": 0,
"silentConcealedSamples": 0,
"totalAudioEnergy": 0,
"totalSamplesDuration": 0,
"totalSamplesReceived": 0,
"trackIdentifier": "b887152b-4a78-439d-b98c-0062993b60d5-1"
},
"IT01V3190702610": {
"timestamp": 0,
"type": "inbound-rtp",
"kind": "video",
"mediaType": "video",
"ssrc": 3190702610,
"transportId": "T01",
"jitter": 0,
"packetsLost": 0,
"packetsReceived": 0,
"bytesReceived": 0,
"firCount": 0,
"framesAssembledFromMultiplePackets": 0,
"framesDecoded": 0,
"framesDropped": 0,
"framesReceived": 0,
"freezeCount": 0,
"headerBytesReceived": 0,
"jitterBufferDelay": 0,
"jitterBufferEmittedCount": 0,
"jitterBufferMinimumDelay": 0,
"jitterBufferTargetDelay": 0,
"keyFramesDecoded": 0,
"mid": "4",
"nackCount": 0,
"pauseCount": 0,
"pliCount": 0,
"rtxSsrc": 2661524669,
"totalAssemblyTime": 0,
"totalDecodeTime": 0,
"totalFreezesDuration": 0,
"totalInterFrameDelay": 0,
"totalPausesDuration": 0,
"totalProcessingDelay": 0,
"totalSquaredInterFrameDelay": 0,
"trackIdentifier": "f17449f0-f57c-41b6-8f27-ef939ea78166-1"
},
"IUjjFb9rJ": {
"timestamp": 0,
"type": "local-candidate",
"address": "x.x.x.x",
"candidateType": "prflx",
"foundation": "2924890005",
"ip": "x.x.x.x",
"isRemote": false,
"networkType": "wifi",
"port": 55424,
"priority": 1853824767,
"protocol": "udp",
"relatedAddress": "192.168.1.252",
"relatedPort": 55424,
"transportId": "T01",
"usernameFragment": "vsc3"
},
"IdyZkkZI+": {
"timestamp": 0,
"type": "remote-candidate",
"address": "x.x.x.x",
"candidateType": "srflx",
"foundation": "2",
"ip": "x.x.x.x",
"isRemote": true,
"port": 10000,
"priority": 1694498815,
"protocol": "udp",
"relatedAddress": "0.0.0.0",
"relatedPort": 9,
"transportId": "T01",
"usernameFragment": "7a6r21hs2uapre"
},
"P": {
"timestamp": 0,
"type": "peer-connection",
"dataChannelsClosed": 0,
"dataChannelsOpened": 0
},
"T01": {
"timestamp": 0,
"type": "transport",
"bytesReceived": 1359,
"bytesSent": 859,
"dtlsCipher": "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"dtlsRole": "client",
"dtlsState": "connected",
"iceLocalUsernameFragment": "vsc3",
"iceRole": "controlled",
"iceState": "connected",
"localCertificateId": "CFE7:F9:82:B9:75:46:14:2F:C4:5D:50:EF:B5:12:0C:C8:00:05:89:0B:52:4A:A8:C0:08:66:4A:F1:90:16:69:6E",
"packetsReceived": 12,
"packetsSent": 3,
"remoteCertificateId": "CF1F:17:48:0F:2E:4D:E4:BA:0C:3E:3A:F8:94:FB:44:34:C0:64:0E:99:25:3F:A0:AE:82:84:C6:83:9E:B1:6A:0C",
"selectedCandidatePairChanges": 1,
"selectedCandidatePairId": "CPUjjFb9rJ_dyZkkZI+",
"srtpCipher": "AEAD_AES_128_GCM",
"tlsVersion": "FEFD"
},
"timestamp": 1713790871827.155
},
1713790871832,
38
]
Important mention regarding getStats entries, because they can be of considerable length, we delta compressed.
In other words only the difference from the previous entry is sent and stored, thus some custom logic is required to make sense of them.
The service decoding these files would need to always keep the previous entry of getStats in order to regenerate the complete data.
The following code snippet when applied on a getStats entry will generate the complete data view. Please be advised that the snippet is meant as a guideline.
/**
* The function will use `baseStats` as the base line for decompression, where
* `newStats` is expected to contain just the differences from the base line.
* It will go through each entry of `newStats` and reconstruct data.
*
* @param {Object} baseStats - Complete WebRTC statistics entry.
* @param {Object} newStats - Delta compressed statistics entry.
* @returns {Object} - Decompressed `newStats` entry.
*/
decompress(baseStats, newStats) {
const timestamp = newStats.timestamp;
Object.keys(baseStats).forEach(id => {
// If the new statistic data does not contain a certain report we consider it was removed.
// e.g. a ssrc was removed from the connection.
if (!newStats[id]) {
delete baseStats[id];
}
});
// Iterate through the new entry and reconstruct it with data that hasn't changed from the base
// stat
Object.keys(newStats).forEach(id => {
if (baseStats[id]) {
const report = newStats[id];
// Timestamp will usually be set to 0 in reports but will be saved at the stats level
// so we don't send it unecessarely for each report in the stats object.
report.timestamp = timestamp;
Object.keys(report).forEach(name => {
baseStats[id][name] = report[name];
});
// If there is a new report in the stats data we add the complete structure as there is no
// base line to construct it from.
} else {
if (newStats[id].timestamp === 0) {
newStats[id].timestamp = timestamp;
}
baseStats[id] = newStats[id];
}
});
return baseStats;
}
Updated 8 months ago