Commit Graph

21 Commits

Author SHA1 Message Date
Gabriel Borlea 4b969cf4ab
feat(face-landmarks): add face landmarks timeline (#12561)
* feat(face-landmarks): add face landmarks timeline

fixes after rebase

* fixes after rebase compiling and linting

* fix: change keyboard shorcut for participants stats

* fix: label for emotions switch

* fix: linting issues

* code review changes

* fix linting issues

* code review changes 2

* fix typo
2022-11-22 15:56:37 +02:00
Hristo Terezov f0a45a9976 feat: Handle dominant speaker silence 2022-09-14 12:13:41 -05:00
damencho 6fe353cfa4 fix: Fixes lua log lines.
When using directly log( the log lines come from `general` not from the module itself.
`Aug 29 19:38:08 general	info	Starting speakerstats for conference...`
2022-08-30 11:24:28 -05:00
abora8x8 15d453de1d
Fix: Speaker stats are not delivered for the breakout rooms (#11644)
* Send speaker stats for brk room

* Fix comments component loaded
2022-06-09 13:15:04 -05:00
Avram Tudor 42703fed47
ref(face-landmarks) refactor namings (#11307)
* ref(face-landmarks) refactor namings

* code review
2022-04-06 12:10:31 +03:00
Gabriel Borlea 61684b1071
feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006)
* Initial implementation; Happy flow

* Maybe revert this

* Functional prototype

* feat(facial-expressions): get stream when changing background effect and use presenter effect with camera

* add(facial-expressions): array that stores the expressions durin the meeting

* refactor(facial-expressions): capture imagebitmap from stream with imagecapture api

* add(speaker-stats): expression label

* fix(facial-expression): expression store

* revert: expression leabel on speaker stats

* add(facial-expressions): broadcast of expression when it changes

* feat: facial expression handling on prosody

* fix(facial-expressions): get the right track when opening and closing camera

* add(speaker-stats): facial expression column

* fix(facial-expressions): allow to start facial recognition only after joining conference

* fix(mod_speakerstats_component): storing last emotion in speaker stats component and sending it

* chore(facial-expressions): change detection from 2000ms to 1000ms

* add(facial-expressions): send expression to server when there is only one participant

* feat(facial-expressions): store expresions as a timeline

* feat(mod_speakerstats_component): store facial expresions as a timeline

* fix(facial-expressions): stop facial recognition only when muting video track

* fix(facial-expressions): presenter mode get right track to detect face

* add: polyfils for image capture for firefox and safari

* refactor(facial-expressions): store expressions by counting them in a map

* chore(facial-expressions): remove manually assigning the backend for tenserflowjs

* feat(facial-expressions): move face-api from main thread to web worker

* fix(facial-expressions): make feature work on firefox and safari

* feat(facial-expressions): camera time tracker

* feat(facial-expressions): camera time tracker in prosody

* add(facial-expressions): expressions time as TimeElapsed object in speaker stats

* fix(facial-expresions): lower the frequency of detection when tf uses cpu backend

* add(facial-expressions): duration to the expression and send it with durantion when it is done

* fix(facial-expressions): prosody speaker stats covert fro string to number and bool values set by xmpp

* refactor(facial-expressions): change expressions labels from text to emoji

* refactor(facial-expressions): remove camera time tracker

* add(facial-expressions): detection time interval

* chore(facial-expressions): add docs and minor refactor of the code

* refactor(facial-expressions): put timeout in worker and remove set interval in main thread

* feat(facial-expressions): disable feature in the config

* add(facial-expressions): tooltips of labels in speaker stats

* refactor(facial-expressions): send facial expressions function and remove some unused functions and console logs

* refactor(facial-expressions): rename action type when a change is done to the track by the virtual backgrounds to be used in facial expressions middleware

* chore(facial-expressions): order imports and format some code

* fix(facial-expressions): rebase issues with newer master

* fix(facial-expressions): package-lock.json

* fix(facial-expression): add commented default value of disableFacialRecognition flag and short description

* fix(facial-expressions): change disableFacialRecognition to enableFacialRecognition flag in config

* fix: resources load-test package-lock.json

* fix(facial-expressions): set and get facial expressions only if facial recognition enabled

* add: facial recognition resources folder in .eslintignore

* chore: package-lock update

* fix: package-lock.json

* fix(facial-expressions): gpu memory leak in the web worker

* fix(facial-expressions): set cpu time interval for detection to 6000ms

* chore(speaker-stats): fix indentation

* chore(facial-expressions): remove empty lines between comments and type declarations

* fix(facial-expressions): remove camera timetracker

* fix(facial-expressions): remove facialRecognitionAllowed flag

* fix(facial-expressions): remove sending interval time to worker

* refactor(facial-expression): middleware

* fix(facial-expression): end tensor scope after setting backend

* fix(facial-expressions): sending info back to worker only on facial expression message

* fix: lint errors

* refactor(facial-expressions): bundle web worker using webpack

* fix: deploy-facial-expressions command in makefile

* chore: fix load test package-lock.json and package.json

* chore: sync package-lock.json

Co-authored-by: Mihai-Andrei Uscat <mihai.uscat@8x8.com>
2021-11-17 16:33:03 +02:00
Gabriel Borlea af01072827 fix(speaker-stats): calculate total dominant speaker time if user is dominant and has no previous speaker time 2021-11-01 07:53:10 -07:00
Дамян Минков 4075e5deb7 fix: Fix speakerstats reports. 2021-07-15 17:03:05 +03:00
damencho e6cdeb31ff feat: Optimizes speaker stats to skip occupants with 0 talk time.
Skips adding focus user and skips sending an empty message.
Keep sending participant info for those left the room, to keep existing behaviour.
2021-06-22 09:44:26 +03:00
Дамян Минков 9f65ae52f1
fix: Prosody modules - drop unused and duplicate code and drop chatty debug statements (#8027)
* chore: Updates mod_smacks.lua version to latest.

https://hg.prosody.im/prosody-modules/file/db75772afb28/mod_smacks/mod_smacks.lua

* Drop unused modules.

* Update docs.

* Move utility functions away from domain mapper.

* Remove some chatty debug log messages.

* Drops not needed patch for mod_websocket.
2020-11-04 08:25:03 -06:00
emrah da33d8a033 fix: speakerstats_component, attempt to index (a nil value) 2020-10-20 07:48:23 -05:00
Boris Grozev 76e1217439 ref: Make is_healthcheck_room more generic. 2020-05-12 18:22:14 -05:00
damencho eea8fef044 Initial Lobby backend implementation. 2020-04-30 16:34:46 -05:00
Saúl Ibarra Corretgé 073fdc7b0e
sperakerstats: prevent access of nil object (#5112)
If the dominant speaker leaves their object will be gone from the mapping.
2020-03-06 12:33:41 +01:00
damencho b4be1bcd05 Adds some checks about async.
There are modules that will not work with prosody 0.10 as they depend on util.async. Adds a safeguard and print error about it in the logs.
And others that just do not work because of the muc module API that they use.
2019-12-10 10:55:56 +01:00
drimovecz ffded8d82a Drimovecz/speakerstats (#4851)
* Correctly process speaker stats events when the conference contains a subdomain
2019-11-13 15:37:09 +00:00
drimovecz 6ecd150f75 Add context user on speaker stats 2019-10-23 09:24:43 +01:00
damencho a5fc62b920 Updates correct loading and fix checking is dominant speaker. 2019-07-26 11:18:55 +01:00
damencho 380d9c75d1 Simplifies logic and renames a method. 2018-12-28 13:54:29 +00:00
damencho 7a09befd87 Updates time to be in ms and sends update of stats when user joins. 2018-12-28 13:54:29 +00:00
damencho 3b4037553a Adds server-side speaker stats handling.
Adds the component which receives the messages from client and a module which enabled on a virtual host will start advertising the component. When clients discover the component they will send message to the component with the name of the room where the dominant speaker event happen.
2018-12-28 13:54:29 +00:00