jiti-meet/resources/prosody-plugins
Gabriel Borlea 61684b1071
feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006)
* Initial implementation; Happy flow

* Maybe revert this

* Functional prototype

* feat(facial-expressions): get stream when changing background effect and use presenter effect with camera

* add(facial-expressions): array that stores the expressions durin the meeting

* refactor(facial-expressions): capture imagebitmap from stream with imagecapture api

* add(speaker-stats): expression label

* fix(facial-expression): expression store

* revert: expression leabel on speaker stats

* add(facial-expressions): broadcast of expression when it changes

* feat: facial expression handling on prosody

* fix(facial-expressions): get the right track when opening and closing camera

* add(speaker-stats): facial expression column

* fix(facial-expressions): allow to start facial recognition only after joining conference

* fix(mod_speakerstats_component): storing last emotion in speaker stats component and sending it

* chore(facial-expressions): change detection from 2000ms to 1000ms

* add(facial-expressions): send expression to server when there is only one participant

* feat(facial-expressions): store expresions as a timeline

* feat(mod_speakerstats_component): store facial expresions as a timeline

* fix(facial-expressions): stop facial recognition only when muting video track

* fix(facial-expressions): presenter mode get right track to detect face

* add: polyfils for image capture for firefox and safari

* refactor(facial-expressions): store expressions by counting them in a map

* chore(facial-expressions): remove manually assigning the backend for tenserflowjs

* feat(facial-expressions): move face-api from main thread to web worker

* fix(facial-expressions): make feature work on firefox and safari

* feat(facial-expressions): camera time tracker

* feat(facial-expressions): camera time tracker in prosody

* add(facial-expressions): expressions time as TimeElapsed object in speaker stats

* fix(facial-expresions): lower the frequency of detection when tf uses cpu backend

* add(facial-expressions): duration to the expression and send it with durantion when it is done

* fix(facial-expressions): prosody speaker stats covert fro string to number and bool values set by xmpp

* refactor(facial-expressions): change expressions labels from text to emoji

* refactor(facial-expressions): remove camera time tracker

* add(facial-expressions): detection time interval

* chore(facial-expressions): add docs and minor refactor of the code

* refactor(facial-expressions): put timeout in worker and remove set interval in main thread

* feat(facial-expressions): disable feature in the config

* add(facial-expressions): tooltips of labels in speaker stats

* refactor(facial-expressions): send facial expressions function and remove some unused functions and console logs

* refactor(facial-expressions): rename action type when a change is done to the track by the virtual backgrounds to be used in facial expressions middleware

* chore(facial-expressions): order imports and format some code

* fix(facial-expressions): rebase issues with newer master

* fix(facial-expressions): package-lock.json

* fix(facial-expression): add commented default value of disableFacialRecognition flag and short description

* fix(facial-expressions): change disableFacialRecognition to enableFacialRecognition flag in config

* fix: resources load-test package-lock.json

* fix(facial-expressions): set and get facial expressions only if facial recognition enabled

* add: facial recognition resources folder in .eslintignore

* chore: package-lock update

* fix: package-lock.json

* fix(facial-expressions): gpu memory leak in the web worker

* fix(facial-expressions): set cpu time interval for detection to 6000ms

* chore(speaker-stats): fix indentation

* chore(facial-expressions): remove empty lines between comments and type declarations

* fix(facial-expressions): remove camera timetracker

* fix(facial-expressions): remove facialRecognitionAllowed flag

* fix(facial-expressions): remove sending interval time to worker

* refactor(facial-expression): middleware

* fix(facial-expression): end tensor scope after setting backend

* fix(facial-expressions): sending info back to worker only on facial expression message

* fix: lint errors

* refactor(facial-expressions): bundle web worker using webpack

* fix: deploy-facial-expressions command in makefile

* chore: fix load test package-lock.json and package.json

* chore: sync package-lock.json

Co-authored-by: Mihai-Andrei Uscat <mihai.uscat@8x8.com>
2021-11-17 16:33:03 +02:00
..
token fix: Fixes errors in prosody about string formatting and nil values. (#10037) 2021-09-27 14:48:13 -05:00
ext_events.lib.lua Adds server-side speaker stats handling. 2018-12-28 13:54:29 +00:00
mod_auth_jitsi-anonymous.lua fix(prosody) fix username not being set in mod_auth_jitsi-anonymous 2021-08-18 15:16:53 +02:00
mod_auth_token.lua call module that checks the jwt against the access service 2021-10-04 14:11:54 -05:00
mod_av_moderation.lua fix(prosody): av_moderation_component default val 2021-08-27 07:58:36 -05:00
mod_av_moderation_component.lua feat: Simplify AV moderation participant approve/remove message and fix array usage. (#10062) 2021-09-30 13:35:55 -05:00
mod_client_proxy.lua Remove the "focus" external component, use client_proxy instead. (#8381) 2021-01-11 15:45:00 -06:00
mod_conference_duration.lua fix: Fixes default values in speakerstats and conf duration modules. 2021-08-27 08:42:59 -05:00
mod_conference_duration_component.lua Conference timer should start counting at 0 2020-04-23 10:22:02 -05:00
mod_external_services.lua JiConOp2 (#9052) 2021-04-22 12:54:34 -05:00
mod_filter_iq_jibri.lua feat: Adds an option to validate a recording token. 2020-07-02 12:51:14 -05:00
mod_filter_iq_rayo.lua feat: Throttle out call attempts to the max number per minute (#7742) 2020-09-22 10:53:43 -05:00
mod_jiconop.lua feat(external-api) Expose deployment info through API 2021-10-07 09:41:42 -05:00
mod_jitsi_session.lua feat: jitsi_session: extracts URL parameters from BOSH or WS into session 2021-05-13 06:29:42 -05:00
mod_limits_exception.lua fix: Adds a null check in mod_limits_exceptions. 2021-08-10 14:34:20 -05:00
mod_muc_allowners.lua fix: Fixes kick when allowners is enabled. 2021-10-20 08:10:40 -07:00
mod_muc_call.lua fix: Optimizes hot paths in prosody modules, string comparisons. 2020-11-06 13:33:14 -06:00
mod_muc_census.lua feat: new prosody module to report census of all rooms (#9901) 2021-09-09 13:24:04 -05:00
mod_muc_domain_mapper.lua fix: Clear prosody utils and removes a debug statement. 2021-06-22 09:44:26 +03:00
mod_muc_lobby_rooms.lua feat: Filters moderators presence in lobby room. (#10316) 2021-11-08 16:38:22 -07:00
mod_muc_max_occupants.lua feat: Dynamically limit the number of participants in a room (#9880) 2021-09-09 08:15:14 -05:00
mod_muc_meeting_id.lua ref: Make is_healthcheck_room more generic. 2020-05-12 18:22:14 -05:00
mod_muc_poltergeist.lua feat: jitsi_session: extracts URL parameters from BOSH or WS into session 2021-05-13 06:29:42 -05:00
mod_muc_rate_limit.lua feat: Adds a prosody module limiting the join rate in a room. 2021-10-19 16:52:32 -07:00
mod_muc_size.lua prosody muc_size plugin, room get info error fix (Traceback[httpserver]: /usr/lib/prosody/util/async.lua:137: /prosody-plugins/mod_muc_size.lua:141: attempt to concatenate local 'subdomain' (a nil value) 2020-07-01 08:04:41 -05:00
mod_muc_transcription_filter.lua Added module for filtering transcription requests from presence stanz… (#6404) 2020-04-28 09:11:58 -05:00
mod_polls.lua feat(polls) fix spacing and send answer identifier 2021-11-05 07:34:52 -07:00
mod_poltergeist_component.lua A poltergeist module. 2017-07-14 22:18:23 -05:00
mod_presence_identity.lua Update poltergeist's presence with identity information. (#2650) 2018-03-23 13:58:05 -07:00
mod_reservations.lua chore(lint) tame the (uppdated) linter 2021-11-09 09:43:55 +01:00
mod_roster_command.lua Remove the "focus" external component, use client_proxy instead. (#8381) 2021-01-11 15:45:00 -06:00
mod_roster_command.patch Remove the "focus" external component, use client_proxy instead. (#8381) 2021-01-11 15:45:00 -06:00
mod_smacks.lua chore: Updates mod_smacks.lua version to latest. 2021-06-22 09:44:26 +03:00
mod_speakerstats.lua fix: Fixes default values in speakerstats and conf duration modules. 2021-08-27 08:42:59 -05:00
mod_speakerstats_component.lua feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
mod_token_verification.lua fixed admin check for token verification 2021-01-05 12:56:33 -06:00
mod_turncredentials.lua Introduces installing coturn as turn server for jitsi-meet (#4959) 2020-01-09 16:51:27 +00:00
mod_websocket_session_event.patch add mod_websocket patch for session event 2020-07-23 16:29:45 -05:00
muc_owner_allow_kick.patch Move miscellaneous files to resources 2017-02-23 10:01:19 +01:00
poltergeist.lib.lua Adding state to poltergeist store for correlating external resources with calls. 2018-06-29 14:51:48 -05:00
util.lib.lua call module that checks the jwt against the access service 2021-10-04 14:11:54 -05:00