Go to file
Gabriel Borlea 61684b1071
feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006)
* Initial implementation; Happy flow

* Maybe revert this

* Functional prototype

* feat(facial-expressions): get stream when changing background effect and use presenter effect with camera

* add(facial-expressions): array that stores the expressions durin the meeting

* refactor(facial-expressions): capture imagebitmap from stream with imagecapture api

* add(speaker-stats): expression label

* fix(facial-expression): expression store

* revert: expression leabel on speaker stats

* add(facial-expressions): broadcast of expression when it changes

* feat: facial expression handling on prosody

* fix(facial-expressions): get the right track when opening and closing camera

* add(speaker-stats): facial expression column

* fix(facial-expressions): allow to start facial recognition only after joining conference

* fix(mod_speakerstats_component): storing last emotion in speaker stats component and sending it

* chore(facial-expressions): change detection from 2000ms to 1000ms

* add(facial-expressions): send expression to server when there is only one participant

* feat(facial-expressions): store expresions as a timeline

* feat(mod_speakerstats_component): store facial expresions as a timeline

* fix(facial-expressions): stop facial recognition only when muting video track

* fix(facial-expressions): presenter mode get right track to detect face

* add: polyfils for image capture for firefox and safari

* refactor(facial-expressions): store expressions by counting them in a map

* chore(facial-expressions): remove manually assigning the backend for tenserflowjs

* feat(facial-expressions): move face-api from main thread to web worker

* fix(facial-expressions): make feature work on firefox and safari

* feat(facial-expressions): camera time tracker

* feat(facial-expressions): camera time tracker in prosody

* add(facial-expressions): expressions time as TimeElapsed object in speaker stats

* fix(facial-expresions): lower the frequency of detection when tf uses cpu backend

* add(facial-expressions): duration to the expression and send it with durantion when it is done

* fix(facial-expressions): prosody speaker stats covert fro string to number and bool values set by xmpp

* refactor(facial-expressions): change expressions labels from text to emoji

* refactor(facial-expressions): remove camera time tracker

* add(facial-expressions): detection time interval

* chore(facial-expressions): add docs and minor refactor of the code

* refactor(facial-expressions): put timeout in worker and remove set interval in main thread

* feat(facial-expressions): disable feature in the config

* add(facial-expressions): tooltips of labels in speaker stats

* refactor(facial-expressions): send facial expressions function and remove some unused functions and console logs

* refactor(facial-expressions): rename action type when a change is done to the track by the virtual backgrounds to be used in facial expressions middleware

* chore(facial-expressions): order imports and format some code

* fix(facial-expressions): rebase issues with newer master

* fix(facial-expressions): package-lock.json

* fix(facial-expression): add commented default value of disableFacialRecognition flag and short description

* fix(facial-expressions): change disableFacialRecognition to enableFacialRecognition flag in config

* fix: resources load-test package-lock.json

* fix(facial-expressions): set and get facial expressions only if facial recognition enabled

* add: facial recognition resources folder in .eslintignore

* chore: package-lock update

* fix: package-lock.json

* fix(facial-expressions): gpu memory leak in the web worker

* fix(facial-expressions): set cpu time interval for detection to 6000ms

* chore(speaker-stats): fix indentation

* chore(facial-expressions): remove empty lines between comments and type declarations

* fix(facial-expressions): remove camera timetracker

* fix(facial-expressions): remove facialRecognitionAllowed flag

* fix(facial-expressions): remove sending interval time to worker

* refactor(facial-expression): middleware

* fix(facial-expression): end tensor scope after setting backend

* fix(facial-expressions): sending info back to worker only on facial expression message

* fix: lint errors

* refactor(facial-expressions): bundle web worker using webpack

* fix: deploy-facial-expressions command in makefile

* chore: fix load test package-lock.json and package.json

* chore: sync package-lock.json

Co-authored-by: Mihai-Andrei Uscat <mihai.uscat@8x8.com>
2021-11-17 16:33:03 +02:00
.github feat(ci) use Node 16 / npm 8 for testing 2021-11-03 12:44:45 +01:00
android chore(deps) update uuid package to 8.3.2 2021-11-10 11:35:20 +01:00
connection_optimization feat(reload): Preserve local track mute state. 2020-05-07 18:18:21 -05:00
css feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
debian fix: Fixes upgrading component to muc prosody config. 2021-10-29 13:53:01 -07:00
doc feat: Adds a prosody module limiting the join rate in a room. 2021-10-19 16:52:32 -07:00
flow-typed/npm chore(deps) update uuid package to 8.3.2 2021-11-10 11:35:20 +01:00
fonts feat: SVG icons 2019-09-13 14:07:53 +02:00
images feat(audio-screen-share): improved ux flow (#9411) 2021-07-07 11:07:30 +03:00
ios chore(deps) update uuid package to 8.3.2 2021-11-10 11:35:20 +01:00
lang feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
modules feat(e2ee) add externally managed key mode 2021-11-16 12:12:10 +01:00
patches chore(Deps) update Babel and eslint to their latest versions 2021-11-09 09:43:55 +01:00
react feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
resources feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
service/UI feat(share-video) Add capability for sharing any direct link video 2021-05-20 13:00:54 +03:00
sounds fix(reactions) Updated sounds (#10321) 2021-11-08 16:16:09 +02:00
static Improve premeeting screens ux (#9726) 2021-08-20 11:53:11 +03:00
twa fix(twa) update template 2021-01-12 14:56:04 +01:00
.buckconfig Merge jitsi-meet-react's mobile support 2016-10-12 10:31:52 -05:00
.editorconfig The Makefile file format requires tab characters 2016-11-23 15:54:03 -06:00
.eslintignore feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
.eslintrc.js chore(deps) adapt to eslint-config-jitsi rename 2021-11-10 13:10:15 +01:00
.flowconfig update RN to 61 2019-09-25 17:31:52 +02:00
.gitattributes react-native 0.40.0 2017-01-22 12:32:08 -06:00
.gitignore feat(rn) switch to XCFramework and WebRTC M87 2020-11-25 16:39:26 +01:00
.npmrc chore(deps) run npm audit fix 2021-10-13 14:41:26 +02:00
.nvmrc feat(misc) add .nvmrc 2021-11-03 12:44:45 +01:00
.travis.yml ci: use Xcode 11.1 on Travis 2019-10-14 16:47:07 +02:00
.watchmanconfig Merge jitsi-meet-react's mobile support 2016-10-12 10:31:52 -05:00
CONTRIBUTING.md Update contributing file 2021-05-07 17:23:01 +02:00
LICENSE Moves original contribution note 2015-09-19 12:23:54 -05:00
Makefile feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
README.md fix(doc) fix typo in README 2021-10-06 15:06:55 +02:00
SECURITY.md doc: add H1 to SECURITY.md 2020-06-30 09:07:57 +02:00
analytics-ga.js fix(analytics-ga): Ignore some events 2018-10-22 16:18:47 -05:00
app.js chore(deps) @matrix-org/olm@latest 2021-08-24 15:40:18 +02:00
babel.config.js feat(rn,ui) introduce react-native-paper 2021-05-11 10:01:45 +02:00
base.html fix(static): Add base. 2020-03-26 12:34:00 -05:00
body.html Adds a hook to insert body & head html. (#4843) 2019-11-12 13:37:48 +00:00
conference.js feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
config.js feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
connection.js chore(deps) adapt to logger package rename 2021-11-10 14:40:17 +01:00
eslint feat(av-moderation) Updated Advanced moderation (#9875) 2021-09-10 14:05:16 +03:00
favicon.ico Added favicon.ico for 1) better jitsi brand recognition and 2) getting rid of warnings in web server log files 2014-06-05 13:22:44 +03:00
head.html Adds a hook to insert body & head html. (#4843) 2019-11-12 13:37:48 +00:00
index.android.js [RN] Drop no longer needed polyfills 2018-08-05 17:04:19 -05:00
index.html fix(iframe) don't register service worker when in an iframe 2021-08-24 11:58:43 +02:00
index.ios.js feat(web/RN): update React to v16 and React Native to 0.49 2017-11-02 14:32:35 +01:00
interface_config.js chore(interface-config) Group deprecated configs 2021-09-24 17:08:24 +03:00
logging_config.js fix(logging): Set the log level to debug again. 2021-08-06 12:46:29 -04:00
manifest.json fix(pwa) move manifest to the root 2020-10-22 16:17:18 +02:00
metro.config.js feat: SVG icons 2019-09-13 14:07:53 +02:00
package-lock.json feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
package.json feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00
plugin.head.html Adds empty SSI templates to avoid 404. 2015-03-31 11:17:26 +02:00
pwa-worker.js feat(pwa) add pwa specifics 2020-10-13 13:40:31 +02:00
readme-img1.png feat(doc) revamp README 2021-09-09 16:37:32 +02:00
title.html Adjust some paths to respect base. 2019-03-14 11:22:37 +00:00
webpack.config.js feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006) 2021-11-17 16:33:03 +02:00

README.md

Jitsi Meet

Jitsi Meet is a set of Open Source projects which empower users to use and deploy video conferencing platforms with state-of-the-art video quality and features.



Amongst others here are the main features Jitsi Meet offers:

  • Support for all current browsers
  • Mobile applications
  • Web and native SDKs for integration
  • HD audio and video
  • Content sharing
  • End-to-End Encryption
  • Raise hand and reactions
  • Chat with private conversations
  • Polls
  • Virtual backgrounds

And many more!

Using Jitsi Meet

Using Jitsi Meet is straightforward, as it's browser based. Head over to meet.jit.si and give it a try. It's anonymous, scalable and free to use. All browsers are supported!

Using mobile? No problem, you can either use your mobile web browser or our fully-featured mobile apps:

Android Android (F-Droid) iOS

If you are feeling adventurous and want to get an early scoop of the features as they are being developed you can also sign up for our open beta testing here:

Running your own instance

If you'd like to run your own Jitsi Meet installation head over to the handbook to get started.

We provide Debian packages and a comprehensive Docker setup to make deployments as simple as possible. Advanced users also have the possibility of building all the components from source.

You can check the latest releases here.

Jitsi as a Service

If you like the branding capabilities of running your own instance but you'd like to avoid dealing with the complexity of monitoring, scaling and updates, JaaS might be for you.

8x8 Jitsi as a Service (JaaS) is an enterprise-ready video meeting platform that allows developers, organizations and businesses to easily build and deploy video solutions. With Jitsi as a Service we now give you all the power of Jitsi running on our global platform so you can focus on building secure and branded video experiences.

Documentation

All the Jitsi Meet documentation is available in the handbook.

Security

For a comprehensive description of all Jitsi Meet's security aspects, please check this link.

For a detailed description of Jitsi Meet's End-to-End Encryption (E2EE) implementation, please check this link.

For information on reporting security vulnerabilities in Jitsi Meet, see SECURITY.md.

Contributing

If you are looking to contribute to Jitsi Meet, first of all, thank you! Please see our guidelines for contributing.



Built with ❤️ by the Jitsi team at 8x8 and our community.