Commit Graph

111 Commits

Author SHA1 Message Date
Andrei Gavrilescu 06491e2406
feat(noise-suppression): Add noise suppression effect. (#11547)
* add denoise effect

* denoise prototype

* improve rnnoise / add comments

* revert some unnecessary changes

* Add noise suppressor worklet

* Send notification on failure

* address code review

* additional comments

* additional comments

* update package-lock

* fix rebase changes

* update rnnoise npm package

* sort lang

* adjust webpack performance hint

* address code review

* address code review

* switch ns files to typescript

* fix null-loader version, lang sort

* fix lint

* missing import

* fix lint / address code review

* use single action for ns state

* move activation to thunk

* increase node heap

* copy noise-suppressor to deploy

* fix ts lint
2022-07-20 15:31:17 +03:00
Gabriel Borlea 0c021868b5
feat(face-landmarks): integrate human library
It replaces face-api.

* feat(face-landmarks): integrate human library

* feat(face-landmarks): rewrite worker in typescript

* fix(face-landmarks): allow worker bundle size up to 2 mib

* fix: remove unwanted comment

* code review
2022-05-06 14:41:08 +02:00
Saúl Ibarra Corretgé 0b57bcb20b feat(load-test) split to a separate repository
It now lives here https://github.com/jitsi/jitsi-meet-load-test
2022-05-05 13:48:58 +02:00
Saúl Ibarra Corretgé 9dd44fc48e feat(local-recordings) drop old "local recordings" implementation
It's about to become very confusing, since we are going to add actual
local recordings with video.

This feature was never fully finalizeed since it required manual
processing of the files, as they were not uploaded anywhere.

In addition, unless one opens the local audio device without any audio
processing first, any tracks opened later will have audio proceessing
turned on, something not desirable for the scenario this feature was
designed for in the first place: podcasts.

This feature will likely come back as a JaaS demo / MVP where the local
recording is made outside of the Jitsi Meet iframe.
2022-05-03 14:40:12 +02:00
Gabriel Borlea 856ef757d4
fix(face-landmarks): get face models from node_modules instead of having resoruces folder (#11326) 2022-04-07 13:46:04 +03:00
Avram Tudor 42703fed47
ref(face-landmarks) refactor namings (#11307)
* ref(face-landmarks) refactor namings

* code review
2022-04-06 12:10:31 +03:00
Avram Tudor 2c165d4313
feat(face-landmarks) merge face expressions and face centering (#11283)
* feat(face-landmarks) merge face expressions and face centering

* code review

* code review

* code review
2022-04-04 16:09:14 +03:00
Tudor-Ovidiu Avram d718d9d8fb feat(face-centering) implement centering of faces in a video
Config options:

faceCoordinatesSharing.enabled
faceCoordinatesSharig.threshold
faceCoordinatesSharing.captureInterval
2022-03-04 18:13:36 -06:00
Saúl Ibarra Corretgé a7529aca7f fix(build) remove double slashes 2022-01-25 12:15:35 +01:00
Saúl Ibarra Corretgé d5f0e3c062 feat(ts) introduce TypeScript
This is the stepping stone for TypeScript support across Jitsi Meet. Currently
it makes it possible for lib-jitsi-meet to use TypeScript without adding any TS
code here yet.
2022-01-25 12:15:35 +01:00
Saúl Ibarra Corretgé b65eb7d01a fix(build) exit with error if any CSS step fails
Before:

~~~
$ make deploy-css
./node_modules/.bin/sass css/main.scss css/all.bundle.css && \
        ./node_modules/.bin/cleancss --skip-rebase css/all.bundle.css > css/all.css ; \
        rm css/all.bundle.css
Error: Can't find stylesheet to import.
   ╷
78 │ @import 'filmstrip/vertical_filmstrip_overrides2';
   │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ╵
  css/main.scss 78:9  root stylesheet
$
(exitcode is 0)
~~~

After:

~~~
$ make deploy-css
./node_modules/.bin/sass css/main.scss css/all.bundle.css && \
        ./node_modules/.bin/cleancss --skip-rebase css/all.bundle.css > css/all.css && \
        rm css/all.bundle.css
Error: Can't find stylesheet to import.
   ╷
78 │ @import 'filmstrip/vertical_filmstrip_overrides2';
   │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ╵
  css/main.scss 78:9  root stylesheet
make: *** [deploy-css] Error 65
$
(exitcode is != 0)
~~~
2022-01-12 10:31:04 +01:00
Saúl Ibarra Corretgé cd166b5f19 fix(build) fix make dev with facial recognition worker 2021-11-19 11:00:57 +01:00
Gabriel Borlea 61684b1071
feat(facial-expressions): add the facial expression feature and display them in speakerstats (#10006)
* Initial implementation; Happy flow

* Maybe revert this

* Functional prototype

* feat(facial-expressions): get stream when changing background effect and use presenter effect with camera

* add(facial-expressions): array that stores the expressions durin the meeting

* refactor(facial-expressions): capture imagebitmap from stream with imagecapture api

* add(speaker-stats): expression label

* fix(facial-expression): expression store

* revert: expression leabel on speaker stats

* add(facial-expressions): broadcast of expression when it changes

* feat: facial expression handling on prosody

* fix(facial-expressions): get the right track when opening and closing camera

* add(speaker-stats): facial expression column

* fix(facial-expressions): allow to start facial recognition only after joining conference

* fix(mod_speakerstats_component): storing last emotion in speaker stats component and sending it

* chore(facial-expressions): change detection from 2000ms to 1000ms

* add(facial-expressions): send expression to server when there is only one participant

* feat(facial-expressions): store expresions as a timeline

* feat(mod_speakerstats_component): store facial expresions as a timeline

* fix(facial-expressions): stop facial recognition only when muting video track

* fix(facial-expressions): presenter mode get right track to detect face

* add: polyfils for image capture for firefox and safari

* refactor(facial-expressions): store expressions by counting them in a map

* chore(facial-expressions): remove manually assigning the backend for tenserflowjs

* feat(facial-expressions): move face-api from main thread to web worker

* fix(facial-expressions): make feature work on firefox and safari

* feat(facial-expressions): camera time tracker

* feat(facial-expressions): camera time tracker in prosody

* add(facial-expressions): expressions time as TimeElapsed object in speaker stats

* fix(facial-expresions): lower the frequency of detection when tf uses cpu backend

* add(facial-expressions): duration to the expression and send it with durantion when it is done

* fix(facial-expressions): prosody speaker stats covert fro string to number and bool values set by xmpp

* refactor(facial-expressions): change expressions labels from text to emoji

* refactor(facial-expressions): remove camera time tracker

* add(facial-expressions): detection time interval

* chore(facial-expressions): add docs and minor refactor of the code

* refactor(facial-expressions): put timeout in worker and remove set interval in main thread

* feat(facial-expressions): disable feature in the config

* add(facial-expressions): tooltips of labels in speaker stats

* refactor(facial-expressions): send facial expressions function and remove some unused functions and console logs

* refactor(facial-expressions): rename action type when a change is done to the track by the virtual backgrounds to be used in facial expressions middleware

* chore(facial-expressions): order imports and format some code

* fix(facial-expressions): rebase issues with newer master

* fix(facial-expressions): package-lock.json

* fix(facial-expression): add commented default value of disableFacialRecognition flag and short description

* fix(facial-expressions): change disableFacialRecognition to enableFacialRecognition flag in config

* fix: resources load-test package-lock.json

* fix(facial-expressions): set and get facial expressions only if facial recognition enabled

* add: facial recognition resources folder in .eslintignore

* chore: package-lock update

* fix: package-lock.json

* fix(facial-expressions): gpu memory leak in the web worker

* fix(facial-expressions): set cpu time interval for detection to 6000ms

* chore(speaker-stats): fix indentation

* chore(facial-expressions): remove empty lines between comments and type declarations

* fix(facial-expressions): remove camera timetracker

* fix(facial-expressions): remove facialRecognitionAllowed flag

* fix(facial-expressions): remove sending interval time to worker

* refactor(facial-expression): middleware

* fix(facial-expression): end tensor scope after setting backend

* fix(facial-expressions): sending info back to worker only on facial expression message

* fix: lint errors

* refactor(facial-expressions): bundle web worker using webpack

* fix: deploy-facial-expressions command in makefile

* chore: fix load test package-lock.json and package.json

* chore: sync package-lock.json

Co-authored-by: Mihai-Andrei Uscat <mihai.uscat@8x8.com>
2021-11-17 16:33:03 +02:00
Vlad Piersec 9ab16a0a5c chore(webpack): Fix source maps on hot reloading 2021-11-01 11:01:52 +02:00
Saúl Ibarra Corretgé a313f5cde2 feat(build) migrate to Webpack 5 2021-10-07 20:22:44 +02:00
Christoph Settgast 2f1105e6d2 chore(deps) @matrix-org/olm@latest
Bugfixes since 3.2.1, see https://gitlab.matrix.org/matrix-org/olm/-/blob/master/CHANGELOG.rst
In 3.2.2 the package name changed to @matrix-org/olm, thus
updating in Makefile & imports

Signed-off-by: Christoph Settgast <csett86@web.de>
2021-08-24 15:40:18 +02:00
Saúl Ibarra Corretgé 30d0aabaca feat(build,rnnoise) don't use an external bundle for the effect
The majority of the code is in the WASM file, the JS is just 9KB.
It's so little, in fact, that the performance hint for the main bundle didn't
have to be adjusted.
2021-03-12 23:00:50 +01:00
Saúl Ibarra Corretgé 22b6d32174 feat(build,virtual-background) don't use an external bundle for the effect
The majority of the code is in the WASM file and models, this is just a few KB.
It's so little, in fact, that the performance hint for the main bundle didn't
have to be adjusted.
2021-03-12 23:00:50 +01:00
tudordan7 194d357005 feat(virtual-backgrounds) add virtual background support 2021-03-12 15:05:20 +01:00
Tudor D. Pop 946339a52e
feat(blur) replace BodyPix with TFLite
Use the Google Meet model and SIMD optimized WASM.
2021-02-17 16:03:33 +01:00
Hristo Terezov 7dc899ace1 ref(DeviceSelectionPopup): remove. 2021-02-08 15:53:38 -06:00
Дамян Минков 67a01364d3
Move load-test to resources (#8560)
* fix: Move load-test to resources.

* squash: Updates load-test dependencies.

* squash: Fix load-test build.
2021-02-05 09:12:45 -06:00
Jonathan Lennox 12680c35ca
Add lightweight load-test webpage, disabled by default (#8514)
Co-authored-by: Hristo Terezov <hristo@jitsi.org>
Co-authored-by: damencho <damencho@jitsi.org>
2021-02-04 18:04:36 -05:00
Saúl Ibarra Corretgé 42d559de93 deps: replace node-sass with sass
The former is no longer actively maintained.

Fixes: https://github.com/jitsi/jitsi-meet/issues/6427
2020-09-17 16:44:01 +02:00
Saúl Ibarra Corretgé 7cafa205ee e2ee: stage 2
Adapt to E2EE changes in lib-jitsi-meet. Notably:

---
    e2ee: introduce per-participant randomly generated keys

    This the second stage in our E2EE journey.

    Instead of using a single pre-shared passphrase for deriving the key used for
    E2EE, we now establish a secure E2EE communication channel amongst peers.

    This channel is implemented using libolm, using XMPP groupchat or JVB channels
    as the transport.

    Once the secure E2EE channel has been established each participant will generate
    a random 32 byte key and exchange it over this channel.

    Keys are rotated (well, just re-created at the moment) when a participant joins
    or leaves.
---
2020-09-10 16:06:25 +02:00
Saúl Ibarra Corretgé 2b4f33bef8 e2ee: use a separate bundle for the worker 2020-09-10 16:06:25 +02:00
Hristo Terezov 5e85b5f63a fix(close3): Add close3.js 2020-08-28 11:33:19 -05:00
Saúl Ibarra Corretgé 4cfc8cd7a2 deps: update clean-css (reland)
Fix the incorrect paths (was a breaking change in version 4) by using the
`--skip-rebase` cli option.
2020-07-06 21:16:14 +02:00
Saúl Ibarra Corretgé 6e679f952f redux: refactor loading of middlewares and reducers
Up until now we relied on implicit loading of middlewares and reducers, through
having imports in each feature's index.js.

This leads to many complex import cycles which result in (sometimes) hard to fix
bugs in addition to (often) breaking mobile because a web-only feature gets
imported on mobile too, thanks to the implicit loading.

This PR changes that to make the process explicit. Both middlewares and reducers
are imported in a single place, the app entrypoint. They have been divided into
3 categories: any, web and native, which represent each of the platforms
  respectively.

Ideally no feature should have an index.js exporting actions, action types and
components, but that's a larger ordeal, so this is just the first step in
getting there. In order to both set example and avoid large cycles the app
feature has been refactored to not have an idex.js itself.
2020-06-16 11:24:15 +02:00
Saúl Ibarra Corretgé 4faeb84d2e build: fix using parallell make
We don't support it, so make sure targets run serially.

Fixe: https://github.com/jitsi/jitsi-meet/issues/6647
2020-05-14 11:14:43 +02:00
ludovicm67 bdd129b9a2 feat(analytics): include MatomoHandler by default 2020-05-06 15:32:49 +02:00
ludovicm67 4c635a2a63 feat(analytics): add Matomo support 2020-05-06 15:32:49 +02:00
Pierre Pronchery 73d948d150
Also ship robots.txt in the source package (#5811) 2020-04-10 12:19:11 -05:00
Andrei Gavrilescu 761ac6a730 feat: integrate rnnoise based service for voice activity (VAD) detection 2019-10-04 12:55:18 +02:00
Cristian Florin Ghita 3b750ddd5a Add video background blur 2019-07-04 06:26:32 -07:00
Hristo Terezov 59fc3642a6 feat(amplitude): add mobile implementation 2019-03-12 16:48:08 +01:00
Hristo Terezov e5a8d95f1f feat(Amplitude): Integration. 2019-01-07 14:32:31 +00:00
Radium Zheng 0827e02de9 use official repo for libflac.js 2018-08-08 09:19:53 +10:00
Radium Zheng 5daa91ec1b update libflac.js to 4 and use proper fork 2018-08-08 09:19:28 +10:00
Radium Zheng 07bc70c2f5 Implement local recording
index.js of local recording

local-recording(ui): recording button

local-recording(encoding): flac support with libflac.js

Fixes in RecordingController; integration with UI

local-recording(controller): coordinate recording on different clients

local-recording(controller): allow recording on remote participants

local-recording(controller): global singleton

local-recording(controller): use middleware to init LocalRecording

cleanup and documentation in RecordingController

local-recording(refactor): "Delegate" -> "Adapter"

code style

stop eslint and flow from complaining

temp save: client status

fix linter issues

fix some docs; remove global LocalRecording instance

use node.js packaging for libflac.js; remove vendor/ folder

code style: flacEncodeWorker.js

use moment.js to do time diff

remove the use of console.log

code style: flac related files

remove excessive empty lines; and more docs

remove the use of clockTick for UI updates

initalize flacEncodeWorker properly, to avoid premature audio data transmission

move the realization of recordingController events
from LocalRecordingButton to middleware

i18n strings

minor markup changes in LocalRecordingInfoDialog

fix documentation
2018-08-08 09:18:16 +10:00
damencho 31dec2cf99 Cleans libs folder before create it and deploy to it. 2018-05-01 09:15:57 -05:00
Saúl Ibarra Corretgé 169c47ac7f misc: add helper make dev command
Running webpack-dev-server is not enough, so add a helper which takes are of
doing the needful.
2018-04-20 11:37:28 +02:00
virtuacoplenny 59d046dca9 feat(info): new dialog design (#2452)
* feat(info): new dialog design

- Add display of a dial in number.
- Add a static page to show a full list of dial in numbers.
- Add password management.
- Invite modal will be changed soon to remove password and
  dial-in.

* squash: add classes for torture tests

* squash: class for local lock for torture tests

* squash: more classes for torture tests

* squash: more classes, work around linter

* squash: remove unused string?

* squash: work around linter and avoid react warnings

* squash: pixel push, add bold

* squash: font size bump

* squash: NumbersTable -> NumbersList

* squash: document response from fetching numbers

* squash: showEdit -> editEnabled, pixel push padding for alignment

* squash: pin -> conferenceID

* squash: prepare to receive defaultCountry from api
2018-02-13 13:46:47 -06:00
hristoterezov 083f6b400b chore(capabilities.json): deploy 2018-02-01 17:21:52 -06:00
hristoterezov 4c65262a87 fix(browser-caps): Deploy. 2018-01-25 10:44:24 -06:00
bgrozev 090f2f9ccb Restructures the analytics events (#2333)
* ref: Restructures the pinned/unpinned events.

* ref: Refactors the "audio only disabled" event.

* ref: Refactors the "stream switch delay" event.

* ref: Refactors the "select participant failed" event.

* ref: Refactors the "initially muted" events.

* ref: Refactors the screen sharing started/stopped events.

* ref: Restructures the "device list changed" events.

* ref: Restructures the "shared video" events.

* ref: Restructures the "start muted" events.

* ref: Restructures the "start audio only" event.

* ref: Restructures the "sync track state" event.

* ref: Restructures the "callkit" events.

* ref: Restructures the "replace track".

* ref: Restructures keyboard shortcuts events.

* ref: Restructures most of the toolbar events.

* ref: Refactors the API events.

* ref: Restructures the video quality, profile button and invite dialog events.

* ref: Refactors the "device changed" events.

* ref: Refactors the page reload event.

* ref: Removes an unused function.

* ref: Removes a method which is needlessly exposed under a different name.

* ref: Refactors the events from the remote video menu.

* ref: Refactors the events from the profile pane.

* ref: Restructures the recording-related events.

Removes events fired when recording with something other than jibri
(which isn't currently supported anyway).

* ref: Cleans up AnalyticsEvents.js.

* ref: Removes an unused function and adds documentation.

* feat: Adds events for all API calls.

* fix: Addresses feedback.

* fix: Brings back mistakenly removed code.

* fix: Simplifies code and fixes a bug in toggleFilmstrip

when the 'visible' parameter is defined.

* feat: Removes the resolution change application log.

* ref: Uses consistent naming for events' attributes.

Uses "_" as a separator instead of camel case or ".".

* ref: Don't add the user agent and conference name

as permanent properties. The library does this on its own now.

* ref: Adapts the GA handler to changes in lib-jitsi-meet.

* ref: Removes unused fields from the analytics handler initializaiton.

* ref: Renames the google analytics file and add docs.

* fix: Fixes the push-to-talk events and logs.

* npm: Updates lib-jitsi-meet to 515374c8d383cb17df8ed76427e6f0fb5ea6ff1e.

* fix: Fixes a recently introduced bug in the google analytics handler.

* ref: Uses "value" instead of "delay" since this is friendlier to GA.
2018-01-03 13:24:07 -08:00
hristoterezov 3b35bbd5cf chore(Makefile): add alwaysontop.js to Makefile 2017-11-10 12:02:13 -06:00
hristoterezov 96e83989a5 feat(device_selection): Implement popup 2017-06-07 09:23:40 -05:00
Lyubo Marinov 5106f9f958 Process do_external_connect.js through webpack 2017-04-11 18:29:18 -05:00
Saúl Ibarra Corretgé bc60bd23b2 build: remove no longer needed Makefile rule
- we now use pinned dependencies, so there is no need to run npm update
- AFAICT the node-sass workaround is no longer needed
2017-03-24 11:02:09 +01:00