Commit Graph

65 Commits

Author SHA1 Message Date
Andrei Gavrilescu 06491e2406
feat(noise-suppression): Add noise suppression effect. (#11547)
* add denoise effect

* denoise prototype

* improve rnnoise / add comments

* revert some unnecessary changes

* Add noise suppressor worklet

* Send notification on failure

* address code review

* additional comments

* additional comments

* update package-lock

* fix rebase changes

* update rnnoise npm package

* sort lang

* adjust webpack performance hint

* address code review

* address code review

* switch ns files to typescript

* fix null-loader version, lang sort

* fix lint

* missing import

* fix lint / address code review

* use single action for ns state

* move activation to thunk

* increase node heap

* copy noise-suppressor to deploy

* fix ts lint
2022-07-20 15:31:17 +03:00
Saúl Ibarra Corretgé 8e65fab544 fix(virtual-background) don't treat timeout as fatal failure
If downloading the model tiemouts, it may succeed in the (near) future,
don't just give up.
2022-04-29 10:07:26 +02:00
Roshan Pulapura 09441c2632
feat(virtual-backgrounds) use new Open Source model
https://google.github.io/mediapipe/solutions/models.html#selfie-segmentation
2022-01-06 10:16:55 +01:00
Saúl Ibarra Corretgé 00ae2dc6a9 fix(virtual-backgrounds) make error message translatable 2021-12-08 15:12:32 +01:00
Saúl Ibarra Corretgé a182f53cdc fix(virtual-backgrounds) fix error if we failed to load the model
If another try is made, show an error notification instead of failing with an
exception.
2021-12-08 15:12:32 +01:00
Tudor D. Pop a618697e34
feat(notifications) revisit timeouts and make them configurable 2021-11-24 12:05:27 +01:00
Saúl Ibarra Corretgé 162a67fe8b chore(lint) tame the (uppdated) linter 2021-11-09 09:43:55 +01:00
Avram Tudor b4ba887d92
Revert "Update Virtual Background Model (#9867)" (#10247)
This reverts commit 9b6b335c60.
2021-10-25 14:48:18 +03:00
Roshan Pulapura 9b6b335c60
Update Virtual Background Model (#9867)
* update virtual background

* remove comments

* remove general model
2021-10-25 12:59:11 +03:00
Avram Tudor f24e0f3622
fix(virtual-backgrounds) cache wasm and model (#10190)
Prevents downloading wasm module and module each time background is changed
2021-10-18 15:19:21 +03:00
robertpin 001ae54a7c feat(screenshot-capture) Updated screensharing screenshot capture
Changed screen capture to non effect. Effects are used to alter the stream, this feature does not need to alter the stream, it just needs access to it

Changed image diff library. Previous library diff’ed the whole image, the new one has en early return threshold

Use ImageCaptureAPI to take the screenshot. Added polyfill for it and polyfill for createImageBitmap

Added analytics
2021-10-05 10:25:27 +02:00
Tudor D. Pop eb4aefbca1
fix(virtual-background) upload bkg image on poor connection 2021-08-26 15:33:43 +02:00
Andrei Gavrilescu d47e67c28c
fix(screen-share): never mute audio screen share track #9725 2021-08-16 12:24:01 +03:00
tudordan7 bb8c30a6c9 fix(virtual-background): Add logs on virtual background actions. 2021-07-16 10:31:35 +03:00
Tudor D. Pop d72b27d46d
fix(virtual-background): Fix mirror behavior for remote participants. 2021-07-08 13:20:12 -04:00
Tudor D. Pop 74d65ff596
fix(virtual-background): Fix resize action and prevent mirror behavio… (#9441)
* fix(virtual-background): Fix resize action and prevent mirror behaviour on desktop share as a virtual background.
Co-authored-by: tudordan7 <tudor.pop@decagon.tech>
2021-06-24 11:21:34 -04:00
Tudor D. Pop 2ed2a8d41f
fix(wasm-support): WebAssembly browser support. (#9410)
* fix(wasm-support): WebAssembly browser support.

* fix(wasm-support): WebAssembly browser support.

Co-authored-by: tudordan7 <tudor.pop@decagon.tech>
2021-06-17 09:03:39 -05:00
tudordan7 06c527b6fe feat(virtual-background): Desktop share as virtual background 2021-05-26 12:36:29 -04:00
tudordan7 7f020a1107 feat(virtual-background): Desktop share as virtual background 2021-05-26 12:36:29 -04:00
tudordan7 4872ce83ba feat(virtual-background): Desktop share as virtual background 2021-05-26 12:36:29 -04:00
tudordan7 99f61ca2cd feat(virtual-background): Desktop share as virtual background 2021-05-26 12:36:29 -04:00
tudordan7 dffe2316d4 feat(virtual-background): Desktop share as virtual background wip 2021-05-26 12:36:29 -04:00
tudordan7 04a464dfee feat(virtual-background): Desktop share as virtual background wip 2021-05-26 12:36:29 -04:00
tudordan7 be0632783d feat(virtual-background): Desktop share as virtual background wip 2021-05-26 12:36:29 -04:00
Andrei Gavrilescu 6d3d65da03
feat(screenshare): Audio only screenshare (#8922)
* audio only screen share implementation

* clean up

* handle stop screen share from chrome window

* update icon
2021-04-12 10:37:39 +03:00
Tudor D. Pop af28080058
feat(virtual-background) add slight blur option 2021-04-09 14:17:06 +02:00
Saúl Ibarra Corretgé 8eed42c273
fix(virtual-backgrounds) add segmentation model license information
Fixes: https://github.com/jitsi/jitsi-meet/issues/8792
2021-04-07 17:15:17 +02:00
Tudor D. Pop 77ee4b13e1
feat(virtual-backgrounds) add ability to upload custom images 2021-03-24 17:32:45 +01:00
luz paz d3680bbebd fix(misc) follow-up typos
Found via `codespell -q 3 -S ./lang`
2021-03-22 10:41:41 +01:00
Saúl Ibarra Corretgé 1395f84550 fix(virtual-background) fix tainted canvas when using the CDN
When we use a CDN the images come from an origin different than the site so
unless we mark them for CORS the canvas where they are painted will be tainted.

Ref: https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_enabled_image
2021-03-17 16:32:16 +01:00
Saúl Ibarra Corretgé 4fa47c8070 fix(virtual-background) use a DOM element for storing the image
THis will reuse the previously cached image and obey the base href.

Ref:
https://stackoverflow.com/questions/6241716/is-there-a-difference-between-new-image-and-document-createelementimg
2021-03-16 11:27:27 +01:00
Andrei Gavrilescu bad1bc91cf
fix(screenshare): audio screen share muted state (#8785)
* AudioMixerEffect muted state

* update lib-jitsi-meet
2021-03-15 11:44:03 +02:00
Saúl Ibarra Corretgé 31ace267ce fix(virtual-background) use tighter edge smoothing 2021-03-12 15:05:20 +01:00
tudordan7 194d357005 feat(virtual-backgrounds) add virtual background support 2021-03-12 15:05:20 +01:00
Tudor D. Pop dd1f8339b1
fix(blur-effect) enable blur effect on all platforms supporting canvas filters
That means all browsers except Safari, for now.

In addition, use the 96p model (instead of the 144p one) on browsers without SIMD support.
2021-02-25 13:21:03 +01:00
Tudor D. Pop f69a31d9c6
fix(blur) check model response status and catch errors 2021-02-19 15:00:07 +01:00
Saúl Ibarra Corretgé 861935c9d7 fix(blur) fix model paths 2021-02-18 15:57:01 +01:00
Tudor D. Pop 946339a52e
feat(blur) replace BodyPix with TFLite
Use the Google Meet model and SIMD optimized WASM.
2021-02-17 16:03:33 +01:00
Pawel Domas a0806716ae fix(JitsiStreamPresenterEffect): frozen on Safari
Canvas rendering does not work as expected on Safari - the image stays
still on the first frame. Calling play() on the video tags seems to help.
2021-01-07 15:10:27 -05:00
Jaya Allamsetty 3381cf4422 fix(screenshare): Fixes for the blurry desktop share issues.
Do not resize the desktop share to 720p by default when the desktop track resolution is higher than 720p. This is causing bluriness when presenter is turned on.
Remove the 'detail' contentHint setting for the desktop+presenter canvas stream as it forcing chrome to send only 5 fps stream for high resolution desktop tracks.
Move the desktop resizing logic behind a config.js option - videoQuality.resizeDesktopForPresenter.
2020-11-06 17:04:00 -05:00
paweldomas ab5c8d49c3 fix(flow): ignore contentHint
I don't know how to modify built-in MediaStreamTrack type, so ignore the error.
2020-10-15 15:10:40 -04:00
paweldomas 820d9b2ba8 feat(presenter): apply 'text' contentHint
...so that the text is more readable in the presenter mode. Chrome by
default uses 'detail' for screen sharing. I went with the 'text' here,
because the docs[1] say "may take advantage of encoder tools that
optimize for text rendering." - whether that's good specifically for
the presenter mode I don't know. It looked good for me when tested
on Chrome.

https://www.w3.org/TR/mst-content-hint/
2020-10-15 15:10:40 -04:00
Josh Brown ebb1b8d76b
fix(background-blur) refactor to improve performance 2020-09-17 18:25:06 +02:00
Tristian Flanagan 9e7a477797
feat(blur): terminate blur web worker when disabled (#7347)
* feat(blur): terminate blur web worker when disabled
2020-07-22 10:37:17 -04:00
Đặng Minh Tiến eb1ef0fa9c
Update JitsiStreamPresenterEffect.js (#7362)
* Update JitsiStreamPresenterEffect.js
Create/terminate the Web Worker on effect start/stop so that we don't leak them.
2020-07-21 10:01:38 -04:00
Saúl Ibarra Corretgé 7646618e5a rnnoise: simplify initialization 2020-06-16 10:49:33 +02:00
Bettenbuk Zoltan 0598e7369b ref: eslint-config-jitsi@1.0.3 2020-05-20 17:14:29 +02:00
Saúl Ibarra Corretgé 6ce27ef10d misc: set worker names 2020-04-28 12:02:58 +02:00
Jaya Allamsetty 6fc7456196 fix(blur): Decrease the blur amount and improve the accuracy 2020-03-26 16:49:51 -04:00
Andrei Gavrilescu f502e13edc
feat(screenSharing): Add system audio screen sharing 2020-03-26 14:17:44 +02:00