haskal
/
jiti-meet
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
jiti-meet/react/features/stream-effects/virtual-background/vendor
History
Saúl Ibarra Corretgé 8eed42c273
fix(virtual-backgrounds) add segmentation model license information 
Fixes: https://github.com/jitsi/jitsi-meet/issues/8792
 		2021-04-07 17:15:17 +02:00
..
models feat(virtual-backgrounds) add virtual background support 2021-03-12 15:05:20 +01:00
tflite feat(virtual-backgrounds) add virtual background support 2021-03-12 15:05:20 +01:00
README.md fix(virtual-backgrounds) add segmentation model license information 2021-04-07 17:15:17 +02:00

README.md

Virtual Background on stream effects

Inspired from https://ai.googleblog.com/2020/10/background-features-in-google-meet.html and https://github.com/Volcomix/virtual-background.git

Canvas 2D + CPU

This rendering pipeline is pretty much the same as for BodyPix. It relies on Canvas compositing properties to blend rendering layers according to the segmentation mask.

Interactions with TFLite inference tool are executed on CPU to convert from UInt8 to Float32 for the model input and to apply softmax on the model output.

The framerate is higher and the quality looks better than BodyPix

SIMD and non-SIMD

How to test on SIMD:

  1. Go to chrome://flags/
  2. Search for SIMD flag
  3. Enable WebAssembly SIMD support(Enables support for the WebAssembly SIMD proposal).
  4. Reopen Google Chrome

More details:

LICENSE

The mdoels vendored here were downloaded early January (they were available as early as the 4th), before Google switched the license away from Apache 2. Thus we understand they are not covered by the new license which according to the model card dates from the 21st of January.

We are not lawyers so do get legal advise if in doubt.

References: