jiti-meet/react/features/stream-effects/virtual-background/vendor/README.md

# Virtual Background on stream effects

> From https://google.github.io/mediapipe/solutions/models.html#selfie-segmentation

#### Canvas 2D + CPU

This rendering pipeline is pretty much the same as for BodyPix. It relies on Canvas compositing properties to blend rendering layers according to the segmentation mask.

Interactions with TFLite inference tool are executed on CPU to convert from UInt8 to Float32 for the model input and to apply softmax on the model output.

The framerate is higher and the quality looks better than BodyPix

#### SIMD and non-SIMD

How to test on SIMD:
1. Go to chrome://flags/
2. Search for SIMD flag
3. Enable WebAssembly SIMD support(Enables support for the WebAssembly SIMD proposal).
4. Reopen Google Chrome

More details:
- [WebAssembly](https://webassembly.org/)
- [WebAssembly SIMD](https://github.com/WebAssembly/simd)
- [TFLite](https://blog.tensorflow.org/2020/07/accelerating-tensorflow-lite-xnnpack-integration.html)
feat(blur) replace BodyPix with TFLite Use the Google Meet model and SIMD optimized WASM. 2021-02-17 15:03:33 +00:00			`# Virtual Background on stream effects`

feat(virtual-backgrounds) use new Open Source model https://google.github.io/mediapipe/solutions/models.html#selfie-segmentation 2022-01-06 09:16:55 +00:00			`> From https://google.github.io/mediapipe/solutions/models.html#selfie-segmentation`
feat(blur) replace BodyPix with TFLite Use the Google Meet model and SIMD optimized WASM. 2021-02-17 15:03:33 +00:00
			`#### Canvas 2D + CPU`

			`This rendering pipeline is pretty much the same as for BodyPix. It relies on Canvas compositing properties to blend rendering layers according to the segmentation mask.`

			`Interactions with TFLite inference tool are executed on CPU to convert from UInt8 to Float32 for the model input and to apply softmax on the model output.`

			`The framerate is higher and the quality looks better than BodyPix`

			`#### SIMD and non-SIMD`

			`How to test on SIMD:`
			`1. Go to chrome://flags/`
			`2. Search for SIMD flag`
			`3. Enable WebAssembly SIMD support(Enables support for the WebAssembly SIMD proposal).`
			`4. Reopen Google Chrome`

			`More details:`
			`- [WebAssembly](https://webassembly.org/)`
			`- [WebAssembly SIMD](https://github.com/WebAssembly/simd)`
fix(virtual-backgrounds) add segmentation model license information Fixes: https://github.com/jitsi/jitsi-meet/issues/8792 2021-04-07 15:15:17 +00:00			`- [TFLite](https://blog.tensorflow.org/2020/07/accelerating-tensorflow-lite-xnnpack-integration.html)`