Background Blurring with Semantic Image Segmentation using DeepLabv3+

Few days ago Microsoft released a version of Teams which uses AI to blur your background during video calls.

Here’s a video on YouTube

This gave me an idea to try building this myself using AI. A few months ago Google open sourced DeepLab, a state of the art research for semantic image segmentation

From DeepLab Github

DeepLab: Deep Labelling for Semantic Image Segmentation

DeepLab is a state-of-art deep learning model for semantic image segmentation, where the goal is to assign semantic labels (e.g., person, dog, cat and so on) to every pixel in the input image.

So I used a Keras implementation of DeepLabv3+ to blur my background when I use my webcam


In this post I’ll show you how to build this


After installing DeepLabV3+ lets import it along with required libraries


Then we create an instance of DeepLabv3 model, initialize our webcam stream, create a window using OpenCV where our result will be displayed and set a value for the background blur that is required


In the above code we initialize a loop and read every frame streaming from our webcam. We then resize the input to be 512px wide.
In the following lines a padding is added to each frame to resize it to 512×512 pixels required by DeepLabv3 model, please note that can be changed when initializing the model by specifying your own desired input size.
Another dimension is added to the input frame using numpy expand_dims before calling the predict function of the model.

A labels array containing indices of the maximum values is created after we remove single dimensions from the output result.


We then cleanup labels array by padding some values from the bottom of the image. We create a mask over all the black pixels in the output array. Then we resize the input frame from our webcam to match the size of our output labels for fitting our mask we just created
Then we blur this resized frame, in the original frame I replace the values of every dark pixel with values from our blurred frame and show the results. We exit while loop and close the stream after pressing ‘q’ to clean up OpenCV and streaming video frames objects

You can grab the code from my github and if you like it dont forget to give it a Star 🙂

2 Replies to “Background Blurring with Semantic Image Segmentation using DeepLabv3+”

  1. Hi good work out there however when i am trying to import Deeplabv3 i get error. i looked in and i didn’t find any Deeplabv3. Any Idea

    ImportError: cannot import name ‘Deeplabv3’ from ‘deeplab.model’ from I tried search Deeplabv3 in but i found nothing any idea.

    When i run python deeplab/ i get following logs

    [ OK ] DeeplabModelTest.testBuildDeepLabWithDensePredictionCell
    [ RUN ] DeeplabModelTest.testBuildDeepLabv2
    [ OK ] DeeplabModelTest.testBuildDeepLabv2
    [ RUN ] DeeplabModelTest.testForwardpassDeepLabv3plus
    W0821 01:05:20.922364 4398204352] From deeplab/ The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

    [ OK ] DeeplabModelTest.testForwardpassDeepLabv3plus
    [ RUN ] DeeplabModelTest.testWrongDeepLabVariant
    [ OK ] DeeplabModelTest.testWrongDeepLabVariant
    [ RUN ] DeeplabModelTest.test_session
    [ SKIPPED ] DeeplabModelTest.test_session

    1. It could be the directory structure and
      Coincidentally I just faced this issue a short while ago
      ‘Import Error: Cannot import Name XXXX’ and I had to remove XXXX from the import statement to look like
      From YYYY.ZZZ import AAAA

      Maybe you should try that or post your question in their Github issue tracker to get instant help, did you try searching in there?

Leave a Reply

Your email address will not be published. Required fields are marked *