• Stars
    star
    235
  • Rank 171,079 (Top 4 %)
  • Language
    Python
  • Created about 6 years ago
  • Updated over 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fast (aimed to "real time") Portrait Segmentation on mobile phone

Fast_Portrait_Segmentation

Fast (aimed to "real time") Portrait Segmentation at mobile phone

This project is not normal semantic segmentation but focus on real-time protrait segmentation.All the experimentals works with pytorch.

I hope to find a effcient network which can run on mobile phone. Currently, successfull application of person body/protrait segmentation can be find in APP like SNOW&B612, whose technology is proposed by a Korea company Nalbi.

Models

  • mobilenet_dilate_unet[1][2][7][9]

    Encoder : mobilenet_v2(os: 32)

    Decoder : unet(concat low level feature) use dilate convolution at different stage(d = 2, 6, 12, 18)

  • Shuffle_Seg_SkipNet[4][10][18]

    Encoder : shufflenet

    Decoder : skip connection (add low level feature)

  • esp_dense_seg[20][10][15][19]

  • residualdense_bisenet[15][23][24]

    Attention model is a potential module in the segmentation task. I use a very light residual-dense net as the backbone of the Context Path. The details about fussion of last features in Contxt Path is not clear in the paper(BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation).

  • Segmentation + Matting [7][12][15]

    Hard segmentation + Soft matting.(coming soon)

update 2019/04/10: The code and pre_trained model of final version of the portrait_segmentation is released ! ! ! mobile_phone_human_matting

Speed Analysis

⚡ Real-time ! ! ! 🎉🎉🎉

Platform : ncnn.

Mobile phone: Samsung Galaxy S8+(cpu).

model size (M) time(ms)
model_seg_matting 3.3 ~40

update : 2018/12/27: Demo video on my iphone 6 (baiduyun)

Result Examples

HUAWEI Mate 20 released recently can keep color on human and make the bacgrand gray in real time (click to view ). I test my model using cpu on my MAC, getting some videos here.

References

papers