• Stars
    star
    464
  • Rank 94,450 (Top 2 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 5 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Computer vision utils for Blender (generate instance annoatation, depth and 6D pose by one line code)

bpycv: Computer Vision and Deep Learning Utils for Blender

Contents: Features | Install | Demo | Tips

0
Figure.1 Render instance annoatation, RGB image and depth

▮ Features

  • Render annotations for semantic segmentation, instance segmentation and panoptic segmentation
  • Generate 6DoF pose ground truth
  • Render depth ground truth
  • Pre-defined domain randomization:
  • Easy installation and demo running
  • Docker support: docker run -v /tmp:/tmp diyer22/bpycv (see Dockerfile)
  • A Python Codebase for building synthetic datasets (see example/ycb_demo.py)
  • Conversion to Cityscapes annotation format
  • Easy development and debugging due to:
    • No complicated packaging
    • Use of Blender's native API and calling methods

News: We win 🥈2nd place in IROS 2020 Open Cloud Robot Table Organization Challenge (OCRTOC)

▮ Install

bpycv support Blender 2.9, 3.0, 3.1+

  1. Download and install Blender here.

  2. Open Blender dir in terminal and run install script:

# For Windows user: ensure powershell has administrator permission

# Ensure pip: equl to /blender-path/3.xx/python/bin/python3.10 -m ensurepip
./blender -b --python-expr "from subprocess import sys,call;call([sys.executable,'-m','ensurepip'])"
# Update pip toolchain
./blender -b --python-expr "from subprocess import sys,call;call([sys.executable]+'-m pip install -U pip setuptools wheel'.split())"
# pip install bpycv
./blender -b --python-expr "from subprocess import sys,call;call([sys.executable]+'-m pip install -U bpycv'.split())"
# Check bpycv ready
./blender -b -E CYCLES --python-expr "import bpycv,cv2;d=bpycv.render_data();bpycv.tree(d);cv2.imwrite('/tmp/try_bpycv_vis(inst-rgb-depth).jpg', d.vis()[...,::-1])"

▮ Demo

1. Fast Instance Segmentation and Depth Demo

Copy-paste this code to Scripting/Text Editor and click Run Script button(or Alt+P)

import cv2
import bpy
import bpycv
import random
import numpy as np

# remove all MESH objects
[bpy.data.objects.remove(obj) for obj in bpy.data.objects if obj.type == "MESH"]

for index in range(1, 20):
    # create cube and sphere as instance at random location
    location = [random.uniform(-2, 2) for _ in range(3)]
    if index % 2:
        bpy.ops.mesh.primitive_cube_add(size=0.5, location=location)
        categories_id = 1
    else:
        bpy.ops.mesh.primitive_uv_sphere_add(radius=0.5, location=location)
        categories_id = 2
    obj = bpy.context.active_object
    # set each instance a unique inst_id, which is used to generate instance annotation.
    obj["inst_id"] = categories_id * 1000 + index

# render image, instance annoatation and depth in one line code
result = bpycv.render_data()
# result["ycb_meta"] is 6d pose GT

# save result
cv2.imwrite(
    "demo-rgb.jpg", result["image"][..., ::-1]
)  # transfer RGB image to opencv's BGR

# save instance map as 16 bit png
cv2.imwrite("demo-inst.png", np.uint16(result["inst"]))
# the value of each pixel represents the inst_id of the object

# convert depth units from meters to millimeters
depth_in_mm = result["depth"] * 1000
cv2.imwrite("demo-depth.png", np.uint16(depth_in_mm))  # save as 16bit png

# visualization instance mask, RGB, depth for human
cv2.imwrite("demo-vis(inst_rgb_depth).jpg", result.vis()[..., ::-1])

Open ./demo-vis(inst_rgb_depth).jpg:

demo-vis(inst_rgb_depth)

2. YCB Demo

mkdir ycb_demo
cd ycb_demo/

# prepare code and example data
git clone https://github.com/DIYer22/bpycv
git clone https://github.com/DIYer22/bpycv_example_data

cd bpycv/example/

blender -b -P ycb_demo.py

cd dataset/vis/
ls .  # visualize result here
# 0.jpg

Open visualize result ycb_demo/bpycv/example/dataset/vis/0.jpg:
0
instance_map | RGB | depth

example/ycb_demo.py Inculding:

  • Domain randomization for background, light and distractor (from ShapeNet)
  • Codebase for building synthetic datasets base on YCB dataset

3. 6DoF Pose Demo

Generate and visualize 6DoF pose GT: example/6d_pose_demo.py

▮ Tips

Blender may can't direct load .obj or .dea file from YCB and ShapeNet dataset.
It's better to transefer and format using meshlabserver by run meshlabserver -i raw.obj -o for_blender.obj -m wt



suggestion and pull request are welcome 😊

More Repositories

1

boxx

Tool-box for efficient build and debug in Python. Especially for Scientific Computing and Computer Vision.
Python
475
star
2

add_service

🚀 Effortlessly create and manage systemd startups with just one command.
Python
65
star
3

hik_camera

Easy-to-use Python API for Hikrobot Ethernet Industrial Camera (Pythonic 风格的海康网口工业相机 SDK)
Python
62
star
4

retail_product_checkout_tools

Tools for "Retail Product Checkout Dataset"(RPC)
Python
38
star
5

calibrating

Calibrate camera's intrinsic/extristric, and build stereo depth camera with OpenCV python.
Python
33
star
6

smartImgProcess

手工实现的智能图片处理系统 包含基础的图片处理功能 各类滤波 seam carving算法 以及结合精细语义分割信息 实现智能去除目标的功能
Python
27
star
7

process_raw

A python package to process raw and dng file, supporting demosaicing with gamma correction.
Python
25
star
8

auto_merge_ppt

自动排序并合并ppt
Python
10
star
9

ShowFoldersSize

Windows 上查看文件夹大小及文件结构
Python
10
star
10

zcs

Z Configuration System: a flexible powerful configuration system which takes advantage of both `argparse` and `yacs`
Python
9
star
11

jetInKinect

use Kinect to control jet in *GTA: San Andreas*
Python
8
star
12

stereo_matching_crestereo

Effortlessly deploy CREStereo in PyTorch with a simple pip install.
Python
7
star
13

BuctLibCrx

北化图书馆助手 Chrome 插件:将北化图书馆检索结果自动增加豆瓣图片,评分以及简介
JavaScript
7
star
14

salience-ELM

code of "Saliency Detection via CNN Coarse Learning and Compactness Based ELM Refinement"
Python
7
star
15

calibdiff

A PyTorch-based differential camera calibration library for intrinsic, extrinsic, and stereo camera calibration.
Python
6
star
16

RetroPython

用手机的重力感应操作 在树莓派的LED阵列上运行 类似贪吃蛇的双人对战游戏
JavaScript
4
star
17

Backpropagation

纯numpy实现的人工神经网络及反向传播算法
Python
3
star
18

photo_on_web

用Pyhton 和 JS 开发,对照片的GPS信息映射到地图上的照片展示网页
Python
2
star
19

MyMarkDown

我的文章
2
star
20

dataMining

基于Python的 apriori,FP tree fp growth算法实现及求其强关系
Python
2
star
21

quickSeg

(stop maintain)A deep learning Semantic Segmentation scaffolding(图像分割脚手架+模型库)
Python
2
star
22

bpycv_example_data

Example data for try bpycv
HTML
2
star
23

securityAlgorithm

Python实现 Vigenere, DES, RSA, MD5, DSA 算法,其中 RSA算法手工完成
Python
2
star
24

diyer22.github.com

HTML
1
star
25

poster_hackathon

Poster Hackathon's Python code
Python
1
star
26

graph_optimization

Example code for graph optimization
Python
1
star
27

MyLinear

为熟悉线性代数,将矩阵常用运算用Python实现
Python
1
star
28

fisheye_calibration_undistortion

Fisheye calibration and undistortion example by Python OpenCV
Python
1
star
29

ram_manage_visualization

模拟实现动态分区存储管理可视化
HTML
1
star
30

dete-toolbox

My detection toolbox
Python
1
star
31

paper-note

Write Paper Note in Issue
1
star
32

ELM

基于ELM的图像显著区域检测
Python
1
star