• Stars
    star
    253
  • Rank 160,776 (Top 4 %)
  • Language
    Python
  • Created about 5 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Tools for converting Label Studio annotations into common dataset formats

Label Studio Converter

Website โ€ข Docs โ€ข Twitter โ€ข Join Slack Community

Table of Contents

Introduction

Label Studio Format Converter helps you to encode labels into the format of your favorite machine learning library.

Examples

JSON

Running from the command line:

pip install -U label-studio-converter
python label-studio-converter export -i exported_tasks.json -c examples/sentiment_analysis/config.xml -o output_dir -f CSV

Running from python:

from label_studio_converter import Converter

c = Converter('examples/sentiment_analysis/config.xml')
c.convert_to_json('examples/sentiment_analysis/completions/', 'tmp/output.json')

Getting output file: tmp/output.json

[
  {
    "reviewText": "Good case, Excellent value.",
    "sentiment": "Positive"
  },
  {
    "reviewText": "What a waste of money and time!",
    "sentiment": "Negative"
  },
  {
    "reviewText": "The goose neck needs a little coaxing",
    "sentiment": "Neutral"
  }
]

Use cases: any tasks

CSV

Running from the command line:

python label_studio_converter/cli.py --input examples/sentiment_analysis/completions/ --config examples/sentiment_analysis/config.xml --output output_dir --format CSV --csv-separator $'\t'

Running from python:

from label_studio_converter import Converter

c = Converter('examples/sentiment_analysis/config.xml')
c.convert_to_csv('examples/sentiment_analysis/completions/', 'output_dir', sep='\t', header=True)

Getting output file tmp/output.tsv:

reviewText	sentiment
Good case, Excellent value.	Positive
What a waste of money and time!	Negative
The goose neck needs a little coaxing	Neutral

Use cases: any tasks

CoNLL 2003

Running from the command line:

python label_studio_converter/cli.py --input examples/named_entity/completions/ --config examples/named_entity/config.xml --output tmp/output.conll --format CONLL2003

Running from python:

from label_studio_converter import Converter

c = Converter('examples/named_entity/config.xml')
c.convert_to_conll2003('examples/named_entity/completions/', 'tmp/output.conll')

Getting output file tmp/output.conll

-DOCSTART- -X- O
Showers -X- _ O
continued -X- _ O
throughout -X- _ O
the -X- _ O
week -X- _ O
in -X- _ O
the -X- _ O
Bahia -X- _ B-Location
cocoa -X- _ O
zone, -X- _ O
...

Use cases: text tagging

COCO

Running from the command line:

python label_studio_converter/cli.py --input examples/image_bbox/completions/ --config examples/image_bbox/config.xml --output tmp/output.json --format COCO --image-dir tmp/images

Running from python:

from label_studio_converter import Converter

c = Converter('examples/image_bbox/config.xml')
c.convert_to_coco('examples/image_bbox/completions/', 'tmp/output.conll', output_image_dir='tmp/images')

Output images could be found in tmp/images

Getting output file tmp/output.json

{
  "images": [
    {
      "width": 800,
      "height": 501,
      "id": 0,
      "file_name": "tmp/images/62a623a0d3cef27a51d3689865e7b08a"
    }
  ],
  "categories": [
    {
      "id": 0,
      "name": "Planet"
    },
    {
      "id": 1,
      "name": "Moonwalker"
    }
  ],
  "annotations": [
    {
      "id": 0,
      "image_id": 0,
      "category_id": 0,
      "segmentation": [],
      "bbox": [
        299,
        6,
        377,
        260
      ],
      "ignore": 0,
      "iscrowd": 0,
      "area": 98020
    },
    {
      "id": 1,
      "image_id": 0,
      "category_id": 1,
      "segmentation": [],
      "bbox": [
        288,
        300,
        132,
        90
      ],
      "ignore": 0,
      "iscrowd": 0,
      "area": 11880
    }
  ],
  "info": {
    "year": 2019,
    "version": "1.0",
    "contributor": "Label Studio"
  }
}

Use cases: image object detection

Pascal VOC XML

Running from the command line:

python label_studio_converter/cli.py --input examples/image_bbox/completions/ --config examples/image_bbox/config.xml --output tmp/voc-annotations --format VOC --image-dir tmp/images

Running from python:

from label_studio_converter import Converter

c = Converter('examples/image_bbox/config.xml')
c.convert_to_voc('examples/image_bbox/completions/', 'tmp/output.conll', output_image_dir='tmp/images')

Output images can be found in tmp/images

Corresponding annotations could be found in tmp/voc-annotations/*.xml:

<?xml version="1.0" encoding="utf-8"?>
<annotation>
<folder>tmp/images</folder>
<filename>62a623a0d3cef27a51d3689865e7b08a</filename>
<source>
<database>MyDatabase</database>
<annotation>COCO2017</annotation>
<image>flickr</image>
<flickrid>NULL</flickrid>
</source>
<owner>
<flickrid>NULL</flickrid>
<name>Label Studio</name>
</owner>
<size>
<width>800</width>
<height>501</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>Planet</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>299</xmin>
<ymin>6</ymin>
<xmax>676</xmax>
<ymax>266</ymax>
</bndbox>
</object>
<object>
<name>Moonwalker</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>288</xmin>
<ymin>300</ymin>
<xmax>420</xmax>
<ymax>390</ymax>
</bndbox>
</object>
</annotation>

Use cases: image object detection

YOLO to Label Studio converter

Usage:

label-studio-converter import yolo -i /yolo/root/directory -o ls-tasks.json

Help:

label-studio-converter import yolo -h

usage: label-studio-converter import yolo [-h] -i INPUT [-o OUTPUT]
                                          [--to-name TO_NAME]
                                          [--from-name FROM_NAME]
                                          [--out-type OUT_TYPE]
                                          [--image-root-url IMAGE_ROOT_URL]
                                          [--image-ext IMAGE_EXT]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        directory with YOLO where images, labels, notes.json
                        are located
  -o OUTPUT, --output OUTPUT
                        output file with Label Studio JSON tasks
  --to-name TO_NAME     object name from Label Studio labeling config
  --from-name FROM_NAME
                        control tag name from Label Studio labeling config
  --out-type OUT_TYPE   annotation type - "annotations" or "predictions"
  --image-root-url IMAGE_ROOT_URL
                        root URL path where images will be hosted, e.g.:
                        http://example.com/images or s3://my-bucket
  --image-ext IMAGE_EXT
                        image extension to search: .jpg, .png

YOLO export folder example:

yolo-folder
  images
   - 1.jpg
   - 2.jpg
   - ...
  labels
   - 1.txt
   - 2.txt

  classes.txt

classes.txt example

Airplane
Car

Contributing

We would love to get your help for creating converters to other models. Please feel free to create pull requests.

License

This software is licensed under the Apache 2.0 LICENSE ยฉ Heartex. 2020

More Repositories

1

labelImg

LabelImg is now part of the Label Studio community. The popular image annotation tool created by Tzutalin is no longer actively being developed, but you can check out Label Studio, the open source data labeling tool for images, text, hypertext, audio, video and time-series data.
Python
20,885
star
2

label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
JavaScript
16,524
star
3

awesome-data-labeling

A curated list of awesome data labeling tools
3,470
star
4

label-studio-frontend

Data labeling react app that is backend agnostic and can be embedded into your applications โ€” distributed as an NPM package
JavaScript
318
star
5

label-studio-ml-backend

Configs and boilerplates for Label Studio's Machine Learning backend
Python
263
star
6

label-studio-transformers

Label data using HuggingFace's transformers and automatically get a prediction service
Python
176
star
7

RLHF

Collection of links, tutorials and best practices of how to collect the data and build end-to-end RLHF system to finetune Generative AI models
Jupyter Notebook
62
star
8

label-studio-sdk

Label Studio SDK
Python
51
star
9

dm2

Full-fledged Data Exploration Tool for Label Studio
JavaScript
35
star
10

pyheartex

Heartex Python SDK - Connect your own models to Heartex Data Labeling
Python
28
star
11

brand-sentiment-analysis

Scripts utilizing Heartex platform to build brand sentiment analysis from the news
CSS
22
star
12

label-studio-evalme

Evaluation metrics package
Python
7
star
13

label-studio-terraform

HCL
5
star
14

label-studio-examples

Example Code to Supplement the Label Studio Blog
Python
5
star
15

label-studio-tools

Python
4
star
16

text-classifier

Tensorflow-based text classifier that could be integrated with Heartex/Label Studio
Python
4
star
17

awesome-human-in-the-loop

Awesome List of Human in the Loop resources and references for retraining models.
4
star
18

smartfew

SmartFew is your swiss knife for semi-supervised structuring of unlabeled data using Few Shot Learning.
Python
4
star
19

charts

3
star
20

heartexlabs.github.io

Label Studio website with the documentation
HTML
2
star
21

awesome-active-learning

A curated list of awesome active learning related topics
2
star
22

label-studio-addon-dicom

DICOM format annotation and labeling support for Label Studio
2
star
23

articles

Materials we publish on Medium and other resources about labeling, machine learning, active learning, etc
1
star