• Stars
    star
    127
  • Rank 282,790 (Top 6 %)
  • Language
    Ruby
  • License
    MIT License
  • Created over 7 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Visualize different Unicode encodings in the terminal

unibits | Reveal the Unicode [version] [ci]

Ruby library and CLI command that visualizes various Unicode and ASCII/single byte encodings in the terminal:

  • Makes analyzing encodings easier
  • Helps you with debugging strings
  • Highlights invalid/special/blank bytes/characters/codepoints
  • Supports UTF-8, UTF-16LE/UTF-16BE, UTF-32LE/UTF-32BE, ISO-8859-X, Windows-125X, IBMX, CP85X, macX, TIS-620/Windows-874, KOI8-R/KOI8-U, 7-Bit ASCII/GB1988, and arbitrary BINARY data

Color Coding

Each byte of the given string is highlighted using the following mechanism (characters -> codepoints):

  • Red for invalid bytes
  • Light blue for blanks
  • Blue for control characters
  • Non-control formatting characters in pink
  • Green for marks (Unicode only)
  • Orange for unassigned codepoints
  • Lighter orange for unassigned codepoints which are also ignorable
  • Random color for all other codepoints

The same colors are used in the higher-level companion tool uniscribe.

Setup

Make sure you have Ruby installed and installing gems works properly. Then do:

$ gem install unibits

Usage

Pass the string to debug to unibits:

From CLI

$ unibits "🌫 Idiosyncrätic ℜսᖯʏ"

From Ruby

require 'unibits/kernel_method'
unibits "🌫 Idiosyncrätic ℜսᖯʏ"

Advanced Options

unibits takes some optional options:

  • encoding (e): The encoding of the given string (uses the string's default encoding if none given)
  • convert (c): An encoding the string should be converted to before visualizing it
  • stats: Whether to show a short stats header (default: true), you can deactivate on the CLI with --no-stats
  • wide-ambiguous: Treat characters of ambiguous width as 2 spaces instead of 1 (more info)
  • width (w): Set a custom column width, if not set, unibits will retrieve it from the terminal or just use 80

Examples of Valid Encodings

UTF-8

CLI: $ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"

Ruby: unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'

Screenshot UTF-8

UTF-16LE

CLI: $ unibits -e utf-8 -c utf-16le "🌫 Idiosyncrätic ℜսᖯʏ"

Ruby: unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-16le'

Screenshot UTF-16LE

UTF-32BE

CLI: $ unibits -e utf-8 -c utf-32be "🌫 Idiosyncrätic ℜսᖯʏ"

Ruby: unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-32be'

Screenshot UTF-32BE

BINARY

CLI: $ unibits -e binary "🌫 Idiosyncrätic ℜսᖯʏ"

Ruby: unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'binary'

Screenshot BINARY

ASCII

CLI: $ unibits -e utf-8 -c ascii "ascii"

Ruby: unibits "ascii", encoding: 'utf-8', convert: 'ascii'

Screenshot ASCII

Examples of Invalid Encodings

UTF-8

Example in Ruby: unibits "unexpected \x80 | not enough \xF0\x9F\x8C | overlong \xE0\x81\x81 | surrogate \xED\xA0\x80 | too large \xF5\x8F\xBF\xBF"

Screenshot invalid UTF-8

ASCII

Example in Ruby: unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'ascii'

Screenshot invalid ASCII

Notes

More info

Related gems

Lots of thanks to @damienklinnert for the motivation and inspiration required to build this! 🎆

Copyright (C) 2017-2022 Jan Lelis https://janlelis.com. Released under the MIT license.

More Repositories

1

irbtools

Improvements for Ruby's IRB console 💎︎
Ruby
919
star
2

clipboard

Ruby access to the clipboard on Windows, Linux, macOS, Java, WSL and more platforms 📋︎
Ruby
372
star
3

paint

Ruby gem for ANSI terminal colors 🎨︎ VERY FAST
Ruby
370
star
4

whirly

Colorful Terminal Spinner for Ruby 😀︎
Ruby
324
star
5

idiosyncratic-ruby.com

Documenting All Ruby Specialities 💎︎
JavaScript
312
star
6

uniscribe

Know your Unicode ✀
Ruby
280
star
7

pws

Command-Line Password Safe 🔐︎
Ruby
209
star
8

unicode-emoji

Up-to-date Emoji Regex in Ruby 💥
Ruby
146
star
9

unicode-display_width

Monospace Unicode character width in Ruby
Ruby
118
star
10

sugar_refinery

Tiny refinements for Ruby
Ruby
110
star
11

stdgems

Ruby's default & bundled gems: The new standard library
Ruby
109
star
12

productive-sublime-snippets-ruby

Ruby Snippets for Sublime Text
Ruby
107
star
13

relaxed.ruby.style

A Relaxed Style Guide for Ruby & Configuration for RuboCop
Ruby
72
star
14

unicode-confusable

Unicode::Confusable.confusable? "ℜսᖯʏ", "Ruby"
Ruby
71
star
15

wirb

Ruby Object Inspection for IRB
Ruby
70
star
16

fresh

Fresh Ruby Enhanced SHell
Ruby
70
star
17

sig

Validate Method Arguments & Results in Ruby
Ruby
58
star
18

fancy_irb

Colors & Hash Rockets in IRB
Ruby
47
star
19

rg

A way to integrate AngularJS into a Rails project using CoffeeScript and Bower.
Ruby
46
star
20

debugging

Improve your Print Debugging
Ruby
42
star
21

unicode-x

Unicode Micro Libraries for Ruby
Ruby
38
star
22

characteristics

Character info under different encodings
Ruby
27
star
23

object_shadow

The Shadow of a Ruby Object lets you See and Manipulate its Instance Variables and Methods
Ruby
27
star
24

value_struct

Read-only structs in Ruby
Ruby
25
star
25

redux.rb

A tiny Ruby redux
Ruby
25
star
26

code

Displays a Ruby method's source code
Ruby
24
star
27

has_many_booleans

This Rails plugin/gem allows you to generate virtual boolean attributes, which get saved in the database as a single bitset integer.
Ruby
23
star
28

microevent.rb

Events for Ruby objects (a.k.a objects with Publish-Subscribe capabilities a.k.a. Observer pattern)
Ruby
23
star
29

ruby.style

Collects Ruby Style Guides
CSS
22
star
30

unicopy

Unicode command-line codepoint dumper
Ruby
20
star
31

unicode-blocks

Unicode Blocks of a Ruby String
Ruby
18
star
32

irbtools-more

irbtools-more adds gems to IRB that may not build out-of-the-box
18
star
33

character.construction

Notable characters, codepoints, and resources
Ruby
16
star
34

ruby_version

RubyVersion | Better than RUBY_VERSION
Ruby
15
star
35

better-array

Unobtrusive JavaScript Array Extras
JavaScript
15
star
36

render_react

Pre-render and mount React components from Ruby
Ruby
15
star
37

yyid.ex

Almost a random UUID in Elixir
Elixir
14
star
38

rubybuntu-gedit

Ruby/Rails/Web related gedit language definitions, mime types, styles and snippets.
Ruby
14
star
39

slim_migrations

Let's you write slightly slimmer Rails migrations.
Ruby
14
star
40

unicode-name

Unicode character names in Ruby
Ruby
13
star
41

uke

𝄝 Ukulele CLI Support
Ruby
13
star
42

gedit-external-tools

A repository for useful and handy snippets for gedit's external tools plugin
Shell
13
star
43

boolean2

Boolean2 is a Ruby constant that is an ancestor of true and false.
Ruby
11
star
44

derb

Dockerfile.erb
Ruby
11
star
45

unicode-scripts

Unicode Scripts / Script Extensions of a Ruby String
Ruby
11
star
46

az

From A to Z
Ruby
10
star
47

symbolify

␀ ␁ ␂ ␃ ␄ ␅ ␆ ␇ ␈ ␉ ␊ ␋ ␌ ␍ ␎ ␏ ␐ ␑ ␒ ␓ ␔ ␕ ␖ ␗ ␘ ␙ ␚ ␛ ␜ ␝ ␞ ␟ ␠ ␡
Ruby
9
star
48

micrologger

A minimal logger based on MicroEvent.rb
Ruby
9
star
49

watchbuffy

Which Buffy episode to put on next?
Ruby
8
star
50

ruby_info

RubyInfo | Better than SCRIPT_LINES__
Ruby
8
star
51

productive-sublime-snippets-erb

Productive Sublime Snippets for ERB
Ruby
8
star
52

unicoder

(wip)
Ruby
7
star
53

clipboard_formatter

A clipboard formatter for RSpec
Ruby
7
star
54

unicode-categories

Unicode General Categories of a Ruby String
Ruby
7
star
55

microgem

more gems
Ruby
6
star
56

rubynetz

Example Usage of Harvester
6
star
57

unicode-sequence_name

Unicode sequence names in Ruby
Ruby
6
star
58

unicode-types

Basic Unicode Types of a Ruby String
Ruby
6
star
59

Deutsch.rb

Like English.rb
Ruby
6
star
60

rubybuntu-language-specs

gtksourceview language specifications for Ruby/Web devoloper's gedit
Ruby
6
star
61

unicode-numeric_value

Convert a Unicode character into its numeric value
Ruby
6
star
62

ripl-multi_line

This ripl plugin allows you to evaluate multiple lines of code.
Ruby
6
star
63

ruby_engine

RubyEngine | Better than RUBY_ENGINE
Ruby
6
star
64

rubybuntu-mime

gnome mime types for Ruby/Web developer's gedit
5
star
65

promiseUserMedia.js

Promisified access to getUserMedia & vendor prefixes.
JavaScript
5
star
66

added

Module#added
Ruby
5
star
67

ripltools

This meta gem installs a bunch of ripl plugins for a nice-to-use general purpose ripl.
Ruby
5
star
68

every_day_irb

Ruby
4
star
69

procstar

Provides to_proc implementations for other Ruby classes than just Symbol
Ruby
4
star
70

rusty_clipboard

Ruby 🡪 Rust 🡪 System Clipboard
Ruby
4
star
71

unicode-age

Determine Unicode version required to display a string
Ruby
4
star
72

yyid.rb

Almost a random UUID in Ruby
Ruby
4
star
73

multi_block

Pass multiple blocks to a Ruby method
Ruby
4
star
74

ripl-color_result

This ripl plugin colorizes your results.
Ruby
4
star
75

nem

npm + gem = nem
Ruby
4
star
76

rubybuntu-editor-styles

gtksourceview styles for Ruby/Web devoloper's gedit
4
star
77

named_proc

Named procs and lambdas
Ruby
3
star
78

local_port

Returns the next free local port number to use for your shiny new service
Ruby
3
star
79

nomore

Blocks your computer from accessing domains on the internet
Ruby
3
star
80

cd

Enhanced cd command for the Ruby console.
Ruby
3
star
81

ripl-auto_indent

This ripl plugin indents your multi-line Ruby input.
Ruby
3
star
82

unicode-version

Which level of Unicode and Emoji support is included with Ruby?
Ruby
3
star
83

egonil

Egocentric Nil
Ruby
2
star
84

talk-ruby-unconf-surprises

Ruby is Full of Surprises (Ruby Unconf 2018)
JavaScript
2
star
85

iterate

Kernel#iterate
Ruby
2
star
86

ripl-rocket

Lets you display the ripl result as a comment on the same line.
Ruby
2
star
87

website

Ruby
2
star
88

instance_variables_from

Turn bindings, hashes or arrays into instance variables
Ruby
2
star
89

yyid.js

yyid() generates a random uuid* in the browser, uses the crypto api when available
JavaScript
2
star
90

pws-otp

Experimental OTP support for PWS
Ruby
2
star
91

null_plus

+nil
Ruby
2
star
92

ripl-color_streams

This ripl plugin colorizes your stdout and stderr streams.
Ruby
2
star
93

wcswidth-ruby

FFI bindings to libc's wcswidth() to determine the actual display width of strings
Ruby
2
star
94

unicode-category.js

Get the General Category of a Unicode character
JavaScript
1
star
95

yyid.go

Almost a random UUID in Go
Go
1
star
96

yyid-node.js

Almost a random UUID in node.js
JavaScript
1
star
97

communication-map

WebRTC based Location Sharing
CSS
1
star
98

null_question

Adds the null? predicate to Ruby's nil
Ruby
1
star
99

ripl-profiles

This ripl plugin adds a --profile option to ripl that loads profile files in ~/.ripl/profiles before starting ripl
Ruby
1
star
100

exists

Turns null objects into nil
Ruby
1
star