Redis Analyzer
Console tool to scan keys in Redis database in real time and aggregate count and memory usage statistics by key prefixes.
Features:
- Keys count and memory usage statistics
- Concurrent scanning of multiple Redis databases
- Low memory signature
- Fast (scans instances with dozens of millions of keys in few minutes)
Motivation
There are already good tools out there doing similar job with even more features:
But using them on big Redis databases with millions of keys is not viable because of memory requirements and time it takes to complete the full scan.
Installation
You'll need rust and cargo. See here for instructions on how to get them.
Then you can just run cargo install redis-analyzer
to install it.
Alternatively, to build it yourself, clone the repository and run cargo build --release
.
Usage
# redis-analyzer --help
redis-analyzer 0.2.0
Analyzes keys in Redis to produce breakdown of the most frequent prefixes.
USAGE:
redis-analyzer [FLAGS] [OPTIONS] --urls <URL1,URL2>
FLAGS:
--full-keys Shows full keys in result instead of just suffixes.
-h, --help Prints help information
--progress Shows progress
-V, --version Prints version information
OPTIONS:
-c, --concurrency <CONCURRENCY>
Maximum number of hosts scanned at the same time. [default: number of logical CPUs]
-d, --depth <DEPTH> Maximum key depth to examine.
-f, --format <plain|json> Output format. (default: plain)
--min-count-percentage <MIN_PREFIX_COUNT_PERCENTAGE>
Minimum prefix frequency in percentages for prefix to be included in the result. [default: 1.0]
--order <count|memory> Sort order. [default: memory]
--scan-size <SCAN_SIZE>
Configures how many keys are fetched at a time. [default: 100]
-s, --separators <SEPARATORS> List of key separators. [default: :/|]
-u, --urls <URL1,URL2> List of URLs to scan.
Examples
Example output:
$ redis-analyzer -u 127.0.0.1:6379/0,127.0.0.1:6379/2
Took 976ms 142us
Keys Count Memory Usage
ALL -------------------15155 (100.00%) --------------26.88MB (100.00%)
ββ cache --------------ββ 294 (1.94%) ---------------ββ 2.04MB (7.60%)
β ββ Touchify --------β--ββ 239 (81.29%) -----------β--ββ 20.62KB (0.99%)
β ββ internal -----β-----ββ 239 (100.00%) -------β-----ββ 20.62KB (100.00%)
β ββ User ------β--------ββ 239 (100.00%) ----β--------ββ 20.62KB (100.00%)
ββ feed ---------------ββ 158 (1.04%) ---------------ββ 1.60MB (5.97%)
β ββ feed ------------β--ββ 158 (100.00%) ----------β--ββ 1.60MB (100.00%)
β ββ feeds --------β-----ββ 155 (98.10%) --------β-----ββ 1.60MB (99.93%)
ββ hovno --------------ββ 13808 (91.11%) ------------ββ 1.55MB (5.75%)
ββ sidekiq ------------ββ 399 (2.63%) ---------------ββ 1.27MB (4.74%)
β ββ stat ------------β--ββ 388 (97.24%) -----------β--ββ 28.21KB (2.16%)
β ββ processed ----β-----ββ 194 (50.00%) --------β-----ββ 14.58KB (51.68%)
β ββ failed -------β-----ββ 194 (50.00%) --------β-----ββ 13.63KB (48.32%)
ββ stat ---------------ββ 176 (1.16%) ---------------ββ 11.24KB (0.04%)
ββ counts -------------ββ 120 (0.79%) ---------------ββ 9.68KB (0.04%)
ββ [other] ------------ββ 200 (1.32%) ---------------ββ 20.39MB (75.86%)
$ redis-analyzer -u 127.0.0.1:6379/0 -d 1
Took 752ms 708us
Keys Count Memory Usage
ALL -----------14971 (100.00%) ------7.29MB (100.00%)
ββ cache ------ββ 294 (1.96%) -------ββ 2.04MB (28.02%)
ββ feed -------ββ 158 (1.06%) -------ββ 1.60MB (22.00%)
ββ hovno ------ββ 13808 (92.23%) ----ββ 1.55MB (21.19%)
ββ sidekiq ----ββ 399 (2.67%) -------ββ 1.27MB (17.42%)
ββ counts -----ββ 120 (0.80%) -------ββ 9.68KB (0.13%)
ββ [other] ----ββ 192 (1.28%) -------ββ 840.05KB (11.25%)
$ redis-analyzer -u 127.0.0.1:6379/0 -d 2 --order count --full-keys
Took 838ms 811us
Keys Count Memory Usage
ALL ---------------------14971 (100.00%) --------7.29MB (100.00%)
ββ hovno ----------------ββ 13808 (92.23%) ------ββ 1.55MB (21.19%)
ββ sidekiq --------------ββ 399 (2.67%) ---------ββ 1.27MB (17.42%)
β ββ sidekiq:stat ------β--ββ 388 (97.24%) -----β--ββ 28.21KB (2.17%)
ββ cache ----------------ββ 294 (1.96%) ---------ββ 2.04MB (28.02%)
β ββ cache:Touchify ----β--ββ 239 (81.29%) -----β--ββ 20.62KB (0.99%)
ββ feed -----------------ββ 158 (1.06%) ---------ββ 1.60MB (22.00%)
β ββ feed:feed ---------β--ββ 158 (100.00%) ----β--ββ 1.60MB (100.00%)
ββ counts ---------------ββ 120 (0.80%) ---------ββ 9.68KB (0.13%)
ββ [other] --------------ββ 192 (1.28%) ---------ββ 840.05KB (11.25%)