Description
lq, short for logseq-query, is a commandline tool for querying
logseq knowledge graphs. lq makes it easy to define
custom datalog queries and rules and invoke them from the commandline. Rules and
queries are just EDN data and can be
composed to make complex queries easy to read and write.
Setup
Install lq from npm:
npm install logseq-query -g
Usage
Note: This section assumes basic familiarity with datalog queries. For a primer on them, see http://www.learndatalogtoday.org/. For the visual learners, check out the demo!
lq knows about your local logseq graphs in ~/.logseq. For example:
$ lq graphs
| :name | :path |
|--------------+-------------------------------------------------------------------------------------------------|
| lambda | /Users/me/.logseq/graphs/logseq_local_++Users++me++code++work++lambda.transit |
| logseq-notes | /Users/me/.logseq/graphs/logseq_local_++Users++me++code++priv++logseq-notes.transit |
| test-notes | /Users/me/.logseq/graphs/logseq_local_++Users++me++code++repo++logseq-query++test-notes.transit |
Total: 3lq runs queries against one of these graphs. To specify a default graph (and
to avoid having to specify one on every query), add this to your lq config with
your GRAPH:
echo '{:default-options {:graph "GRAPH"}}' > ~/.lq/config.ednLet's look at some query and rule commands:
# List queries including ones you define
$ lq queries
| :name | :namespace | :parent | :desc |
|-----------------+------------+------------------+------------------------------------------------------------|
| content-search | lq | | Full text search on :block/content |
| has-property | lq | | List blocks that have given property |
| property | lq | | List all blocks that have property equal to property value |
| property-all | lq | | List all blocks that have properties |
| property-counts | lq | :lq/property-all | Counts for all block properties |
| property-search | lq | | Full text search on property |
| task | lq | | Todos that contain one of the markers |
Total: 7
# Pull up query-specific help
$ lq q content-search -h
Usage: lq q [OPTIONS] content-search QUERY
Full text search on :block/content
...
# Queries run by their :name
$ lq q content-search foo
...
# If two queries have the same :name, invoke their full name i.e. :namespace/:name
$ lq q lq/content-search foo
# Queries can be exported and used in logseq! Here we copy that to osx's clipboard
$ lq q content-search -e | pbcopy
# List rules including ones you define
$ lq rules
| :name | :namespace | :desc |
|-------------------+------------+-------------------------------------------------------------------|
| page-property | logseq | Pages that have property equal to value or that contain the value |
| block-content | logseq | Blocks that have given string in :block/content |
| namespace | logseq | |
| page | logseq | |
| has-property | logseq | Blocks that have given property |
| all-page-tags | logseq | |
| has-page-property | logseq | Pages that have given property |
| priority | logseq | |
| between | logseq | |
| task | logseq | Tasks that contain one of markers |
| page-tags | logseq | |
Total: 11Queries
The q command runs one of the named queries from the previous section as well as any user-defined queries.
Let's try one of the default queries, property, which finds blocks/lines with a specific property value:
$ lq q property type digital-garden
[{:block/uuid #uuid "620e8da6-e960-4e81-918e-4678db577794", :block/properties {:url "https://note.xuanwo.io/#/page/database", :type "digital-garden", :desc "Great to see one with active use of type"}, :block/left #:db{:id 3436}
...Query results print as EDN by default. This allows tools like babashka to transform results easily e.g.
$ lq q property type digital-garden | bb '(->> *input* (map #(-> % :block/properties :url)))'
("https://note.xuanwo.io/#/page/database" "https://kvistgaard.github.io/sparql/" "https://zettelkasten.sorenbjornstad.com/")lq provides useful transformations with the following options:
-
-c: Prints the count of the results -
-p: Colorizes and pretty prints results with puget, assumingpugetis available as a command -
-C: Prints only the contents of block results. This is useful to search through query results easily, which is not possible with logseq$ lq q content-search babashka -C |grep blog desc:: another #babashka script for a custom blog but this time with netlify ...
-
-t: Prints results in a table. If it's a block, it will print the :block/properties e.g.$ lq q property type digital-garden -t | :id | :url | :type | :desc | |------+------------------------------------------+----------------+------------------------------------------| | 3407 | https://note.xuanwo.io/#/page/database | digital-garden | Great to see one with active use of type | | 6674 | https://kvistgaard.github.io/sparql/ | digital-garden | sparql tutorials made with logseq | | 6600 | https://zettelkasten.sorenbjornstad.com/ | digital-garden | "remnote employee, made with tiddlywiki" | ...
-
--tag-counts: Prints tag counts sorted most to least for any query returning blocks$ lq q content-search babashka --tag-counts (["todo" 5] ["done" 3] ["eng" 2] ...
For more options, see lq q -h.
Short queries
lq provides short queries through the sq command. This command allows you to specify
as little or as much of a query from the commandline.
Some examples:
# A single where clause can be specified as is
$ lq sq '(block-content ?b "github.com/")'
...
# For multiple where clauses, wrap it in a vector
$ lq sq '[(block-content ?b "github.com/") (task ?b #{"DONE"})]'
...
# Queries without a :find default to `(pull ?b [*])`. This can be overridden with an explicit :find
$ lq sq '[:find ?b :where (block-content ?b "github.com/") (task ?b #{"DONE"})]'
...
# To print what the full query looks like
$ lq sq '[:find ?b :where (block-content ?b "github.com/") (task ?b #{"DONE"})]' -eThe sq command supports most of the q options. For the full list of
available options, see lq sq -h.
Create a query
Where lq shines is in how easy it is to define new queries. Referencing this
section, a query is a map entry in queries.edn where the map
key is its name and the value is a map with :query and :desc.edn`. Let's add
the query from the last section:
# Copies the last command's output to clipboard in osx
$ lq sq '[:find ?b :where (block-content ?b "github.com/") (task ?b #{"DONE"})]' -n | pbcopyIn queries.edn, paste the clipboard and add a :desc:
;; cldwalker is my namespace but feel free to choose your own e.g. github username
:cldwalker/github-tasks
{:query
[:find
(pull ?b [*])
:where
(block-content ?b "github.com/")
(task ?b #{"DONE"})]
:desc "Find github tasks"}This query can now be run as lq q github-tasks!
To make this query more useful, let's give this query arguments and transform
them with :in and :args-transform keys respectively. For example:
:cldwalker/github-tasks
{:query
[:find
(pull ?b [*])
;; $ and % are needed for all our queries at the beginning and end respectively
;; and refer to database and rules
:in $ ?markers %
:where
(block-content ?b "github.com/")
(task ?b ?markers)]
:args-transform (fn [args]
(set (map (comp clojure.string/upper-case name) args)))
:desc "Find github tasks"}This query can now be called with arguments e.g. lq q github-tasks todo doing.
It's worth noting that queries can use any of the rules that come with lq e.g.
block-content as well as any you define. Just use the rules and lq
will figure out how to pull the rules into your query.
Create a rule
Datalog rules allow you to bundle multiple where clauses behind one clause. They
are a great way to compose functionality, leverage datalog's terse power and
make queries more readable. Referencing this section, a rule is a
map entry in rules.edn where the key is its name and the value is a map with
:rule and :desc keys. For example, to reuse the github-tasks query in
other queries:
;; cldwalker is my namespace but feel free to choose your own e.g. github username
:cldwalker/github-task
{:rule
[(github-task ?b ?markers)
(block-content ?b "github.com/")
(task ?b ?markers)]
:desc "Github tasks"]}With this rule defined, use it in a short query to find github tasks that contain the word logseq e.g.
lq sq '[(github-task ?b #{"TODO"}) [?b :block/content "logseq"]]'Config
lq has three optional config files under ~/.lq/. Config files allow you to
add functionality to lq.
- config.edn - General configuration
- queries.edn - Define custom queries
- rules.edn - Define custom rules
Note: This tool is alpha and there may be breaking changes with configuration until it stabilizes.
For examples of these configs, see mine.
config.edn
This is the main config file. It is a map with the following keys:
:default-options(map): Provides default values for options toqandsqcommands.
queries.edn
This file defines custom queries similar to logseq's advanced queries. Queries are maps with the following keys:
:query(vector): A logseq/datascript datalog query. Any lq rules can be used in a query:desc(string): A brief description of the query:parent(keyword): Refer to an existing query in order to inherit its key values. The most common use case is to apply different result-transforms on the same query:result-transform(fn): Fn to transforms query results. Same as logseq:default-args(vector): Default arguments to pass to query if none are given:args-transform(fn): Fn to transform arguments:usage(string): Argument string to print for help. Useful when args are transformed
rules.edn
This file defines custom datalog
rules. Rules allow you
to group :where clauses in a query. Rules are maps with the following keys:
:rule(vector) - A datalog rule:desc(string) - A brief description of the rule
Motivation
This project aims to empower logseq users to access and transform their knowledge in fine-grained ways from the commandline. This project is also a great place to experiment with querying. Since this is a commandline tool, hopefully this inspires folks to script their logseq graphs and try useful things with them e.g. querying across graphs, joining graphs with external data sources, running queries in CI, etc.
Development
REPL
Interacting via a REPL is possible if this repository is cloned. Then
nbb-logseq repl to start a repl and lq bb socket-repl PORT to start a socket
repl to connect your editor to. cldwalker.logseq-query.tasks ns is for non
query fns and cldwalker.logseq-query.datascript is for query fns.
Testing
Run all tests with nbb-logseq -cp src:test:resources test/test_runner.cljs.
End to end query tests are in cldwalker.logseq-query.queries-test. These tests
query against the logseq graph test-notes. Each query/test has its own pages
and is isolated from others thanks to datascript.core/filter. To add a new
test:
- Add a new test page and relevant data to the graph, with logseq!
- With logseq >= 0.6.3, run the command
Save current graph to diskto save the graph to ~/.logseq. - Run
bb copy-test-dbto copy the logseq db undertest/.
Contributing
I'm not seeking major contributions to this project though discussion and issues on github are always welcome. I may be interested in a query or rule contribution if it's general enough. For those contributions, I would want a test for the new functionality. See testing for more.
License
See LICENSE.md
Credits
- 🪵 Logseq - For being the fastest, user-friendliest triples editor I've seen yet
- 🔥 Nbb - Opening up blazing ClojureScript CLIs to the NodeJS ecosystem
- 📀 Datascript - For bringing a modern, open-source datalog to the frontend and backend
Additional Links
- Datalevin - another datalog db that can be scripted with babashka
- Zsh autocompletion for lq