delve.el

So what is Delve?

Delve (currently version 0.9.4) is a package on top of Org Roam. It provides tools to collect, inspect and edit Org Roam Nodes in a separate application buffer. It helps you to quickly establish and maintain a project-specific subset of Org Roam Nodes, e.g. when writing a paper or for collecting information on a particular topic. Those subsets (called ‘collections’) can be stored persistently in separate files. Delve also offers functions to ‘edit’ nodes remotely, e.g. by adding or removing tags without having to open the node’s file itself.

Here’s a slightly outdated gif:

Slightly Breaking Change: As of v 0.9.4, the storage system has changed. Previously, Delve storage files had only been accepted in a special user directory relative to the Emacs root directory. The new system accepts multiple storage directories. Since all storage files will be recognized by their extension .delve, you must change all existing storages by renaming the files. There is an interactive function to automate that; see this section.

Table of Contents

• Quick overview: Features
• Installation
• It’s all about collections
• Storing Collections in files
• How items are presented
• Keybindings / Usage
  • Delve
  • Delve Minor Mode
• Customizations
• Upcoming
• Changelog

State of Development

The present version is a complete rewrite in response to Org Roam fundamentally changing its internal structure when migrating from v1 to v2. It has grown much more than its author has expected. Please file an issue if there are functionalities you miss or if you find an error. Also feel free to discuss conceptual issues.

Presently, development is mostly aiming at stabilizing the current basic set of features and covering the present functionality with automated tests. There are also plans for a closer integration with org-roam-ui. Further plans are to add some export options.

Feature List

• Dashboard with customizable queries for quick access (and overview) of nodes with certain tags and of nodes with no links.
• Sort any list with a spiffy interactive transient menu.
• Insert or delete nodes interactively from the Delve buffer, or from any Org Roam file using delve-minor-mode.
• Re-arrange nodes manually by moving them around, or by creating hierarchical trees.
• Copy and paste Delve nodes individually, by marking them or simply using transient regions.
• Yank copied Zettel into Org buffers as links.
• Preview a node in Delve for a rough glance at the content of a node without having to leave the collection buffer.
• Toggle between a short node view and a more extended one.
• Edit the node’s tags from within the Delve buffer without leaving it.
• Insert backlinks or fromlinks of a specific node.
• Hide away nodes by creating “piles”.
• Move marked nodes into a new collection.
• Bookmark stored collections for quick access.

Installation

Delve requires lister, which is available on Melpa. Delve itself, however, is not (yet?) on Melpa. For the time being, you will have to install it manually.

Delve is currently following the org-roam source without taking care of the releases. If there is a DB error or something similar, consider updating org-roam from source even if there is not a new official release yet.

Delve profits from all-the-icons. If it is installed, items will be displayed with nice icons. Install it from there.

Here’s an example using straight.el:

(use-package delve
  :straight (:repo "publicimageltd/delve"
             :host github
             :type git)
  :after (org-roam)
  ;; this is necessary if use-package-always-defer is true
  :demand t
  :bind
  ;; the main entry point, offering a list of all stored collections
  ;; and of all open Delve buffers:
  (("<f12>" . delve))
  :config
  ;; set meaningful tag names for the dashboard query
  (setq delve-dashboard-tags '("Tag1" "Tag2"))
  ;; optionally turn on compact view as default
  (add-hook #'delve-mode-hook #'delve-compact-view-mode)
 ;; turn on delve-minor-mode when Org Roam file is opened:
  (delve-global-minor-mode))

The notion of a ‘collection’

The core idea of Delve is to allow you to add stuff to “collections”, and to edit those. A collection is a list of nodes, either stored in a file or currently visited in a Delve buffer. All commands which ask you to do something “with a collection”, e.g. to add a node to a collection, first ask you to select the collection to act on. In these cases, you can always choose both an open Delve buffer or a storage file. Selecting a storage file effectively causes the file to be read in a new buffer, reading it ‘on the fly’.

Example: Imagine you have a stored collection of nodes referring to the topic Artificial Intelligence. Then when visiting an Org Roam file, you find an interesting node which you would like to add to that collection. You press M-n c (delve-minor-mode-collect) and select the file which contains that collection. Now a new Delve buffer has been created in the background and the node at point been added to it. To explicitly visit that buffer, use C-u f12 (C-u M-x delve).

Storing collections in files

If you call Delve (M-x delve), it will offer you a list of all stored collections. This is useful e.g. when adding Org roam nodes to an existing collection.

Delve tries to mimic Emacs’s established behavior of storing buffers into files. To store a collection, save any Delve buffer with M-x delve-save-buffer (or C-x s from within a Delve buffer). To open a collection (visit it in a new buffer), respectively use M-x delve-find-storage-file (or C-x f from within a Delve buffer). To save a buffer which is already linked to a new file, use M-x delve-write-buffer (or C-x w from within a Delve buffer).

All storage files will be recognized by the file extension .delve. The extension can be changed by setting delve-storage-suffix.

Once a buffer is associated with a file, the file name will be displayed in the header. There will be an asterisk displayed before the file name if the buffer content has been modified.

The default storage directory is defined in the variable delve-storage-paths. It is initially set to a directory delve-store within the local emacs user directory, which will be created when you use the storage feature for the first time. But since it’s Emacs, you can customize it:

;; one file name -- one directory:
(setq delve-storage-paths "~/path/to/directory")
;; a list of file names -- multiple locations:
(setq delve-storage-paths '("~/path1" "~/path2"))

Note that if you provide multiple paths, you will have to make sure by yourself that these directories do actually exist.

All stored files can be bookmarked. Simply set a bookmark in the visiting buffer. Calling the bookmark will jump to an existing buffer containing that collection or load it.

Changed at =0.9.4=: If you had used Delve prior to v 0.9.4, you might want to convert all existing files in the storage directory to the new file name format. You can use M-x delve-convert-storage-directory for that. The function is interactive and will guide you through the conversion process in two steps: It first asks you for the directory name (the default should be fine if you did not yet change delve-storage-paths) and then gives you some information about the files found in this directory. Only after you confirm that will your files be changed. In any case, the function is just a bulk rename, so you can just do it manually.

How items are presented

All nodes which refer to a file (and not to a subheading) look like that:

Here the node has been created as a backlink from the node “Künstliche Intelligenz (AI)”. If you press RET on the button linking to that original node, Delve will look for it and jump to it if it exists.

The other type of nodes (i.e., headings) looks like that:

Also note the list of tags which are associated with that specific node.

Per default, heading notes are displayed with their outline path, including the file title. The variable delve-display-path controls this behaviour, set it to nil to turn this off. Here’s two nodes first with path and then without:

In the dashboard buffer, you’ll find queries:

Press <TAB> to add its contents into the current buffer’s collection.

Keybindings / Usage

Delve

Navigation / The Tab Key

Use the usual nagivation commands.

<TAB> does the following:

• If there is a sublist (indented nodes) below the node at point, hide them outline-style.
• If the current node hides a sublist, open it.
• If there is no sublist, and the current item is a Zettel, insert all backlinks and fromlinks as sublist.
• If there is no sublist, and the current item is a query, execute the query and insert the results as a sublist.
• If there is no sublist, and the current item is a a pile, insert the contents of the pile and remove the pile (!).

As with most commands inserting stuff, pressing C-u before executing the command offers you to add that result into another collection.

Preview or visit the node at point

The preview buffer recognizes all id links in the previewed text and turns them into ‘buttons’. Press RET or click on these buttonized links to visit the node they are referring to. Press i on the links in the preview to directly insert the node referred to after the node at point.

Marking / unmarking nodes

Most functions which work with “marked nodes” also accept regions.

Choosing and inserting nodes

Per default, offer to insert a node from a given list of nodes per completion. If consult is installed, all of the following commands allow to insert multiple nodes at once. Support for other completion packages is lacking, contributions are welcome.

Insert nodes directly

Deleting nodes

Copy and Paste

There is a rudimentary support of copy/paste. Use the usual commands to copy the items within the active region into the kill ring, such as M-w, or to copy and kill them (C-w). A string representing the selected items is pushed onto the kill ring. The yank command (M-y) is remapped to an internal function which interprets this string data and inserts it at point.

There is currently no replacement for yank-pop.

Refresh / Update

Piling Zettel

Like on any good real desktop, you can pile the Zettels:

If you press p while the region is active, pile the nodes in that region.

To insert a pile, either press <TAB> or i.

Insert headings

Use h to insert a heading. A heading is just a simple text item which you can use to internally structure your nodes.

Remote Editing of Org Roam Nodes

Remote editing either applies to all marked nodes and the nodes in the currently active region, or, if nothing is marked, to the node at point.

If editing multiple nodes, you can choose between all tags which are present in all nodes (union of sets). Attempts to remove a tag in a node which does not have this tag are silently skipped.

Press g to refresh after editing.

Sorting

The key s gives access to some sorting commands, which are presented as a transient menu. Sorting (or reversing) applies to the current sublist at point. If there is no sublist, the whole list is sorted.

Delve Minor Mode

If you enable the delve-global-minor-mode, a delve-minor-mode will be locally enabled when visting an Org Roam file. This binds some keys which facilitate ‘collecting’ stuff. All keys are on a transient prefix which defaults to M-n. You can change the binding for this transient by setting the variable delve-minor-mode-prefix-key manually (or using customize).

;; set this /before/ loading Delve!, e.g. in the :init section of a
;; use-package declaration:
  (setq delve-minor-mode-prefix-key (kbd "C-c d"))

Collecting vs. Inspecting

Delve offers two distinct ways of collecting nodes, corresponding to different workflows.

One variant is that you collect nodes while browsing through your note files, that is, you visit Org Roam files and think ‘Yes, that’s interesting, I will use it later!’ You want to copy this node into a list and then move on looking through your notes. For this, Delve Minor Mode commands which have the word collect in their function name are your friends. Per default, they add the nodes to the last selected Delve buffer in the background, not disturbing your evaluation of th nodes.

For these collecting commands, use prefix key (usually C-u) to finetune the selection of the target Delve buffer. If you call the command with no prefix key (the default), Delve will use the last selected buffer or ask you to select one if there is none yet (e.g. by selecting a collection stored into a file). Using one prefix (C-u) will unconditionally prompt you to select the target collection. Using two prefixes (C-u C-u) will create a new buffer for you. Note that in this case, since you do not explicitly select these automatically generated buffer, it will not be recognized as the “last selected buffer” for the following operations.

The second variant is that you want to collect nodes in order to explore their relations to other nodes within a Delve buffer. That is, you encounter an interesting node and think: ‘Hey, I want to look at this node’s backlinks, and their backlinks, and just generally check where this node leads me too!’ In this case, you want to switch immediately to the buffer in which you have just collected the nodes. Functions offering this kind of functionality have the word inspect in their function name. They add the nodes to an automatically created Delve buffer and then switch to it. Additionally, this buffer is marked as the “last selected buffer” so that all collection commands recognize it.

Minor Mode Keys

`M-n’ opens a transient offering the user to either edit, inspect or collect the node at point. The functions finally reached through the transients can easily be bound separately. Have a look at how the transients are defined or post an issue.

For editing the node at point:

For collecting the node at point:

For inspecting:

It is planned to enable the collection keys also in Org Roam Mode buffers (TODO).

Customizations

delve-dashboard-tags 

List of strings (or of lists of strings), from which the initial Dashboard queries are built. E.g., with the setting (setq delve-dashboard-tags '("relevant")); the Dashboard will have a query for all Delve nodes tagged with the tag relevant.

delve-last-modified-limit

Number of nodes to be displayed in the pre-configured Dashboard query ‘last modified items’. Defaults to 20.

delve-compact-view-shows-node-path

In compact view, show the complete path to the node if it is not a file node (that is, if it is a subtree). Defaults to t.

Upcoming

This is a list of stuff I intend to add (soonish):

• Save all stores with suffix ‘.delve’.
• Do not restrict storage to one directory; use list of directories instead.
• Remote editing on all marked lists.
• Make minor mode collection functions also work in org roam mode buffer.
• Add ‘modified’ flag and show it in the header.
• Add sorting according to title and mtime.

Changelog

Current

• Add option delve-compact-view-shows-node-path
• In Delve buffers, remap rename-buffer to edit the collection’s title
• Add new Dashboard query ‘last modified nodes’ with customizable variable delve-last-modified-limit to set the number of nodes to be displayed.
• Add new Dashboard query ‘TODO items’
• Use hand-made completing-read-multiple (found in https://github.com/emacs-citar/citar/compare/simple-crm) since consult’s is now deprecated (see https://github.com/minad/consult/commit/b15c81f7766a8981f2f022fc47bbeb7000696caf).
• Add much more commands to the transient suffix key ‘n’

0.9.4

• Bookmarks.
• Lift storage system restriction to one predefined directory; force all storage files to end in .delve.
• Toggle between extended view (default) and a shorter one.
• Rudimentary copy/paste.
• Yank copied or pasted Org Roam node zettel into org buffers.
• Bugfix because org-roam somwhere lost the function org-roam-node-find-file-noselect.
• Sorting.

0.9.3

• Refactor collecting nodes from outside Delve.
• For non-file nodes, display the outline path. Depends now on Org Roam with DB 18 (merged in Nov., 10th, 21).
• Introduce new item type “Heading” (key h)
• Allow remote editing (add, remove tags) of multiple items.
• Mark list as “modified” if items are deleted, inserted or updated. Storing the list removes that flag.
• Somewhere in between is 0.9.2, I forgot to update all version numbers in all files.

0.9

Complete rewrite; now based on Org Roam v2.