• Stars
    star
    467
  • Rank 93,935 (Top 2 %)
  • Language
    Elixir
  • Created over 4 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A minimal filesystem-based publishing engine with Markdown support and code highlighting

NimblePublisher

Online Documentation.

NimblePublisher is a minimal filesystem-based publishing engine with Markdown support and code highlighting.

use NimblePublisher,
  build: Article,
  from: Application.app_dir(:app_name, "priv/articles/**/*.md"),
  as: :articles,
  highlighters: [:makeup_elixir, :makeup_erlang]

The example above will get all articles in the given directory, call Article.build/3 for each article, passing the filename, the metadata, and the article body, and define a module attribute named @articles with all built articles returned by the Article.build/3 function.

Each article in the articles directory must have the format:

%{
  title: "Hello world"
}
---
Body of the "Hello world" article.

This is a *markdown* document with support for code highlighters:

```elixir
IO.puts "hello world"
```

Options

  • :build - the name of the module that will build each entry

  • :from - a wildcard pattern where to find all entries. Files with the .md or .markdown extension will be converted to Markdown with Earmark. Other files will be kept as is.

  • :as - the name of the module attribute to store all built entries

  • :highlighters - which code highlighters to use. NimblePublisher uses Makeup for syntax highlighting and you will need to add its .css classes. You can generate the CSS classes by calling Makeup.stylesheet(:vim_style, "makeup") inside iex -S mix. You can replace :vim_style by any style of your choice defined here.

  • :earmark_options - an %Earmark.Options{} struct

  • :parser - custom module with a parse/2 function that receives the file path and content as params. See Custom parser for more details.

  • :html_converter - custom module with a convert/4 function that receives the extension, body, and attributes of the markdown file, as well as all options as params. See Custom HTML converter for more details.

Examples

Let's see a complete example. First add nimble_publisher with the desired highlighters as a dependency:

def deps do
  [
    {:nimble_publisher, "~> 1.0"},
    {:makeup_elixir, ">= 0.0.0"},
    {:makeup_erlang, ">= 0.0.0"}
  ]
end

In this example, we are building a blog. Each post stays in the "posts" directory with the format:

/posts/YEAR/MONTH-DAY-ID.md

A typical blog post will look like this:

# /posts/2020/04-17-hello-world.md
%{
  title: "Hello world!",
  author: "Josรฉ Valim",
  tags: ~w(hello),
  description: "Let's learn how to say hello world"
}
---
This is the post.

Therefore, we will define a Post struct that expects all of the fields above. We will also have a :date field that we will build from the filename. Overall, it will look like this:

defmodule MyApp.Blog.Post do
  @enforce_keys [:id, :author, :title, :body, :description, :tags, :date]
  defstruct [:id, :author, :title, :body, :description, :tags, :date]

  def build(filename, attrs, body) do
    [year, month_day_id] = filename |> Path.rootname() |> Path.split() |> Enum.take(-2)
    [month, day, id] = String.split(month_day_id, "-", parts: 3)
    date = Date.from_iso8601!("#{year}-#{month}-#{day}")
    struct!(__MODULE__, [id: id, date: date, body: body] ++ Map.to_list(attrs))
  end
end

Now, we are ready to define our MyApp.Blog with NimblePublisher:

defmodule MyApp.Blog do
  alias MyApp.Blog.Post

  use NimblePublisher,
    build: Post,
    from: Application.app_dir(:my_app, "priv/posts/**/*.md"),
    as: :posts,
    highlighters: [:makeup_elixir, :makeup_erlang]

  # The @posts variable is first defined by NimblePublisher.
  # Let's further modify it by sorting all posts by descending date.
  @posts Enum.sort_by(@posts, & &1.date, {:desc, Date})

  # Let's also get all tags
  @tags @posts |> Enum.flat_map(& &1.tags) |> Enum.uniq() |> Enum.sort()

  # And finally export them
  def all_posts, do: @posts
  def all_tags, do: @tags
end

Important: Avoid injecting the @posts attribute into multiple functions, as each call will make a complete copy of all posts. For example, if you want to define recent_posts() as well as all_posts(), DO NOT do this:

def all_posts, do: @posts
def recent_posts, do: Enum.take(@posts, 3)

Instead do this:

def all_posts, do: @posts
def recent_posts, do: Enum.take(all_posts(), 3)

Other helpers

You may want to define other helpers to traverse your published resources. For example, if you want to get posts by ID or with a given tag, you can define additional functions as shown below:

defmodule NotFoundError do
  defexception [:message, plug_status: 404]
end

def get_post_by_id!(id) do
  Enum.find(all_posts(), &(&1.id == id)) ||
    raise NotFoundError, "post with id=#{id} not found"
end

def get_posts_by_tag!(tag) do
  case Enum.filter(all_posts(), &(tag in &1.tags)) do
    [] -> raise NotFoundError, "posts with tag=#{tag} not found"
    posts -> posts
  end
end

Custom parser

You may want to define a custom function to parse the content of your files.

  use NimblePublisher,
    ...
    parser: Parser,

defmodule Parser do
  def parse(path, contents) do
    [attrs, body] = :binary.split(contents, ["\n---\n"])
    {Jason.decode!(attrs), body}
  end
end

The parse/2 function from this module receives the file path and content as params. It must return:

  • a 2 element tuple with attributes and body - {attrs, body}
  • a list of 2 element tuple with attributes and body - [{attrs, body} | _]

Custom HTML converter

You can also define a custom HTML converter that will be used to convert the file body (typically Markdown) into HTML. For example, you may wish to use an alternative Markdown parser such as md. If you want to use the built-in highlighting, you need to call it manually.

  use NimblePublisher,
    ...
    html_converter: MarkdownConverter,
    highlighters: [:makeup_elixir]
defmodule MarkdownConverter do
  def convert(filepath, body, _attrs, opts) do
    if Path.extname(filepath) in [".md", ".markdown"] do
      highlighters = Keyword.get(opts, :highlighters, [])
      body |> Md.generate() |> NimblePublisher.highlight(highlighters)
    end
  end
end

The convert/4 function from this module receives an extension name, a body, the parsed attributes from the file, and the options passed to NimblePublisher. It must return the converted body as a string.

Live reloading

If you are using Phoenix, you can enable live reloading by simply telling Phoenix to watch the โ€œpostsโ€ directory. Open up "config/dev.exs", search for live_reload: and add this to the list of patterns:

live_reload: [
  patterns: [
    ...,
    ~r"posts/*/.*(md)$"
  ]
]

Learn more

Nimble*

All nimble libraries by Dashbit:

  • NimbleCSV - simple and fast CSV parsing
  • NimbleOptions - tiny library for validating and documenting high-level options
  • NimbleParsec - simple and fast parser combinators
  • NimblePool - tiny resource-pool implementation
  • NimblePublisher - a minimal filesystem-based publishing engine with Markdown support and code highlighting
  • NimbleTOTP - tiny library for generating time-based one time passwords (TOTP)

License

Copyright 2020 Dashbit

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

More Repositories

1

broadway

Concurrent and multi-stage data ingestion and data processing with Elixir
Elixir
2,414
star
2

flow

Computational parallel flows on top of GenStage
Elixir
1,544
star
3

mox

Mocks and explicit contracts in Elixir
Elixir
1,343
star
4

nimble_parsec

A simple and fast library for text-based parser combinators
Elixir
807
star
5

nimble_csv

A simple and fast CSV parsing and dumping library for Elixir
Elixir
771
star
6

nimble_options

A tiny library for validating and documenting high-level options. ๐Ÿ’ฝ
Elixir
505
star
7

nimble_totp

A tiny Elixir library for time-based one time passwords (TOTP)
Elixir
396
star
8

nimble_pool

A tiny resource-pool implementation for Elixir
Elixir
344
star
9

bytepack_archive

Archive of bytepack.io
Elixir
328
star
10

broadway_kafka

A Broadway connector for Kafka
Elixir
232
star
11

broadway_dashboard

Keep track of your Broadway pipelines from Phoenix LiveDashboard
Elixir
218
star
12

mix_phx_gen_auth_demo

Example repository for mix phx.gen.auth
Elixir
211
star
13

broadway_rabbitmq

A Broadway producer for RabbitMQ
Elixir
192
star
14

broadway_sqs

A Broadway producer for Amazon SQS
Elixir
92
star
15

broadway_cloud_pub_sub

A Broadway producer for Google Cloud Pub/Sub
Elixir
70
star
16

nimble_ownership

Elixir
55
star
17

broadway_bike_sharing_rabbitmq_example

An example of a Broadway pipeline for a bike sharing app with RabbitMQ and PostgreSQL
Elixir
50
star
18

nimble_strftime

A simple and fast strftime-based datetime formatter
Elixir
37
star
19

unpickler

A library for loading data in the Python's pickle format
Elixir
14
star
20

broadway_website

Landing page for the Elixir Broadway library.
HTML
6
star