dmytrostriletskyi/diagrams-as-code

Diagrams as code: declarative configurations using YAML for drawing cloud system architectures.

Table of content:

• Introduction
• Roadmap
• Getting Started
  • How to Install
  • Examples
  • Syntax Highlighting
    • PyCharm
• Usage
  • Command Line Interface
  • Guide
• Disclaimer

Introduction

Diagrams as code is essentially the process of managing diagrams through code rather than interactively drawing them on specific web services such as draw.io. It lets you generate the cloud system architecture in a declarative way with widely used YAML syntax (which is de facto a standard for infrastructure and configurations).

Declarative method of describing things means that a user simply describes the solution they need, how it should look, and everything that would be in the state of the final solution, leaving the process for the software to decide.

Diagrams as code brings you the following benefits compared to drawing architecture on your own, it:

• Does not require any knowledge about how to properly draw an architecture diagram. Basically, you just define a set of resources, compose them into groups and set relationships, the rest is done for you.
• Allows you to track architecture diagram changes with a version control systems such as Git.
• Moves collaboration to the next level: updating an architecture diagram through a pull request with a code review instead of a video session and/or screen sharing.
• Reduces costs on further updating of an initial architecture diagram. Basically, when you create an image on a web service you have to eventually store two files: PNG-like to put into your documentation and XML to be able to adjust your image in the future. So, there is no need to care about XML anymore.
• Backups you in case of losing XML files as YAML files are always stored in a repository.
• Improves consistency as now a diagram is stored along the code in a repository, the place you visit and work on frequently, and it is easier to keep it up-to-date.

Currently, the following components are provided:

• Major cloud providers: AWS, Azure, GCP, IBM, Alibaba, Oracle, OpenStack, DigitalOcean and so on.
• On-Premise, Kubernetes, Firebase, Elastic, SaaS.
• Programming languages and frameworks.

Roadmap

• Add support of C4.
• Add support of Custom.
• Add IDEs plugins and/or web user interface for live editing.
• Add the JSON Schema to Json Schema Store.
• Research Confluence integration to update images from the CI-builds directly.
• Research ChatGRT integration.

Getting Started

How to install

As the project uses Graphviz to render the diagram, you need to install it:

• For Linux — https://graphviz.gitlab.io/download/#linux
• For Windows — https://graphviz.gitlab.io/download/#windows
• For macOS — https://graphviz.gitlab.io/download/#mac

After, you can install the project itself with the following command using pip3:

$ pip3 install diagrams-as-code

Examples

You can find examples of YAML configurations in the examples folder. Below are placed are few of them (click on the name to redirect to the configurations file).

Web Services on AWS	Web Services On-Premise	Exposed Pods on Kubernetes

Message Collecting on GCP	Events Processing on AWS	Workers on AWS

Syntax Highlighting

When you will be writing your own YAML files, you likely need a syntax highlighting. Currently, there is no mapping of the YAML files to a specific schema to enable the syntax highlighting automatically. So, there is a need for manual operation here.

PyCharm

For PyCharm, open the settings and proceed to Languages & Frameworks, then to Scheams and DTDs, then to JSON Schema Mappings. After, create a new schema, name it Diagrams as code, choose JSON Schema version 7, paste https://raw.githubusercontent.com/dmytrostriletskyi/diagrams-as-code/main/json-schemas/0.0.1.json` to the Schema file or URL field and click Apply:

Right after then, open a YAML file and click on No JSON schema at to bottom-right corner:

It will open a panel where you can choose the newly created schema with the name Diagrams as code:

As a result, you will experience syntax highlighting when typing:

Usage

Command Line Interface

To draw an architecture, call diagrams-as-code command line interface, providing a path to a YAML file with configurations. The drawing will be saved in the folder the command line interface was executed from.

$ diagrams-as-code examples/web-services-aws.yaml

Guide

Please, check all-fields.yaml as the example to see all possible configurations before diving into a detailed explanation about them (and this is the result image).

A YAML file conceptually contains two configurations: generic information such as a name of a diagram, a format of an image to be generated, style and resources themselves such as AWS and Kubernetes resources, Nginx, ElasticSearch and all other things you actually want to draw, and relationships among them.

diagram:
  name: Web Services Architecture on AWS
  file_name: web-services-architecture-aws
  format: jpg
  direction: left-to-right
  style:
    graph:
      splines: ortho
    node:
      shape: circle
    edge:
      color: '#000000'
  label_resources: false
  open: true
  resource:
    ...

Generic information schema looks like this:

style is responsible for styling overall diagram such as a background color or choosing between curvy or straight arrows. It should go as a nested object of parameters as key/value of Graphviz's attributes.

Its schema looks like:

resources is responsible for specifying a list of resources on a diagram and relationships among them. Each resource has a unique identifier, name and type. Name will be shown on a diagram under the specific resource:

diagram:
  resources:
    - id: elb
      name: ELB
      type: aws.network.ELB

Identifier will be used by other resources to set a relationship direction's type among them:

diagram:
  resources:
    - id: dns
      name: DNS
      type: aws.network.Route53
      relates:
        - to: elb
          direction: outgoing
    - id: elb
      name: ELB
      type: aws.network.ELB

There is also a type named group. It is not a specific resource, it is rather a group of resources. It is needed for other resources to be able to have a relationship with all group's resources at once. Each group's resource can have separate relationships as well:

diagram:
  resources:
    - id: elb
      name: ELB
      type: aws.network.ELB
      relates:
        - to: graphql-api
          direction: outgoing
    - id: graphql-api
      name: GraphQL API
      type: group
      of:
        - id: first-api
          name: GraphQL API №1
          type: aws.compute.ECS
        - id: second-api
          name: GraphQL API №2
          type: aws.compute.ECS
        - id: third-api
          name: GraphQL API №3
          type: aws.compute.ECS

There is also a type named cluster. It is needed to separate multiple resources or groups logically: for instance, there might be a cluster of different APIs composed as groups, a cluster of databases and a cluster of caches.

Pay attention that to refer a cluster resource, there are the following identifiers web-services.graphql-api and web-services.rest-api what means that you need to chain identifiers of nested resources through a dot to identify a resource you build a relationship with.

diagram:
  resources:
    - id: elb
      name: ELB
      type: aws.network.ELB
      relates:
        - to: web-services.graphql-api
          direction: outgoing
        - to: web-services.rest-api
          direction: outgoing
    - id: web-services
      name: Web Services
      type: cluster
      of:
        - id: graphql-api
          name: GraphQL API
          type: group
          of:
            - id: first-api
              name: GraphQL API №1
              type: aws.compute.ECS
            - id: second-api
              name: GraphQL API №2
              type: aws.compute.ECS
            - id: third-api
              name: GraphQL API №3
              type: aws.compute.ECS
        - id: rest-api
          name: REST API
          type: group
          of:
            - id: first-api
              name: REST API №1
              type: aws.compute.EC2
            - id: second-api
              name: REST API №2
              type: aws.compute.EC2
            - id: third-api
              name: REST API №3
              type: aws.compute.EC2
    - id: databases
      name: Databases
      type: cluster
      of:
        - id: leader
          name: Leader
          type: aws.database.RDS
          relates:
            - to: databases.follower
              direction: undirected
        - id: follower
          name: Follower
          type: aws.database.RDS

Basically, to recap and also clarify:

• There are resources, groups (of resources or other groups) and clusters (of resources, groups or other clusters).
• You can build a relationship between resource to resource and resource to group (and vice versa), it is impossible to relate to a cluster.
• You should chain identifiers of nested resources through a dot to identify a resource you build a relationship to.

resources schema looks like this:

This is the table of all available types by a category:

relates schema looks like:

Disclaimer

diagrams-as-code is a wrapper around the original diagrams. The original diagrams lets you draw the cloud system architecture in Python code. It was born for prototyping a new system architecture design without any design tools. Under the hood, diagrams-as-code parse a YAML file and map to a specific set of diagrams's functions and classes, and execute them in proper order.

But you don't have to worry about diagrams because diagrams-as-code is self-contained and encapsulates it well.