Parser for aeronautical information available online.
This gem incluces executables to download and parse aeronautical information (HTML, PDF, XSLX, ODS and CSV), then build and export is as AIXM or OFMX.
- Homepage
- Rubydoc
- Author: Sven Schwyn - Bitcetera
Thank you for supporting free and open-source software by sponsoring on GitHub or on Donorbox. Any gesture is appreciated, from a single Euro for a ☕️ cup of coffee to 🍹 early retirement.
Install
Usage
Regions
Storage
References
Development
This gem is cryptographically signed in order to assure it hasn't been tampered with. Unless already done, please add the author's public key as a trusted certificate now:
gem cert --add <(curl -Ls https://raw.github.com/svoop/aipp/main/certs/svoop.pem)
Make sure to have the latest version of Ruby and then install this gem:
gem install aipp --trust-policy MediumSecurity
If you're familiar with Bundler powered Ruby projects, you might prefer to add the following to your Gemfile or gems.rb:
gem 'aipp'
And then install the bundle:
bundle install --trust-policy MediumSecurity
AIPP parses different kind of information sources and converts them to different output formats depending on which executable you use:
Executable | Output Format |
---|---|
aip2aixm |
AIXM |
aip2ofmx |
OFMX |
The parsers are organized in three levels:
region ⬅︎ aeronautical region such as "LF" (France)
└── scope ⬅︎ scope such as "AIP" or "NOTAM"
└── section ⬅︎ section of the scope such as "ENR-2.1" or "aerodromes"
The following scopes are currently available:
Scope | Content | Cache |
---|---|---|
AIP (default) | aeronautical information publication | by AIRAC cycle |
NOTAM | notice to airmen | by effective date and hour |
To list all available scopes, regions and sections:
aip2aixm --list
See the built-in help for all options:
notam2aixm --help
Example: You wish to build the complete OFMX file for the current AIRAC cycle AIP of the region LF:
aip2ofmx -r LF
You'll find the OFMX file in the current directory if the binary exits successfully.
To implement a region, you have to create a directory lib/aipp/regions/{REGION}/ off the gem root and then subdirectories for each scope as well as for support files. Here's a simplified overview for the region "LF" (France):
LF/ ⬅︎ region "LF"
├── README.md
├── aip ⬅︎ scope "AIP"
│ ├── AD-2.rb ⬅︎ section "AD-2"
│ └── ENR-4.3.rb ⬅︎ section "ENR-4.3"
├── notam ⬅︎ scope "NOTAM"
│ ├── AD.rb ⬅︎ section "AD"
│ └── ENR.rb ⬅︎ section "ENR"
├── borders
│ ├── france_atlantic_coast.geojson
│ └── france_atlantic_territorial_sea.geojson
├── fixtures
│ └── aerodromes.yml
└── helpers
├── base.rb
└── surface.rb
All paths from here on forward are relative to the region directory. |
Say, you want to parse AIP ENR-4.1. You have to create the file aip/ENR-4.1.rb which defines the class ENR41
as follows:
module AIPP::LF::AIP
class ENR41 < AIPP::AIP::Parser
depends_on :ENR21, :ENR22 # declare dependencies to other parsers
(...)
end
end
Another parser might target en-route NOTAM and therefore has to go to notam/ENR.rb like so:
module AIPP::LF::NOTAM
class ENR < AIPP::NOTAM::Parser
(...)
end
end
Parser files and classes may follow AIP naming conventions such as ENR-4.1.rb. However, you're free to use arbitrary naming for parser files like navaids.rb (e.g. if you're working with one big data source which contains the full AIP dataset you'd like to split into smaller parts). |
The class has to implement some methods either in the class itself or in a helper included by the class.
The class must implement the parse
method which contains the code to read, parse and write the data:
module AIPP::LF::AIP
class ENR41 < AIPP::AIP::Parser
def parse
html = read # read the Nokogiri::HTML5 document
feature = (...) # build the feature
add(feature: feature) # add the feature to AIXM::Document
end
end
end
Some AIP may be split over several files which require a little more code to load the individual HTML source files:
module AIPP::LF::AIP
class AD2 < AIPP::AIP::Parser
def parse
%i(one two three).each do |part|
html = read("#{aip}.#{part}") # read with a non-standard name
support_html = read('AD-0.6') # maybe read necessary support documents
(...)
end
end
end
end
The parser has access to the following methods:
Method | Description |
---|---|
section |
current section (e.g. ENR-2.1 or aerodromes ) |
read |
download, cache and read a document from source |
add |
add a AIXM::Feature |
find |
find previously written AIXM::Feature s by object |
find_by |
find previously written AIXM::Feature s by class and attribute values |
unique |
prevent duplicate AIXM::Feature s |
given |
inline condition for assignments |
link_to |
optionally checked Markdown link |
Equally available is the current runtime environment. All of the following objects behave like OpenStruct
:
Method | Description |
---|---|
AIPP.cache |
cache to make transient objects available across AIPs |
AIPP.borders |
borders of the current region |
AIPP.fixtures |
fixtures of the current region |
AIPP.options |
arguments read from aip2aixm or aip2ofmx respectively |
AIPP.config |
configuration read from config.yml |
To make the parser code more readable, a few core extensions are provided:
Object#blank
(ActiveSupport)NilClass
Integer
String
(ActiveSupport)String
Array
Hash
Enumerable
DateTime
(ActiveSupport)Nokogiri
The class must implement the origin_for
method which returns an origin object describing how to download the source data (e.g. an AIP file or NOTAM message):
module AIPP::LF::AIP
class AD2 < AIPP::AIP::Parser
def origin_for(document)
# build and return the origin object
end
end
end
Return any of the following origin objects best explained by example:
AIPP::Downloader::File.new(
file: "file.dat", # relative path to file
type: :pdf # optional: file type if different from extension
)
AIPP::Downloader::File.new(
archive: "foobar.zip", # relative path to archive
file: "subdir/file.dat", # file to extract from archive
type: :pdf # optional: file type if different from extension
)
See Downloader for more on recognised file and archive types.
AIPP::Downloader::HTTP.new(
file: "https://example.com/foobar.zip", # URL where the file is located
type: :pdf, # optional: file type if different from extension
headers: "Cookie: name=value", # optional: additional headers e.g. for session
)
AIPP::Downloader::HTTP.new(
archive: "https://example.com/foobar.zip", # URL where the archive is located
file: "subdir/file.dat", # file to extract from archive
type: :pdf, # optional: file type if different from extension
headers: "Cookie: name=value", # optional: additional headers e.g. for session
)
The excon gem is used to perform HTTP requests.
AIPP::Downloader::GraphQL.new(
client: MyAPI::Client, # GraphQL client class
query: MyAPI::Name::Query, # GraphQL query class
variables: { # dynamic query parameters
first_name: 'Geronimo',
age: 50
}
)
For this GraphQL downloader to work, you have to declare a GraphQL client class beforehand. See the graphql-client gem documentation for details, the following example fits the downloader above:
module MyAPI
HttpAdapter = GraphQL::Client::HTTP.new(ENV['MY_API_URL']) do
def headers(context)
{ "Authorization": "Bearer #{ENV['MY_API_AUTHORIZATION']}" }
end
end
Schema = GraphQL::Client.load_schema(HttpAdapter)
Client = GraphQL::Client.new(schema: Schema, execute: HttpAdapter)
class Name
Query = Client.parse <<~END
query ($first_name: String!, $age: Int!) {
queryNOTAMs(
filter: {first_name: $first_name, age: $age}
) {
name
}
}
END
end
end
For performance, all downloads are cached and subsequent runs will use the cached data rather than fetching the sources anew. Each scope defines a cache time window, see the table of scopes above. You can discard existing and rebuild caches by use of the --clean
command line argument.
The class may implement the setup
method. If present, it will be called when this parser is instantiated:
module AIPP::LF::AIP
class AD2 < AIPP::AIP::Parser
def setup
AIXM.config.voice_channel_separation = :any
AIPP.cache.setup_at ||= Time.now
end
end
end
Helpers are mixins defined in the helpers/ subdirectory. All helpers are required automatically in alphabetic order.
AIXM knows named borders for country boundaries. However, you might need additional borders which don't exist as named borders.
You can define additional borders as AIPP::Border
objects in two ways.
Create simple GeoJSON files in the borders/ subdirectory, for example this my_border_1.geojson
:
{
"type": "GeometryCollection",
"geometries": [
{
"type": "LineString",
"coordinates": [
[6.009531650000042, 45.12013319700009],
[6.015747738000073, 45.12006702600007]
]
},
{
"type": "LineString",
"coordinates": [
[4.896732957592112, 43.95662950764992],
[4.005739165537195, 44.10769266295027]
]
}
]
}
The GeoJSON file must consist of exactly one `GeometryCollection` which may contain any number of `LineString` geometries. Only `LineString` geometries are recognised! To define a closed polygon, the first coordinates of a `LineString` must be identical to the last coordinates. |
It's also possible to create a AIPP::Border
objects on the fly:
my_border_2 = AIPP::Border.from_array(
[
["6.009531650000042 45.12013319700009", "6.015747738000073 45.12006702600007"],
["4.896732957592112 43.95662950764992", "4.005739165537195 44.10769266295027"]
]
)
The coordinate pairs must be separated with whitespaces and/or commas. If you want to use this border everywhere, make sure you add it to the others:
borders["my_border_2"] = my_border_2
In the parser, the borders
method gives you access to all borders read from GeoJSON files:
borders # => { "my_border_1" => #<AIPP::Border>, "my_border_2" => #<AIPP::Border> }
The border object implements simple nearest point and segment calculations to create arrays of AIXM::XY
which can be used with AIXM::Component::Geometry
.
See AIPP::Border
for more on this.
Fixtures are static YAML data files in the fixtures/ subdirectory. All fixtures are read automatically, e.g. the contents of the lib/aipp/regions/{REGION}/fixtures/aerodromes.yml will be available from AIPP.fixtures.aerodromes
.
Read on for how to best use fixtures.
When parsed data is faulty or missing, you may fall back to fixtures instead. This is where patches come in. You can patch any AIXM attribute setter by defining a patch block inside the AIP parser and accessing the static data via parser.fixture
:
module AIPP::LF::AIP
class AD2 < AIP
patch AIXM::Feature::Airport, :z do |object, value|
throw(:abort) unless value.nil?
throw(:abort, 'fixture missing') unless z = AIPP.fixtures.aerodromes.dig(object.id, 'z')
AIXM.z(z, :qnh)
end
end
end
The patch receives the object and the value which is about to be assigned. It should implement something along these lines:
- If the value is okay,
throw(:abort)
to leave the patch block without touching anything. - Otherwise, try to fetch a better value e.g. from the fixtures. If no better value can be found (e.g. outdated fixtures),
throw(:abort, "reason")
to leave the patch block and fail with a useful error message which contains the reason thrown. - At last, build and return the value object which will be assigned instead of the original value.
In order to reference the source of an AIXM/OFMX feature, it's necessary to know the line number where a particular node occurs in the HTML source file. You can ask any HTML element as follows:
tr.line
You should fail
on fatal problems which must be fixed. The --debug-on-error
command line argument will open a debug session when such an error occurs. Issue errors as usual:
fail "my message"
You should warn
on non-fatal problems which should be fixed (default) or might be ignored (severe: false
). The --debug-on-warning ID
command line argument will open a debug session when then warning with the given ID occurs. To issue a warning:
warn("my message")
Less important warnings are only shown if the --verbose
mode is set:
warn("my message", severe: false)
Use info
for essential info messages:
info("my message") # displays "my message" in black
info("my message", color: :green) # displays "my message" in green
Use verbose_info
for in-depth info messages which are only shown if the --verbose
mode is set:
verbose_info("my message") # displays "my message" in blue
The default Ruby debugger is enabled by default, you can add a breakpoint as usual with:
debugger
AIPP uses a storage directory for configuration, caching and in order to keep the results of previous runs. The default location is ~/.aipp
, however, you can pass a different directory with the --storage
argument.
You'll find a directory for each region and scope which contains the following items:
sources/
ZIP archives which cache all source files used to build.builds/
ZIP archives of successful builds containing:- the built AIXM/OFMX file
build.yaml
– context of the build processmanifest.csv
– diffable manifest (see below)
config.yml
This file contains configuration which will be read on subsequent runs, most notably the namespace UUID used to identify the creator of OFMX files.
The manifest is a CSV which lists every feature on a separate line along with its hashes, AIP and comment. You can diff
or git diff
two manifests:
$ git diff -U0 2019-09-12/manifest.csv 2019-10-10/manifest.csv
--- a/2019-09-12/manifest.csv
+++ b/2019-10-10/manifest.csv
@@ -204 +204 @@ AD-1.3,Ahp,9e9f031e,d6f22057,Airport: LFLJ COURCHEVEL
-AD-1.3,Ahp,9f1eed18,37ddbbde,Airport: LFQD ARRAS ROCLINCOURT
+AD-1.3,Ahp,9f1eed18,f0e60105,Airport: LFQD ARRAS ROCLINCOURT
@@ -312 +312 @@ AD-2,Aha,4250c9ee,04d49dc7,Address: RADIO for LFHV
-AD-2,Aha,6b381b32,fb947716,Address: RADIO for LFPO
+AD-2,Aha,6b381b32,b9723b7e,Address: RADIO for LFPO
@@ -664 +663,0 @@ AD-2,Ser,3920a7fd,4545c5eb,Service: AFIS by LFGA TWR
-AD-2,Ser,39215774,1f13f2cf,Service: APP by LFCR APP
@@ -878 +876,0 @@ AD-2,Ser,bb5228d7,7cfb4572,Service: TWR by LFMH TWR
-AD-2,Ser,bc72caf2,0a15b39c,Service: FIS by LFCR FIC
(...)
The advantage of git diff
is its ability to highlight exactly which part of a line has changed. Check out this post to learn how.
- Geo Maps – programmatically generated GeoJSON maps
- open flightmaps – open-source aeronautical maps
- AIXM Rubygem – AIXM/OFMX generator for Ruby
To install the development dependencies and then run the test suite:
bundle install
bundle exec rake # run tests once
bundle exec guard # run tests whenever files are modified
Please submit issues on:
https://github.com/svoop/aipp/issues
To contribute code, fork the project on GitHub, add your code and submit a pull request:
https://help.github.com/articles/fork-a-repo
The gem is available as open source under the terms of the MIT License.