# SHUTDOWN DUE TO COVID-19
# baxx.dev
check it out `ssh [email protected]`
[ work in progress ]
* https://baxx.dev/help
* TODO.txt
* infra and pricing.txt
* stat (disk usage, mem, mdadm) https://baxx.dev/stat
# backup service
(also i am learning how to build a product without a website haha)
# screenshots
βββββββββββββββββββββββββββββββββββββββββββββββββ
β β
β βββββββ ββββββ βββ ββββββ βββ β
β ββββββββββββββββββββββββββββββββ β
β ββββββββββββββββ ββββββ ββββββ β
β ββββββββββββββββ ββββββ ββββββ β
β βββββββββββ βββββββ βββββββ βββ β
β βββββββ βββ ββββββ ββββββ βββ β
β β
β Storage 10G β
β Trial 1 Month 0.1 EUR β
β Subscription: 5 EUR per Month β
β Availability: ALPHA β
β β
β Contact Us: β
β * Slack https://baxx.dev/join/slack β
β * Google Groups https://baxx.dev/join/groups β
β β
β E-mail β
β β
β Password β
β β
β Confirm Password β
β β
β β
β Registering means you agree with β
β the terms of service! β
β β
β [Register] [Login] β
β β
β [Help] [What/Why/How] [Terms Of Service] β
β β
β [Quit] β
βββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β β
β βββββββ ββββββ βββ ββββββ βββ β
β ββββββββββββββββββββββββββββββββ β
β ββββββββββββββββ ββββββ ββββββ β
β ββββββββββββββββ ββββββ ββββββ β
β βββββββββββ βββββββ βββββββ βββ β
β βββββββ βββ ββββββ ββββββ βββ β
β β
β β
β Email: [email protected] β
β Verification pending. β
β Please check your spam folder. β
β β
β Subscription: β
β Activate at https://baxx.dev/sub/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX β
β β
β Refreshing.. - β
β β
β [βHelp] [Resend Verification Email] [Quit] β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
# who watches the watchers
the current baxx infra progress is
2 machines, each running only docker and ssh
[ b.baxx.dev ]
* ssh
* docker
+ postgres-master
+ nginx + letsencrypt
+ who watches the watchers [πΉjob]
+ run notification rules [πΉjob]
+ process email queue [πΉjob]
+ collect memory/disk/mdadam stats [privileged] [πΉjob] (priv because mdadm)
+ baxx-api
+ judoc [localhost]
+ scylla [privileged] (priv because of io tunning)
[ a.baxx.dev ]
* ssh
* docker
+ postgres-slave
+ nginx + letsencrypt
+ who watches the watchers [πΉjob]
+ process email queue [πΉjob]
+ collect memory/disk/mdadam stats [privileged] [πΉjob] (priv because mdadm)
+ baxx-api
+ judoc [localhost]
+ scylla [privileged] (priv because of io tunning)
as you can see both machines are in the scylla cluster, and both of
them are sending the notification emails (using select for update locks)
and only one of them is running the notification rules.
I have built quite simple yet effective monitoring system for baxx.
Each process with [πΉjob] tag is something like:
(using πΉ because of daemon)
for {
work
sleep X
}
What I did is:
setup("monitoring key", X+5)
for {
work
tick("monitoring key")
sleep X
}
Then the 'who watches the watchers' programs check if "monitoring key"
is executed at within X+5 seconds per node(), and if not they send
slack message
The 'who watches the watchers' then sends notifications (both watchers
send notifications on their own, so i receive the notification twice
but that is ok)
The watchers themselves also use the system, so if one of them dies,
the other one will send notification.
# testing
all the β checks are tested (manually) and the alerts are performing
really good
## shut down postgres
* β shutdown postgres and see if notifications are sent
## shut down one machine
* β aa.baxx.dev
* β bb.baxx.dev
## mdadm
* β make it fail
mdadm -f /dev/md2 /dev/nvme1n1p3
* β wait for panic message
* β remove the disk
mdadm --remove /dev/md2 /dev/nvme1n1p3
* β add the disk back
mdadm --add /dev/md2 /dev/nvme1n1p3
* β wait to see it is acknowledged
works really nice
## test disk thresh
* β start the status tool with with 1% disk threshold
and wait for alert
## test memory thresh
* start the status tool with with 1% memory threshold
and wait for alert
## test health of baxx api
* query /status which should
+ query postgres
+ query judoc
