Everbrent/cis
9
0
Fork 0
mirror of https://github.com/m8tin/cis.git synced 2025-12-06 15:58:26 +01:00
Code Issues Packages Projects Releases Wiki Activity

Added basic Monitoring

Browse Source
This commit is contained in:
Martin Berghaus
2025-03-07 18:28:57 +01:00
parent e782848efd
commit bf19c05148
9 changed files with 486 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

126
script/monitor/README.md Normal file
View File

@@ -0,0 +1,126 @@
How to setup a monitoring dashboard
===================================
Inspired by: https://pimylifeup.com/ubuntu-chromium-kiosk/
Steps
-----
### 1.) Install Ubuntu Server (no desktop) on your computer than set hostname and timezone.
```sh
hostnamectl set-hostname check.local
timedatectl set-timezone Europe/Berlin
```
### 2.) Install minimal GUI and Tools.
```sh
apt install ubuntu-desktop-minimal
apt install language-pack-gnome-de
apt install xdotool
apt install dbus-x11
```
### 3.) Create a kiosk user with home-directory.
```sh
useradd -m kiosk
```
and disable Welocme-Screen
```sh
echo "yes" > /home/kiosk/.config/gnome-initial-setup-done
```
### 4.) Edit following file `nano /etc/gdm3/custom.conf` to turn of wayland and turn on autologin for user 'kiosk'.
```
[daemon]
# Uncomment the line below to force the login screen to use Xorg
#WaylandEnable=false
WaylandEnable=false
# Enabling automatic login
# AutomaticLoginEnable = true
# AutomaticLogin = user1
AutomaticLoginEnable = true
AutomaticLogin = kiosk
```
### 5.) Configure GUI of user kiosk to prevent monitor from sleeping
```sh
#gsettings list-recursively
# Does not work
#sudo -u kiosk gsettings set org.gnome.desktop.session idle-delay 0
# Set idle-delay from "uint32 300" to "uint32 0", needs 'apt install dbus-x11'
# You can check the value in "GUI-Session of kiosk -> Settings -> Power"
sudo -u kiosk dbus-launch dconf write /org/gnome/desktop/session/idle-delay "uint32 0"
```
### 6.) Create custom service to start firefox loading the page.
Therefore create a file `/etc/systemd/system/kiosk.service` with this content:
```
[Unit]
Description=Firefox Kiosk
Wants=graphical.target
After=graphical.target
[Service]
Environment=DISPLAY=:0
# Set firefox language, needs 'apt install language-pack-gnome-de'
Environment=LANG=de_DE.UTF-8
Type=simple
# Always a fresh firefox ('-' allow error if common does not exist)
ExecStartPre=-/usr/bin/rm -r /home/kiosk/snap/firefox/common
# Move Mouse (should also work on small screens), needs 'apt install dbus-x11'
ExecStartPre=/usr/bin/xdotool mousemove 4096 2160
# See: https://wiki.mozilla.org/Firefox/CommandLineOptions (just -kiosk URL => Start-Assistant, so use -url too)
ExecStart=/usr/bin/firefox -fullscreen -kiosk -url http://monitor.example.net/check.html
Restart=always
RestartSec=30
User=kiosk
Group=kiosk
[Install]
WantedBy=graphical.target
```
### 7.) Enable the service and reboot
```sh
systemctl enable kiosk
reboot
```
Troubleshouting
---------------
```
systemctl disable pd-mapper.service
apt purge cloud-init -y && apt autoremove --purge -y
```

204
script/monitor/check.html Normal file
View File

@@ -0,0 +1,204 @@
<html>
<head>
<meta charset="utf-8">
<title>Monitoring Dashboard</title>
<style>
html, body {
--background-theme-color: #001EA0;
--cell-space: 20px;
--logo-height: 50px;
background-color: #cccccc;
font-family: Verdana;
font-size: 14pt;
color: #ffffff;
height: 100%;
margin: 0;
}
@media screen and (orientation: portrait) {
body {
zoom: 200%
}
}
#header {
background-color: var(--background-theme-color);
position: sticky;
top: 0;
height: calc(var(--logo-height) + (2 * var(--cell-space)));
width: 100%;
}
#header img {
height: var(--logo-height);
margin: var(--cell-space);
vertical-align: middle;
}
#header h1 {
display: inline;
font-weight: normal;
vertical-align: middle;
}
#content {
min-height: 100%;
}
#footer {
background-color: var(--background-theme-color);
position: sticky;
bottom: 0px;
padding: var(--cell-space);
text-align: center;
vertical-align: middle;
font-size: 22pt;
}
#checks {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(350px, 1fr));
padding: var(--cell-space);
grid-gap: var(--cell-space);
}
#checks > div {
border: 1px solid black;
border-radius: 10px;
padding: 10px;
text-align: center;
box-shadow: 0 2px 5px 0 rgba(0, 0, 0, 0.3), 0 2px 10px 0 rgba(0, 0, 0, 0.2);
}
#checks > div.ok {
background-color: #66aa22;
color: #222222;
}
#checks > div.info {
background-color: #88cc44;
color: #222222;
}
#checks > div.warn {
background-color: #ffdd00;
color: #222222;
}
#checks > div.fail {
background-color: #ff0000;
}
#checks > div.timeout {
background-color: var(--background-theme-color);
}
</style>
</head>
<body>
<div id="header">
<img src="logo.png"></img>
<h1>Monitoring</h1>
</div>
<div id="content">
<div id="checks">
<div class="check warn">Loading...</div>
</div>
</div>
<div id="footer">
Köln, <span id="datetime"></span>
</div>
<script>
var connectionAlive = true;
function downloadCheckFile(callback) {
var xmlHttp = new XMLHttpRequest();
xmlHttp.open('GET', "check.txt", false);
xmlHttp.onreadystatechange=function() {
if(xmlHttp.readyState==4) {
callback(xmlHttp.responseText);
}
}
try {
xmlHttp.send(null);
if (xmlHttp.status >= 200 && xmlHttp.status < 304) {
return true;
} else {
return false;
}
} catch (e) {
return false;
}
}
function convertToHtml(checkText) {
if (!connectionAlive) {
return '<div class="fail">CONNECTION FAILED</div>';
}
var html = "";
var lines = checkText.split(/\n/);
for(var lineNo = 2; lineNo < lines.length; lineNo++) {
var line = lines[lineNo];
var parts = line.split('?');
if (parts.length > 1) {
var name = parts[0].trim().split("_").join(" ");
var resultParts = parts[1].trim().split('#');
var result = resultParts[0];
var message = resultParts[1];
if(name == 'MISSED') {
var fileTimeParts = message.split('-');
var fileTime = new Date();
fileTime.setHours(parseInt(fileTimeParts[0]));
fileTime.setMinutes(parseInt(fileTimeParts[1]));
fileTime.setSeconds(parseInt(fileTimeParts[2]));
var scriptTime = new Date();
scriptTime.setMinutes(scriptTime.getMinutes() - 2);
if (scriptTime.getTime() < fileTime.getTime()){
if (result == "0") {
html += '<div class="ok">EVERYTHING OK<br/>' + fileTime.toLocaleTimeString() + '</div>';
} else {
html += '<div class="fail">FAILED: ' + result + '<br/>' + fileTime.toLocaleTimeString() + '</div>';
}
} else {
html += '<div class="check fail">CHECKS TOO OLD<br/>' + fileTime.toLocaleTimeString() + '</div>';
}
} else {
if(result.indexOf('OK') >= 0) {
html += '<div class="ok">'+ name;
} else if(result.indexOf('INFO') >= 0) {
html += '<div class="info">'+ name;
} else if(result.indexOf('TIMEOUT') >= 0) {
html += '<div class="timeout">'+ name;
} else if(result.indexOf('WARN') >= 0) {
html += '<div class="warn">'+ name;
} else {
html += '<div class="fail">' + name;
}
if(message) {
html += '<br/>' + message.trim();
}
html += '</div>';
}
}
}
return html;
}
function exchangeChecks(text) {
document.getElementById("checks").innerHTML = convertToHtml(text);
}
function refreshTime() {
document.getElementById("datetime").innerHTML = new Date().toLocaleString("de-DE", {timeZone: "Europe/Berlin"});
}
function refreshChecks() {
connectionAlive = downloadCheckFile(exchangeChecks);
}
function reloadIfAlive() {
if (connectionAlive) {
location.reload();
}
}
setInterval(refreshTime, 1000);
setInterval(refreshChecks, 5000);
setInterval(reloadIfAlive, 300000);
refreshTime();
refreshChecks();
</script>
</body>
</html>

86
script/monitor/check.sh Executable file
View File

@@ -0,0 +1,86 @@
#!/bin/bash
function doChecks(){
local readonly _TMPDIR="${1:?"doChecks(): Missing parameter TMPDIR:"}"
local readonly _COLOR="${2:-"monocrom"}"
local _DATETIME=$(date +%H-%M-%S)
mkdir -p ${_TMPDIR}
rm ${_TMPDIR}/* > /dev/null 2>&1
for check in /monitoring/checks/*.on
do
local _CHECK_FILENAME="${check##*/}"
echo -n "${_CHECK_FILENAME%%.on}?" > "${_TMPDIR}/${_CHECK_FILENAME}"
timeout -k 10s 20s bash ${check} >> "${_TMPDIR}/${_CHECK_FILENAME}" 2> /dev/null || echo "TIMEOUT#Timeout" >> "${_TMPDIR}/${_CHECK_FILENAME}" &
done
wait
local _FAILED=0
echo "CHECK?RESULT[#MESSAGE]:"
echo "-----------------------"
for resultFile in ${_TMPDIR}/*
do
cat "${resultFile}"
grep -q "FAIL" ${resultFile} && _FAILED=$(expr ${_FAILED} + 1)
done
if [ "${_COLOR}" == "color" ]; then
#color is for console-output
echo "-----------------------"
if [ ${_FAILED} -ne 0 ]; then
echo "MISSED?${_FAILED}#${_DATETIME}"
else
echo "MISSED?${_FAILED}#${_DATETIME}"
fi
else
echo "MISSED?${_FAILED}#${_DATETIME}"
fi
rm -r ${_TMPDIR} > /dev/null 2>&1
return 0
}
function usage(){
printf "\nUsage: /monitoring/check.sh <command> <options>"
echo
echo "possible commands:"
echo
echo "- all"
echo " Executes all checks."
echo "- auto <out_file>"
echo " Executes quiet all checks and saves the result in the given out_file."
return 0
}
main(){
case "${1:-""}" in
all)
printf "Checks werden ausgeführt..." \
&& doChecks "/tmp/checks" color \
&& printf "Success" \
&& return 0
;;
auto)
# If just a filename is given it is created in /tmp, because of 'cd /tmp'
cd /tmp \
&& doChecks "/tmp/checks$(date +%N)" > "$2.new" \
&& mv -f "$2.new" "$2" \
&& return 0
return 1
;;
*)
[ "${1:+isset}" == "isset" ] \
&& printf "Parameter '${1}' ist kein gültiger Befehl.\n"
usage
return 0
;;
esac
return 1
}
main "$@" || exit 1

14
script/monitor/checks/GENERIC_NGINX_CHECK.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
_SERVER="${1:?"FQDN of server missing"}"
_PORT="${2:-"22"}"
_USER="monitoring"
#grep:
# -F Use fixed text, no regexp which has to be interpreted
#cut:
# -d Delimiter, marker where to cut (here ;)
# -f Index of column to show (One based, so there is no -f0)
_RESULT="$(ssh -p "${_PORT}" "${_USER}"@"${_SERVER}" 'systemctl status nginx.service' | grep -F Active: | grep -F running | cut -d';' -f2)"
! [ -z "${_RESULT}" ] && echo "OK#UPTIME:${_RESULT}" || echo "FAIL"

9
script/monitor/checks/GENERIC_PING_CHECK.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/bin/bash
_SERVER="${1:?"FQDN of server missing"}"
# -4 Use IPv4
# -W SECONDS Wait seconds for an answer
# -c COUNT_VALUE Count of pings being executed
_RESULT="$(ping -4 -W 1 -c 1 "${_SERVER}" | grep "time=" | cut -d'=' -f4)"
! [ -z "${_RESULT}" ] && echo "OK#RTT: ${_RESULT}" || echo "FAIL#PLEASE USE FALLBACK!"

13
script/monitor/checks/GENERIC_POOL_SIZE_CHECK.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
SERVER="${1:?"FQDN of server missing"}"
FILE="pool-size.txt"
# --connect-timeout SECONDS Maximum time allowed for connection
# -k Allow connections to SSL sites without certs (H)
# -L Follow redirects (H)
# --max-time SECONDS Maximum time allowed for the transfer
# -s Silent mode. Don't output anything
# -f Fail fast with no output on HTTP errors (otherwise no exit-code > 0 on 404)
RESULT="$(curl --connect-timeout 10 --max-time 10 -k -s -f https://$SERVER/monitoring/$FILE || echo WARN#404 on $FILE check HTTPS)"
echo $RESULT

17
script/monitor/checks/GENERIC_REMOTECHECK Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/bash
# --connect-timeout SECONDS Maximum time allowed for connection
# -k Allow connections to SSL sites without certs (H)
# -L Follow redirects (H)
# --max-time SECONDS Maximum time allowed for the transfer
# -s Silent mode. Don't output anything
URL="${1:?"URL missing"}"
RESULTS="$(curl --connect-timeout 10 --max-time 10 -k -s "$URL" 2>/dev/null)"
CURTIME="$[ $(date +%s) - 10 * 60 ]"
TIME="$(echo "$RESULTS" | tail -n 1)"
if (echo $TIME | grep -E "[^0-9"] > /dev/null); then echo "FAIL"; exit; fi
RES="$(([ "$CURTIME" -gt "$TIME" ] && echo "TIMEOUT") || (echo "$RESULTS" | head -n 1))"
echo $RES
echo "$RESULTS" | tail -n +2 | head -n -1

17
script/monitor/checks/GENERIC_URL_CHECK.sh Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/bash
_URL="${1:?"URL of site missing"}"
#curl:
# --connect-timeout SECONDS Maximum time allowed for connection
# -k Allow connections to SSL sites without certs (H)
# -L Follow redirects (H)
# --max-time SECONDS Maximum time allowed for the transfer
# -s Silent mode. Don't output anything
# --head Show head information only
# --no-progress-meter Clean output for grep
#grep:
# -q Quite, no output just status codes
# -F Interpret search term as plain text
((curl --connect-timeout 10 --max-time 10 -k -s --head --no-progress-meter "${_URL}" | grep -qF '200 OK') && echo OK) || echo FAIL

BIN
script/monitor/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.2 KiB