entanglement.garden issues
https://git.callpipe.com/groups/entanglement.garden/-/issues
2024-03-27T13:43:00Z
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1067
Firing alert `AptUpdatesPending`
2024-03-27T13:43:00Z
Entanglement Monitoring Bot
Firing alert `AptUpdatesPending`
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
...
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2024-03-17 13:12:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:19:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:20:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:44:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:35:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 09:00:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:24:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:15:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1066
Firing alert `NotUpShort`
2024-03-26T11:12:51Z
Entanglement Monitoring Bot
Firing alert `NotUpShort`
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
...
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.103", job="ipmi", module="MXQ2150CK9", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.106", job="ipmi", module="MXQ2510GK0", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.165", job="ipmi", module="MXQ2510GKK", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.212", job="ipmi", module="MXQ2150CJJ", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9225", job="minecraft", server_name="finnanic", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9100", job="node-exporter", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.102.123:9100", job="node-exporter", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1065
Firing alert `AnsiblePullFailedShort`
2024-03-26T07:27:40Z
Entanglement Monitoring Bot
Firing alert `AnsiblePullFailedShort`
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotatio...
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="gateway", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-gateway.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.102.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="main", playbook_url="https://codeberg.org/EntanglementInfrastructure/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.103.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2024-02-12 16:35:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1064
Firing alert `FailedSystmedService`
2024-03-25T21:57:02Z
Entanglement Monitoring Bot
Firing alert `FailedSystmedService`
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `...
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `warn`
- **`state`**: `failed`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:46:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.101.1:9100", job="node-exporter", name="ansible-on-boot.service", severity="warn", state="failed", }`
- **``**
- **Starts at**: 2024-02-10 22:27:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.103.1:9100", job="node-exporter", name="networking.service", severity="warn", state="failed", type="oneshot", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1063
Firing alert `AptUpdatesPending`
2024-03-22T13:28:00Z
Entanglement Monitoring Bot
Firing alert `AptUpdatesPending`
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
...
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2024-03-17 13:12:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:19:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:20:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:44:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:35:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 09:00:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:24:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:15:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1062
Firing alert `NotUpShort`
2024-03-21T10:57:51Z
Entanglement Monitoring Bot
Firing alert `NotUpShort`
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
...
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.103", job="ipmi", module="MXQ2150CK9", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.106", job="ipmi", module="MXQ2510GK0", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.165", job="ipmi", module="MXQ2510GKK", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.212", job="ipmi", module="MXQ2150CJJ", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9225", job="minecraft", server_name="finnanic", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9100", job="node-exporter", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.102.123:9100", job="node-exporter", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1061
Firing alert `AnsiblePullFailedShort`
2024-03-21T07:12:40Z
Entanglement Monitoring Bot
Firing alert `AnsiblePullFailedShort`
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotatio...
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="gateway", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-gateway.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.102.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="main", playbook_url="https://codeberg.org/EntanglementInfrastructure/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.103.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2024-02-12 16:35:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1060
Firing alert `FailedSystmedService`
2024-03-20T21:42:01Z
Entanglement Monitoring Bot
Firing alert `FailedSystmedService`
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `...
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `warn`
- **`state`**: `failed`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:46:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.101.1:9100", job="node-exporter", name="ansible-on-boot.service", severity="warn", state="failed", }`
- **``**
- **Starts at**: 2024-02-10 22:27:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.103.1:9100", job="node-exporter", name="networking.service", severity="warn", state="failed", type="oneshot", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1059
Firing alert `AptUpdatesPending`
2024-03-17T13:12:59Z
Entanglement Monitoring Bot
Firing alert `AptUpdatesPending`
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
...
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2024-03-17 13:12:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:19:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:20:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:44:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:35:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 09:00:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:24:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:15:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1058
Firing alert `NotUpShort`
2024-03-16T10:42:50Z
Entanglement Monitoring Bot
Firing alert `NotUpShort`
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
...
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.103", job="ipmi", module="MXQ2150CK9", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.106", job="ipmi", module="MXQ2510GK0", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.165", job="ipmi", module="MXQ2510GKK", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.212", job="ipmi", module="MXQ2150CJJ", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9225", job="minecraft", server_name="finnanic", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9100", job="node-exporter", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.102.123:9100", job="node-exporter", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1057
Firing alert `AnsiblePullFailedShort`
2024-03-16T06:57:38Z
Entanglement Monitoring Bot
Firing alert `AnsiblePullFailedShort`
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotatio...
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="gateway", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-gateway.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.102.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="main", playbook_url="https://codeberg.org/EntanglementInfrastructure/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.103.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2024-02-12 16:35:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1056
Firing alert `FailedSystmedService`
2024-03-15T21:27:00Z
Entanglement Monitoring Bot
Firing alert `FailedSystmedService`
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `...
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `warn`
- **`state`**: `failed`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:46:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.101.1:9100", job="node-exporter", name="ansible-on-boot.service", severity="warn", state="failed", }`
- **``**
- **Starts at**: 2024-02-10 22:27:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.103.1:9100", job="node-exporter", name="networking.service", severity="warn", state="failed", type="oneshot", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1055
Firing alert `AptUpdatesPending`
2024-03-12T21:42:59Z
Entanglement Monitoring Bot
Firing alert `AptUpdatesPending`
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
...
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2024-02-21 08:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:19:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:20:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:44:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:35:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 09:00:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:24:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:15:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1054
Firing alert `NotUpShort`
2024-03-11T10:27:50Z
Entanglement Monitoring Bot
Firing alert `NotUpShort`
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
...
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.103", job="ipmi", module="MXQ2150CK9", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.106", job="ipmi", module="MXQ2510GK0", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.165", job="ipmi", module="MXQ2510GKK", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.212", job="ipmi", module="MXQ2150CJJ", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9225", job="minecraft", server_name="finnanic", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9100", job="node-exporter", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.102.123:9100", job="node-exporter", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1053
Firing alert `AnsiblePullFailedShort`
2024-03-11T06:42:37Z
Entanglement Monitoring Bot
Firing alert `AnsiblePullFailedShort`
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotatio...
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="gateway", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-gateway.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.102.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="main", playbook_url="https://codeberg.org/EntanglementInfrastructure/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.103.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2024-02-12 16:35:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1052
Firing alert `FailedSystmedService`
2024-03-10T21:11:59Z
Entanglement Monitoring Bot
Firing alert `FailedSystmedService`
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `...
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `warn`
- **`state`**: `failed`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:46:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.101.1:9100", job="node-exporter", name="ansible-on-boot.service", severity="warn", state="failed", }`
- **``**
- **Starts at**: 2024-02-10 22:27:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.103.1:9100", job="node-exporter", name="networking.service", severity="warn", state="failed", type="oneshot", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1051
Firing alert `AptUpdatesPending`
2024-03-07T21:27:58Z
Entanglement Monitoring Bot
Firing alert `AptUpdatesPending`
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
...
# `warn` alert `AptUpdatesPending` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AptUpdatesPending`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2024-02-21 08:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:18:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.100.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.1:9100", job="node-exporter", origin="Debian:10.13/oldstable,Debian-Security:10/oldstable", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.101.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:26:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.1:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:19:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:20:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.102.2:9100", job="node-exporter", origin="gitlab-runner:1/bullseye", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 20:44:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 07:35:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", instance="172.18.103.2:9100", job="node-exporter", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 09:00:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="all", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", origin="Debian:11-updates/oldstable-updates", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:24:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
- **``**
- **Starts at**: 2024-02-21 08:15:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=apt_upgrades_pending+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AptUpdatesPending", arch="amd64", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", origin="Debian-Security:11/oldstable-security", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1050
Firing alert `NotUpShort`
2024-03-06T10:12:49Z
Entanglement Monitoring Bot
Firing alert `NotUpShort`
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
...
# `warn` alert `NotUpShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `NotUpShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.103", job="ipmi", module="MXQ2150CK9", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.106", job="ipmi", module="MXQ2510GK0", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.165", job="ipmi", module="MXQ2510GKK", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.16.33.212", job="ipmi", module="MXQ2150CJJ", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9225", job="minecraft", server_name="finnanic", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.101.162:9100", job="node-exporter", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:42:10.517335409 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=up+%21%3D+1&g0.tab=1)
- **Labels**: `{alertname="NotUpShort", instance="172.18.102.123:9100", job="node-exporter", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1049
Firing alert `AnsiblePullFailedShort`
2024-03-06T06:27:37Z
Entanglement Monitoring Bot
Firing alert `AnsiblePullFailedShort`
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotatio...
# `warn` alert `AnsiblePullFailedShort` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `AnsiblePullFailedShort`
- **`severity`**: `warn`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.1:9100", job="node-exporter", playbook="gateway", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-gateway.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.101.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.102.2:9100", job="node-exporter", playbook="hypervisor", playbook_branch="main", playbook_url="https://codeberg.org/EntanglementInfrastructure/ansible-hypervisor.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", instance="172.18.103.2:9100", job="node-exporter", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.100.76:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="basic", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/ansible-basic.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.101.27:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2024-02-12 16:35:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.102.84:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
- **``**
- **Starts at**: 2023-12-23 14:41:57.643056553 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=ansible_playbook_exit_code+%3E+0&g0.tab=1)
- **Labels**: `{alertname="AnsiblePullFailedShort", clusterid="keycloak", instance="172.18.103.63:9100", job="postgresql-node", playbook="postgresql", playbook_branch="master", playbook_url="https://git.callpipe.com/entanglement.garden/vm-images/postgres.git", severity="warn", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*
https://git.callpipe.com/entanglement.garden/infrastructure/-/issues/1048
Firing alert `FailedSystmedService`
2024-03-05T20:56:59Z
Entanglement Monitoring Bot
Firing alert `FailedSystmedService`
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `...
# `warn` alert `FailedSystmedService` occurred
**Alertmanager link:** [http://monitoring:9093](http://monitoring:9093)
### Common labels:
- **`alertname`**: `FailedSystmedService`
- **`job`**: `node-exporter`
- **`severity`**: `warn`
- **`state`**: `failed`
### Common annotations:
---
## Alerts
- **``**
- **Starts at**: 2023-12-23 14:46:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.101.1:9100", job="node-exporter", name="ansible-on-boot.service", severity="warn", state="failed", }`
- **``**
- **Starts at**: 2024-02-10 22:27:18.518582447 +0000 UTC
- **Ends at**: 0001-01-01 00:00:00 +0000 UTC
- **Generator URL**: [https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1](https://prometheus.entanglement.garden/graph?g0.expr=node_systemd_unit_state%7Bstate%3D%22failed%22%7D+%3D%3D+1&g0.tab=1)
- **Labels**: `{alertname="FailedSystmedService", instance="172.18.103.1:9100", job="node-exporter", name="networking.service", severity="warn", state="failed", type="oneshot", }`
*this issue was automatically generated using the default template that came with [prometheus-gitlab-notifier](https://github.com/FUSAKLA/prometheus-gitlab-notifier), if it sucks please edit `/etc/prometheus/gitlab-issue.tmpl` on monitoring.fruit-0*