Alert Manager not found in both sources

[dpewsey]

Hello,

trying to set up an alert for mimir

cat redis_up.yaml
namespace: SRE
groups:
    - name: Redis
      rules:
        - alert: redis status
          expr: redis_up{job="redis_exporter_targets"} == 1
          for: 1m
          labels:
            severity: warning
            type: redis
          annotations:
            description: A Redis instance is down\n VALUE= {{ $value }}
            summary: A redis instance down (instance {[ $labels.instance }})

mimirtool rules load redis_up.yaml uploads the job
however the alert doesn't show up via the alertmanager ui

I also get this when trying to verify

mimirtool alerts verify
INFO[0000] alert found that was not in both sources      alertname="redis status" state="map[__name__:ALERTS alertname:redis status alertstate:firing instance:redis://api-redis.service.consul:6382 job:redis_exporter_targets severity:warning type:redis]"

How does one create an alert to match? Is there another place the rules need to be created?

[pracucci]

👋 Hi! When you configure an alert, the alert is visible in the Alertmanager UI only when it's firing. Given you've defined your alert expression as redis_up{job="redis_exporter_targets"} == 1 I guess it may be firing (1 means up, I guess because of testing the alert).

That being said, have you configured the Alertmanager URL in the MImir ruler config? Basically the Mimir ruler component evaluates the rules, then send a notification of firing alerts to the Mimir Alertmanager component. The Mimir ruler needs to know the Mimir Alertmanager component address, so you have to configure the right address using:

# Comma-separated list of URL(s) of the Alertmanager(s) to send notifications
# to. Each URL is treated as a separate group. Multiple Alertmanagers in HA per
# group can be supported by using DNS service discovery format. Basic auth is
# supported as part of the URL.
# CLI flag: -ruler.alertmanager-url
[alertmanager_url: <string> | default = ""]

See config reference doc at:
https://grafana.com/docs/mimir/latest/operators-guide/configure/reference-configuration-parameters/