Skip to content

Discovery

Jump to bottom
Yaroslav Pogrebnyak edited this page Jun 22, 2019 · 19 revisions

Service discovery solves the problem of automatic determining locations of service instances. There are two major types of service discovery: client-side discovery and server-side discovery. gobetween has build-in server-side discovery and acts as a router that can have static configuration, or query different kinds of "service registries", depending on how it's configured.

Currently gobetween supports the following discovery types used for building backends pool for the server:

Discovery is defined in [servers.<name>.discovery] section in kind property.

There are some common properties for each discovery type (expect static):

kind = "<kind>"         # (required)
failpolicy = "keeplast" # (optional) "keeplast" | "setempty" - what to do with backends if discovery fails
interval = "0s"         # (required) backends cache invalidation interval; 0 means never.
timeout = "5s"          # (optional) max time to wait for discover until falling to failpolicy

static

It's a simplest way to build load balancer backends. In this case actually no discovery is happened and backends list are static and managed only by health checks. static_list can contain as many backends as needed.

[servers.example]

# ...

  [servers.example.discovery]
  kind = "static"
  static_list = [
    "localhost:8000 weight=5 sni=example.com",  # "host:port weight=N priority=M", weight, priority and sni are optional
    "localhost:8001"            # and = 1 by default
  ]

srv

This discovery method uses DNS lookup to build backends list. gobetween will query DNS server defined in srv_lookup_server property and filter services using pattern defined in srv_lookup_server. You can use any custom DNS server or Consul DNS.

[servers.example]

# ...

  [servers.example.discovery]
  kind = "srv"
  srv_lookup_server = "some.server:53"  # (required)
  srv_lookup_pattern = "some.service."  # (required)
  srv_dns_protocol = "udp"  # (optional, since v0.2.0) - udp | tcp (default: "udp")

Gobetween uses SRV record target as SNI hostname, when configured. For example:

dig @192.168.0.5 foo.service.consul SRV

;; ANSWER SECTION:
foo.service.consul.	0 IN	SRV	1 1 8443 foo.com.
foo.service.consul.	0 IN	SRV	1 1 8443 bar.com.
foo.service.consul.	0 IN	SRV	1 1 8443 baz.com.

so that there will be 3 sni hostnames: foo.com, bar.com and baz.com

docker

Docker discovery can work both with stand-alone Docker server or with Docker Swarm. gobetween will call Docker / Swarm API endpoint defined in docker_endpoint to build backends list. It may be HTTP endpoint (like http://0.0.0.0:2375) or Unix socket (unix:///var/run/docker.sock). To select only needed containers you can set docker_container_label property and containers will be filtered based on provided container label and label value. You should also set docker_container_private_port so gobetween will use corresponding container public port while adding it to backends list.Containers should have label sni with hostname as a value in order to use sni (if enabled)

[servers.example]

# ...

  [servers.example.discovery]
  kind = "docker"
  docker_endpoint = "http://localhost:2375" # (required) Endpoint to docker API
  docker_container_label = "proxied=true"   # (optional) Label to filter containers
  docker_container_private_port = 80        # (required) Private port of container to use
  docker_container_host_env_var = ""        # (optional) (since v0.2.0) Take container host from container env variable

json

JSON discovery is useful for custom setups when you have your own service registry implementation that can provide backends list in JSON format. gobetween will make HTTP query to json_endpoint, expecting valid JSON in response, parse it and build backends list. By defaults JSON should have the following format:

[ 
  {"host": "0.0.0.0", "port": "1231", "weight": 1, "priority": 1, "sni": "foo.com"},
  {"host": "1.1.1.1", "port": "1232", "weight": 2, "priority": 1},
  ...
]

But it may be overridden to fit your custom JSON structure.

[servers.example]

# ...

  [servers.example.discovery]
  kind = "json"
  json_endpoint = "http://localhost:8080"  # (required) JSON discovery Url
  json_host_pattern = "some.level.host"    # (optional) path to host value in JSON object, by default "host"
  json_port_pattern = "some.level.port"    # (optional) path to port value in JSON object, by default "port"
  json_weight_pattern = "some.level.weight" # (optional) path to weight in JSON object, by default "weight"
  json_priority_pattern = "some.level.priority" # (optional) path to priority in JSON object, by default "priority"
  json_sni_pattern = "some.levele.sni" # (optional) path to sni in JSON object, by default "sni"

plaintext

This is even simpler way to integrate custom discovery registries. Line in json discovery, gobetween will query plaintext_endpoint to get newline separated list of nodes in plain text format. Then it will be parsed using regexps line-by-line (one backend in line).A ll necessary values like host, port, etc will captured from named regexp groups. By default regexp plain discovery use

(?P<host>\S+):(?P<port>\d+)(\sweight=(?P<weight>\d+))?(\spriority=(?P<priority>\d+))?(\ssni=(?P<sni>[^\s]+))?

So you can use parse the following responses by default:

0.0.0.0:1234 weight=0 priority=1 
0.0.0.0:4321 weight=1 priority=0 sni=host.com

You can override regexp used to capture values using plaintext_regex_pattern property, keeping in mind groups names:

  • (?P<host>...)
  • (?P<port>...)
  • (?P<weight>...)
  • (?P<priority>...)
  • (?P<sni>...)

All another captured groups will be ignored.

[servers.example]

# ...

  [servers.example.discovery]
  kind = "plaintext"
  plaintext_endpoint = "http://some.url.com"   # (required) Url to plain text discovery
  plaintext_regex_pattern = ""                 # (optional) Regex with named capturing groups

exec

This is most powerful discovery method. In this case backends list will be parser from the stdout of arbitrary script / program. gobetween will execute exec_command (first element is full path to the program, all others are optional arguments. Expected output of the script should be in the following format:

host1:port1 weight=N priority=Q sni=host.com
host2:port2 weight=M

Weight, priority and sni are optional. Lines should be separated by newline (\n).

Here is an example of script /path/to/script.sh:

#!/usr/bin/env bash

echo localhost:8000 weight=1
echo localhost:8001 weight=2

gobetween process should have execute permission to the script.

[servers.example]

# ...

  [servers.example.discovery]
  kind = "exec"
  exec_command = ["/path/to/script.sh", "arg1", "arg2"] # (required) command to exec and variable-length arguments

consul

(since v0.3.0) Consul discovery uses Consul API to retrieve list of backends. If you're relying on Consul healthchecks and using consul_service_passing_only = true if makes sense to turn off gobetween healthchecks.

If sni is enabled, gobetween parses tags associated with each service and processes tag that starts with sni=, extracting hostname until end of tag or space character.

[servers.example]

# ...

  [servers.example.discovery]
  consul_host = "localhost:8500"       # (required) Consul host:port
  consul_service_name = "myservice"    # (required) Service name
  consul_service_tag = ""              # (optional) Service tag
  consul_service_passing_only = true   # (optional) Get only services with passing healthchecks
  consul_service_datacenter = ""       # (optional) Datacenter to use

  consul_auth_username = ""   # (optional) HTTP Basic Auth username
  consul_auth_password = ""   # (optional) HTTP Basic Auth password
  consul_acl_token     = ""   # (optional) ACL token used in requests to Consul (since 0.8.0)

  consul_tls_enabled = false                    # (optional) enable client tls auth
  consul_tls_cert_path = "/path/to/cert.pem"
  consul_tls_key_path = "/path/to/key.pem"
  consul_tls_cacert_path = "/path/to/cacert.pem"

lxd

(since v0.5.0)

LXD discovery provides the ability to query an LXD server for hosted containers and then forward traffic to those containers. There are two ways of using LXD discovery: running gobetween on the LXD server itself and using unix sockets or having gobetween query a remote LXD server via https.

Local LXD Server (unix sockets)

[servers.example]

# ...

  [servers.example.discovery]
  kind = "lxd"
  lxd_server_address = "unix:///var/lib/lxd/unix.socket"

  # (optional) Filter containers with specified setting key and value
  lxd_container_label_key = "user.gobetween.label"
  lxd_container_label_value = "web"

  # Take server port from specified setting key
  lxd_container_port_key = "user.gobetween.port"

  # OR use specified port
  lxd_container_port = 12345

Next, launch a container:

$ lxc launch <image_name> <container_name> -c user.gobetween.label=web -c user.gobetween.port=80

Or without port setting, if you decided to hardcode port in lxd_container_port

$ lxc launch <image_name> <container_name> -c user.gobetween.label=web

Or without any setting at all, in this case gobetween will take all containers and use lxd_container_port port.

$ lxc launch <image_name> <container_name>

gobetween will query the local LXD server for all containers with the metadata user.gobetween.label of web forward traffic to the port set in user.gobetween.port.

Remote LXD Server (https)

Querying a remote LXD server requires some extra configuration. First, you must make sure your LXD server is remotely accessible. See this blog post for instructions on how to do that.

Second, the server that will host gobetween must authenticate with the LXD server. The same blog post provides instructions on how to do that using the lxc client, but gobetween can also handle this internally, which means you do not have to install any LXC/LXD packages on the gobetween host.

To configure gobetween to query a remote LXD server, use the following configuration:

[servers.example]

# ...

  [servers.example.discovery]
  kind = "lxd"
  lxd_server_address = "https://lxd-01.example.com:8443"

  lxd_server_remote_name = "lxd-01"
  lxd_server_remote_password = "password"
  lxd_generate_client_certs = true
  lxd_accept_server_cert = true

  # (optional) Filter containers with specified setting key and value
  lxd_container_label_key = "user.gobetween.label"
  lxd_container_label_value = "web"

  # Take server port from specified setting key
  lxd_container_port_key = "user.gobetween.port"

  # OR use specified port
  lxd_container_port = 12345

One important thing to keep in mind is that gobetween must be able to access the containers remotely. This means that your containers must be attached to a network accessible to gobetween or you have port forwarding rules in place to forward traffic to internally hosted containers.

Notes

  • Only one LXD server per server entry is possible.
  • By default, the container's eth0 interface is used. You can choose a different a different interface on your containers by either:
    1. Setting lxd_container_interface to an interface name (such as eth1) which will apply to all containers on the LXD server.
    2. Setting lxd_container_interface_key to a value such as user.gobetween.iface and then setting this metadata in each container to a value such as eth1.
  • You can use IPv6 by setting lxd_container_address_type to IPv6.
  • If you are communicating with local lxd server via socket and you installed LXD via snap, you have to do one of two things:
    1. Make sure you run the lxd.migrate script or
    2. Update the socket path in the gobetween config lxd_server_address field.
Clone this wiki locally