Add initial scripts (#1)

LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2017-2020 The stuvus-config contributors
+Copyright (c) 2020-now The ansible_config_repo_scripts contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,223 @@
+# ansible_config_repo_scripts
+A collection of useful scripts for ansible config repos
+
+This is intended to be included as a submodule in an ansible config repo.
+
+## Requirements
+
+- Ansible
+- git
+- Python 3 with the modules PyYAML, netaddr, and JMESPath
+
+### Ubuntu/Debian or other APT based systems
+
+```bash
+sudo apt install ansible git python3 python3-netaddr python3-jmespath python3-yaml
+```
+
+### Nix/NixOS
+
+If you have [Nix](https://nixos.org/nix/), then `nix-shell` automatically makes the requirements available.
+The playbook script will automatically call `nix-shell` if it's present or you can launch it in a shell yourself.
+You may want to link/copy the shell.nix to the root of your config repo.
+
+## Usage
+
+Ansible config repos are typically heavily based on git submodules.
+To update them (and also initialize them if they are not initialized yet), the script `submodules` may help.
+
+---
+
+The `playbook.sh` script is designed to run playbooks from anywhere.
+
+```
+$ # Syntax
+$ playbook.sh <ansible_args> <playbook_name_without_yml>
+$
+$ # Example
+$ playbook.sh -l hypervisor01 all
+```
+
+The special playbook `all` is generated by `scripts/mkplaybook.py` and doesn't exist physically.
+
+### mkplaybook
+
+`mkplaybook` generates a playbook based on group memberships and `roles.yml`.
+For every role in the `roles/` directory, a host group with the same name is assumed and a proper play is generated.
+The facts are only gathered once.
+For every role a tag is generated named `_ROLENAME`.
+
+The `roles.yml` contains a dict with the role name as key and settings as values.
+
+| `Name`     | `Description`                                  |
+|------------|------------------------------------------------|
+| `early`    | Run this role before all other non-early roles |
+| `late`     | Run this role after all other non-late roles   |
+| `hosts`    | Also run this role on these hosts/groups       |
+| `excludes` | Short form for `hosts: [ !HOST ]`              |
+| `after`    | Always run this role after the specified roles |
+| `tags`     | Apply these additional tags                    |
+
+## Expected directory structure of parent repo
+
+The script in this repository expect the following directories in the parent repo:
+
+### hosts
+
+`hosts` should contain all *host files* with their variables.
+Each host has an own file `hosts/<hostname>.yml`.
+Additionally you can add files in `hosts/<hostname>/`.
+For example,
+
+```bash
+hosts/myhost/first.yml
+hosts/myhost/second.yml
+```
+
+will result in a host `myhost` with the variables from `first` and `second` merged.
+
+There is the special variable `_groups` which is the list of names of groups which the host is a member of.
+Every other variable is just passed to Ansible as variables applying to that host.
+
+### groups
+
+`groups/` should contain *group files* which specify *group variables*
+Each group has an own file, called `groups/<groupname>.yml`.
+Just as with hosts, you can additionally add files in `groups/<groupname>/`
+
+There are three special groups:
+
+- `all` contains all hosts (even though they don't list `all` in their `_groups`).
+  Therefore `groups/all.yml` contains the default variables which apply to all hosts.
+- `ungrouped` contains all hosts without any groups.
+  This should however never be the case.
+- `virtual` contains all hosts with a `vm` variable defined.
+
+### playbooks
+
+All Playbook files should be located in `groups/playbooks/`.
+Each one should begin with a comment describing what it's supposed to do.
+
+### roles
+
+All roles (typically as submodules) used in the repo.
+
+### files
+
+Files which are used in the repo (e.g. config files used by roles).
+
+### modules
+
+Custom modules used in the repo.
+
+### user.yml
+The file `user.yml` is used to configure user specific settings like the `ansible_user`.
+This file should be in the `.gitignore`
+
+### roles.yml
+A file containing some rules about the order of roles in the repo
+
+#### Example:
+```yaml
+---
+# All roles with non-default values.
+# Please sort the keys alphabetically within the three sections.
+
+#####
+# Early roles
+#####
+
+apt_sources:
+  early: true
+  hosts:
+    - all
+  after:
+    - fstab
+  tags:
+    - common
+
+fstab:
+  early: true
+  hosts:
+    - all
+  tags:
+    - common
+
+sudo:
+  early: true
+  hosts:
+    - all
+  tags:
+    - common
+
+#####
+# Late roles
+#####
+
+upgrade:
+  late: true
+  hosts:
+    - all
+
+#####
+# Normal roles
+#####
+
+grub2:
+  after:
+    - crypttab
+    - fstab
+
+icinga2-client:
+  hosts:
+    - all
+  after:
+    - icinga2-master
+    - icinga2-scripts
+  tags:
+    - common
+```
+
+### ansbile.cfg
+The ansible config.
+It is important that the variable `inventory` is set to `<path-to-this-submodule>/inventory.py`
+
+### environment
+A file with some environment variables. See below.
+
+## Special variables
+
+Some important special variables are:
+
+- `_groups` is a mandatory variable per host and is a list of groups that host is a member of
+- `ansible_host` is a mandatory variable per host and is the IP address which ansible uses to
+  connect to that host
+- `ansible_user` is the username used to connect to the server.
+
+The [Ansible documentation](https://docs.ansible.com/ansible/latest/intro_inventory.html#list-of-behavioral-inventory-parameters)
+lists more variables which are recognized by Ansible.
+
+## Environment
+
+To ensure consistent playbook executions, the entire environment is dropped before the Playbook is
+ran.
+However, as some environment variables are needed for correct playbook exection, they can be set
+from files in the repository.
+There are two files both located in the repository root:
+`./environment` and `./environment.local`.
+While `environment` must exist and has to be committed, `environment.local` is optional and must not be committed.
+Both work in the same way having simple assignments in bash style.
+Lines may be empty or start with `#` to indicate a comment.
+Comments after assignments are treated as parts of the assignments.
+Assignments may contain blank characters.
+
+Example:
+```bash
+# This is a comment followed by an empty line
+
+SOME_VAR=some value
+LANG=
+```
+
+Assignments with an empty value are filled from the parent environment (your shell).
+If the variable doesn't exist in the parent environment, it is ignored.

generate_ip_doc.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+import yaml
+import glob
+import os
+from copy import deepcopy
+from datetime import datetime
+
+# cd into stuvus_config
+rpath=os.path.dirname(os.path.realpath(__file__))
+os.chdir(rpath+'/..')
+
+# get a list of all host configuration files
+hostvar_files=glob.glob("./hosts/*.yml")
+hostvar_files.extend(glob.glob("./hosts/*/*.yml"))
+
+data = {}
+hostvar_keys_of_interest = ['ip', 'hostname', 'type', 'description', 'organisation', 'groups']
+
+# build hostname from host configuration path
+def get_hostname(host_config_path):
+    hostname = host_config_path.replace('.yml','')
+    hostname = hostname.replace('./hosts/','')
+    hostname = hostname.split('/')[0] # get the hostname not the filename (needed for hosts with multiple config files)
+    return hostname
+
+# get all ips from host configuration
+def get_all_host_ips(host_config):
+    ips = []
+
+    if 'ansible_host' in host_config:
+        ips.append(host_config['ansible_host'])
+
+    # go over all configured interface types
+    for interface_type in [ interface_type for interface_type in ['interfaces', 'bridges'] if interface_type in host_config]:
+        for interface in host_config[interface_type]:
+            if 'ip' in interface:
+                ips.append(interface['ip'])
+            if 'ips' in interface:
+                for ip in interface['ips']:
+                    ips.append(ip)
+    ips = [ ip.split('/')[0] for ip in ips ] # get ips without CIDR
+    return ips
+
+# iterate over all hosts
+for host_config_path in hostvar_files:
+    # parse host configuration
+    host_config_file = open(host_config_path)
+    host_config = yaml.safe_load(host_config_file)
+    host_config_file.close()
+
+    # get and set the hostname
+    host_config['hostname'] = get_hostname(host_config_path)
+
+    host_ips = get_all_host_ips(host_config)
+    for ip in host_ips:
+        sort_ip = ''.join([ ip_part.zfill(3) for ip_part in ip.split('.') ])
+        data[sort_ip] = deepcopy(host_config)
+        data[sort_ip]['ip'] = ip
+        try:
+            data[sort_ip]['organisation'] = host_config['vm']['org']
+        except KeyError:
+            data[sort_ip]['organisation'] = '___-___'
+        if 'vm' in data[sort_ip]:
+            data[sort_ip]['type'] = ' vm '
+        else:
+            data[sort_ip]['type'] = ' hw '
+        data[sort_ip]['groups'] = ", ".join(data[sort_ip]['_groups']) # pretty formate groups
+
+format_string = '|'
+separator_string = '|'
+header_string = '|'
+for info_key in hostvar_keys_of_interest:
+    # maximum string length for relevant data
+    max_key_length = max([ len(data[sort_ip][info_key]) for sort_ip in data])
+    format_string += ' {%s:<%d} |' % (info_key, max_key_length) # can't use .format here since i need to build a format string
+    separator_string += '{row_separator:{row_separator}<{length}}|'.format(row_separator = '-', length = max_key_length+2)
+    header_string += ' {info_key:<{length}} |'.format(info_key = info_key, length = max_key_length)
+
+# print table head
+print("letztes Update {}\n".format(datetime.now()))
+print(header_string)
+print(separator_string)
+
+# print all host information
+for sort_ip in sorted(data):
+    print(format_string.format(**data[sort_ip]))