Symlists extractor

README.md

@@ -27,3 +27,4 @@ Tools for sysadmins.
 * [sympatoldap](utils/sympatoldap/README.md): creates LDAP entries for every list (and its aliases) whose status is open on the LDAP server
 * [splitting_daemons_logs](utils/splitting_daemons_logs/README.md): split Sympa processus logs into separated files
 * [sqlite2mysql](utils/sqlite2mysql/README.md): attempt to convert sqlite to mysql
+* [symlists_extractor](utils/symlists_extractor/README.md) : extract mailing lists how are subscribers to other mailing lists 

utils/symlists_extractor/README.md

@@ -0,0 +1,65 @@
+# CLI utility: symlists_extractor
+
+## Objective
+
+Retrieve from a SYMPA server the list of mailing lists registered as a member of another mailing list and not *included*
+as good practice dictates... ;)
+
+This script uses the [sympasoap](https://gitlab.com/animath/si/py-sympa-soap) , a python sympa SOAP wrapper library to work.
+
+## Use 
+* Have python installed (version >=v3.10)
+* Create a virtual environment: 
+ * ``$ python -m venv .venv``
+* To use it: 
+ * ``$ source .venv/bin/activate`` (Linux)
+ * ``$ call C:\path\to\project\env_project_name\Scripts\activate.bat`` (windows)
+ * or ``C:\path\to\project\env_project_name\Scripts\Activate.ps1`` (windows powershell)
+* Install the dependencies of the requirements.txt file with pip
+ * ``$ pip install -r requirements.txt``
+* In a terminal, at the root of the script, user the following command :
+
+``
+$ python symlist_extractor.py *arg1 *arg2 *arg3
+``
+
+* The arguments are **MANDATORY** in the following strict order: 
+ * arg1 = server url without **"/wsdl"** at the end (ex: http://nom_de_domaine.fr/sympa)
+ * arg2 = administrator login (ex: [email protected]_de_domaine.fr)
+ * arg3 = administrator password (ex: mon_beau_mdp)
+
+**/!\ Note 1:**
+* More than 3 arguments => not taken into account 
+* Less than 3 arguments => the script will be in error
+
+After a few minutes, you will get a JSON file with the following structure in the script folder: 
+
+**/!\ Note 2:**
+The dictionary keys are the existing mailing lists on your SYMPA server and the values are the mailing lists that
+are subscribed to the list and are not included in a data source.
+
+```json=
+{
+ "security@domain_name.fr": [ 
+ "abuse@domain_name.fr",
+ "report@domain_name.fr",
+ "....",
+ ] 
+}
+```
+
+## Changlog
+
+v0.0.1: initial release
+
+## Licence
+
+GPLv3
+
+## Author
+
+Nicolas.renard_at_uha.fr
+
+## Original source
+
+* [github source project](https://github.com/nicorenard/symlists_extractor)

utils/symlists_extractor/symlists_extractor.py

@@ -0,0 +1,99 @@
+import json
+import os
+import sys
+from sympasoap import Client
+from sympasoap.client import Fault
+
+
+class SympaController:
+
+ def __init__(self, url: str, login: str, password: str):
+ self.sympa_url = url
+ self.login = login
+ self.password = password
+ self.client = SympaController.get_connect_to_sympa(self)
+
+ def get_connect_to_sympa(self) -> Client:
+ """
+ Méthode de connexion au serveur Sympa.
+
+ :returns: Un client de connexion
+ """
+ try:
+ client = Client(self.sympa_url)
+ client.login(self.login, self.password)
+ return client
+ except Fault as e:
+ msg = f"Erreur, pas de connexion à Sympa : {str(e)}."
+ raise Fault(msg)
+
+ def get_list_in_list(self) -> None:
+ """
+ Méthode pour récupérer des listes de diffusion comme abonnées dans des listes de diffusion.
+ """
+ conn = self.client
+ sympa_lists = []
+ # get all lists
+ all_lists = conn.all_lists()
+ if not len(all_lists) > 0:
+ print(f"Warning - This is probably not an error but the result is empty.")
+ sys.exit(1)
+ all_lists_clean = [mailing_list.list_address for mailing_list in all_lists]
+
+ # get all subscribers
+ for mailing_list in all_lists_clean:
+ subscribers = conn.get_subscribers(mailing_list, emails_only=True)
+ datas = {
+ "name": mailing_list,
+ "members": subscribers,
+ }
+ sympa_lists.append(datas)
+ list_in_list = []
+
+ # cleaning subscribers
+ for mailing_list in all_lists_clean:
+ for list_element in sympa_lists:
+ members = list_element.get("members")
+ if mailing_list in members:
+ datas = {
+ "list_address": list_element.get("name"),
+ "list_hidden_in_list": mailing_list
+ }
+ list_in_list.append(datas)
+
+ # merging infos
+ combined_dict = {}
+ for item in list_in_list:
+ list_address = item["list_address"]
+ list_hidden_in_list = item["list_hidden_in_list"]
+ if list_address not in combined_dict:
+ combined_dict[list_address] = []
+ combined_dict[list_address].append(list_hidden_in_list)
+
+ # report
+ file_name = "lists_hidden_in_lists.json"
+ file_path = os.path.join(file_name)
+ folder = os.path.join(os.path.abspath(os.path.dirname(__file__)), file_path)
+ try:
+ with open(folder, "w") as file:
+ json.dump(combined_dict, file, indent=2)
+ except FileExistsError as e:
+ msg = f"Error, can't access to the JSON file: {str(e)}"
+ raise FileExistsError(msg)
+
+
+def main(kwargs: list[str]) -> None:
+ # sympa env
+ sympa_url = kwargs[0]
+ sympa_login = kwargs[1]
+ sympa_password = kwargs[2]
+ sc = SympaController(sympa_url, sympa_login, sympa_password)
+ print("Research in run....")
+ sc.get_list_in_list()
+ print("Research done.\nA JSON file was generated and is available in the script folder.")
+ sys.exit(0)
+
+
+if __name__ == "__main__":
+ args = sys.argv[1:4]
+ main(args)