diff --git a/src/content/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration.mdx b/src/content/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration.mdx index 0b6eff3474e..867d53066a4 100644 --- a/src/content/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration.mdx +++ b/src/content/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration.mdx @@ -6,12 +6,12 @@ tags: - Private locations metaDescription: Customize your New Relic synthetics job manager. redirects: -freshnessValidatedDate: never +freshnessValidatedDate: 2024-07-29 --- This doc will guide you through configuring your [synthetics job manager](/docs/synthetics/synthetic-monitoring/private-locations/install-job-manager) by showing you how to: -* Use [environment-variables](#environment-variables) to configure your synthetics job manager. +* Use [environment variables](#environment-variables) to configure your synthetics job manager. * Set up [custom modules](#custom-modules) for [scripted API](/docs/synthetics/synthetic-monitoring/scripting-monitors/write-synthetic-api-tests/) or [scripted browser](/docs/synthetics/new-relic-synthetics/scripting-monitors/write-scripted-browsers) monitors. * Provide [user-defined variables](#user-defined-vars) in your configuration. @@ -734,57 +734,60 @@ Private synthetics job managers let you configure environment variables for scri - The user may create a JSON-formatted file and mount the volume where the file is located to a specified target path in the SJM container. - - The file must have read permissions and contain a JSON-formatted map. Example user-defined variables file: + id="user-file-example" + title="Mounting JSON file" + > + The user may create a JSON-formatted file and mount the volume where the file is located to a specified target path in the SJM container. + + The file must have read permissions and contain a JSON-formatted map. Example user-defined variables file: - ``` - { - "KEY": "VALUE", - "user_name": "MINION", - "my_password": "PASSW0RD123", - "my_URL": "https://newrelic.com/", - "ETC": "ETC" - } - ``` - - Place the file in the source directory on the host. The SJM is expecting the file name to be user_defined_variables.json - - Docker example: - - The expected target directory is: `/var/lib/newrelic/synthetics/variables/` - - ``` - docker run ... -v /variables:/var/lib/newrelic/synthetics/variables:rw ... - ``` + ``` + { + "KEY": "VALUE", + "user_name": "MINION", + "my_password": "PASSW0RD123", + "my_URL": "https://newrelic.com/", + "ETC": "ETC" + } + ``` - Kubernetes example: - - The user has two options when providing a file to the SJM pod in Kubernetes. They may: - 1. pass in a local file. - 2. provide a PersistentVolume that includes the user_defined_variables.json. + Place the file in the source directory on the host. The SJM is expecting the file name to be user_defined_variables.json - ### Pass in a local file -This option creates a ConfigMap Kubernetes resource and mounts that to the SJM pod. + Docker example: -``` - helm install newrelic/synthetics-job-manager ... --set-file "synthetics.userDefinedVariables.userDefinedFile=[local-path]/user_defined_variables.json" ... -``` + The expected target directory is: `/var/lib/newrelic/synthetics/variables/` - ### Mount a PersistentVolume -This option requires the user to provide a PersistentVolume that includes the user_defined_variables.json file or a PersistentVolumeClaim to the same. For more details on helm chart installation using a PersistentVolume, follow the instructions at [permanent data storage](/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration#permanent-data-storage). - -Once the user has prepared a PersistentVolume as described below, launch the SJM, setting the path where the user_defined_variables.json file is located and setting any other `synthetics.persistence` variables as necessary. + ``` + docker run ... -v /variables:/var/lib/newrelic/synthetics/variables:rw ... + ``` -``` -helm install newrelic/synthetics-job-manger ... --set synthetics.userDefinedVariables.userDefinedPath="variables" -``` - + Kubernetes example: + + The user has two options when providing a file to the SJM pod in Kubernetes. They may: + * Pass in a local file. + * Provide a PersistentVolume that includes the `user_defined_variables.json`. + + ### Pass in a local file + + This option creates a ConfigMap Kubernetes resource and mounts that to the SJM pod. + + ``` + helm install newrelic/synthetics-job-manager ... --set-file "synthetics.userDefinedVariables.userDefinedFile=[local-path]/user_defined_variables.json" ... + ``` + + ### Mount a `PersistentVolume` + + This option requires the user to provide a `PersistentVolume` that includes the `user_defined_variables.json` file or a `PersistentVolumeClaim` to the same. For more details on helm chart installation using a `PersistentVolume`, follow the instructions at [permanent data storage](/docs/synthetics/synthetic-monitoring/private-locations/job-manager-configuration#permanent-data-storage). + + Once the user has prepared a `PersistentVolume` as described below, launch the SJM, setting the path where the `user_defined_variables.json` file is located, and set any other `synthetics.persistence` variables as necessary. + + ``` + helm install newrelic/synthetics-job-manger ... --set synthetics.userDefinedVariables.userDefinedPath="variables" + ``` + + - @@ -802,14 +805,12 @@ Use the `--set-literal` flag to pass in the JSON formatted string. ``` helm install newrelic/synthetics-job-manager ... --set-literal synthetics.userDefinedVariables.userDefinedJson='{"key":"value","name":"sjm"}' ... ``` - + ### Accessing user-defined environment variables from scripts [#env-vars-scripts] -To reference a configured user-defined environment variable, use the reserved `$env.USER_DEFINED_VARIABLES` followed by the name of a given variable with dot notation. - -For example, `$env.USER_DEFINED_VARIABLES.MY_VARIABLE` +To reference a configured user-defined environment variable, use the reserved `$env.USER_DEFINED_VARIABLES` followed by the name of a given variable with dot notation (for example, `$env.USER_DEFINED_VARIABLES.MY_VARIABLE`). User-defined environment variables are not sanitized from logs. Consider using the [secure credentials](/docs/synthetics/new-relic-synthetics/using-monitors/secure-credentials-store-credentials-information-scripted-browsers) feature for sensitive information. @@ -823,36 +824,31 @@ To set up the modules: 1. Create a directory with a `package.json` file following [npm official guidelines](https://docs.npmjs.com/files/package.json) in the root folder. The SJM will install any dependencies listed in the package.json's `dependencies` field. These dependencies will be available when running monitors on the private synthetics job manager. See an example of this below. - - - In this example, a custom module directory is used with the following structure: +### Example - ``` - /example-custom-modules-dir/ - ├── counter - │ ├── index.js - │ └── package.json - └── package.json ⇦ the only mandatory file - ``` +In this example, a custom module directory is used with the following structure: + +``` +/example-custom-modules-dir/ + ├── counter + │ ├── index.js + │ └── package.json + └── package.json ⇦ the only mandatory file +``` - The `package.json` defines `dependencies` as both a local module (for example, `counter`) and any hosted modules (for example, `smallest` version `1.0.1`): +The `package.json` defines `dependencies` as both a local module (for example, `counter`) and any hosted modules (for example, `smallest` version `1.0.1`): - ``` - { - "name": "custom-modules", - "version": "1.0.0", ⇦ optional - "description": "example custom modules directory", ⇦ optional - "dependencies": { - "smallest": "1.0.1", ⇦ hosted module - "counter": "file:./counter" ⇦ local module - } - } - ``` - - +``` +{ + "name": "custom-modules", + "version": "1.0.0", ⇦ optional + "description": "example custom modules directory", ⇦ optional + "dependencies": { + "smallest": "1.0.1", ⇦ hosted module + "counter": "file:./counter" ⇦ local module + } +} +``` 2. Once you create the custom modules directory and the `package.json`, apply it to your SJM for Docker and Kubernetes. @@ -872,23 +868,216 @@ To set up the modules: id="kubernetes" title="Kubernetes" > - Complete the following: - - 1. Launch the SJM, setting a value for the `persistence.customModules` configuration value either in the command line or in a YAML file during installation. The value should specify the subpath on your synthetics job manager Persistent Volume where your custom modules files exist. For example: - - ``` - helm install ... --set persistence.customModules= ... - ``` - 2. Make sure that your custom modules directory is available on the Minion Pod. You can use `kubectl cp` as one method to copy the directory from your host to the Minion. For example: - - ``` - kubectl cp /example-custom-modules-dir /:/var/lib/newrelic/synthetics/modules - ``` + For Kubernetes, the directory at `/var/lib/newrelic/synthetics/modules` needs to exist on a PV prior to launching the SJM with custom modules enabled. + + + The PV access mode should be ReadWriteMany if you need to share storage across multiple pods. + + + One method is to create a pod that mounts the PV just for the purpose of copying your custom modules directory to the PV. The following example uses Amazon EFS with Amazon EKS: + + ### Create the namespace, persistent volume, and persistent volume claim + + 1. Make sure you've already set up your EFS filesystem and installed the [EFS CSI driver](https://github.com/kubernetes-sigs/aws-efs-csi-driver) on your cluster. You will also need your EFS filesystem ID for the PV's `spec.csi.volumeHandle`. + + ```sh + kubectl apply -f - < + + --- + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: custom-modules-pvc + namespace: newrelic + spec: + accessModes: + - ReadWriteMany + storageClassName: efs-sc + resources: + requests: + storage: 5Gi + EOF + ``` + + 2. Switch to the `newrelic` namespace in your `~/.kube/config`. + + ```sh + kubectl config get-contexts + kubectl config set-context --namespace=newrelic + kubectl config view --minify | grep namespace: + ``` + + 3. At this point, the PVC should be bound to the PV with RWX access mode. + + ```sh + $ kubectl get pv,pvc + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE + persistentvolume/custom-modules-pvc 5Gi RWX Retain Bound newrelic/custom-modules-pvc efs-sc 4m46s + + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE + persistentvolumeclaim/custom-modules-pvc Bound custom-modules-pvc 5Gi RWX efs-sc 4m10s + ``` + + ### Create `mount-custom-mods-pod` to copy your custom-modules directory + + ```sh + kubectl apply -f - < + Release "synthetics-job-manager" does not exist. Installing it now. + NAME: synthetics-job-manager + LAST DEPLOYED: Fri Jun 28 16:53:28 2024 + NAMESPACE: newrelic + STATUS: deployed + REVISION: 1 + TEST SUITE: None + ``` + + The `custom-modules` directory should now contain the installed packages in `node_modules`. + + ```sh + $ kubectl exec -it mount-custom-mods-pod -- bash + root@mount-custom-mods-pod:/# cd /var/lib/newrelic/synthetics/modules/ + root@mount-custom-mods-pod:/var/lib/newrelic/synthetics/modules# ls -l custom-modules/ + total 16 + -rw-r--r-- 1 root root 836 Jun 29 03:51 README + drwxr-xr-x 18 root root 6144 Jun 29 03:51 node_modules + -rw-r--r-- 1 501 staff 299 Jun 29 03:49 package.json + -rw-r--r-- 1 root root 190 Jun 29 03:51 package.json.shasum + ``` + + 2. Look for the following in `synthetics-job-manager` pod logs: + + ```log + 2024-06-29 03:51:28,407{UTC} [main] INFO c.n.s.j.p.options.CustomModules - Detected mounted path for custom node modules + 2024-06-29 03:51:28,408{UTC} [main] INFO c.n.s.j.p.options.CustomModules - Validating permission for custom node modules package.json file + 2024-06-29 03:51:28,409{UTC} [main] INFO c.n.s.j.p.options.CustomModules - Installing custom node modules... + 2024-06-29 03:51:44,670{UTC} [main] INFO c.n.s.j.p.options.CustomModules - Custom node modules installed successfully. + ``` + + If custom node modules are not detected, adjust permissions on the `custom-modules` directory and `package.json` file. + + ```sh + $ kubectl exec -it mount-custom-mods-pod -- bash + root@mount-custom-mods-pod:/# cd /var/lib/newrelic/synthetics/modules/ + root@mount-custom-mods-pod:/var/lib/newrelic/synthetics/modules# chmod -R 777 custom-modules + root@mount-custom-mods-pod:/var/lib/newrelic/synthetics/modules# chown -R 2000:2000 custom-modules + ``` -3. To check if the modules were installed correctly or if any errors occurred, review the [SJM logs](/docs/synthetics/new-relic-synthetics/private-locations/job-manager-maintenance-monitoring#monitor-docker-logs) for the section titled `"... Initialization of Custom Modules ..."`. These logs will include the npm installation logs, providing information regarding the installation process and any potential errors encountered. +To check if the modules were installed correctly or if any errors occurred, review the [SJM logs](/docs/synthetics/new-relic-synthetics/private-locations/job-manager-maintenance-monitoring#monitor-docker-logs) for the section titled `"... Initialization of Custom Modules ..."`. These logs will include the npm installation logs, providing information regarding the installation process and any potential errors encountered. Now you can add `"require('smallest');"` into the [script](/docs/synthetics/new-relic-synthetics/scripting-monitors/write-scripted-browsers) of monitors you send to this private location. @@ -933,8 +1122,8 @@ Example: 2. Provide an existing PersistentVolume (PV) name, setting the `synthetics.persistence.existingVolumeName` configuration value. Helm will generate a PVC for the user. The user may optionally set the following values as well: -- `synthetics.persistence.storageClass`: the storage class of the existing PV. If not provided, Kubernetes will use the default storage class. -- `synthetics.persistence.size`: the size for the claim. If not set, the default is currently 2Gi. +- `synthetics.persistence.storageClass`: The storage class of the existing PV. If not provided, Kubernetes will use the default storage class. +- `synthetics.persistence.size`: The size for the claim. If not set, the default is currently 2Gi. ``` helm install ... --set synthetics.persistence.existingVolumeName=sjm-volume --set synthetics.persistence.storageClass=standard ...