HTTP lookup data adapter error reporting is inconsistent

[kroepke]

When debugging a customer issue, we found that there are no usable error notifications in the UI, if the HTTP server returns unsuccessful status codes.

graylog2-server/graylog2-server/src/main/java/org/graylog2/lookup/adapters/HTTPJSONPathDataAdapter.java

Lines 193 to 220 in d1fab26

	final Timer.Context time = httpRequestTimer.time();
	try (final Response response = httpClient.newCall(request).execute()) {
	if (!response.isSuccessful()) {
	LOG.warn("HTTP request for key <{}> failed: {}", key, response);
	httpRequestErrors.mark();
	return getErrorResult();
	}
	final LookupResult result = parseBody(singleJsonPath, multiJsonPath, response.body().byteStream());
	if (result == null) {
	return getErrorResult();
	}
	return result;
	} catch (IOException e) {
	LOG.error("Data adapter <{}>: HTTP request error for key <{}> from URL <{}>", name(), key, urlString, e);
	httpRequestErrors.mark();
	Notification systemNotification = notificationService.buildNow()
	.addType(Notification.Type.GENERIC)
	.addSeverity(Notification.Severity.NORMAL)
	.addDetail("title", "HTTP data adapter lookup failure")
	.addDetail("description", f("Data adapter <%s>: HTTP request error from URL <%s>: %s", name(), urlString, e.getMessage()));
	notificationService.publishIfFirst(systemNotification);
	return getErrorResult();
	} finally {
	time.stop();
	}

In this case an API returned 403 and the customer cannot see this happening in the UI.
Together with #19159 this lead to massive processing latency.