Document "headers" as part of the config options

[jaijhala]

Currently this is what we have as a list of Config options for this plugin - https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html#plugins-inputs-http_poller-options

But I don't see any description for headers field. Looking at issue (#44) there seems to be a confusion on how this looks like.

So it would be a good idea to document this.

[yaauie]

I don't have time at the moment to make this top priority, but I'll add notes here based on what I know to be true so that someone else can do it if they get to it before me:

headers falls under urls, which only really documents the simple string-value option:

A Hash of urls in this format : "name" ⇒ "url". The name and the url will be passed in the outputed event

Separately, there's a note:

# Supports all options supported by ruby's Manticore HTTP client

That's right, we blindly pass through anything we parse to Manticore, and hope it can deal with it correctly.

So, the values in urls can be either:

• a string url (which will be issued as an HTTP GET).
• a sub-hash containing many useful keys provided by the Manticore backend:
  • url: the String url
  • method: (optional) the HTTP method to use (defaults to GET)
  • user: (optional) the HTTP Basic Auth user
  • password: (optional) the HTTP Basic Auth password (note: in Manticore, this must be under an auth sub-hash, but this plugin accepts it either way)
  • headers: a hash containing key-value pairs of headers (note: in Manticore, this must be under an auth sub-hash, but this plugin accepts it either way)
  • body: a string (supported only on POST and PUT requests)
  • probably any of the other options mentioned in the Manticore::Client#http(method,url,options) docs (although: these are not thoroughly tested and therefore liable to break in unexpected ways if we ever end up replacing the backend)

A couple more notes:

• passwords specified in this way are prone to leaking in plugin log output, since the plugin does not declare them as such and therefore doesn't wrap them in leak-reducing wrappers as we do elsewhere.
• I'm not confident that boolean-type options like follow_redirects are supported correctly, as the strings true or false may get passed through and in ruby any string is "truthy".
• our implementation precludes the ability to specify auth[:eager] as anything other than true

[karenzone]

@karenzone - Opportunity for doc improvements