Pulsar event listener

[dgreene1]

What is the underlying problem you're trying to solve?

We want to have OPA be up to date with the newest event in a Pulsar topic.

Describe the ideal solution

The same as https://docs.styra.com/enterprise-opa/tutorials/using-data/streaming-kafka but for Pulsar.

Describe a "Good Enough" solution

A generic endpoint for pushing data. I believe this already exists. The problem here is that how does the central “pusher” know of all of the OPA instances? How does it know that they all accepted the push?

Additional Context

[srenatus]

Thanks for the request!

A generic endpoint for pushing data. I believe this already exists. The problem here is that how does the central “pusher” know of all of the OPA instances? How does it know that they all accepted the push?

For completeness' sake, there's the Data API which lets you PUT data updates into OPA. But you're correct, a central pusher service -- which is outside of the scope of OPA -- would need to know all OPA instances that it would push data to. Alternatively, it could distribute bundles (or delta bundles) and the OPA instances would fetch them and keep themselves updated (periodically, or via long polling).

Since OPA itself has no data plugins like EOPA, I don't know if this feature request has much chances. It would require quite a bit of effort to bring dynamic data plugins to OPA. It's more feasible as a feature request to EOPA, but I'm afraid we don't track them here (but it's been noted now 📝).

[ashutosh-narkar]

Closing this for now as it seems outside the scope of OPA. Feel free to re-open if anything OPA-specific would be needed. Thanks.

[dgreene1]

No problem. I will either enhance permit’s OPAL, wait for Enterprise OPA to implement it, or go elsewhere entirely.