Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for different log formats apart from JSON #17

Open
KonstantinGasser opened this issue May 8, 2023 · 0 comments
Open

Support for different log formats apart from JSON #17

KonstantinGasser opened this issue May 8, 2023 · 0 comments
Labels
feature an addition to the tool help wanted Extra attention is needed

Comments

@KonstantinGasser
Copy link
Owner

KonstantinGasser commented May 8, 2023

scotty currently only supports JSON logs when it comes to formatting of the logs. However, there are other common formats out there scotty should support.

This issue should not only focus on including a new log format but further establish a clean concept and APIs such that further formats can be added by only adding a new "formatter/parser".

For the sake of this issue I would suggest to implement a parser/formatter for logfmt logs.

With this issue it would be great to solve the following:

  1. remove the direct dependency within the store/formatter.go that only uses a JSON parser towards a generic solution.
  2. implement the option to parse logfmt logs

Side notes:

When browsing and thereby formatting the logs we display indented JSON and it would be nice the use a different type of visualisation for the key/value pairs of the logfmt logs.

Maybe out of scope but something we need to think about is how knows which parse to use. It would be wasteful to try the parsers in sequence until one works. Much rather I would suggest - and that is out of scope - that beam is extended with a cmd flag to indicate which format is streamed. In case of ideas please open an issue in beam and cross-reference the issue in here.

Lastly, formats such as CLF or ELF should be treated as plain text and not be parsed nor formatted. As a matter of fact if the flag in beam is omitted, hence the format is unknown to scotty, scotty should treat the logs as plain text by default.

@KonstantinGasser KonstantinGasser added help wanted Extra attention is needed feature an addition to the tool labels May 8, 2023
@KonstantinGasser KonstantinGasser changed the title Support for different log formats apart of JSON Support for different log formats apart from JSON May 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature an addition to the tool help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant