-
Notifications
You must be signed in to change notification settings - Fork 863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aggregation of tables registered in data collector during batch runs #2055
Comments
Sorry for the late response.
The table output just hasn't been worked out yet, and not due to a complication AFAICT. You are welcome to help integrate your custom workaround into |
Thanks for reporting. Linking a related/similar discussion here: |
What's the problem this feature will solve?
Consideration of table aggregation during batch runs.
Describe the solution you'd like
Concatenate rows of tables over batch runs while adding an additional column corresponding to the run id in the collect_data method of batch_run.
Additional context
Hello,
In a project, I am using the tables attribute of the datacollector. I am logging completed "jobs" that only happen sometimes in a step and also does only concern a specific type of agent. I wanted to avoid adding unnecessary rows to the agent reporters that would log nonsense data for a "job" when there are no new jobs to log or when jobs do not have properties that the actual agents have. This keeps the agent reporter data compact.
However, looking at the batch_run source code, it seems that only model and agent reporter data is aggregated over runs. Since I was not able to use the batch_run function, I had to write a workaround with a model run method that saves tables to file. I was wondering if there is a deeper reason or a complication that is not that obvious?
Best Regards
The text was updated successfully, but these errors were encountered: