Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-objective optimization configs no longer allow overrides via globals #847

Open
bpkroth opened this issue Aug 19, 2024 · 2 comments
Open
Labels
bug Something isn't working

Comments

@bpkroth
Copy link
Contributor

bpkroth commented Aug 19, 2024

Prior to #725 we could specify something like this in a reusable optimizer config:

// MLOS Core optimizer
{
    "class": "mlos_bench.optimizers.MlosCoreOptimizer",

    // Note: these variables can be overridden by the experiment global config variables.
    "config": {
        // Override the default backend optimizer choice:
        // "optimizer_type": "FLAML",
        // "optimizer_type": "SMAC",

        // Optionally reduce the high dimension search space using LlamaTune.
        // "space_adapter_type": "LLAMATUNE",
        // "space_adapter_type": null,

        "optimization_target": "99th Percentile Latency (microseconds)",
        "optimization_direction": "min",

        "max_suggestions": 100
    }
}

and then override it in the --globals like so:

{
    "optimization_target": "Throughput",
    "optimization_direction": "max"
}

Now, optimization targets are specified like so:

    "optimization_targets": {
        "99th Percentile Latency (microseconds)": "min",
        // ...
    }

That makes it harder to overwrite them with globals and leads to the need to specify additional config files rather than keeping them in experiment files. Do we have a good solution for this?

@bpkroth
Copy link
Contributor Author

bpkroth commented Aug 19, 2024

Wonder if we can add an "optimization_targets": {...} block in the globals and special case that to be passed to the optimizer maybe? Seems a little messy ...

@bpkroth
Copy link
Contributor Author

bpkroth commented Aug 19, 2024

Actually, --max-suggestions instead of --max_suggestions also no longer works. Might submit a PR to fix that one ...

motus added a commit that referenced this issue Aug 19, 2024
@bpkroth bpkroth added the bug Something isn't working label Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant