Make warmstarting easier

[benjamc]

Provide a helper function receiving list[Configuration] and float costs.
Use tell under the hood and convert stuff into TrialInfo and TrialValue.

[armandieu]

Hello @benjamc, with a group of friend from University studying optimization, we would like to collaborate on this issue if possible.
Can you please tell me more about the details.

[eddiebergman]

Heyo, just leaving this for context #950 as I've provided some sample code there in a project where I wrap SMAC to be able to do this with my own trial-like objects.

[eddiebergman]

I tried this out in the Colab notebook at the fall school and can verify it's at least consistent with the tell() interface, i.e. use one instantiation to generate 10 trial/value pairs. Create a new optimizer instance and tell it the 10 trial/value pairs. Confirm that the 11th trial produced by both is the same.

Doesn't solve the original problem but good to confirm its correctness that it can be used to warmstart correctly.

[armandieu]

Hello Dear @eddiebergman , can you please share this Colab notebook to have an idea of where to start. Thanks.

[armandieu]

Hello,
For example purposes, I was thinking of a method included in the helper class similar to this.

#self smac class
#configList: List of Configuration
#costList: List of floats
def warm_starting(self, configList, costList):
  trialInfos = [SMACTrialInfo(config=config) for config in configList]
  trialValues = [SMACTrialValue(cost=cost) for cost in costList]
  for i in range(len(trialInfo)):
    self.facade.tell(info=trialInfos[i], value=trialValues[i], save=True)

As I am learning from the documentation, please let me know if it makes sense.

[armandieu]

Hello SMAC community, as part of this approach, I have made an example of how the implementation of a warmstarting from a list of configuration and cost values will look like.
Any comments and direction are really appreciated.

from ConfigSpace import Configuration
from smac.runhistory.dataclasses import TrialValue, TrialInfo
# For the moment we assume the values 1, 2, -0.5 are points the model must know to solve it faster.
x_values = [1, 2, -0.5]
# In this case I initialize the cost for each point as specific values, but it could be the result of the
# target function evaluated in the point model.train(config)
cost_list = [3, 4, 1]
# Finally we will transform the x_values to warm start into TrialInfo objects which also needs. Here we assume x is an hyperparameter already setup in the configspace of the model
trialInfos = [TrialInfo(config=Configuration(model.configspace, {'x': x})) for x in x_values]
trialValues = [TrialValue(cost=cost) for cost in cost_list]
# Then it is necessary to use the tell interface of SMAC for warmstart the model, 
# https://automl.github.io/SMAC3/main/examples/1_basics/3_ask_and_tell.html#ask-and-tell
# In this case, assuming we have an smac facade objecti named scam, we will warmstart all the objects Trial Info in the list ?
# trialInfos and trialValues
for i in range(len(trialInfos)):
  smac.tell(info=trialInfos[i], value=trialValues[i], save=True)