Performance Constraints

Integrating performance baselines and performance expectations in the hyperparameter optimization process is furthermore helpful to increase the overall speed and efficiency. The user can specify to skip the testing of a specific hyperparameter configuration in further inner-cross-validation folds if the given configuration performs worse than a given static or dynamic threshold.

           
from sklearn.datasets import load_boston
from sklearn.model_selection import KFold

from photonai.base import Hyperpipe, PipelineElement, OutputSettings
from photonai.optimization import MinimumPerformance, IntegerRange

# WE USE THE BREAST CANCER SET FROM SKLEARN
X, y = load_boston(True)

# DESIGN YOUR PIPELINE
my_pipe = Hyperpipe(name='basic_svm_pipe_no_performance',
                    optimizer='sk_opt',
                    optimizer_params={'n_configurations': 25},
                    metrics=['mean_squared_error', 'pearson_correlation'],
                    best_config_metric='mean_squared_error',
                    outer_cv=KFold(n_splits=3, shuffle=True),
                    inner_cv=KFold(n_splits=3),
                    eval_final_performance=True,
                    verbosity=1,
                    output_settings=OutputSettings(project_folder='./result_folder'),
                    performance_constraints=[MinimumPerformance('mean_squared_error', 35, 'first'),
                                             MinimumPerformance('pearson_correlation', 0.7, 'all')])

my_pipe += PipelineElement('StandardScaler')
my_pipe += PipelineElement('RandomForestRegressor', hyperparameters={'n_estimators': IntegerRange(5, 50)})


# NOW TRAIN YOUR PIPELINE
my_pipe.fit(X, y)