Error updating hyperparameters in "Best so far"

[senyang1107]

Dear Kerastuner Team,

First and foremost, I'd like to express my appreciation for the Kerastuner project. Your work has been immensely beneficial to the machine learning community, enabling us to fine-tune models with greater ease and efficiency.
However, I've encountered a minor issue while using Kerastuner and would like to bring it to your attention, along with a suggestion for improvement.
Describe the bug
In my experiments, I found a hyperparameter error in the Best so far after using randomsearch. As you can see from the picture, when the num_layers = 11 in Best so far, we should expect unit_11=none, but the opposite is true. I don't know why this problem has occurred, and I really hope to get your help.

To Reproduce
def build_model(hp):
model = keras.Sequential()
for i in range(hp.Int('num_layers', 1, 12)):
model.add(layers.Dense(units=hp.Int('units_' + str(i), min_value=16,max_value=512,step=16), activation='relu'))
model.add(Dropout(rate=hp.Float('dropout_rate', min_value=0.0, max_value=0.5, step=0.1)))
batch_size = hp.Int('batch_size', min_value=4, max_value=64, step=4) #
model.add(Dense(1,kernel_initializer=initializer)) #
model.compile(optimizer='adam', loss='mean_squared_error', metrics=[keras.metrics.RootMeanSquaredError()])
return model
tuner = RandomSearch(
build_model,
objective=keras_tuner.Objective("val_root_mean_squared_error", direction="min"),
max_trials=100, #
executions_per_trial=1,
directory='randomsearch',
project_name='randomsearch_choice_L1_L2',
seed=42, #
overwrite=True, #
allow_new_entries=True, #
)
early_stopping = EarlyStopping(monitor='val_root_mean_squared_error', patience=10, restore_best_weights=True)
tuner.search(train_validation_X.values,train_validation_Y.values, epochs=700, validation_split=0.25, callbacks=[early_stopping], verbose=1)
Expected behavior
The performance of the DNN model is mainly affected by the number of hidden layers and the number of neurons in each hidden layer. Therefore, we hope that the hyperparameters will be updated, that is, the num_layers in Best so far will be consistent with the number of units. In addition, we use keras_tuner_cv 1.1.1 (https://github.com/VZoche-Golob/keras-tuner-cv/tree/50da4f1e2f3d552f47ee54092d14845a5b646cbf) based on kerastuner (version 1.3.5) to perform cross-validation on each set of hyperparameters to obtain a hyperparameter model with strong generalization ability. Therefore, I hope to modify this bug based on version 1.3.5 of kerastuner to match keras_tuner_cv. Additionally, given the extensive utilization of Kerastuner, I firmly believe that integrating cross-validation functionality based on keras-tuner-cv into Kerastuner would constitute a highly advantageous feature. This integration would not only address the issues outlined in #663 but also yield significantly superior results with less time compared to sklearn cross-validation #139, which lacks GPU support. Finally, this would allow users to better assess the robustness of their models and select the most optimal hyperparameter configurations with greater accuracy.

Would you like to help us fix it?
I'm a Python novice, but I'll do my best to assist with bug fixing.