Wrong Pretrained model saved in Pretraining and FineTuning BERT

[joHussien]

Issue Type

Bug

Source

source

Keras Version

Keras 2.14

Custom Code

Yes

OS Platform and Distribution

No response

Python version

No response

GPU model and memory

No response

Current Behavior?

Save this base model for further finetuning.

encoder_model.save("encoder_model.keras")

This line under https://github.com/keras-team/keras-io/tree/master/guides/keras_nlp/transformer_pretraining.py

Here you save the model architecture (encoder_model), but instead you should save pretrained_model which is the final pretrained model that you should use for finetuning afterwards

Standalone code to reproduce the issue or tutorial link

# Save this base model for further finetuning.
encoder_model.save("encoder_model.keras")

This line under https://github.com/keras-team/keras-io/tree/master/guides/keras_nlp/transformer_pretraining.py

Here you save the model architecture (encoder_model), but instead you should save pretrained_model which is the final pretrained model that you should use for finetuning afterwards

Relevant log output

# Save this base model for further finetuning.
encoder_model.save("encoder_model.keras")

This line under https://github.com/keras-team/keras-io/tree/master/guides/keras_nlp/transformer_pretraining.py

Here you save the model architecture (encoder_model), but instead you should save pretrained_model which is the final pretrained model that you should use for finetuning afterwards