As we know,
tf.estimator.Estimator.train just do training and
tf.estimator.Estimator.evaluate just do evaluation, what if we want to regularly evaluate half-trained model while training to get a learning curve or select a best performance model?
tf.estimator.train_and_evaluate( estimator, train_spec, eval_spec )
This utility function trains, evaluates, and (optionally) exports the model by using the given
estimator. All training related specification is held in
train_spec, including training input_fn and training max steps, etc. All evaluation and export related specification is held in
eval_spec, including evaluation input_fn, steps, etc.
Estimatorinstance to train and evaluate.
TrainSpecinstance to specify the training specification.
EvalSpecinstance to specify the evaluation and export specification.
tupleof the result of the evaluate call to the Estimator and the export results using the specified ExportStrategy. Currently, the return value is undefined for distributed training mode.
ValueError: if environment variable
TF_CONFIGis incorrectly set.
Below is an example for applying
tf.estimator.train_and_evaluate to bert run_classifier.py:
# Create train_spec train_input_fn = file_based_input_fn_builder( input_file=train_file, seq_length=FLAGS.max_seq_length, is_training=True, drop_remainder=True) train_spec = tf.estimator.TrainSpec( input_fn=train_input_fn, max_steps=num_train_steps) # Create eval_spec exporters = [tf.estimator.BestExporter( name="best_exporter", serving_input_receiver_fn=serving_input_receiver_fn, exports_to_keep=5)] eval_drop_remainder = True if FLAGS.use_tpu else False eval_input_fn = file_based_input_fn_builder( input_file=eval_file, seq_length=FLAGS.max_seq_length, is_training=False, drop_remainder=eval_drop_remainder) eval_spec = tf.estimator.EvalSpec( input_fn=eval_input_fn, steps=FLAGS.save_checkpoints_steps, exporters=exporters, start_delay_secs=0, throttle_secs=600) # Call train_and_evaluate eval_results, export_results = tf.estimator.train_and_evaluate( estimator, train_spec=train_spec, eval_spec=eval_spec)
train_spec is simple, for creating
input_fn, below args are worth mentioning:
steps: Int. Positive number of steps for which to evaluate model. If
None, evaluates until
input_fnraises an end-of-input exception. See
exporters: Iterable of
Exporters, or a single one, or
exporterswill be invoked after each evaluation.
start_delay_secs: Int. Start evaluating after waiting for this many seconds.
throttle_secs: Int. Do not re-evaluate unless the last evaluation was started at least this many seconds ago. Of course, evaluation does not occur if no new checkpoints are available, hence, this is the minimum.
FLAGS.save_checkpoints_steps so we can evaluate each checkpoint saved while training.
BestExporter with default
exports_to_keep will export 5 best checkpoints with least loss to saved model. I will write another article about implementing an
BestExporter which saves best checkpoint instead of exporting saved model.
Python量化投资网携手4326手游为资深游戏玩家推荐：《《封印战记》：次元兽研究计划 | 食尸鬼军团已锁定！》