[Tensorflow] tf.estimator.train_and_evaluate

阿里云双11来了!从本博客参与阿里云,服务器最低只要86元/年!

As we know, tf.estimator.Estimator.train just do training and tf.estimator.Estimator.evaluate just do evaluation, what if we want to regularly evaluate half-trained model while training to get a learning curve or select a best performance model?

There is a method tf.estimator.train_and_evaluate at the same level of tf.estimator.Estimator in tensorflow API docs, which is exact what we want.

tf.estimator.train_and_evaluate(
    estimator,
    train_spec,
    eval_spec
)

This utility function trains, evaluates, and (optionally) exports the model by using the given estimator. All training related specification is held in train_spec, including training input_fn and training max steps, etc. All evaluation and export related specification is held in eval_spec, including evaluation input_fn, steps, etc.

Args:

  • estimator: An Estimator instance to train and evaluate.
  • train_spec: A TrainSpec instance to specify the training specification.
  • eval_spec: A EvalSpec instance to specify the evaluation and export specification.

Returns:

A tuple of the result of the evaluate call to the Estimator and the export results using the specified ExportStrategy. Currently, the return value is undefined for distributed training mode.

Raises:

  • ValueError: if environment variable TF_CONFIG is incorrectly set.

Below is an example for applying tf.estimator.train_and_evaluate to bert run_classifier.py:

# Create train_spec
train_input_fn = file_based_input_fn_builder(
    input_file=train_file,
    seq_length=FLAGS.max_seq_length,
    is_training=True,
    drop_remainder=True)
train_spec = tf.estimator.TrainSpec(
    input_fn=train_input_fn,
    max_steps=num_train_steps)
# Create eval_spec
exporters = [tf.estimator.BestExporter(
    name="best_exporter",
    serving_input_receiver_fn=serving_input_receiver_fn,
    exports_to_keep=5)]
eval_drop_remainder = True if FLAGS.use_tpu else False
eval_input_fn = file_based_input_fn_builder(
    input_file=eval_file,
    seq_length=FLAGS.max_seq_length,
    is_training=False,
    drop_remainder=eval_drop_remainder)
eval_spec = tf.estimator.EvalSpec(
    input_fn=eval_input_fn,
    steps=FLAGS.save_checkpoints_steps,
    exporters=exporters,
    start_delay_secs=0,
    throttle_secs=600)
# Call train_and_evaluate
eval_results, export_results = tf.estimator.train_and_evaluate(
    estimator,
    train_spec=train_spec,
    eval_spec=eval_spec)

Creating train_spec is simple, for creating eval_spec, besides input_fn, below args are worth mentioning:

  • steps: Int. Positive number of steps for which to evaluate model. If None, evaluates until input_fn raises an end-of-input exception. See Estimator.evaluate for details.

  • exporters: Iterable of Exporters, or a single one, or None. exporters will be invoked after each evaluation.

  • start_delay_secs: Int. Start evaluating after waiting for this many seconds.

  • throttle_secs: Int. Do not re-evaluate unless the last evaluation was started at least this many seconds ago. Of course, evaluation does not occur if no new checkpoints are available, hence, this is the minimum.

I set steps to FLAGS.save_checkpoints_steps so we can evaluate each checkpoint saved while training. BestExporter with default compare_fn and exports_to_keep will export 5 best checkpoints with least loss to saved model. I will write another article about implementing an Exporter based BestExporter which saves best checkpoint instead of exporting saved model.

https://www.jianshu.com/p/6fd5adb41b8f

Python量化投资网携手4326手游为资深游戏玩家推荐:《《封印战记》:次元兽研究计划 | 食尸鬼军团已锁定!

「点点赞赏,手留余香」

    还没有人赞赏,快来当第一个赞赏的人吧!
0 条回复 A 作者 M 管理员
    所有的伟大,都源于一个勇敢的开始!
欢迎您,新朋友,感谢参与互动!欢迎您 {{author}},您在本站有{{commentsCount}}条评论