Empty predictions?

[ioana-blue]

I see quite often the following error. See below for a stack trace (ignore the line numbers, I have a local copy of the code that has debug prints, more output, etc.). It looks as if there are no predictions performed (since targets is non-empty) and the code dies in rouge.

Traceback (most recent call last):
  File "train_and_eval.py", line 607, in <module>
    main()
  File "train_and_eval.py", line 175, in main
    train_and_eval(model, args)
  File "train_and_eval.py", line 284, in train_and_eval
    valid_targets)
  File "train_and_eval.py", line 335, in evaluate
    rouge = compute_rouge(valid_predictions, targets)
  File "train_and_eval.py", line 568, in compute_rouge
    scores = rouge.get_scores(hyps=predictions, refs=targets, avg=True)
  File "/opt/conda/lib/python3.6/site-packages/rouge/rouge.py", line 86, in get_scores
    return self._get_avg_scores(hyps, refs)
  File "/opt/conda/lib/python3.6/site-packages/rouge/rouge.py", line 112, in _get_avg_scores
    sc = fn(hyp, ref)
  File "/opt/conda/lib/python3.6/site-packages/rouge/rouge.py", line 55, in <lambda>
    "rouge-1": lambda hyp, ref: rouge_score.rouge_n(hyp, ref, 1),
  File "/opt/conda/lib/python3.6/site-packages/rouge/rouge_score.py", line 159, in rouge_n
    raise ValueError("Collections must contain at least 1 sentence.")
ValueError: Collections must contain at least 1 sentence.

Any hunch why that would happen?

[ioana-blue]

Funnily enough, I get this only when I use learning rate decay.

[ioana-blue]

Not sure why this error happens. I set a try/catch and I printed the number of predictions/targets, they are both non-zero (and equal, as expected). One thing that might happen is some prediction to be non-empty, but really empty (as in whitespaces?)

[BUCTwangkun]

Not sure why this error happens. I set a try/catch and I printed the number of predictions/targets, they are both non-zero (and equal, as expected). One thing that might happen is some prediction to be non-empty, but really empty (as in whitespaces?)

Have you solved the problem? My predicetions is always same. Just like:

prediction: test performed target: set
prediction: test performed target: set
prediction: test performed target: cpy
prediction: test performed target: len
prediction: test performed target: len
prediction: test performed target: to string
prediction: test performed target: set euler angles
prediction: test performed target: set euler angles rad
prediction: test performed target: get gimbal pole
prediction: test performed target: get roll rad
prediction: test performed target: get roll
prediction: test performed target: get pitch rad
prediction: test performed target: get pitch
prediction: test performed target: get yaw rad
prediction: test performed target: get yaw