How to use BERT embedding?

[nashid]

Is it possible to use BERT word embeddings along with this NMT implementation?

The goal is to use a pre-trained BERT language model so the contextualized embedding could be leveraged.

I am wondering whether anyone implemented or was able to run this model with any other contextualized embedding like ELMO or BERT.

[nashid]

@mommi84 any idea?

[namavar-marjane]

That would be a very useful feature.

[nashid]

This would be very useful as contextual embedding became a norm nowadays. Wondering anyone implemented or was able to run this model with any contextualized embedding?