Enhancing context through contrast
This project aimed to introduce a context-enhancement step after pre-training to enhance source-sentence representations for translation. The goal was to learn an encoder that projects embeddings from different languages into a shared semantic space, effectively capturing meaning. To achieve this, we employed a contrastive learning strategy without relying on explicit augmentations of the parallel sentence dataset.
This work was accepted at the ML Pre-registration Workshop at NeurIPS 2022.