Compare how contextual embeddings change meaning based on context
I went to the bank to deposit money.
The river bank was covered with flowers.
She sat on the bank of the stream fishing.
The bank approved our loan application.
We walked along the muddy river bank.
• Fixed representation regardless of context
• Trained on large corpus (Word2Vec, GloVe)
• One embedding per word type
• Cannot handle polysemy effectively
Contextual models capture different word senses
Static embeddings have fixed representations
BERT uses bidirectional attention
Context variance indicates sensitivity
Identify correct meaning in context
Better handling of ambiguous words
Context-aware understanding