- Natural Language Processing with TensorFlow
- Thushan Ganegedara
- 312字
- 2021-06-25 21:28:22
Chapter 3. Word2vec – Learning Word Embeddings
In this chapter, we will discuss a topic of paramount importance in NLP—Word2vec, a technique to learn word embeddings or distributed numerical feature representations (that is, vectors) of words. Learning word representations lies at the very foundation of many NLP tasks because many NLP tasks rely on good feature representations for words that preserve their semantics as well as their context in a language. For example, the feature representation of the word forest should be very different from oven as these words are rarely used in similar contexts, whereas the representations of forest and jungle should be very similar.
Note
Word2vec is called a distributed representation, as the semantics of the word is captured by the activation pattern of the full representation vector, in contrast to a single element of the representation vector (for example, setting a single element in the vector to 1 and rest to 0 for a single word).
We will go step by step from the classical approach to solving this problem to modern neural network-based methods that deliver state-of-the-art performance in finding good word representations. We visualize (using t-SNE, a visualization technique for high-dimensional data) such learned word embeddings for a set of words on a 2D canvas in Figure 3.1. If you take a closer look, you will see that similar things are placed close to each other (for example, numbers in the cluster in the middle):

Figure 3.1: An example visualization of learned word embeddings using t-SNE
Note
t-Distributed Stochastic Neighbor Embedding (t-SNE)
This is a dimensionality reduction technique that projects high-dimensional data to a two-dimensional space. This allows us to imagine how high-dimensional data is distributed in space, and it is quite useful as we cannot visualize beyond three dimensions easily. You will learn about t-SNE in more detail in the next chapter.
- JavaScript語言精髓與編程實踐(第3版)
- 區(qū)塊鏈:以太坊DApp開發(fā)實戰(zhàn)
- Getting Started with SQL Server 2012 Cube Development
- HTML5 and CSS3 Transition,Transformation,and Animation
- JavaScript 程序設(shè)計案例教程
- QGIS By Example
- Advanced Oracle PL/SQL Developer's Guide(Second Edition)
- FPGA Verilog開發(fā)實戰(zhàn)指南:基于Intel Cyclone IV(進階篇)
- C++從入門到精通(第5版)
- Java EE企業(yè)級應(yīng)用開發(fā)教程(Spring+Spring MVC+MyBatis)
- Python 3.7從入門到精通(視頻教學(xué)版)
- 軟件測試綜合技術(shù)
- SQL Server 2008 R2數(shù)據(jù)庫技術(shù)及應(yīng)用(第3版)
- 大學(xué)計算機基礎(chǔ)實訓(xùn)教程
- Python Digital Forensics Cookbook