The total amount of digital data generated worldwide is increasing at a rapid rate. Approximately 80% of this newly generated data is unstructured data – data that does not conform to a table- or object-based model. Examples of unstructured data include text, images, protein structures, geospatial information, and IoT data streams. Despite this, most organizations lack effective methods for storing and analyzing these vast quantities of unstructured data. Embeddings – high-dimensional, dense vectors that represent the semantic content of unstructured data – can remedy this.
In this tutorial, we’ll introduce embeddings and vector search from both an ML- and application-level perspective. We’ll start with a high-level overview of embeddings and vector search and discuss best practices around embedding generation and usage. We’ll then apply this knowledge to build two systems: semantic text search and reverse image search. Finally, we’ll explore how we can put our application into production using Milvus, the world’s most popular open-source vector database.
Director of Operations & ML Architect at Zilliz