Modern semantic search does not have to require a separate vector database. Data architects, database engineers, developers, or platform leaders can integrate Vertex AI and Cloud SQL vector indexes directly into their existing MySQL workflow.
DBTA recently held a webinar, Why Your Existing MySQL Instance Is Ready for Semantic Search, with Isabella Lubin, group product manager at Google Cloud and Shu Zhou, engineer at Google Cloud, who explored how to bring embeddings and vector search capabilities into Cloud SQL for MySQL, enabling semantic queries and generative AI use cases without introducing additional infrastructure.
According to Lubin and Zhou, Vector embeddings are numerical representations of text, images, and complex data types mapped to n-dimensional space [generated by embedding models]. Vector search/Similarity search is used to find relevant and similar content (distance between two vectors). And Vector indexing speeds up the search by implementing approximate nearest neighbor (ANN) search.
Vector search use cases include:
Retail: identifying best products for a natural language search
Financial Services: fraud detection
Healthcare: identifying potential illnesses based on patient symptoms
Vector search enables you to process more ambiguous inputs and identify potential matches. This means that almost every application has the potential to introduce vector search—the question becomes what the requirements for accuracy are, and the cost to transform and maintain, Lubin and Zhou explained.
Vectors work in Cloud SQL by enabling customers to generate embeddings for their existing data without the need to go outside of the database. Vectors and their indexes are managed and utilized as normal database objects, supporting full transactional semantics and crash safety. Customers can perform ANN searches in a native SQL way, supporting both relational and vector predicates.
Things to consider when designing your application include:
- The right time to switch to ANN from KNN is all about query performance and accuracy for your application. With Cloud SQL for MySQL, you can’t add an index until you have at least 1000 rows.
- Most embedding models are not interchangeable/backwards-compatible. If you switch to a new model, you’ll need to re-calculate embeddings for existing data.
- The right number of partitions for your index might change as your data size increases.
For the full webinar, featuring a more in-depth discussion, Q&A, and more, you can view an archived version of the webinar here.