--- title: Overview | OpenSearch description: What is OpenSearch? An open source search and analytics engine for log aggregation, monitoring, and data exploration. url: https://canonical.com/data/opensearch/what-is-opensearch?format=md --- # What is OpenSearch? OpenSearch is an open source search and analytics suite that developers use to build solutions for search, data observability, data ingestion, Security Event and Information Management (SIEM), vector database and more. It is designed for scalability, offering powerful full-text search capabilities and supporting various data types, including structured and unstructured data. OpenSearch has rapidly developed into a standalone platform with unique features and capabilities. --- [Contact us](https://canonical.com/contact-us) [Datasheet](https://assets.ubuntu.com/v1/6dc3440a-Charmed%20OpenSearch%20-%20Datasheet%20-%20amended%2004%2006%202024.pdf) --- ## Why choose OpenSearch * Transactionality and speed * Scalability and high availability * Data security --- ## Why do companies use OpenSearch? --- ### Advanced analytics OpenSearch provides powerful analytics capabilities, enabling companies to analyse large datasets in real-time. This is useful for monitoring trends, generating business insights, and making data-driven decisions. --- ### The flexibility of open source As an open source project, OpenSearch offers the flexibility to customise and extend the software according to specific business needs without licensing costs. This also encourages a community-driven approach to development and innovation. --- ### Compatibility and integration OpenSearch is compatible with a variety of data sources and can integrate with other tools and platforms. This interoperability makes it easy to incorporate OpenSearch into your existing technology stack. --- ## How do companies use OpenSearch? --- ##### Search OpenSearch enhances your website or e-commerce search capabilities with full-text querying, autocomplete, scroll search, and customisable scoring and ranking. --- ##### Analytics and machine learning You can use OpenSearch in multiple analytics solutions such as events analytics, trace analytics, and machine learning, which uses algorithms such as anomaly detection and data clustering. --- ##### Security Security information and event management (SIEM) solutions can use OpenSearch to investigate, detect, analyse, and respond to security threats that can jeopardise the success of businesses and organisations and their online operations. --- ##### Observability You can use OpenSearch to create observability applications through the OpenSearch Dashboard. You can also use it to schedule, export and share reports. --- ## How does OpenSearch work? --- ### Distributed search and analytics engine OpenSearch Service offers real-time document search capabilities that surpass traditional database search. Based on Lucene, it is a portable, platform-agnostic, open source search engine supporting various features like keyword search, natural language processing, synonyms, and multiple languages. Its capabilities include: --- * Acquiring data from databases and content management systems * Providing search APIs * Enabling searches across numerous attributes * Utilising built-in machine learning (ML) algorithms for k-nearest neighbours (k-NN) search, facilitating vector search, similarity search, semantic search and more * Applying built-in ML algorithms for Learning to Rank to compute relevance scores * Supporting multiple query languages, including SQL --- OpenSearch’s distributed design means that users and applications interact with OpenSearch clusters. Each cluster is a collection of one or more nodes running on servers that store data and process search requests. OpenSearch can also run on a local laptop with minimal system requirements, so it’s easy to get started. The diagram below illustrates an example of an OpenSearch cluster, displaying OpenSearch nodes, OpenSearch Dashboard, and data sources. End users can interact directly with the OpenSearch Dashboard, for example to perform data analysis tasks in order to improve business processes. However, before users can access the Dashboard, data sources need to be ingested into the OpenSearch cluster. This data source can be in different formats like log files, metrics, JSON documents, etc. A cluster can contain various types of nodes: main, coordinating and data nodes. Each node has a different role: --- ### Cluster managers Manage the overall operation of a cluster and keep track of the cluster state. This includes creating and deleting indexes, keeping track of the nodes that join and leave the cluster, checking the health of each node in the cluster (by running ping requests), and allocating shards to nodes. --- ### Data nodes Store and search data. These nodes perform all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of a cluster and need more disk space than any other node type. --- ### Coordinating nodes Delegate client requests to shards on the data nodes, collect and aggregate the results into one final result, and send this result back to the client. Coordinating nodes manage outside requests like the OpenSearch Dashboard and other client libraries. --- ## Feature breakdown --- ### Search engine Data must be indexed before it can be searched. Indexing is the process that search engines use to organise data for quick retrieval, creating a structure known as an index. In OpenSearch, the fundamental unit of data is a JSON document, and each document within an index is identified by a unique ID. To improve OpenSearch's indexing capabilities, it uses the Index State Management (ISM) plugin. This plugin automates periodic administrative tasks by triggering actions based on the index's age, size, or document count. With the ISM plugin, you can create policies that automatically manage index rollovers or deletions according to your specific needs. --- ### OpenSearch Dashboard [OpenSearch Dashboard](https://github.com/opensearch-project/OpenSearch-Dashboards) is an open source, integrated visualisation tool that allows users to explore their data in OpenSearch. From real-time application monitoring, threat detection, and incident management to personalised search, OpenSearch Dashboards represent trends, outliers, and patterns in data graphically. --- ### Vector database Utilising OpenSearch as a vector database enables you to combine traditional search, analytics, and vector search in one solution. OpenSearch's vector database capabilities have the potential to speed up the development of artificial intelligence (AI) by serving as a knowledge base. It can be used in various use cases such as semantic search, multimodal search, and more. --- ## The role of OpenSearch vector databases in LLM applications Vector databases facilitate efficient data representation, retrieval and manipulation, enabling AI systems to generate high-fidelity outputs across various domains, from natural language processing to image synthesis. This webinar discusses various concepts, such as generative AI, retrieval augmented generation (RAG), the importance of search engines like OpenSearch, and efficient open source tooling that enables developers and enthusiasts to build their generative AI applications. --- [Watch the webinar to learn more  ›](https://www.youtube.com/watch?v=G5XzI_NwZwk) --- ## OpenSearch plugins OpenSearch has several features and plugins to help index, secure, monitor and analyse data. Most OpenSearch plugins have associated OpenSearch Dashboard plugins that provide a convenient, unified user interface. * [Anomaly detection](https://opensearch.org/docs/latest/monitoring-plugins/ad/) – Identify atypical data and receive automatic notifications * [KNN](https://opensearch.org/docs/latest/search-plugins/knn/) – Find “nearest neighbours” in your vector data * [Performance Analyzer](https://opensearch.org/docs/latest/monitoring-plugins/pa/) – Monitor and optimise your cluster * [SQL](https://opensearch.org/docs/latest/search-plugins/sql/) – Use SQL or a piped processing language to query your data * [Index State Management](https://opensearch.org/docs/latest/im-plugin/) – Automate index operations * [ML Commons plugin](https://opensearch.org/docs/latest/ml-commons-plugin/index/) – Train and execute machine-learning models * [Asynchronous search](https://opensearch.org/docs/latest/search-plugins/async/) – Run search requests in the background * [Cross-cluster replication](https://opensearch.org/docs/latest/replication-plugin/index/) – Replicate your data across multiple OpenSearch clusters --- ## Canonical’s Charmed OpenSearch Secure and automate the deployment, maintenance and upgrades of your search and analytics suite across private and public clouds. --- [Contact Us](https://canonical.com/data/opensearch#get-in-touch) [Datasheet](https://assets.ubuntu.com/v1/6dc3440a-Charmed%20OpenSearch%20-%20Datasheet%20-%20amended%2004%2006%202024.pdf?_gl=1*76e20f*_gcl_au*MTY4OTkyMTA1My4xNzE2NTQxNjg0) --- ### Charmed OpenSearch Included in [Ubuntu Pro](https://ubuntu.com/pro) + Support When you purchase an Ubuntu Pro + Support plan, you also get support for the full Charmed OpenSearch solution. --- * Up to 15 years of OpenSearch support per release track * 24/7 or weekday phone and ticket support * Up to 15 years of security maintenance for OpenSearch covering critical and high severity CVEs --- Charmed OpenSearch allows you to automate deployment and operation of OpenSearch at web scale in the environment of your choice – on the cloud or in your data centre. --- [Explore Managed OpenSearch Solutions ›](https://canonical.com/data/OpenSearch/managed) --- ### OpenSearch OCI-compliant container image Included in Ubuntu Pro + Support Also included in Ubuntu Pro + Support, you get support for Canonical’s OCI-compliant container image for OpenSearch, based on Ubuntu LTS. --- * Up to 15 years of support per release track * Same 24/7 or weekday phone and ticket support commitment * Same 15 years of security maintenance covering critical and high severity CVEs in the image --- ### OpenSearch consultancy and support ### Advanced professional services for OpenSearch, when you need them Get help designing, planning and building and even operating a hyper automated production OpenSearch service that perfectly fits your needs, with Canonical’s expert services. --- * Help with design and build of both production and non-production OpenSearch environments with Charmed OpenSearch * Managed services for OpenSearch in your cloud tenancy or data centre, backed by an SLA * Firefighting support with a OpenSearch operations expert, who works alongside your team when crisis hits --- [Access the datasheet  ›](https://assets.ubuntu.com/v1/4220f815-Data%20Solutions%20Advisory%20version%20_updated%20-%20v3-2.pdf) --- ## Installing Charmed OpenSearch Charmed OpenSearch is operated via Juju, an open source orchestration engine for software operators that makes OpenSearch easy to operate. [Access the tutorial here ›](https://canonical.com/data/opensearch/docs/latest) --- ## Learn more about Opensearch [Future-proof AI applications with OpenSearch as a vector database](https://ubuntu.com/engage/opensearch-ai-webinar) This webinar explains the role of vector databases in LLMs. In addition, it gives an overview of how OpenSearch functions for vector embedding storage and search. --- [Large Language Models (LLMs) Retrieval Augmented Generation (RAG) using Charmed OpenSearch](https://ubuntu.com/blog/vector-database-large-language-models-opensearch) Retrieval-augmented generation (RAG) is a method that enables users to converse with data repositories. It’s a tool that can revolutionise how you access and utilise data. Charmed OpenSearch is a simple and robust technology that can enable RAG capabilities. --- [What is NoSQL and what are database operators?](https://ubuntu.com/blog/what-is-nosql) Introduction to database operators and the value proposition of different operators, such as Charmed OpenSearch. --- OpenSearch is a registered trademark of Amazon Web Services. Other trademarks are property of their respective owners. Charmed OpenSearch is not sponsored, endorsed, or affiliated with Amazon Web Services. Close ### Talk to our OpenSearch® experts --- Fill in this form and a member of our team will be in touch shortly.