What is OpenSearch?
OpenSearch is an open source search and analytics suite that developers use to build solutions for search, data observability, data ingestion, Security Event and Information Management (SIEM), vector database and more. It is designed for scalability, offering powerful full-text search capabilities and supporting various data types, including structured and unstructured data. OpenSearch has rapidly developed into a standalone platform with unique features and capabilities.
Contact usabout OpenSearch for the enterprise Datasheet
Why choose OpenSearch
-
Transactionality and speed
-
Scalability and high availability
-
Data security
Why do companies use OpenSearch?
Advanced analytics
OpenSearch provides powerful analytics capabilities, enabling companies to analyse large datasets in real-time. This is useful for monitoring trends, generating business insights, and making data-driven decisions.
The flexibility of open source
As an open source project, OpenSearch offers the flexibility to customise and extend the software according to specific business needs without licensing costs. This also encourages a community-driven approach to development and innovation.
Compatibility and integration
OpenSearch is compatible with a variety of data sources and can integrate with other tools and platforms. This interoperability makes it easy to incorporate OpenSearch into your existing technology stack.
How do companies use OpenSearch?
Search
OpenSearch enhances your website or e-commerce search capabilities with full-text querying, autocomplete, scroll search, and customisable scoring and ranking.
Analytics and machine learning
You can use OpenSearch in multiple analytics solutions such as events analytics, trace analytics, and machine learning, which uses algorithms such as anomaly detection and data clustering.
Security
Security information and event management (SIEM) solutions can use OpenSearch to investigate, detect, analyse, and respond to security threats that can jeopardise the success of businesses and organisations and their online operations.
Observability
You can use OpenSearch to create observability applications through the OpenSearch Dashboard. You can also use it to schedule, export and share reports.
How does OpenSearch work?
Distributed search and analytics engine
OpenSearch Service offers real-time document search capabilities that surpass traditional database search. Based on Lucene, it is a portable, platform-agnostic, open source search engine supporting various features like keyword search, natural language processing, synonyms, and multiple languages.
Its capabilities include:
- Acquiring data from databases and content management systems
- Providing search APIs
- Enabling searches across numerous attributes
- Utilising built-in machine learning (ML) algorithms for k-nearest neighbours (k-NN) search, facilitating vector search, similarity search, semantic search and more
- Applying built-in ML algorithms for Learning to Rank to compute relevance scores
- Supporting multiple query languages, including SQL
OpenSearch’s distributed design means that users and applications interact with OpenSearch clusters. Each cluster is a collection of one or more nodes running on servers that store data and process search requests. OpenSearch can also run on a local laptop with minimal system requirements, so it’s easy to get started.
The diagram below illustrates an example of an OpenSearch cluster, displaying OpenSearch nodes, OpenSearch Dashboard, and data sources.
End users can interact directly with the OpenSearch Dashboard, for example to perform data analysis tasks in order to improve business processes. However, before users can access the Dashboard, data sources need to be ingested into the OpenSearch cluster. This data source can be in different formats like log files, metrics, JSON documents, etc.
A cluster can contain various types of nodes: main, coordinating and data nodes. Each node has a different role:
Cluster managers
Manage the overall operation of a cluster and keep track of the cluster state. This includes creating and deleting indexes, keeping track of the nodes that join and leave the cluster, checking the health of each node in the cluster (by running ping requests), and allocating shards to nodes.
Data nodes
Store and search data. These nodes perform all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of a cluster and need more disk space than any other node type.
Coordinating nodes
Delegate client requests to shards on the data nodes, collect and aggregate the results into one final result, and send this result back to the client. Coordinating nodes manage outside requests like the OpenSearch Dashboard and other client libraries.
Feature breakdown
Search engine
Data must be indexed before it can be searched. Indexing is the process that search engines use to organise data for quick retrieval, creating a structure known as an index.
In OpenSearch, the fundamental unit of data is a JSON document, and each document within an index is identified by a unique ID.
To improve OpenSearch's indexing capabilities, it uses the Index State Management (ISM) plugin. This plugin automates periodic administrative tasks by triggering actions based on the index's age, size, or document count. With the ISM plugin, you can create policies that automatically manage index rollovers or deletions according to your specific needs.
OpenSearch Dashboard
OpenSearch Dashboard is an open source, integrated visualisation tool that allows users to explore their data in OpenSearch. From real-time application monitoring, threat detection, and incident management to personalised search, OpenSearch Dashboards represent trends, outliers, and patterns in data graphically.
Vector database
Utilising OpenSearch as a vector database enables you to combine traditional search, analytics, and vector search in one solution. OpenSearch's vector database capabilities have the potential to speed up the development of artificial intelligence (AI) by serving as a knowledge base. It can be used in various use cases such as semantic search, multimodal search, and more.
The role of OpenSearch vector databases in LLM applications
Vector databases facilitate efficient data representation, retrieval and manipulation, enabling AI systems to generate high-fidelity outputs across various domains, from natural language processing to image synthesis.
This webinar discusses various concepts, such as generative AI, retrieval augmented generation (RAG), the importance of search engines like OpenSearch, and efficient open source tooling that enables developers and enthusiasts to build their generative AI applications.
OpenSearch plugins
OpenSearch has several features and plugins to help index, secure, monitor and analyse data. Most OpenSearch plugins have associated OpenSearch Dashboard plugins that provide a convenient, unified user interface.
- Anomaly detection – Identify atypical data and receive automatic notifications
- KNN – Find “nearest neighbours” in your vector data
- Performance Analyzer – Monitor and optimise your cluster
- SQL – Use SQL or a piped processing language to query your data
- Index State Management – Automate index operations
- ML Commons plugin – Train and execute machine-learning models
- Asynchronous search – Run search requests in the background
- Cross-cluster replication – Replicate your data across multiple OpenSearch clusters
Canonical’s Charmed OpenSearch
Secure and automate the deployment, maintenance and upgrades of your search and analytics suite across private and public clouds.
Charmed OpenSearch
Included in Ubuntu Pro + Support
When you purchase an Ubuntu Pro + Support plan, you also get support for the full Charmed OpenSearch solution.
- Up to 10 years of OpenSearch support per release track
- 24/7 or weekday phone and ticket support
- Up to 10 years of security maintenance for OpenSearch covering critical and high severity CVEs
Charmed OpenSearch allows you to automate deployment and operation of OpenSearch at web scale in the environment of your choice – on the cloud or in your data centre.
OpenSearch Rock container image
Included in Ubuntu Pro + Support
Also included in Ubuntu Pro + Support, you get support for Canonical’s container image OpenSearch, based on Ubuntu LTS. So solid and secure, we call it a Rock.
- Up to 10 years of support per release track
- Same 24/7 or weekday phone and ticket support commitment
- Same 10 years of security maintenance covering critical and high severity CVEs in the image
OpenSearch consultancy and support
Advanced professional services for OpenSearch, when you need them
Get help designing, planning and building and even operating a hyper automated production OpenSearch service that perfectly fits your needs, with Canonical’s expert services.
- Help with design and build of both production and non-production OpenSearch environments with Charmed OpenSearch
- Managed services for OpenSearch in your cloud tenancy or data centre, backed by an SLA
- Firefighting support with a OpenSearch operations expert, who works alongside your team when crisis hits
Installing Charmed OpenSearch
Charmed OpenSearch is operated via Juju, an open source orchestration engine for software operators that makes OpenSearch easy to operate.
Learn more about Opensearch
Future-proof AI applications with OpenSearch as a vector database
This webinar explains the role of vector databases in LLMs. In addition, it gives an overview of how OpenSearch functions for vector embedding storage and search.
Large Language Models (LLMs) Retrieval Augmented Generation (RAG) using Charmed OpenSearch
Retrieval-augmented generation (RAG) is a method that enables users to converse with data repositories. It’s a tool that can revolutionise how you access and utilise data. Charmed OpenSearch is a simple and robust technology that can enable RAG capabilities.
What is NoSQL and what are database operators?
Introduction to database operators and the value proposition of different operators, such as Charmed OpenSearch.
OpenSearch is a registered trademark of Amazon Web Services. Other trademarks are property of their respective owners. Charmed OpenSearch is not sponsored, endorsed, or affiliated with Amazon Web Services.