Run production-grade Apache Spark® workloads on Kubernetes
snap install spark-client snap install microk8s microk8s enable dns rbac microk8s.kubectl create namespace spark spark-client.service-account-registry create --username spark
--namespace spark --conf spark.executor.instances=3 spark-client.spark-shell
Simplify the provisioning and maintenance of Apache Spark on Kubernetes with Charmed Spark. Run it on any Public or Private Cloud.
Try it now Contact us
The best way to run Spark
Adopting Kubernetes as your Spark cluster resource manager grants you access to the huge community as well as the innovation, scalability and flexibility that Kubernetes has to offer.
- Easy deployment and integration versus legacy Hadoop YARN
- Advanced, pluggable networking options, including Cilium, Calico, Contrail and Flannel
- Support for industry standard authentication protocols including OIDC
- Converge your big data workloads with other applications on the same infrastructure
- Get the same user experience in your lab as you get in the cloud with MicroK8s
If you're already using Spark on Kubernetes, Charmed Spark from Canonical offers a drop in replacement for upstream Spark.
New users can get up and running with Charmed Spark on MicroK8s in just a few commands.
Access the docs
Charmed Spark features
Native Kubernetes support
Charmed Spark runs on CNCF conformant Kubernetes including AWS EKS and Canonical Kubernetes.
Cost-effective
Charmed Spark is free to use on any Kubernetes platform. Pay only for support.
Security
Includes up to 10 years of security maintenance per track with coverage for critical CVEs.
Management tooling
Includes additional tooling for managing Spark on Kubernetes including observability integration, the spark8t toolkit and a Juju operator for Spark History Server.
Frequently asked questions
Is this a drop-in replacement for Apache Spark?
Charmed Spark can be used as a direct replacement for upstream Apache Spark with no or minimal application changes.
Can I run Charmed Spark on any Kubernetes?
You can run Charmed Spark on CNCF conformant Kubernetes.
Can I run Charmed Spark on Hadoop?
Charmed Spark is designed ground up for Kubernetes and does not support deployment to Hadoop.
Do you offer support?
We offer technical support with a choice of 24/7 or weekday SLAs.
Can I use Charmed Spark for free?
Charmed Spark is free to use. Pay only for support.
Can you deliver a complete solution?
We also offer a complete solution for big data based on Canonical Kubernetes and Charmed Ceph.
Request a demo
Learn more
Watch
Webinar on Spark security for big data operations
Uncover best practices and practical steps to Spark security.
Download
Guide to implementing big data solutions with open source
A security-maintained and fully supported solution for Apache Spark on Kubernetes.
Discover
Learn how our our Kubernetes solutions can accelerate data lake deployments in hybrid cloud environments.
Apache®, Apache Spark, Spark®, and the Spark logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.