Blog posts tagged
"Big Data"

robgibbon
17 October 2023

Why we built a Spark solution for Kubernetes

Data Platform Article

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported distribution of Apache Spark and optimises it for deployment to Kubernetes, which is where most of the industry is moving these days. Reimagining how to work with big data ...

Canonical
17 October 2023

Canonical announces supported solution for Apache Spark® on Kubernetes

Canonical announcements Article

17 October 2023 Today, Canonical announced the release of Charmed Spark – an advanced solution for Apache Spark® that provides everything users need to run Apache Spark on Kubernetes. Apache Spark is suitable for use in diverse data processing applications including predictive analytics, data warehousing, machine learning data preparatio ...

robgibbon
10 August 2023

Write a Spark big data job with ChatGPT

AI Article

I’ve read and watched more than a few articles about ChatGPT in the last couple of months. It seems the large language model AI hype machine just can’t stop. As somebody with a passion for music production, some of the more interesting things I’ve seen included a guy using ChatGPT to build a virtual effect ...

robgibbon
3 July 2023

Charmed Spark beta release is out – try it today

AI Article

The Canonical Data Fabric team is pleased to announce the first beta release of Charmed Spark, our solution for Apache Spark. Apache Spark is a free, open source software framework for developing distributed, parallel processing jobs. It’s popular with data engineers and data scientists alike when building data pipelines for both batch an ...

robgibbon
3 May 2023

Big data security foundations in five steps

Data Platform Article

We’ve all read the headlines about spectacular data breaches and other security incidents, and the impact that they have had on the victim organisations. And in some ways there’s no place more vulnerable to attack than a big data environment like a data lake. ...

robgibbon
16 November 2022

Apache Kafka service design for low latency and no data loss

Apps Article

Designing a production service environment around Apache Kafka that delivers low latency and zero-data loss at scale is non-trivial. Indeed, it’s the holy grail of messaging systems. In this blog post, I’ll outline some of the fundamental service design considerations that you’ll need to take into account in order to get your service arch ...

robgibbon
31 August 2022

Kubernetes operators – the top 5 things to watch for

Charms Article

Software operators are steadily revolutionising how we deploy and run complex distributed systems. They offer the promise of low-intervention, self-driving software – ideally leading to service reliability gains and better uptime. For an introduction to Kubernetes operators, check out our introductory webinar or download our guide to Kube ...

robgibbon
6 December 2021

Canonical Data Platform 2021 winter roundup

AI Article

Canonical Data Platform: that was 2021 It’s that time of the year again: many folks are panic buying cans of windscreen de-icer spray and thermal underwear, bringing pine trees into the front room and preparing to enjoy an extended break with the family. So we thought to ourselves, what better time than now to take ...

robgibbon
10 November 2021

SQL Server on Ubuntu Pro: bringing it all back home

Cloud and server Article

Not going to lie, the Microsoft SQL Server is my all-time favourite Microsoft product. For a long time, SQL Server was only available for Windows, but not much is really sacred. So now Microsoft, in collaboration with Canonical, are distributing and supporting several flavours of SQL Server on Ubuntu Pro for Azure. I’m going to ...

robgibbon
26 October 2021

In defence of pet servers

Apps Article

We all know the drill by now: modern compute infrastructure needs to be deterministic, disposable, commoditised and repeatable. We’re all farmers now, and our server estates must be treated like cattle – ready for slaughter at a moment’s notice. However, we must remember that the driver behind the new design rationale is primarily the unr ...

robgibbon
30 August 2021

Cloud PaaS through the lens of open source – opinion

Data Systems Article

Opinion piece by Rob Gibbon – Product Manager at Canonical. All views expressed are the author’s own. The open source perspective viz. PaaS Open source software, as the name suggests, is developed in the open. The software can be freely inspected by anyone, and can be freely patched as required to suit the security requirements ...

robgibbon
18 August 2021

How to run Apache Spark on MicroK8s and Ubuntu Core, in the cloud: Part 4

Cloud and server Article

In this series, we’ve been building up an Apache Spark cluster on Kubernetes using MicroK8s, Ubuntu Core OS, LXD and GCP. We’ve learned about and set up nested virtualisation on the cloud, and had some fun. But right, it’s retrospective time: in Part 1, we saw how to get MicroK8s up on LXD, on Ubuntu ...

Blog posts tagged "Big Data"

Blog posts tagged
"Big Data"