--- title: Accelerating AI with open source machine learning infrastructure description: The landscape of artificial intelligence is rapidly evolving, demanding robust and scalable infrastructure. To meet these challenges, we’ve developed a comprehensive reference architecture (RA) that leverages the power of open-source tools and cutting-edge hardware. This architecture, built on Canonical’s MicroK8s and Charmed Kubeflow, ru […] url: https://canonical.com/blog/accelerating-ai-with-open-source-machine-learning-infrastructure?format=md --- 1. [Blog](https://canonical.com/blog) 2. Article --- [Andreea Munteanu](https://canonical.com/blog/author/munteanuandreea "More about Andreea Munteanu") 20 March 2025 # Accelerating AI with open source machine learning infrastructure [Dell](https://canonical.com/blog/tag/dell) [Dell Technologies](https://canonical.com/blog/tag/dell-technologies-2) [Kubeflow](https://canonical.com/blog/tag/kubeflow) [kubernetes](https://canonical.com/blog/tag/kubernetes) [machine learning](https://canonical.com/blog/tag/machine-learning) [MicroK8s](https://canonical.com/blog/tag/microk8s) [NIM Microservices](https://canonical.com/blog/tag/nim-microservices) [nvidia](https://canonical.com/blog/tag/nvidia) [NVIDIA AI Enterprise](https://canonical.com/blog/tag/nvidia-ai-enterprise) [PowerEdge](https://canonical.com/blog/tag/poweredge) --- Share the article The landscape of artificial intelligence is rapidly evolving, demanding robust and scalable infrastructure. To meet these challenges, we’ve developed a comprehensive reference architecture (RA) that leverages the power of open-source tools and cutting-edge hardware. This architecture, built on Canonical’s MicroK8s and Charmed Kubeflow, running on [Dell PowerEdge](https://www.dell.com/en-us/shop/dell-poweredge-servers/sc/servers) R7525 servers, and accelerated by [NVIDIA NIM microservices](https://www.nvidia.com/en-us/ai-data-science/products/nim-microservices/), provides a streamlined path for deploying and managing machine learning workloads. ## Empowering data scientists and engineers This solution is designed to empower data scientists and machine learning engineers, enabling them to iterate faster, scale seamlessly, and maintain robust security. For infrastructure builders, solution architects, DevOps engineers, and CTOs, this RA offers a clear path to advance AI initiatives while addressing the complexities of large-scale deployments. At the heart of this architecture lies the synergy between Canonical and NVIDIA. Our collaboration ensures that the software stack, from Ubuntu Server and Ubuntu Pro to Charmed Kubeflow, is optimized for NVIDIA-Certified Systems. This integration delivers exceptional performance and reliability, allowing organizations to maximize their AI efficiency. ## Dell PowerEdge R7525: the foundation for high-performance AI The Dell PowerEdge R7525 server plays a crucial role in this architecture, providing the robust hardware foundation needed for demanding AI workloads. As a 2U rack server, it’s engineered for high-performance computing, virtualization, and data-intensive tasks. Featuring dual-socket AMD EPYC processors, the R7525 delivers exceptional scalability, advanced memory capabilities, and flexible storage options. This makes it ideal for AI and machine learning environments, where processing large datasets and complex models is essential. The R7525’s design ensures that organizations can virtualize traditional IT applications alongside transformative AI systems, providing a unified platform for diverse workloads. ## Leveraging NVIDIA NIM and A100 GPUs The architecture leverages NVIDIA NIM microservices included with [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/) software platform, for secure and reliable AI model inferencing. This, combined with the power of [NVIDIA A100 GPUs](https://www.nvidia.com/en-us/data-center/h100/), provides the necessary computational muscle for demanding AI workloads. By deploying an LLM with NVIDIA NIM on Charmed Kubeflow, organizations can seamlessly transition from model development to production. ## Canonical’s open-source components Canonical’s MicroK8s, a CNCF-certified Kubernetes distribution, provides a lightweight and efficient container orchestration platform. Charmed Kubeflow simplifies the deployment and management of AI workflows, offering an extensive ecosystem of tools and frameworks. This combination ensures a smooth and efficient machine learning lifecycle. ## Key benefits of the architecture The benefits of this architecture are numerous. Faster iteration, enhanced scalability, and robust security are just a few. The deep integrations between NVIDIA and Canonical ensure that the solution works seamlessly out of the box, with expedited bug fixes and prompt security updates. Moreover, the foundation of Ubuntu provides a secure and stable operating environment. This reference architecture is more than just a blueprint— it’s a practical guide. The document includes hardware specifications, software versions, and a step-by-step tutorial for deploying an LLM with NIM. It also addresses cluster monitoring and management, providing a holistic view of the system. ## Unlocking new opportunities By leveraging the combined expertise of Canonical, Dell, and NVIDIA, organizations can unlock new opportunities in their respective domains. This solution enhances data analytics, optimizes decision-making processes, and revolutionizes customer experiences. ## Get started today This RA is a solid foundation for deploying AI workloads. By combining Canonical, Dell, and NVIDIA’s expertise, organizations can enhance data analytics, optimize decision-making, and revolutionize customer experiences. As the RA concludes, organizations can confidently embrace this solution to drive innovation and accelerate AI adoption. Ready to elevate your AI initiatives? [Download it now](https://ubuntu.com/engage/ra-machine-learning-infrastructure) ### **Further Information** * * [https://microk8s.io](https://microk8s.io/) * * ## Sign up for our newsletter Get the latest Canonical news and updates in your inbox. Work email: \*I agree to receive information about Canonical's products and services. By submitting this form, I confirm that I have read and agree to [Canonical's Privacy Policy](https://canonical.com/legal/dataprivacy). Sign up ## Share on --- ## Related posts [### How to set up a micro lab: four principles for a reliable homelab](https://canonical.com/blog/how-to-set-up-a-reliable-homelab) After over a decade of running a homelab, I have learned a few difficult lessons. Although it begins as a “lab,” you inevitably end up with something you want to keep. If a... [Jake Nabasny](https://canonical.com/blog/author/slapcat) 17 March 2026 [### Canonical announces Ubuntu support for the NVIDIA Rubin platform](https://canonical.com/blog/nvidia-vera-rubin-ubuntu-support) Official Ubuntu support for the NVIDIA Rubin platform, including the NVIDIA Vera Rubin NVL72 rack-scale systems, announced at CES 2026 CES 2026, Las Vegas. – Canonical, the... [Canonical](https://canonical.com/blog/author/canonical) 5 January 2026 [### Canonical joins the Open Secure AI Alliance](https://canonical.com/blog/open-secure-ai-alliance) Canonical is now part of the Open Secure AI Alliance, announced by NVIDIA with partners across cloud computing, cybersecurity, enterprise software, open source foundations, and... [Canonical](https://canonical.com/blog/author/canonical) 28 August 2026 [### Securing AI agent workflows on Ubuntu with the new NVIDIA OpenShell snap](https://canonical.com/blog/nvidia-openshell-ubuntu-announcement) By packaging OpenShell as a snap, Canonical is enabling enterprises to confidently run next-generation agentic workflows across local devices, hybrid environments, and private clouds. [Canonical](https://canonical.com/blog/author/canonical) 1 June 2026