When Kubernetes first got here onto the scene, it was a significant turning level, a revision of the infrastructure and operations house that reworked the best way builders and ops personnel construct, deploy, and preserve purposes within the cloud. It has since develop into the clear commonplace for a way fashionable purposes are constructed and operated. Because the CNCF famous in its newest Annual Cloud Native Survey report, “Amongst container customers, 82% are utilizing Kubernetes in manufacturing in 2025, up from 66% in 2023. This represents near-universal adoption throughout the container ecosystem.”
Over the previous few years, one other revision within the house has occurred with Kubernetes’s evolution from a container orchestrator to an AI infrastructure platform. In keeping with the CNCF survey, “The rise of Kubernetes because the de facto AI platform represents a elementary shift in how organizations method machine studying operations. . .[with Kubernetes] offering a unified orchestration layer that handles each conventional software workloads and compute-intensive AI duties.” The emergence of seismic applied sciences like generative AI and agentic AI has solely accelerated this transformation.
The intersection of AI with Kubernetes is undoubtedly some of the impactful developments within the operations house. As Jonathan Johnson, software program architect at Dijure, observes, “AI on K8s could be very, essential, and there’s not sufficient [resources] on the market.” Raju Gandhi, senior technical architect at Edward Jones, echoes this evaluation, noting that “operationalizing AI/ML on K8s is an enormous difficulty, [and it’s only] getting larger. This can be a subject that wants consideration.” However what are a number of the issues that it’s best to find out about this development to maintain abreast and keep forward within the recreation?
Generative AI
Anybody with entry to a pc or a smartphone has possible used some iteration of generative AI, a surprising truth when you think about that GenAI was on the outer edges of mainstream discourse and consumption a scant 5 years in the past. However on the finish of 2022, the debut of ChatGPT marked the start of a technological revolution, one that might affect and reshape practically each side of our working and private lives. Unsurprisingly, there at the moment are 1000’s of generative AI fashions, a proliferation that naturally has its personal set of complexities. Deciding on a mannequin is easy, however should you’re an software developer or MLOps engineer, how do you go about working that mannequin in a manufacturing system? Not solely do you must be cognizant of things like resilience, scalability, safety, and operational prices, however there’s the truth that bringing a mannequin from experimentation into manufacturing might be arduous if not achieved correctly. That’s the place Kubernetes comes into play.
As Roland Huß and Daniele Zonca, distinguished engineers at Purple Hat, word, “GenAI/LLM fashions are useful resource intensive, requiring substantial computational energy and huge datasets. Given its scalability and extensibility, Kubernetes is uniquely suited to operate as an environment friendly platform for AI and LLM mannequin pretraining, fine-tuning, deployment, and immediate engineering.” They additional elaborate that “this integration with Kubernetes not solely simplifies the adoption of cutting-edge AI applied sciences but in addition ensures a seamless and environment friendly operational stream. Kubernetes, with its strong scalability and administration capabilities, stands as a really perfect platform for generative AI tasks, aligning DevOps and MLOps practices in a cohesive ecosystem.”
This sentiment is already shared by a large swath of the business. In keeping with the CNCF survey above, as of 2025, 66% of organizations run generative AI workloads on Kubernetes. These organizations embrace OpenAI, which makes use of Kubernetes for its AI/LLM software experimenting and testing; Tesla, which makes use of KServe to handle production-grade LLM inference; and Adobe, which makes use of Kubernetes to energy its suite of generative artistic fashions. Different corporations taking this method embrace Uber, Intuit, and Google. With extra corporations adopting this observe for his or her generative AI and LLMs operations, it’d be prudent for any group to leverage Kubernetes for their very own GenAI and LLM workflows.
Agentic AI
Practically coinciding with the rise of GenAI has been the regular progress of agentic AI. Not like GenAI, agentic AI goes past answering easy prompts and producing textual content in its capability to function autonomously to carry out complicated, multistep actions, make the most of instruments, and make impartial selections. With its capability to help each conventional ML processes and GenAI and LLM operations, it ought to come as no shock that Kubernetes has a job within the agentic AI ecosystem as properly.
In keeping with Ronald Petty, principal advisor at RX-M, “Kubernetes has been leveraged to host machine studying pipelines, together with AI mannequin coaching and inference. As inference choices have develop into plentiful and inexpensive, on and off-premise, now we have seen the rise of brokers. Coupling cloud native applied sciences and standard protocols, we now see brokers shifting from advert hoc demos to complicated fleets of brokers on programs like Kubernetes.” So what are some examples of the mixing between these two applied sciences?
One notable providing is Kagent, an OS programming framework that runs AI brokers in Kubernetes and “helps engineers construct highly effective inside platforms by tackling cloud native duties reminiscent of configuration, troubleshooting, complicated deployment situations, observability pipelines and dashboards, and safely enabling community safety.” Working alongside comparable strains is K8sGPT, an AI-powered device that leverages clever insights and automatic troubleshooting to investigate Kubernetes clusters for configuration issues and safety points, in addition to generates options to issues found in evaluation.
A more moderen entry within the area is Sympozium, a Kubernetes-native coordination layer for multi-agent AI programs that “solves the identical drawback Kubernetes solved for containers, however for brokers that must share context, hand off duties, and preserve shared situational consciousness.” One other newer providing is Agent Sandbox, which lets you run AI brokers as remoted, stateful workloads with a local API on Kubernetes.
The basics
Whereas it’s necessary to pay attention to the newest developments and traits affecting your area, that shouldn’t come on the expense of foundational information and abilities. As basketball nice Michael Jordan as soon as stated, “Get the basics down and the extent of all the pieces you do will rise.” One of the crucial elementary abilities for working with Kubernetes is networking, and frustratingly sufficient, it’s one of many tougher ones to grasp. As Cisco senior workers engineer Nico Vibert observes, “Platform engineers are usually snug with Linux networking however much less so with protocols like BGP and IPv6; community directors know these protocols properly however discover Kubernetes abstractions unfamiliar. Each personas battle to navigate the handfuls of networking instruments seemingly required to satisfy connectivity and safety necessities.” But as organizations transfer mission-critical workloads, AI coaching pipelines, and controlled monetary providers onto Kubernetes, the engineers who can design, safe, and troubleshoot the community layer have develop into a number of the most sought-after professionals within the business.
In recognition of each the significance and troublesome nature of the Kubernetes networking talent, the CNCF lately introduced a brand new certification centered on the Kubernetes community engineer position. The certification is designed to validate hands-on networking experience throughout the entire aforementioned layers, filling a niche that the Kubernetes group has lengthy acknowledged.
For organizations that use Kubernetes to develop and ship purposes, leaders and decision-makers should be conscious that using Kubernetes at the side of the newest AI instruments is not a luxurious however a mandatory observe that may permit their corporations to thrive. The same onus must be positioned on the fundamentals. When hiring your subsequent DevOps, community, or website reliability engineer, make sure that their capability to design, safe, and troubleshoot the Kubernetes community layer is second to none.
If you wish to dive deeper, try Roland Huß and Daniele Zonca’s Generative AI on KubernetesJonathan Johnson’s GPU Kubernetes Homelab dwell course, Alex Corvin, Taneem Ibrahim, and Kyle Stratis’s Scalable Kubernetes Infrastructure for AI PlatformsAshok Srirama and Sukirti Gupta’s Kubernetes for Generative AI Optionsand Yogesh Raheja’s K8sGPT Necessities on-demand course. They’re all on O’Reilly. If you happen to’re not a member, you will get began with a free trial.
