Getting Started with Amazon EKS Anywhere: A Practical Guide for On-Premise Kubernetes Deployment
AWS Community Builder / Cloud Architect / IT Lead / MLOps
Introduction
As businesses increasingly move towards hybrid and multi-cloud environments, managing infrastructure across multiple platforms has become more complex. However, Amazon Web Services (AWS) has introduced a game-changer for organizations that want the power and flexibility of Kubernetes on their on-premise infrastructure. This is where Amazon EKS Anywhere comes into play. In this article, we’ll explore what EKS Anywhere is, its benefits, and how you can set up and manage Kubernetes clusters on your own on-prem servers using VMware vSphere.
Having recently tested EKS Anywhere with my on-prem servers, I can confidently say that it streamlines the process of deploying and managing Kubernetes clusters without the need for complicated third-party tools. Let's walk through the process, from setup to deployment, with some real-world examples.
What is Amazon EKS Anywhere?
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service provided by AWS for running containerized applications. While it simplifies the Kubernetes management process, it traditionally required cloud infrastructure like AWS EC2 instances. However, with EKS Anywhere, AWS now offers a deployment option for customers to create and manage Kubernetes clusters on their on-premise hardware.
The key benefits of EKS Anywhere are:
Consistent Management Experience: It offers the same management tools and experience as Amazon EKS in the AWS Cloud.
Open Source: Built on Amazon EKS Distro, an open-source Kubernetes distribution, it allows users to deploy Kubernetes clusters with minimal effort.
Integration with AWS Tools: Seamlessly integrates with AWS services like AWS Systems Manager (SSM) for monitoring and operations.
In essence, EKS Anywhere allows you to run Kubernetes on your existing infrastructure, ensuring you still benefit from the rich ecosystem and features AWS provides.
Key Features of EKS Anywhere
Hardware Support: It runs on your own hardware or on VMware vSphere, making it ideal for on-premise deployments.
Control Plane: Unlike EKS, where the control plane is managed by AWS, with EKS Anywhere, you manage the control plane yourself.
Cluster Lifecycle Management: EKS Anywhere includes tooling for automating cluster creation, scaling, updates, and even the destruction of Kubernetes clusters.
AWS Integration: Easily view and manage your on-prem Kubernetes clusters using the EKS console, integrating seamlessly with AWS Cloud services.
Support for Third-party Tools: EKS Anywhere supports integrations with tools like Flux for GitOps, eksctl for cluster management, and Cilium for networking.
Setting Up EKS Anywhere on VMware vSphere
For this guide, I’ll walk you through setting up an EKS Anywhere cluster on your on-prem VMware vSphere infrastructure. While you can set up a test cluster on your desktop, here we focus on a more realistic production setup.
Prerequisites:
VMware vSphere version 7.0 or higher.
EKS Anywhere tools installed on your machine.
At least three control plane nodes and three worker nodes for high availability.
Step 1: Install EKS Anywhere CLI Tools
Start by installing the necessary CLI tools. On a Mac, you can do this via Homebrew.
$ brew install aws/tap/eks-anywhere
$ eksctl anywhere version
v0.5.0
Step 2: Generate Cluster Config and Create a Cluster
Let’s create a Kubernetes cluster using eksctl. First, you need to generate a cluster configuration file.
$ CLUSTER_NAME=my-eks-cluster
$ eksctl anywhere generate clusterconfig $CLUSTER_NAME --provider vsphere > $CLUSTER_NAME.yaml
Now that we have the configuration, we can create the cluster on vSphere.
$ eksctl anywhere create cluster -f $CLUSTER_NAME.yaml
The CLI will handle the setup of the control plane, the worker nodes, and the networking components for your cluster. Once the cluster is created, it will be fully operational, and you can use kubectl to interact with it.
Step 3: Export Kubeconfig and Deploy a Test App
Once the cluster is created, you'll have a kubeconfig file to connect to your Kubernetes cluster:
$ export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
$ kubectl get ns
You can now deploy a simple test application to verify everything is working:
$ kubectl apply -f "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml"
$ kubectl get pods -l app=hello-eks-a
This will deploy a basic pod that you can access locally:
$ kubectl port-forward deploy/hello-eks-a 8000:80
$ curl localhost:8000
You should see a simple “Hello from EKS Anywhere” message, confirming that the cluster is up and running.
Managing Your Cluster: High Availability and Updates
In a production environment, you’ll want to ensure high availability and smooth updates for your clusters. EKS Anywhere allows you to scale your cluster as needed and manage rolling updates.
For high availability, it's recommended to have at least three control plane nodes and three worker nodes. You can scale the cluster using:
$ eksctl anywhere scale cluster --control-plane-nodes 3 --worker-nodes 3
To update the cluster, use the built-in update tools provided by EKS Anywhere, which work much like the updates on AWS-managed EKS clusters. The update process ensures that your cluster remains stable during the upgrade, even with multiple nodes.
Using EKS Connector for Centralized Management
One of the standout features of EKS Anywhere is EKS Connector, which allows you to manage your on-prem clusters directly from the EKS console. This makes it easy to view and monitor all your Kubernetes clusters, whether they’re running on AWS or on-prem.
To connect your EKS Anywhere cluster to the EKS console:
Register the cluster through the EKS console.
Download and apply the necessary eks-connector.yaml configuration to your cluster.
Once applied, your cluster will be available in the AWS Management Console for monitoring and management.
$ kubectl apply -f eks-connector.yaml
This allows you to manage your on-prem clusters alongside your AWS-based clusters in a single interface.
Conclusion
Amazon EKS Anywhere has made managing on-prem Kubernetes clusters much simpler by bringing AWS-level tools and support to local infrastructures. Whether you're running on VMware vSphere or other compatible environments, EKS Anywhere allows you to benefit from a consistent, simplified management experience, without the need for complex, third-party tools. It also integrates seamlessly with AWS services, making it easy to monitor and scale your infrastructure.
If you're looking to bring Kubernetes to your on-prem servers, EKS Anywhere is an excellent choice that I would highly recommend based on my recent hands-on testing.
