From 2b2dd91f0aa92f25eec79ccc88d9996c836a1fbf Mon Sep 17 00:00:00 2001 From: Michele Cereda Date: Mon, 11 Aug 2025 21:53:08 +0200 Subject: [PATCH] chore(kb/aws/eks): add gotchas for managed ec2 worker nodes --- knowledge base/cloud computing/aws/eks.md | 30 +++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/knowledge base/cloud computing/aws/eks.md b/knowledge base/cloud computing/aws/eks.md index 187caff..e037229 100644 --- a/knowledge base/cloud computing/aws/eks.md +++ b/knowledge base/cloud computing/aws/eks.md @@ -7,6 +7,7 @@ 1. [OIDC providers](#oidc-providers) 1. [Create worker nodes](#create-worker-nodes) 1. [Create managed node groups](#create-managed-node-groups) + 1. [Managed node groups' gotchas](#managed-node-groups-gotchas) 1. [Schedule pods on Fargate](#schedule-pods-on-fargate) 1. [Secrets encryption through KMS](#secrets-encryption-through-kms) 1. [Storage](#storage) @@ -64,6 +65,18 @@ roles by leveraging: - [OIDC providers], and/or - [Pod Identity]. +Nodes can have up to a [maximum number of pods][maximum number of pods per EKS instance].
+This number defaults to `N * (M-1) + 2`, where: + +- `N` is the number of Elastic Network Interfaces (ENI) of the instance type. +- `M` is the number of IP addresses of a single ENI. + +[Elastic network interfaces] and its children pages describe values for N and M for each instance type. + +[Since September 2021][amazon vpc cni plugin increases pods per node limits], it is possible to increase the default +maximum limit of pods per node when using the CNI plugin.
+Refer [Assign more IP addresses to Amazon EKS nodes with prefixes]. +
Usage @@ -562,6 +575,19 @@ Procedure:
+#### Managed node groups' gotchas + +- When using ARM-based instance types (e.g., `t4g`), one will need to specify the node group's AMI type since it + defaults to an X86 one.
+ Good examples are `AL2023_ARM_64_STANDARD` and `BOTTLEROCKET_ARM_64`. +- The `BOTTLEROCKET_ARM_64`-type AMIs will create 2 unencrypted EBS disks of type `gp2` by default: + + - A 2 GB root disk at `/dev/xvda`. + - A disk for EKS ephemeral storage at `/dev/xvdb` of whatever size is the default or is defined in the node group. + +- Specifying `blockDeviceMappings.ebs.noDevice="true"` in the node group's launch template to try and avoid creating + disks for ephemeral storage (e.g., the second disk for `BOTTLEROCKET_ARM_64`-type AMIs) has **no** effect. + ### Schedule pods on Fargate Additional requirements: @@ -1507,6 +1533,10 @@ helm upgrade -i --repo 'https://aws.github.io/eks-charts' \ [using iam groups to manage kubernetes cluster access]: https://archive.eksworkshop.com/beginner/091_iam-groups/ [using service-linked roles for amazon eks]: https://docs.aws.amazon.com/eks/latest/userguide/using-service-linked-roles.html [view resource usage with the kubernetesmetrics server]: https://docs.aws.amazon.com/eks/latest/userguide/metrics-server.html +[Maximum number of pods per EKS instance]: https://github.com/awslabs/amazon-eks-ami/blob/main/templates/shared/runtime/eni-max-pods.txt +[Elastic network interfaces]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html +[Amazon VPC CNI plugin increases pods per node limits]: https://aws.amazon.com/blogs/containers/amazon-vpc-cni-increases-pods-per-node-limits/ +[Assign more IP addresses to Amazon EKS nodes with prefixes]: https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html [amazon elastic block store (ebs) csi driver]: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/README.md