mirror of
https://gitea.com/mcereda/oam.git
synced 2026-02-09 05:44:23 +00:00
chore(kb/aws/eks): dump findings discovered during testing
This commit is contained in:
Michele Cereda
@@ -11,6 +11,9 @@
|
||||
|
||||
## TL;DR
|
||||
|
||||
Do *not* use '--max-items' with '--query': the items limit is applied before the query filter, and could lead to no
|
||||
results.
|
||||
|
||||
<details>
|
||||
<summary>Installation and configuration</summary>
|
||||
|
||||
@@ -78,6 +81,12 @@ aws sns list-topics
|
||||
|
||||
# Get information about the current user.
|
||||
aws sts get-caller-identity
|
||||
|
||||
# List IAM users.
|
||||
aws iam list-users
|
||||
aws iam list-users --max-items '1'
|
||||
aws iam list-users --query "Users[?(UserName=='mario')]"
|
||||
aws iam list-users --query "Users[?(UserId=='AIDA…')].UserName"
|
||||
```
|
||||
|
||||
Subcommands not listed here are in their own service-specific article:
|
||||
@@ -142,7 +151,7 @@ See [CLI config files] for examples.
|
||||
## Session Manager integration
|
||||
|
||||
> The instance's IAM role must have at least the required permissions to allow to login.<br/>
|
||||
> The bare minimum is for it to have the _SSM Minimum_ role attached:
|
||||
> The bare minimum is for it to have the *SSM Minimum* role attached:
|
||||
>
|
||||
> ```sh
|
||||
> $ aws iam list-attached-role-policies --role-name 'whatevah'
|
||||
|
||||
@@ -83,7 +83,7 @@ To let other IAM principals have access to the cluster, one needs to add them to
|
||||
|
||||
## Requirements
|
||||
|
||||
- [suggestion] 1 (one) custom _Cluster Service Role_ with the `AmazonEKSClusterPolicy` policy attached or similar custom permissions.
|
||||
- \[suggestion] 1 (one) custom _Cluster Service Role_ with the `AmazonEKSClusterPolicy` policy attached or similar custom permissions.
|
||||
|
||||
Kubernetes clusters managed by EKS make calls to other AWS services on the user's behalf to manage the resources that the cluster uses.<br/>
|
||||
For a cluster to be allowed to make those calls, it **requires** to have the aforementioned permissions.
|
||||
@@ -111,7 +111,7 @@ To let other IAM principals have access to the cluster, one needs to add them to
|
||||
|
||||
</div>
|
||||
|
||||
- [suggestion] 1+ (one or more) custom service role(s) for the pod executors, with the required policies attached or similar permissions.
|
||||
- \[suggestion] 1+ (one or more) custom service role(s) for the pod executors, with the required policies attached or similar permissions.
|
||||
|
||||
The reasons and required permissions vary depending on the type of executor.<br/>
|
||||
It would probably be better to create a custom role instead of assigning permissions to the built-in one.
|
||||
@@ -222,6 +222,7 @@ Some create Cloudformation stacks in the process.
|
||||
</details>
|
||||
<br/>
|
||||
|
||||
1. [Give access to users][access management].
|
||||
1. Connect to the cluster.
|
||||
|
||||
```sh
|
||||
@@ -246,7 +247,7 @@ See [Choosing an Amazon EC2 instance type] and [Managed node groups] for more in
|
||||
|
||||
Additional requirements:
|
||||
|
||||
- [suggestion] 1 (one) custom _Node Service Role_ with the `AmazonEKSWorkerNodePolicy`, `AmazonEC2ContainerRegistryReadOnly` and `AmazonEKS_CNI_Policy` policies attached or similar permissions.
|
||||
- \[suggestion] 1 (one) custom _Node Service Role_ with the `AmazonEKSWorkerNodePolicy`, `AmazonEC2ContainerRegistryReadOnly` and `AmazonEKS_CNI_Policy` policies attached or similar permissions.
|
||||
|
||||
The EKS nodes' `kubelet` makes calls to the AWS APIs on one's behalf.<br/>
|
||||
Nodes receive permissions for these API calls through an IAM instance profile and associated policies.
|
||||
@@ -374,7 +375,7 @@ Procedure:
|
||||
|
||||
Additional requirements:
|
||||
|
||||
- [suggestion] 1 (one) custom _Fargate Service Role_ with the `AmazonEKSFargatePodExecutionRolePolicy` policy attached or similar permissions.
|
||||
- \[suggestion] 1 (one) custom _Fargate Service Role_ with the `AmazonEKSFargatePodExecutionRolePolicy` policy attached or similar permissions.
|
||||
|
||||
To create pods on Fargate, the components running on Fargate must make calls to the AWS APIs on one's behalf.<br/>
|
||||
This is so that it can take actions such as pull container images from ECR or route logs to other AWS services.
|
||||
@@ -506,6 +507,7 @@ See the following to allow others:
|
||||
- [Allowing IAM roles or users access to Kubernetes objects on your Amazon EKS cluster].
|
||||
- [How do I resolve the error "You must be logged in to the server (Unauthorized)" when I connect to the Amazon EKS API server?]
|
||||
- [Identity and Access Management]
|
||||
- [Using IAM Groups to manage Kubernetes cluster access]
|
||||
|
||||
## Secrets encryption through KMS
|
||||
|
||||
@@ -561,7 +563,7 @@ Procedure:
|
||||
1. Open the [runbook](https://console.aws.amazon.com/systems-manager/automation/execute/AWSSupport-TroubleshootEKSWorkerNode).
|
||||
1. Check that the AWS Region in the Management Console is set to the same Region as your cluster.
|
||||
1. In the Input parameters section, specify the name of the cluster and the EC2 instance ID.
|
||||
1. [optional] In the `AutomationAssumeRole` field, specify a role to allow Systems Manager to perform actions.<br/>
|
||||
1. \[optional] In the `AutomationAssumeRole` field, specify a role to allow Systems Manager to perform actions.<br/>
|
||||
If left empty, the permissions of your current IAM entity are used to perform the actions in the runbook.
|
||||
1. Choose `Execute`.
|
||||
1. Check the `Outputs` section.
|
||||
@@ -609,6 +611,7 @@ Debug: see [Identify common issues].
|
||||
-->
|
||||
|
||||
<!-- In-article sections -->
|
||||
[access management]: #access-management
|
||||
[create worker nodes]: #create-worker-nodes
|
||||
[identify common issues]: #identify-common-issues
|
||||
[requirements]: #requirements
|
||||
@@ -649,5 +652,6 @@ Debug: see [Identify common issues].
|
||||
[self-managed nodes]: https://docs.aws.amazon.com/eks/latest/userguide/worker.html
|
||||
[service-linked role permissions for amazon eks]: https://docs.aws.amazon.com/eks/latest/userguide/using-service-linked-roles-eks.html#service-linked-role-permissions-eks
|
||||
[using service-linked roles for amazon eks]: https://docs.aws.amazon.com/eks/latest/userguide/using-service-linked-roles.html
|
||||
[using iam groups to manage kubernetes cluster access]: https://archive.eksworkshop.com/beginner/091_iam-groups/
|
||||
|
||||
<!-- Others -->
|
||||
|
||||
@@ -25,7 +25,7 @@ Hosted by the [Cloud Native Computing Foundation][cncf].
|
||||
1. [Quality of service](#quality-of-service)
|
||||
1. [Containers with high privileges](#containers-with-high-privileges)
|
||||
1. [Capabilities](#capabilities)
|
||||
1. [Privileged containers vs privilege escalation](#privileged-containers-vs-privilege-escalation)
|
||||
1. [Privileged container vs privilege escalation](#privileged-container-vs-privilege-escalation)
|
||||
1. [Sysctl settings](#sysctl-settings)
|
||||
1. [Backup and restore](#backup-and-restore)
|
||||
1. [Managed Kubernetes Services](#managed-kubernetes-services)
|
||||
@@ -40,7 +40,7 @@ Hosted by the [Cloud Native Computing Foundation][cncf].
|
||||
1. [Create an admission webhook](#create-an-admission-webhook)
|
||||
1. [Prometheus on Kubernetes using Helm](#prometheus-on-kubernetes-using-helm)
|
||||
1. [Further readings](#further-readings)
|
||||
1. [Sources](#sources)
|
||||
1. [Sources](#sources)
|
||||
|
||||
## Basics
|
||||
|
||||
@@ -344,7 +344,7 @@ From the feature's `man` page:
|
||||
This also means a Container will be **limited** to its contents, plus the capabilities it has been assigned.
|
||||
|
||||
Some capabilities are assigned to all Containers by default, while others (the ones which could cause more issues) require to be **explicitly** set using the Containers' `securityContext.capabilities.add` property.<br/>
|
||||
If a Container is _privileged_ (see [Privileged container vs privilege escalation](#privileged-container-vs-privilege-escalation)), it will have access to **all** the capabilities, with no regards of what are explicitly assigned to it.
|
||||
If a Container is _privileged_ (see [Privileged container vs privilege escalation]), it will have access to **all** the capabilities, with no regards of what are explicitly assigned to it.
|
||||
|
||||
Check:
|
||||
|
||||
@@ -352,7 +352,7 @@ Check:
|
||||
- [Runtime privilege and Linux capabilities in Docker containers] for the capabilities available **inside Kubernetes**, and
|
||||
- [Container capabilities in Kubernetes] for a handy table associating capabilities in Kubernetes to their Linux variant.
|
||||
|
||||
### Privileged containers vs privilege escalation
|
||||
### Privileged container vs privilege escalation
|
||||
|
||||
A _privileged container_ is very different from a _container leveraging privilege escalation_.
|
||||
|
||||
@@ -596,10 +596,9 @@ Others:
|
||||
|
||||
- The [Build your very own self-hosting platform with Raspberry Pi and Kubernetes] series of articles
|
||||
- [Why separate your Kubernetes workload with nodepool segregation and affinity options]
|
||||
- [RBAC.dev]
|
||||
|
||||
## Sources
|
||||
|
||||
All the references in the [further readings] section, plus the following:
|
||||
### Sources
|
||||
|
||||
- Kubernetes' [concepts]
|
||||
- [How to run a command in a Pod after initialization]
|
||||
@@ -615,6 +614,8 @@ All the references in the [further readings] section, plus the following:
|
||||
- [Cloudzero Kubernetes best practices]
|
||||
- [Scaling K8S nodes without breaking the bank or your sanity - Brandon Wagner & Nick Tran, Amazon]
|
||||
- [Kubernetes Troubleshooting - The Complete Guide]
|
||||
- [Kubernetes cluster autoscaler]
|
||||
- [Common labels]
|
||||
|
||||
<!--
|
||||
References
|
||||
@@ -647,8 +648,8 @@ All the references in the [further readings] section, plus the following:
|
||||
[version skew policy]: https://kubernetes.io/releases/version-skew-policy/
|
||||
|
||||
<!-- In-article sections -->
|
||||
[further readings]: #further-readings
|
||||
[pods]: #pods
|
||||
[privileged container vs privilege escalation]: #privileged-container-vs-privilege-escalation
|
||||
|
||||
<!-- Knowledge base -->
|
||||
[azure kubernetes service]: ../cloud%20computing/azure/aks.md
|
||||
@@ -689,6 +690,7 @@ All the references in the [further readings] section, plus the following:
|
||||
[making sense of taints and tolerations]: https://medium.com/kubernetes-tutorials/making-sense-of-taints-and-tolerations-in-kubernetes-446e75010f4e
|
||||
[no_new_privs linux kernel documentation]: https://www.kernel.org/doc/Documentation/prctl/no_new_privs.txt
|
||||
[prestop hook doesn't work with env variables]: https://stackoverflow.com/questions/61929055/kubernetes-prestop-hook-doesnt-work-with-env-variables#62135231
|
||||
[rbac.dev]: https://rbac.dev/
|
||||
[read-only filesystem error]: https://stackoverflow.com/questions/49614034/kubernetes-deployment-read-only-filesystem-error/51478536#51478536
|
||||
[runtime privilege and linux capabilities in docker containers]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
|
||||
[why separate your kubernetes workload with nodepool segregation and affinity options]: https://medium.com/contino-engineering/why-separate-your-kubernetes-workload-with-nodepool-segregation-and-affinity-rules-cb5225953788
|
||||
|
||||
Reference in New Issue
Block a user