diff --git a/knowledge base/cloud computing/aws/cli.md b/knowledge base/cloud computing/aws/cli.md index cebe75a..4ba3ffb 100644 --- a/knowledge base/cloud computing/aws/cli.md +++ b/knowledge base/cloud computing/aws/cli.md @@ -11,11 +11,13 @@ ## TL;DR +
+ Installation and configuration + ```sh # Install the CLI. brew install 'awscli' - # Configure profiles. aws configure aws configure --profile 'work' @@ -23,12 +25,17 @@ aws configure --profile 'work' # Use specific profiles for the rest of the shell session. export AWS_PROFILE='work' - # Enable auto-prompt mode (like `aws-shell` does). aws configure set 'cli_auto_prompt' 'on-partial' export AWS_CLI_AUTO_PROMPT='on' +``` +
+
+ Usage + +```sh # List applications in CodeDeploy. aws deploy list-applications @@ -40,7 +47,6 @@ aws deploy get-deployment-group --application-name 'batman' \ --deployment-group-name 'production' - # Show RDS instances. aws rds describe-db-instances aws rds describe-db-instances --output 'json' --query "DBInstances[?(DBInstanceIdentifier=='master-prod')]" @@ -70,14 +76,37 @@ aws secretsmanager get-secret-value --secret-id 'ecr-pullthroughcache/github' aws sns list-topics -# Start sessions via Session Manager. -aws ssm start-session --target 'i-0123456789abcdef0' +# Get information about the current user. +aws sts get-caller-identity ``` -Non listed subcommand: +Subcommands not listed here are in their own service-specific article: -- [`aws ecr`][ecr tldr] -- [`aws s3`][s3 tldr] +[`ebs`][ebs tldr] | +[`ec2`][ec2 tldr] | +[`ecr`][ecr tldr] | +[`eks`][eks tldr] | +[`s3`][s3 tldr] | +[`ssm`][ssm tldr] + +
+ +
+ Real world use cases + +```sh +# Get roles' ARN from their name. +aws iam list-roles --query "Roles[?RoleName == 'EKSRole'].[RoleName, Arn]" + +# Assume roles given their name. +aws iam list-roles --query "Roles[?RoleName == 'EKSRole'].Arn" --output 'text' \ +| xargs -I {} \ + aws sts assume-role \ + --role-arn "{}" \ + --role-session-name "AWSCLI-Session" +``` + +
## Profiles @@ -150,6 +179,7 @@ aws ssm start-session --target 'i-0123456789abcdef0' - [AWS] - CLI [quickstart] - [Configure profiles] in the CLI +- [How do I assume an IAM role using the AWS CLI?] ### Sources @@ -162,14 +192,19 @@ aws ssm start-session --target 'i-0123456789abcdef0' [aws]: README.md +[ebs tldr]: ebs.md#tldr +[ec2 tldr]: ec2.md#tldr [ecr tldr]: ecr.md#tldr +[eks tldr]: eks.md#tldr [s3 tldr]: s3.md#tldr +[ssm tldr]: ssm.md#tldr [cli config files]: ../../../examples/dotfiles/.aws [configure profiles]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html +[how do i assume an iam role using the aws cli?]: https://repost.aws/knowledge-center/iam-assume-role-cli [improved cli auto-prompt mode]: https://github.com/aws/aws-cli/issues/5664 [install the session manager plugin for the aws cli]: https://docs.aws.amazon.com/systems-manager/latest/userguide/install-plugin-macos-overview.html#install-plugin-macos-signed [quickstart]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html diff --git a/knowledge base/cloud computing/aws/eks.md b/knowledge base/cloud computing/aws/eks.md index f3bb2c8..f761d4a 100644 --- a/knowledge base/cloud computing/aws/eks.md +++ b/knowledge base/cloud computing/aws/eks.md @@ -6,8 +6,11 @@ 1. [Create worker nodes](#create-worker-nodes) 1. [Create managed node groups](#create-managed-node-groups) 1. [Schedule pods on Fargate](#schedule-pods-on-fargate) +1. [Access management](#access-management) 1. [Secrets encryption through KMS](#secrets-encryption-through-kms) 1. [Troubleshooting](#troubleshooting) + 1. [Identify common issues](#identify-common-issues) + 1. [The worker nodes fail to join the cluster.](#the-worker-nodes-fail-to-join-the-cluster) 1. [Further readings](#further-readings) 1. [Sources](#sources) @@ -132,14 +135,14 @@ Some create Cloudformation stacks in the process. ```json { - "Version": "2012-10-17", - "Statement": [{ - "Effect": "Allow", - "Action": "sts:AssumeRole", - "Principal": { - "Service": "eks.amazonaws.com" - } - }] + "Version": "2012-10-17", + "Statement": [{ + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Principal": { + "Service": "eks.amazonaws.com" + } + }] } ``` @@ -147,6 +150,9 @@ Some create Cloudformation stacks in the process. aws iam create-role \ --role-name 'DeepThinker' \ --assume-role-policy-document 'file://eks-cluster-role-trust-policy.json' + aws iam attach-role-policy \ + --role-name 'DeepThinker' \ + --policy-arn 'arn:aws:iam::aws:policy/AmazonEKSClusterPolicy' ``` @@ -156,20 +162,24 @@ Some create Cloudformation stacks in the process. ```ts const cluster_assumeRole_policy = JSON.stringify({ - Version: "2012-10-17", - Statement: [{ - Effect: "Allow", - Action: "sts:AssumeRole", - Principal: { - Service: "eks.amazonaws.com", - }, - }], + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Action: "sts:AssumeRole", + Principal: { + Service: "eks.amazonaws.com", + }, + }], }); const cluster_service_role = new aws.iam.Role("cluster-service-role", { - assumeRolePolicy: cluster_assumeRole_policy, - name: "DeepThinker", - … + assumeRolePolicy: cluster_assumeRole_policy, + managedPolicyArns: [ + // alternatively, use RolePolicyAttachments + "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy", + ], + name: "DeepThinker", + … }); ``` @@ -195,15 +205,15 @@ Some create Cloudformation stacks in the process. ```ts const cluster = new aws.eks.Cluster("cluster", { - name: "DeepThought", - roleArn: cluster_service_role.arn, - vpcConfig: { - subnetIds: [ - "subnet-11112222333344445", - "subnet-66667777888899990", - ], - }, - … + name: "DeepThought", + roleArn: cluster_service_role.arn, + vpcConfig: { + subnetIds: [ + "subnet-11112222333344445", + "subnet-66667777888899990", + ], + }, + … }); ``` @@ -261,16 +271,16 @@ Procedure: ```json { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": "sts:AssumeRole", - "Principal": { - "Service": "ec2.amazonaws.com" - } - } - ] + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Principal": { + "Service": "ec2.amazonaws.com" + } + } + ] } ``` @@ -296,26 +306,26 @@ Procedure: ```ts const nodes_assumeRole_policy = JSON.stringify({ - Version: "2012-10-17", - Statement: [{ - Effect: "Allow", - Action: "sts:AssumeRole", - Principal: { - Service: "ec2.amazonaws.com", - }, - }], + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Action: "sts:AssumeRole", + Principal: { + Service: "ec2.amazonaws.com", + }, + }], }); const node_service_role = new aws.iam.Role("node-service-role", { - assumeRolePolicy: nodes_assumeRole_policy, - managedPolicyArns: [ - // alternatively, use RolePolicyAttachments - "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly", - "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy", - "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy", - ], - name: "DeepThinkerNode", - … + assumeRolePolicy: nodes_assumeRole_policy, + managedPolicyArns: [ + // alternatively, use RolePolicyAttachments + "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly", + "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy", + "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy", + ], + name: "DeepThinkerNode", + … }); ``` @@ -343,16 +353,16 @@ Procedure: ```ts const nodeGroup_alpha = new aws.eks.NodeGroup("nodeGroup-alpha", { - nodeGroupName: "nodeGroup-alpha", - clusterName: cluster.name, - nodeRoleArn: node_service_role.arn, - scalingConfig: { - minSize: 1, - maxSize: 3, - desiredSize: 1, - }, - subnetIds: cluster.vpcConfig.subnetIds, - … + nodeGroupName: "nodeGroup-alpha", + clusterName: cluster.name, + nodeRoleArn: node_service_role.arn, + scalingConfig: { + minSize: 1, + maxSize: 3, + desiredSize: 1, + }, + subnetIds: cluster.vpcConfig.subnetIds, + … }); ``` @@ -384,21 +394,21 @@ Procedure: ```json { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": "sts:AssumeRole", - "Principal": { - "Service": "eks-fargate-pods.amazonaws.com" - }, - "Condition": { - "ArnLike": { - "aws:SourceArn": "arn:aws:eks:region-code:111122223333:fargateprofile/my-cluster/*" - } - } - } - ] + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Principal": { + "Service": "eks-fargate-pods.amazonaws.com" + }, + "Condition": { + "ArnLike": { + "aws:SourceArn": "arn:aws:eks:region-code:111122223333:fargateprofile/my-cluster/*" + } + } + } + ] } ``` @@ -418,29 +428,29 @@ Procedure: ```ts const fargate_assumeRole_policy = JSON.stringify({ - Version: "2012-10-17", - Statement: [{ - Effect: "Allow", - Action: "sts:AssumeRole", - Principal: { - Service: "eks-fargate-pods.amazonaws.com", - }, - Condition: { - ArnLike: { - "aws:SourceArn": `arn:aws:eks:${region}:${account}:fargateprofile/${cluster.name}/*` - } - }, - }], + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Action: "sts:AssumeRole", + Principal: { + Service: "eks-fargate-pods.amazonaws.com", + }, + Condition: { + ArnLike: { + "aws:SourceArn": `arn:aws:eks:${region}:${account}:fargateprofile/${cluster.name}/*` + } + }, + }], }); const fargate_service_role = new aws.iam.Role("fargate-service-role", { - assumeRolePolicy: fargate_assumeRole_policy, - managedPolicyArns: [ - // alternatively, use RolePolicyAttachments - "arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy", - ], - name: "DeepThinkerFargate", - … + assumeRolePolicy: fargate_assumeRole_policy, + managedPolicyArns: [ + // alternatively, use RolePolicyAttachments + "arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy", + ], + name: "DeepThinkerFargate", + … }); ``` @@ -468,20 +478,33 @@ Procedure: ```ts const fargateProfile_alpha = new aws.eks.FargateProfile("fargateProfile-alpha", { - fargateProfileName: "fargateProfile-alpha", - clusterName: cluster.name, - podExecutionRoleArn: fargate_service_role.arn, - selectors: [ - { namespace: "monitoring" }, - { namespace: "default" }, - ], - subnetIds: cluster.vpcConfig.subnetIds, - … + fargateProfileName: "fargateProfile-alpha", + clusterName: cluster.name, + podExecutionRoleArn: fargate_service_role.arn, + selectors: [ + { namespace: "monitoring" }, + { namespace: "default" }, + ], + subnetIds: cluster.vpcConfig.subnetIds, + … }); ``` +## Access management + +By default, the IAM principal creating the cluster is the only one able to make calls to the cluster's API server.
+To let other IAM principals have access to the cluster, one needs to add them to it. + +See the following to allow others: + +- [Required permissions to view EKS resources]. +- [Enabling IAM principal access to your cluster]. +- [Allowing IAM roles or users access to Kubernetes objects on your Amazon EKS cluster]. +- [How do I resolve the error "You must be logged in to the server (Unauthorized)" when I connect to the Amazon EKS API server?] +- https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html + ## Secrets encryption through KMS See [Enabling secret encryption on an existing cluster]. @@ -510,11 +533,11 @@ TL;DR: ```ts const cluster = new aws.eks.Cluster("cluster", { - encryptionConfig: { - provider: { keyArn: `arn:aws:kms:${region}:${account}:key/${key_id}` }, - resources: [ "secrets" ], - }, - … + encryptionConfig: { + provider: { keyArn: `arn:aws:kms:${region}:${account}:key/${key_id}` }, + resources: [ "secrets" ], + }, + … }); ``` @@ -524,12 +547,41 @@ TL;DR: See [Amazon EKS troubleshooting]. +### Identify common issues + +Use the [AWSSupport-TroubleshootEKSWorkerNode](https://docs.aws.amazon.com/systems-manager-automation-runbooks/latest/userguide/automation-awssupport-troubleshooteksworkernode.html) runbook. + +> For the automation to work, worker nodes **must** have permission to access Systems Manager and have Systems Manager running.
+> Grant this permission by attaching the [`AmazonSSMManagedInstanceCore`](https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-instance-profile.html#instance-profile-policies-overview) policy to the node role. + +Procedure: + +1. Open the [runbook](https://console.aws.amazon.com/systems-manager/automation/execute/AWSSupport-TroubleshootEKSWorkerNode). +1. Check that the AWS Region in the Management Console is set to the same Region as your cluster. +1. In the Input parameters section, specify the name of the cluster and the EC2 instance ID. +1. [optional] In the `AutomationAssumeRole` field, specify a role to allow Systems Manager to perform actions.
+ If left empty, the permissions of your current IAM entity are used to perform the actions in the runbook. +1. Choose `Execute`. +1. Check the `Outputs` section. + +### The worker nodes fail to join the cluster. + +Error message example: + +> NodeCreationFailure: Instances failed to join the kubernetes cluster. + +Debug: see [Identify common issues]. + ## Further readings - [Kubernetes] - [EKS Workshop] - [Pulumi] - [Terraform] +- [How can I get my worker nodes to join my Amazon EKS cluster?] +- [Enabling IAM principal access to your cluster] +- [Allowing IAM roles or users access to Kubernetes objects on your Amazon EKS cluster] +- [How do I resolve the error "You must be logged in to the server (Unauthorized)" when I connect to the Amazon EKS API server?] ### Sources @@ -547,6 +599,7 @@ See [Amazon EKS troubleshooting]. - [Enabling secret encryption on an existing cluster] - [Choosing an Amazon EC2 instance type] - [Private cluster requirements] +- [De-mystifying cluster networking for Amazon EKS worker nodes] [create worker nodes]: #create-worker-nodes +[identify common issues]: #identify-common-issues [requirements]: #requirements [secrets encryption through kms]: #secrets-encryption-through-kms @@ -564,6 +618,7 @@ See [Amazon EKS troubleshooting]. +[allowing iam roles or users access to kubernetes objects on your amazon eks cluster]: https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html [amazon eks add-ons]: https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html [amazon eks cluster iam role]: https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html [amazon eks clusters]: https://docs.aws.amazon.com/eks/latest/userguide/clusters.html @@ -576,12 +631,17 @@ See [Amazon EKS troubleshooting]. [aws eks create-fargate-profile]: https://docs.aws.amazon.com/cli/latest/reference/eks/create-fargate-profile.html [aws eks create-nodegroup]: https://docs.aws.amazon.com/cli/latest/reference/eks/create-nodegroup.html [choosing an amazon ec2 instance type]: https://docs.aws.amazon.com/eks/latest/userguide/choosing-instance-type.html +[de-mystifying cluster networking for amazon eks worker nodes]: https://aws.amazon.com/blogs/containers/de-mystifying-cluster-networking-for-amazon-eks-worker-nodes/ [eks workshop]: https://www.eksworkshop.com/ +[enabling iam principal access to your cluster]: https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html [enabling secret encryption on an existing cluster]: https://docs.aws.amazon.com/eks/latest/userguide/enable-kms.html [fargate]: https://docs.aws.amazon.com/eks/latest/userguide/fargate.html [getting started with amazon eks - aws management console and aws cli]: https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html +[how can i get my worker nodes to join my amazon eks cluster?]: https://repost.aws/knowledge-center/eks-worker-nodes-cluster +[how do i resolve the error "you must be logged in to the server (unauthorized)" when i connect to the amazon eks api server?]: https://repost.aws/knowledge-center/eks-api-server-unauthorized-error [managed node groups]: https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html [private cluster requirements]: https://docs.aws.amazon.com/eks/latest/userguide/private-clusters.html +[required permissions to view eks resources]: https://docs.aws.amazon.com/eks/latest/userguide/view-kubernetes-resources.html#view-kubernetes-resources-permissions [self-managed nodes]: https://docs.aws.amazon.com/eks/latest/userguide/worker.html [service-linked role permissions for amazon eks]: https://docs.aws.amazon.com/eks/latest/userguide/using-service-linked-roles-eks.html#service-linked-role-permissions-eks [using service-linked roles for amazon eks]: https://docs.aws.amazon.com/eks/latest/userguide/using-service-linked-roles.html diff --git a/knowledge base/pulumi.md b/knowledge base/pulumi.md index e7a8993..cf5b6c9 100644 --- a/knowledge base/pulumi.md +++ b/knowledge base/pulumi.md @@ -175,14 +175,31 @@ pulumi state unprotect 'resourceUrn' Data resources ```ts -const cluster_role = aws.iam.getRoleOutput({ - name: "AWSServiceRoleForAmazonEKS", -}); - +const cluster_role = aws.iam.getRoleOutput({ name: "AWSServiceRoleForAmazonEKS" }); const cluster = new aws.eks.Cluster("cluster", { roleArn: cluster_role.arn, … }); + +// If used in JSON documents, the function needs to cover the whole document. +const encryptionKey = aws.kms.getKeyOutput({ + keyId: "00001111-2222-3333-4444-555566667777", +}); +const clusterServiceRole = new aws.iam.Role("clusterServiceRole", { + inlinePolicies: [{ + policy: encryptionKey.arn.apply(arn => JSON.stringify({ + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Action: [ + "kms:CreateGrant", + "kms:DescribeKey", + ], + Resource: arn, + }], + })), + }] +}); ``` @@ -259,6 +276,42 @@ yq -iy '. += {"backend": {"url": "s3://myBucket/prefix"}}' 'Pulumi.yaml' # TODO ``` +```ts +// Merge objects. +tags_base = { + ManagedBy: "Pulumi", + Prod: false, +}; +const fargateProfile = new aws.eks.FargateProfile("fargateProfile", { + tags: { + ...tags_base, + ...{ + Description: "Fargate profile for EKS cluster EksTest", + EksComponent: "Fargate profile", + Name: "eksTest-fargateProfile", + }, + }, + … +}); + +// Default tags with explicit provider. +const provider = new aws.Provider("provider", { + defaultTags: { + tags: { + ManagedBy: "Pulumi", + Owner: "user@company.com", + Team: "Infra", + }, + }, +}); +const fargateProfile = new aws.eks.FargateProfile("fargateProfile", { + … +}, { + provider: provider, + … +}); +``` +
@@ -368,6 +421,7 @@ const cluster = new aws.eks.Cluster("cluster", { - [Documentation] - [State] +- [Assigning tags by default on AWS with Pulumi] +[assigning tags by default on aws with pulumi]: https://blog.scottlowe.org/2023/09/11/assigning-tags-by-default-on-aws-with-pulumi/ diff --git a/knowledge base/template.md b/knowledge base/template.md index c6608eb..e2f5f6b 100644 --- a/knowledge base/template.md +++ b/knowledge base/template.md @@ -13,7 +13,7 @@ Intro