diff --git a/.markdownlint.yaml b/.markdownlint.yaml index 417ec9a..7be4729 100644 --- a/.markdownlint.yaml +++ b/.markdownlint.yaml @@ -13,5 +13,7 @@ MD033: # no-inline-html - br - code - details + - i - summary - sup + - u diff --git a/knowledge base/cloud computing/aws/README.md b/knowledge base/cloud computing/aws/README.md index 285d1a7..2290dd5 100644 --- a/knowledge base/cloud computing/aws/README.md +++ b/knowledge base/cloud computing/aws/README.md @@ -105,6 +105,8 @@ One can can rapidly remapping addresses to other instances in one's account and | [S3] | Storage | | [Sagemaker] | Machine learning | | [Security Hub] | Aggregator for security findings | +| [SNS] | Pub/sub message delivery | +| [SQS] | Queues | [Service icons][aws icons] are publicly available for diagrams and such. Public service IP address ranges are [available in JSON form][aws public ip address ranges now available in json form] @@ -510,6 +512,8 @@ machine if not. [route53]: route53.md [s3]: s3.md [sagemaker]: sagemaker.md +[sns]: sns.md +[sqs]: sqs.md [access aws services through aws privatelink]: https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-access-aws-services.html diff --git a/knowledge base/cloud computing/aws/chatbot.md b/knowledge base/cloud computing/aws/chatbot.md new file mode 100644 index 0000000..30c832a --- /dev/null +++ b/knowledge base/cloud computing/aws/chatbot.md @@ -0,0 +1,81 @@ +# Chatbot + +Enables using messaging program chat rooms to monitor, and respond to, operational events in AWS by processing service +notifications from SNS and forwarding them to chat rooms like Slack channels. + +One can also run AWS CLI commands in chat channels using Chatbot. + +1. [TL;DR](#tldr) +1. [Further readings](#further-readings) + 1. [Sources](#sources) + +## TL;DR + +Chatbot has **no** additional charge, minimum fees nor upfront commitments.
+One will pay for the underlying services (SNS, SQS, CloudWatch, …). + +Supports Amazon Chime, Microsoft Teams, and [Slack] at the time of writing. + + + +
+ Usage + +```sh +# List Slack workspaces. +aws chatbot describe-slack-workspaces +aws chatbot describe-slack-workspaces --query 'SlackWorkspaces' + +# Show Slack channel configurations. +aws chatbot describe-slack-channel-configurations +aws chatbot describe-slack-channel-configurations --query 'SlackChannelConfigurations' +``` + +
+ + + +## Further readings + +- [Website] +- [Documentation] +- [Slack] +- [SNS] + +### Sources + +- [What is AWS Chatbot?] + + + + + +[slack]: ../../slack.md +[sns]: sns.md + + + +[documentation]: https://docs.aws.amazon.com/chatbot/ +[website]: https://aws.amazon.com/chatbot/ +[what is aws chatbot?]: https://docs.aws.amazon.com/chatbot/latest/adminguide/what-is.html + + diff --git a/knowledge base/cloud computing/aws/sns.md b/knowledge base/cloud computing/aws/sns.md new file mode 100644 index 0000000..b30a20e --- /dev/null +++ b/knowledge base/cloud computing/aws/sns.md @@ -0,0 +1,187 @@ +# Simple Notification Service + +Managed pub/sub message delivery service. + +1. [TL;DR](#tldr) +1. [Connect a Slack channel to SNS](#connect-a-slack-channel-to-sns) +1. [Further readings](#further-readings) + 1. [Sources](#sources) + +## TL;DR + +_Publishers_ (or _producers_) send messages to a _topic_.
+Topics are logical access points acting as communication channels for multiple _endpoints_ like SQS, emails, lambda, +and others. + +SNS can be also used to send notifications to HTTP(S) endpoints such as Webhook URLs.
+Anyway, SNS does **not** support sending requests using JSON key-value pairs, which some webhooks require (i.e. Slack). + +_Subscribers_ (or _consumers_) subscribe to the topic and receive published messages using a supported endpoint type. + +
+ Standard topics + +- Do **not** enforce strict message ordering, grouping, **nor** deduplication.
+ Consumers of the queue may receive messages out of order, and more than once. +- Support **all** delivery protocols. + +
+
+ FIFO topics + +- Integrate with FIFO queues in [SQS]. +- **Do** enforce strict message ordering, grouping, and deduplication.
+ They always deliver messages to subscribed [SQS] queues in the **exact** order in which the messages are published to + the topic, and **only once**. +- **Do** ensure strict message ordering, message grouping, and deduplication. +- Allowing FIFO and standard queues to subscribe for message processing. + +
+ +By default, only the topic's owner can publish or subscribe to the topic.
+Configure additional access permissions by expanding the topic's Access policy. + + + +
+ Usage + +```sh +# List topics. +aws sns list-topics + +# Get information about topics. +aws sns get-topic-attributes --topic-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic' + +# List subscriptions. +aws sns list-subscriptions +aws sns list-subscriptions --query 'Subscriptions' +aws sns list-subscriptions-by-topic --topic-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic' + +# Get information about subscriptions. +aws sns get-subscription-attributes \ + --subscription-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic:abcdef01-2345-6789-abcd-ef0123456789' +``` + +
+ + + +## Connect a Slack channel to SNS + +Refer [How to Connect AWS SNS to Slack using Webhooks: Easy Step-by-Step Explanation]. + +SNS does **not** currently support integrating directly with third-party applications.
+Leverage [Slack's incoming webhooks] or create a [Chatbot] configuration for the destination Slack channel. + +
+ Webhook + +SNS does **not** currently support sending requests using JSON key-value pairs, and Slack's webhooks require JSON +requests to include a message string as the value of the `text` key.
+To solve this, use a Lambda function to modify the SNS message's body JSON document for the webhook endpoint. + +Procedure: + +1. Ensure the existence of a [Slack incoming webhook][slack's incoming webhooks] to send requests to. +1. Create a topic.
+ The _standard_ type is usually enough. +1. Create a Lambda function.
+ And test it works. +1. Add a topic trigger to the function. + +
+ +
+ Chatbot + +1. Ensure the existence of a [Slack incoming webhook][slack's incoming webhooks] to send requests to. +1. Create a topic.
+ The _standard_ type is usually enough. +1. Create a IAM Role for the Chatbot configuration to use. + +
+ + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "chatbot.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] + } + ``` + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Autogenerated-AWS-Chatbot-NotificationsOnly-Policy", + "Effect": "Allow", + "Action": [ + "cloudwatch:Describe*", + "cloudwatch:Get*", + "cloudwatch:List*" + ], + "Resource": "*" + } + ] + } + ``` + +
+ +1. Create a Chatbot configuration for the destination Slack channel. + +
+ +## Further readings + +- [SQS] + +### Sources + +- [What is Amazon SNS?] +- [How To Set Up A Slack Channel To Be An AWS SNS Subscriber] +- [How to Connect AWS SNS to Slack using Webhooks: Easy Step-by-Step Explanation] + + + + + +[chatbot]: chatbot.md +[sqs]: sqs.md +[slack's incoming webhooks]: ../../slack.md#incoming-webhooks + + + +[what is amazon sns?]: https://docs.aws.amazon.com/sns/latest/dg/welcome.html + + +[how to set up a slack channel to be an aws sns subscriber]: https://medium.com/cohealo-engineering/how-set-up-a-slack-channel-to-be-an-aws-sns-subscriber-63b4d57ad3ea +[how to connect aws sns to slack using webhooks: easy step-by-step explanation]: https://hevodata.com/learn/sns-to-slack/ diff --git a/knowledge base/cloud computing/aws/sqs.md b/knowledge base/cloud computing/aws/sqs.md new file mode 100644 index 0000000..2647433 --- /dev/null +++ b/knowledge base/cloud computing/aws/sqs.md @@ -0,0 +1,62 @@ +# Simple Queue Service + +Managed queues. + +1. [TL;DR](#tldr) +1. [Further readings](#further-readings) + 1. [Sources](#sources) + +## TL;DR + + + + + + + +## Further readings + +- [SNS] + +### Sources + +- [What is Amazon Simple Queue Service?] + + + + + +[sns]: sns.md + + + +[what is amazon simple queue service?]: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html + + diff --git a/knowledge base/slack.md b/knowledge base/slack.md index 0cda7a6..ce82149 100644 --- a/knowledge base/slack.md +++ b/knowledge base/slack.md @@ -3,6 +3,8 @@ 1. [TL;DR](#tldr) 1. [Add custom emoji](#add-custom-emoji) 1. [Give aliases to existing emojis](#give-aliases-to-existing-emojis) +1. [Apps](#apps) +1. [Incoming webhooks](#incoming-webhooks) 1. [Further readings](#further-readings) 1. [Sources](#sources) @@ -23,8 +25,8 @@ mas install '803453959' ```sh # Send notifications to channels -curl -X POST -H 'Content-type: application/json' \ - --data '{"text": "Staging DB restore completed successfully"}' \ +curl -X 'POST' -H 'Content-type: application/json' \ + --data '{"text": "Hello, World!"}' \ 'https://hooks.slack.com/services/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' ``` @@ -57,15 +59,38 @@ Check out [slackmojis] for some common reactions. 1. Choose the image. 1. Give it an alias. +## Apps + +Direct links: + +- [Create a new App](https://api.slack.com/apps?new_app=1). + +## Incoming webhooks + +Refer [Sending messages using incoming webhooks] and [Setting Up Slack Webhook URL Simplified 101]. + +1. Enable Incoming Webhooks for a Slack app.
+ _Features_ → _Incoming Webhooks_ → _Activate Incoming Webhooks_. +1. Create a new Webhook URL and authorize it (or request authorization for it). +1. Install the app in the workspace. +1. Send a test request to the webhook. + + ```sh + curl -X 'POST' -H 'Content-type: application/json' \ + 'https://hooks.slack.com/services/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' --data '{"text": "Hello, World!"}' + ``` + ## Further readings - [Website] +- [Sending messages using incoming webhooks] - [Posting messages using curl] ### Sources - [Slackmojis] - [Slack Notifications for Ansible Tower (AWX)] +- [Setting Up Slack Webhook URL Simplified 101] [posting messages using curl]: https://api.slack.com/tutorials/tracks/posting-messages-with-curl +[sending messages using incoming webhooks]: https://api.slack.com/messaging/webhooks [website]: https://slack.com/ +[setting up slack webhook url simplified 101]: https://hevodata.com/learn/slack-webhook-url/ [slack notifications for ansible tower (awx)]: https://mpolinowski.github.io/docs/DevOps/Ansible/2021-04-30-ansible-tower-slack-notifications/2021-04-30/ [slackmojis]: https://slackmojis.com/ diff --git a/snippets/aws/other commands.fish b/snippets/aws/other commands.fish index e3a2acc..a4436e1 100644 --- a/snippets/aws/other commands.fish +++ b/snippets/aws/other commands.fish @@ -24,6 +24,34 @@ aws autoscaling cancel-instance-refresh --auto-scaling-group-name 'ProductionSer aws autoscaling rollback-instance-refresh --auto-scaling-group-name 'ProductionServers' +### +# Chatbot +# ------------------ +### + +# List Slack workspaces +aws chatbot describe-slack-workspaces +aws chatbot describe-slack-workspaces --query 'SlackWorkspaces' + +# Show Slack channel configurations +aws chatbot describe-slack-channel-configurations +aws chatbot describe-slack-channel-configurations --query 'SlackChannelConfigurations' + + +### +# CloudWatch +# ------------------ +### + +# List available metrics +aws cloudwatch list-metrics --namespace 'AWS/EC2' +aws cloudwatch list-metrics --namespace 'AWS/EC2' --metric-name 'CPUUtilization' +aws cloudwatch list-metrics --namespace 'AWS/EC2' --dimensions 'Name=InstanceId,Value=i-1234567890abcdef0' --query 'Metrics[].MetricName' + +# Show alarms information +aws cloudwatch describe-alarms-for-metric --metric-name 'CPUUtilization' --namespace 'AWS/EC2' --dimensions 'Name=InstanceId,Value=i-1234567890abcdef0' + + ### # Cognito # ------------------ @@ -347,3 +375,23 @@ aws s3api list-objects-v2 --bucket 'backup' aws s3api list-objects-v2 --bucket 'backup' --query "Contents[?LastModified>='2022-01-05T08:05:37+00:00'].Key" aws s3api list-buckets --output 'text' --query 'Buckets[].Name' | xargs -pn '1' aws s3api list-multipart-uploads --bucket + + +### +# SNS +# ------------------ +### + +# List topics +aws sns list-topics + +# Get information about topics +aws sns get-topic-attributes --topic-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic' + +# List subscriptions +aws sns list-subscriptions +aws sns list-subscriptions --query 'Subscriptions' +aws sns list-subscriptions-by-topic --topic-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic' + +# Get information about subscriptions +aws sns get-subscription-attributes --subscription-arn 'arn:aws:sns:eu-west-1:012345678901:aSucculentTopic:abcdef01-2345-6789-abcd-ef0123456789' diff --git a/snippets/pulumi/aws/notify slack channel and take action when instance is impaired.ts b/snippets/pulumi/aws/notify slack channel and take action when instance is impaired.ts new file mode 100644 index 0000000..90df0ca --- /dev/null +++ b/snippets/pulumi/aws/notify slack channel and take action when instance is impaired.ts @@ -0,0 +1,116 @@ +import * as aws from "@pulumi/aws"; + +const instance_output = new aws.ec2.getInstanceOutput({ + filters: [{ + name: "tag:Name", + values: [ "instance-name-tag" ], + }], +}); + +const iamRole = new aws.iam.Role( + "awsChatbot", + { + name: "AWSChatbotRole", + assumeRolePolicy: JSON.stringify({ + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Principal: { + Service: "chatbot.amazonaws.com", + }, + Action: "sts:AssumeRole", + }], + }), + }, +); +new aws.iam.RolePolicy( + "awsChatbot", + { + name: "AllowRoleFunctions", + description: "NotificationsOnly policy for AWS-Chatbot", + policy: JSON.stringify({ + Version: "2012-10-17", + Statement: [{ + Effect: "Allow", + Action: [ + "cloudwatch:Describe*", + "cloudwatch:Get*", + "cloudwatch:List*", + ], + "Resource": "*", + }], + }), + }, +); + +const notifications_snsTopic = new aws.sns.Topic( + "notifications", + { name: "notifications" }, +); +new aws.sns.TopicSubscription( + // FIXME: requires manual confirmation from email + "notifications-email", + { + topic: notifications_snsTopic.arn, + protocol: "email", + endpoint: "infra@example.org", + }, +); +new aws.chatbot.SlackChannelConfiguration( + "notifications-channel", + { + configurationName: "Notifications2Channel", + slackTeamId: "T00000000", + slackChannelId: "C0000000000", + snsTopicArns: [ notifications_snsTopic.arn ], + iamRoleArn: iamRole.arn, + guardrailPolicyArns: [ "arn:aws:iam::aws:policy/ReadOnlyAccess" ], + }, +); + +new aws.cloudwatch.MetricAlarm( + "prometheusServer_systemStatus", + { + name: "PrometheusServer_SystemStatus", + alarmDescription: "Notify and recover when the System status check fails 5 consecutive times over 5 minutes.", + + namespace: "AWS/EC2", + dimensions: { + InstanceId: instance_output.id, + }, + metricName: "StatusCheckFailed_System", + statistic: "Maximum", + unit: "Count", + comparisonOperator: "GreaterThanOrEqualToThreshold", + threshold: 1, + evaluationPeriods: 5, + period: 60, + alarmActions: [ + notifications_snsTopic.arn, + "arn:aws:automate:eu-west-1:ec2:recover", + ], + }, +); +new aws.cloudwatch.MetricAlarm( + "prometheusServer_instanceStatus", + { + name: "PrometheusServer_instanceStatus", + alarmDescription: "Notify and reboot when the Instance status check fails 5 consecutive times over 5 minutes.", + + namespace: "AWS/EC2", + dimensions: { + InstanceId: instance_output.id, + }, + metricName: "StatusCheckFailed_Instance", + statistic: "Maximum", + unit: "Count", + comparisonOperator: "GreaterThanOrEqualToThreshold", + threshold: 1, + evaluationPeriods: 5, + period: 60, + alarmActions: [ + notifications_snsTopic.arn, + "arn:aws:swf:eu-west-1:012345678901:action/actions/AWS_EC2.InstanceId.Reboot/1.0", + ], + }, +); diff --git a/snippets/pulumi/commands.fish b/snippets/pulumi/commands.fish index 610a807..d23e00d 100644 --- a/snippets/pulumi/commands.fish +++ b/snippets/pulumi/commands.fish @@ -87,3 +87,4 @@ pulumi import 'aws:ec2/securityGroup:SecurityGroup' 'internalOps' 'sg-0123456789 pulumi import 'aws:iam/user:User' 'jimmy' 'jimmy' --generate-code='false' pulumi import 'aws:route53/record:Record' 'hoppscotch' 'ZGG4442BC3E8M_hoppscotch.example.org_A' pulumi import 'aws:cloudwatch/metricAlarm:MetricAlarm' 'prometheus-ec2-CPUUtilization' 'prometheus-ec2-CPUUtilization-drc5644' +pulumi import 'aws:chatbot/slackChannelConfiguration:SlackChannelConfiguration' 'alarms' 'arn:aws:chatbot::012345678901:chat-configuration/slack-channel/alarms'