brandon/oam
1
0
Fork 0
mirror of https://gitea.com/mcereda/oam.git synced 2026-02-09 05:44:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(kb/aws): revise and update articles with new findings

Browse Source
This commit is contained in:
Michele Cereda
2025-03-26 18:36:27 +01:00
parent 4dd850545f
commit 7d3b421525
4 changed files with 232 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
knowledge base/cloud computing/aws/README.md
View File

@@ -5,13 +5,13 @@
1. [Elastic IP addresses](#elastic-ip-addresses)
1. [Services](#services)
1. [Billing and Cost Management](#billing-and-cost-management)
1. [CloudWatch](#cloudwatch)
1. [Config](#config)
1. [Detective](#detective)
1. [Global Accelerator](#global-accelerator)
1. [GuardDuty](#guardduty)
1. [EventBridge](#eventbridge)
1. [Inspector](#inspector)
1. [Kinesis](#kinesis)
1. [KMS](#kms)
1. [Security Hub](#security-hub)
1. [Resource constraints](#resource-constraints)
@@ -83,23 +83,24 @@ One can can rapidly remapping addresses to other instances in one's account and
## Services
| Service | Description |
| Service | Summary |
| ----------------------------- | --------------------------------------------- |
| [Billing and Cost Management] | FIXME |
| [CloudWatch] | Observability (logging, monitoring, alerting) |
| [Billing and Cost Management] | Cost management |
| [CloudFront] | Content delivery |
| [CloudWatch] | Observability (logging, monitoring, alerting) |
| [Config] | Compliance |
| [Detective] | FIXME |
| [Detective] | Behaviour anomalies |
| [EC2] | Managed virtual machines |
| [ECR] | Container registry |
| [ECS] | Run containers as a service |
| [EFS] | Serverless file storage |
| [EKS] | Managed Kubernetes clusters |
| [EventBridge] | FIXME |
| [EventBridge] | Stream real time data |
| [GuardDuty] | Threat detection |
| [IAM] | Access control |
| [Image Builder] | Build custom AMIs |
| [Inspector] | FIXME |
| [Inspector] | Security vulnerability assessment |
| [Kinesis] | Video or data streams |
| [KMS] | Key management |
| [OpenSearch] | ELK, logging |
| [RDS] | Databases |
@@ -120,78 +121,6 @@ Costs can be grouped by Tags applied on resources.<br/>
Tags to use for this kind of grouping need to be activated in the _Cost allocation tags_ section.<br/>
New tags might take 24 or 48 hours to appear there.
### CloudWatch
Observability service. with functions for logging, monitoring and alerting.
_Metrics_ are whatever needs to be monitored (e.g. CPU usage).<br/>
_Data points_ are the values of a metric over time.<br/>
_Namespaces_ are containers for metrics.
Metrics only exist in the region in which they are created.
[Many AWS services][services that publish cloudwatch metrics] offer basic monitoring by publishing a default set of
metrics to CloudWatch with no charge.<br/>
This feature is automatically enabled by default when one starts using one of these services.
API calls for CloudWatch are paid.
It's best practice to **distribute** the `ListMetrics` call to avoid throttling.<br/>
The default limit for `ListMetrics` is 25 transactions per second.
The [CloudWatch console] offers some default good queries.
<details>
<summary>Queries of interest</summary>
| What | Section | Tab | How to visualize |
| ------------------------------------ | ----------- | --------------- | ----------------------------------------------------- |
| [Top 10 log groups by written bytes] | All Metrics | Graphed metrics | Add Query > Logs > Top 10 log groups by written bytes |
<details style="padding-left: 1em;">
<summary>Get a dashboard of how much data a <b>small</b> set of log groups ingested in the last 30 days</summary>
> This graph works only with the _Absolute_ time period option.<br/>
> Should you choose _Relative_, the graph returns incorrect data.
1. [CloudWatch console] > _All metrics_ (navigation pane on the left).
1. Choose _Logs_, _Log group metrics_.
1. Select the individual `IncomingBytes` metrics of each log group of interest.
1. Choose the _Graphed metrics_ tab.
1. For each metric:
- Change `Statistic` to `Sum`.
- Change `Period` to `30 Days`.
1. Choose the _Graph options_ tab.
1. Choose the _Number_ option group.
1. At the top right of the graph, choose _Custom_ as the time range.
1. Choose _Absolute_.
1. Select the last 30 days as start and end date.
</details>
<details style="padding-left: 1em;">
<summary>Get a dashboard of how much data <b>all</b> log groups ingested in the last 30 days</summary>
> This graph works only with the _Absolute_ time period option.<br/>
> Should you choose _Relative_, the graph returns incorrect data.
1. [CloudWatch console] > _All metrics_ (navigation pane on the left).
1. Choose the _Graphed metrics_ tab.
1. From the _Add math_ dropdown list, choose _Start with an empty expression_.
1. Paste this as math expression:
```plaintext
SORT(REMOVE_EMPTY(SEARCH('{AWS/Logs,LogGroupName} MetricName="IncomingBytes"', 'Sum', 2592000)),SUM, DESC)
```
1. At the top right of the graph, choose _Custom_ as the time range.
1. Choose _Absolute_.
1. Select the last 30 days as start and end date.
</details>
</details>
### Config
Compliance service for assessing and auditing AWS resources.
@@ -308,6 +237,10 @@ TODO
TODO
### Kinesis
TODO
### KMS
_Key material_ is the cryptographic secret of Keys that is used in encryption operations.
@@ -398,9 +331,7 @@ One pays for data transfer between instances and services in the **same region**
zone**.<br/>
See [Understanding data transfer charges].
One pays for sending logs to [CloudWatch].<br/>
Refer [Which log group is causing a sudden increase in my CloudWatch Logs bill?] to get an idea of what changed in some
time frame.
One pays for sending logs and metrics to [CloudWatch].
## Savings plans
@@ -575,7 +506,6 @@ machine if not.
## Further readings
- [EC2]
- [Services that publish CloudWatch metrics]
- [Best Practices for Tagging AWS Resources]
- [Automating DNS-challenge based LetsEncrypt certificates with AWS Route 53]
- AWS' [CLI]
@@ -586,7 +516,6 @@ machine if not.
### Sources
- [Constraints for tags][constraints tag]
- [What is CloudWatch]
- [What is Amazon VPC?]
- [Subnets for your VPC]
- [What is AWS Config?]
@@ -611,11 +540,9 @@ machine if not.
- [Boto3 resources]
- [Boto3 sessions]
- [Boto3 paginators]
- [Which log group is causing a sudden increase in my CloudWatch Logs bill?]
- [What is AWS Global Accelerator?]
- [How AWS Global Accelerator works]
- [Using Amazon CloudWatch with AWS Global Accelerator]
- [What is AWS CloudWatch? Guide for beginners]
<!--
Reference
@@ -624,18 +551,19 @@ machine if not.
<!-- In-article sections -->
[billing and cost management]: #billing-and-cost-management
[cloudwatch]: #cloudwatch
[config]: #config
[detective]: #detective
[eventbridge]: #eventbridge
[guardduty]: #guardduty
[inspector]: #inspector
[kinesis]: #kinesis
[kms]: #kms
[security hub]: #security-hub
<!-- Knowledge base -->
[cli]: cli.md
[cloudfront]: cloudfront.md
[cloudwatch]: cloudwatch.md
[ec2]: ec2.md
[ecr]: ecr.md
[ecs]: ecs.md
@@ -661,7 +589,6 @@ machine if not.
[boto3 paginators]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html
[boto3 resources]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html
[boto3 sessions]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/session.html
[cloudwatch console]: https://console.aws.amazon.com/cloudwatch/home
[connect to the internet using an internet gateway]: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html
[constraints tag]: https://docs.aws.amazon.com/directoryservice/latest/devguide/API_Tag.html
[creating organization policies with aws organizations]: https://docs.aws.amazon.com/organizations/latest/userguide/orgs_policies_create.html
@@ -676,18 +603,14 @@ machine if not.
[nat gateways]: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html
[rotating aws kms keys]: https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html
[savings plans user guide]: https://docs.aws.amazon.com/savingsplans/latest/userguide/
[services that publish cloudwatch metrics]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html
[subnets for your vpc]: https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html
[test your roles' access policies using the aws identity and access management policy simulator]: https://aws.amazon.com/blogs/security/test-your-roles-access-policies-using-the-aws-identity-and-access-management-policy-simulator/
[tools to build on aws]: https://aws.amazon.com/developer/tools/
[top 10 log groups by written bytes]: https://console.aws.amazon.com/cloudwatch/home#metricsV2?graph=~(view~'timeSeries~stacked~false~metrics~(~(~(expression~'SELECT*20SUM*28IncomingBytes*29*0aFROM*20SCHEMA*28*22AWS*2fLogs*22*2c*20LogGroupName*29*20*0aGROUP*20BY*20LogGroupName*0aORDER*20BY*20SUM*28*29*20DESC*0aLIMIT*2010~label~'!*7bLABEL*7d*20*5bsum*3a*20!*7bSUM*7d*5d~id~'q1)))~region~'eu-west-1~title~'Top*2010*20log*20groups*20by*20written*20bytes~yAxis~(left~(label~'Bytes~showUnits~false))~stat~'Average~period~300)
[understanding data transfer charges]: https://docs.aws.amazon.com/cur/latest/userguide/cur-data-transfers-charges.html
[using amazon cloudwatch with aws global accelerator]: https://docs.aws.amazon.com/global-accelerator/latest/dg/cloudwatch-monitoring.html
[what is amazon vpc?]: https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html
[what is aws config?]: https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html
[what is aws global accelerator?]: https://docs.aws.amazon.com/global-accelerator/latest/dg/what-is-global-accelerator.html
[what is cloudwatch]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html
[which log group is causing a sudden increase in my cloudwatch logs bill?]: https://repost.aws/knowledge-center/cloudwatch-logs-bill-increase
<!-- Others -->
[a guide to tagging resources in aws]: https://medium.com/@staxmarketing/a-guide-to-tagging-resources-in-aws-8f4311afeb46
@@ -697,4 +620,3 @@ machine if not.
[date & time policy conditions at aws - 1-minute iam lesson]: https://www.youtube.com/watch?v=4wpKP1HLEXg
[difference in boto3 between resource, client, and session?]: https://stackoverflow.com/questions/42809096/difference-in-boto3-between-resource-client-and-session
[using aws kms via the cli with a symmetric key]: https://nsmith.net/aws-kms-cli
[what is aws cloudwatch? guide for beginners]: https://www.educative.io/blog/aws-cloudwatch