brandon/oam
1
0
Fork 0
mirror of https://gitea.com/mcereda/oam.git synced 2026-03-04 07:54:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(loki): run on fargate on ecs with config in an efs volume

Browse Source
This commit is contained in:
Michele Cereda
2025-01-20 01:53:43 +01:00
parent d8425ed03b
commit bb5f6a88a1
5 changed files with 379 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
knowledge base/cloud computing/aws/ecs.md
View File

@@ -20,15 +20,16 @@
## TL;DR
_Tasks_ are the basic unit of deployment.<br/>
Their details are specified in _task definitions_.
_Standalone tasks_ are meant to perform some work, then stop much like batch processes.<br/>
_Services_ run and maintain a defined number of instances of the same task simultaneously, and are meant to stay active
much like web servers.
They are instances of the set of containers specified in their own _task definition_.
Tasks model and run one or more containers, much like Pods in Kubernetes.<br/>
Containers **cannot** run on ECS unless encapsulated in a task.
_Standalone tasks_ start a single task, which is meant to perform some work to completion and then stop (much like batch
processes would).<br/>
_Services_ run and maintain a defined number of instances of the same task simultaneously, which are meant to stay
active and act as replicas of some service (much like web servers would).
Tasks are executed depending on their _launch type_ and _capacity providers_:
- On EC2 instances that one owns, manages, and pays for.

73
knowledge base/cloud computing/aws/efs.md
View File

@@ -10,35 +10,91 @@ performance.
## TL;DR
Built to scale on demand growing and shrinking automatically as files are added and removed.<br/>
Accessible across most types of AWS compute instances, including EC2, ECS, EKS, Lambda, and Fargate.
Accessible across EC2, ECS, EKS, Lambda, and Fargate.
Supports the NFS v4.0 and v4.1 protocols.
Supports the **NFS v4.0** and **v4.1** protocols.<br/>
A _mount target_ is required for any file system for clients to be able to use NFS to mount them.
The file system's DNS name automatically resolves to the mount target's IP address in the Availability Zone of the
connecting EC2 instances.<br/>
It follows the `{{ file-system-id }}.efs.{{ aws-region }}.amazonaws.com` convention.
Available file system types:
- _Regional_: redundant across **multiple** geographically separated AZs **within the same Region**.
- _One Zone_: data stored within a **single AZ**, with all the limits it implies.
Default modes:
Available throughput modes:
- _General Purpose Performance_: ideal for latency-sensitive applications.<br/>
- _Elastic_: scales automatically in real time to meet the needs of workloads' activity.<br/>
Only available for file systems using the General Purpose performance mode.<br/>
Default setting when not specified during creation.
- _Provisioned_: statically provides the specified level of throughput independently from the file system's size.
- _Bursting_: scales automatically with the amount of data in Standard storage.
Available performance modes:
- _General Purpose_: lowest per-operation latency.<br/>
Recommended for all file systems. Ideal for latency-sensitive applications.<br/>
Examples: web-serving environments, content-management systems, home directories, and general file serving.
- _Elastic Throughput_: designed to scale throughput performance automatically to meet the needs of workloads' activity.
- _Max I/O_: designed for highly parallelized workloads that **can** tolerate higher latencies than the General Purpose
mode.<br/>
**Not** supported by One Zone file systems or file systems using the Elastic throughput mode.
Lifecycle management settings allow to automatically move files into and out of the lower-cost Infrequent Access storage
class based on access patterns.<br/>
Leverages lifecycle policies.
When creating file systems via the Console, the file system's lifecycle policy is configured by default with the
following settings:
- Transition into IA set to 30 days since last access.
- TransitionToArchive set to 90 days since last access.
- Transition into Standard set to None.
When creating file systems via the CLI or APIs, it is **not** possible to set lifecycle policies at the same time.<br/>
One **must** wait until the file system is created, then use the `PutLifecycleConfiguration` API operation to update the
lifecycle policies.
Provides file-system-access semantics like strong data consistency and file locking.<br/>
Supports controlling access to file systems through POSIX permissions.<br/>
Supports authentication, authorization, and encryption.
Supports:
- Controlling access to file systems through POSIX permissions.
- Authentication and authorization.
- Encryption in transit and at rest.
EFS supports encryption in transit and encryption at rest.<br/>
Encryption at rest is enabled when creating a file system. In such case, all data and metadata is encrypted.<br/>
Encryption in transit is enabled when mounting a file system. Client access via NFS to EFS is controlled by both IAM
policies and network security policies (i.e. security groups).
Windows-based EC2 instances are **not** supported.
Automatic backups are enabled by default when creating file systems using the console.<br/>
When creating file systems via the CLI or the APIs, automatic backups are enabled by default only when setting them up
to be One Zone file systems.
<details>
<summary>Usage</summary>
```sh
# Get mount targets' information.
aws efs describe-mount-targets --query 'MountTargets[]' --file-system-id 'fs-0123456789abcdef0'
# Get mount targets' IP address.
aws efs describe-mount-targets --query 'MountTargets[].IpAddress' --file-system-id 'fs-0123456789abcdef0'
aws efs describe-mount-targets --query 'MountTargets[].IpAddress' --mount-target-id 'fsmt-0123456789abcdef0'
# Mount volumes.
mount -t 'nfs' -o 'nfsvers=4.0,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport' \
'fs-0123456789abcdef0.efs.eu-west-1.amazonaws.com:/' "$HOME/efs"
```
</details>
## Further readings
- [Amazon Web Services]
- [How do I mount, unmount, automount, and on-premises mount my Amazon EFS file system?]
### Sources
@@ -55,6 +111,7 @@ Windows-based EC2 instances are **not** supported.
<!-- Files -->
<!-- Upstream -->
[how do i mount, unmount, automount, and on-premises mount my amazon efs file system?]: https://repost.aws/knowledge-center/efs-mount-automount-unmount-steps
[what is amazon elastic file system?]: https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html
<!-- Others -->