diff --git a/.vscode/settings.json b/.vscode/settings.json
index 7a488b1..9ff8110 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -180,6 +180,7 @@
"hoppscotch",
"imager",
"ingester",
+ "ingesters",
"istio",
"istioctl",
"jdupes",
diff --git a/knowledge base/logging.md b/knowledge base/logging.md
index a76c187..db55678 100644
--- a/knowledge base/logging.md
+++ b/knowledge base/logging.md
@@ -2,46 +2,40 @@
> TODO
-Intro
+Recording of events or actions.
1. [TL;DR](#tldr)
+1. [Level](#level)
1. [Further readings](#further-readings)
1. [Sources](#sources)
## TL;DR
-
+Long term storage is frequently cause of concern.
+Analysis and search is extremely difficult and computationally expensive at scale.
+Locality and standardization are often issues, with each application storing their own logs in their own format on their
+own files in different locations. Finding a way to bring all those into a central location for analysis is one of the
+goals of aggregation solutions.
-
-
-
+| Level | Summary |
+| ----------- | -------------------------------------------------------------------------------------------------- |
+| Debug | **Detailed** contextual information used during development or for troubleshooting |
+| Information | Normal, expected events |
+| Warning | Situations that are unexpected but not errors; can potentially lead to problems if ignored |
+| Error | Issues that need immediate attention, but might not require termination of operations |
+| Critical | Failures or events that require immediate and decisive attention, where operations cannot continue |
## Further readings
@@ -52,6 +46,8 @@ Intro
### Sources
- [Distributed logging for Microservices]
+- [Intro to Logging | Zero to Hero: Loki | Grafana]
+- [Structure of Logs (Part 1) | Zero to Hero: Loki | Grafana]
[distributed logging for microservices]: https://www.geeksforgeeks.org/distributed-logging-for-microservices/
+[Intro to Logging | Zero to Hero: Loki | Grafana]: https://www.youtube.com/watch?v=TLnH7efQNd0
+[Structure of Logs (Part 1) | Zero to Hero: Loki | Grafana]: https://www.youtube.com/watch?v=cnhnoFz6xu0
diff --git a/knowledge base/loki.md b/knowledge base/loki.md
index 8b02eed..5523845 100644
--- a/knowledge base/loki.md
+++ b/knowledge base/loki.md
@@ -1,7 +1,11 @@
# Grafana Loki
-Horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus and designed to be
-very cost-effective and easy to operate.
+Log aggregation system.
+
+Inspired by Prometheus
+Designed to be cost-effective and easy to operate.
+
+Horizontally scalable, highly available, multi-tenant
1. [TL;DR](#tldr)
1. [Components](#components)
@@ -13,7 +17,7 @@ very cost-effective and easy to operate.
1. [Index gateway](#index-gateway)
1. [Compactor](#compactor)
1. [Ruler](#ruler)
-1. [Clients](#clients)
+1. [Logs ingestion](#logs-ingestion)
1. [Logstash](#logstash)
1. [OpenTelemetry](#opentelemetry)
1. [Labels](#labels)
@@ -29,22 +33,38 @@ very cost-effective and easy to operate.
## TL;DR
-It indexes **a set of labels** for each log stream instead of the full logs' contents.
-The log data itself is compressed after indexing, and stored into _chunks_ on the local filesystem or in configured
-object storage solutions.
+Loki does **not** require specific formats in input, but does require log entries to have at least one label.
-Can be executed in (either-or):
+Loki works differently from stores like ElasticSearch.
+For one, Loki indexes log streams **by their labels**, and not by the logs' full contents. This allows for smaller
+indexes and higher parallelization, but prevents full text searches.
+It then compresses the entries' data, and stores it as _chunks_ on the local file system or in configured object storage
+solutions.
-- _single binary_ mode, where all its components run simultaneously in a single process.
-- _simple scalable deployment_ mode, which groups components into _read_, _write_, and _backend_ parts.
-- _microservices mode_, which runs every component by itself in multiple, different processes.
+Loki is built from a set of different components that work together as a set of microservices.
+Those can be executed in one of 3 different modes:
-Files, in Loki, can be _index_es or _chunk_s.
-Indexes are tables of contents, in TSDB format, of where to find logs for specific sets of labels.
+- **Single Binary**.
+ All components run simultaneously in and from a single process.
+ Suggested for testing, and ingesting up to a few GB of logs daily.
+- **Simple Scalable Deployment**.
+ Multiple processes run components in groups, namely _read_, _write_, and _backend_ parts.
+ Suggested when ingesting from a few GB up to a few TB of logs daily.
+- **Microservices**.
+ Multiple processes run each a single component.
+ Designed to be executed in Kubernetes clusters.
+
+Data is stored in 2 main file types: _index_es, or _chunk_s.
+Indexes define where to find logs for specific sets of labels.
Chunks are containers for log entries that are assigned specific sets of labels.
Loki does **not** collect logs itself.
-It needs _agents_, or other logs producers, to collect and push logs to its _ingesters_.
+It needs _agents_, _collectors_, or other logs producers to push entries to it. They recommend using Grafana Alloy.
+
+The _distributors_ receive and check the logs, then forward them to one or more _ingesters_ to ensure proper storage.
+
+The _query frontend_, _query schedulers_, and _queriers_ organize and execute user queries against indexed data.
+Loki uses LogQL for such user queries. One can also query logs from the CLI by using LogCLI.
Setup
@@ -73,8 +93,8 @@ The default configuration file **for package-based installations** is located at
`/etc/loki/loki.yaml`.
The docker image tries using `/etc/loki/local-config.yml` by default (as per the image's `COMMAND` setting).
-Some settings are currently **not** reachable via direct CLI flags (e.g. `schema_configs`, `storage_config.aws.*`).
-Use a configuration file for those.
+Some settings are currently **not** reachable via direct CLI flags (e.g. `schema_configs`, `storage_config.aws.*`), and
+as such _require_ one to use a configuration file.
Disable reporting
@@ -300,10 +320,18 @@ and caching offered by the query frontend.
Multiple rulers will use a consistent hash ring to distribute rule groups amongst available ruler instances.
-## Clients
+## Logs ingestion
Refer [Send log data to Loki].
+Logs are pushed to Loki via the `/api/v1/push` HTTP API endpoint as logproto or JSON payloads.
+The distributors receive and validate the received requests, then forward the data to one or more ingesters.
+Ingesters prepare and write the data into storage.
+
+Loki stores logs as streams of log entries, all in the same format regardless of the format they were when ingested.
+
+Indexes come in the form of labels, distinguishing log streams from one another.
+
### Logstash
Loki provides the `logstash-output-loki` Logstash output plugin to enable shipping logs to a Loki or Grafana Cloud
@@ -609,6 +637,7 @@ analytics:
- [How to Set Up Grafana, Loki, and Prometheus Locally with Docker Compose: Part 1 of 3]
- [Deploying Grafana, Loki, and Prometheus on AWS ECS with EFS and Cloud Formation (Part 3 of 3)]
- [AWS deployment (S3 Single Store)]
+- [Zero to Hero: Loki] video playlist
[deploying grafana, loki, and prometheus on aws ecs with efs and cloud formation (part 3 of 3)]: https://medium.com/@ahmadbilalch891/deploying-grafana-loki-and-prometheus-on-aws-ecs-with-efs-and-cloud-formation-part-3-of-3-24140ea8ccfb