brandon/oam
1
0
Fork 0
mirror of https://gitea.com/mcereda/oam.git synced 2026-02-12 15:04:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
33925bb2d75cda1158c450013970f48b5bcf69dd
oam/knowledge base/cloud computing/aws/opensearch.md
Michele Cereda 33925bb2d7 chore(kb/aws/opensearch): add storage tiers
2024-06-12 23:55:24 +02:00

8.4 KiB
Raw Blame History

Amazon OpenSearch Service

Amazon offering for managed OpenSearch clusters.

  1. Storage
    1. UltraWarm storage
    2. Cold storage
  2. Operations
    1. Migrate indexes to UltraWarm storage
    2. Return warm indexes to hot storage
    3. Migrate indexes to Cold storage
  3. Cost-saving measures
  4. Further readings
    1. Sources

Storage

Standard data nodes use hot storage in the form of instance stores or EBS volumes attached to each node.
Hot storage provides the fastest possible performance for indexing and searching new data.

UltraWarm nodes use S3 and caching.
Useful for indexes that are not actively written to, queried less frequently, or don't need the hot storage's performance.

Warm indexes are read-only unless returned to hot storage.
This makes UltraWarm storage best-suited for immutable data such as logs.

Warm indexes behave like any other index.

Cold storage uses s3 too. It is meant for data accessed only occasionally or no longer in active use.
One can't read from nor write to cold indexes. When one needs it, one can selectively attach it to UltraWarm nodes.

UltraWarm storage

Refer UltraWarm storage for Amazon OpenSearch Service.

Requirements:

  • OpenSearch/ElasticSearch >= v6.8.
  • Dedicated master nodes.
  • No t2 nor t3 instances types as data nodes.
  • When using a Multi-AZ architecture with Standby domain, the number of warm nodes must be a multiple of the number of Availability Zones being used.
  • Others.

Considerations:

  • When calculating UltraWarm storage requirements, consider only the size of the primary shards.
    S3 removes the need for replicas and abstracts away any operating system or service considerations.
  • Dashboards and _cat/indices will still report UltraWarm index size as the total of all primary and replica shards.
  • There are limits to the amount of storage each instance type can address and the maximum number of warm nodes supported by Domains.
  • Amazon recommends a maximum shard size of 50 GiB.
  • Upon enablement, UltraWarm might not be available to use for several hours even if the domain state is Active.
  • Use Index State Management to automate indexes migration to UltraWarm after they meet specific conditions.

Before disabling UltraWarm, one must either delete all warm indexes or migrate them back to hot storage.
After warm storage is empty, wait five minutes before attempting to disable UltraWarm.

Cold storage

Refer Cold storage for Amazon OpenSearch Service.

Requirements:

Operations

Migrate indexes to UltraWarm storage

Indexes' health must be green to perform migrations.

Migrations are executed one index at a time, sequentially.
There can be up to 200 migrations in the queue.
Any request that exceeds the limit will be rejected.

Index migrations to UltraWarm storage require a force merge operation, which purges documents that were marked for deletion.
By default, UltraWarm merges indexes into one segment. One can set this value up to 1000.

Migrations might fail during snapshots, shard relocations, or force merges.
Failures during snapshots or shard relocation are typically due to node failures or S3 connectivity issues.
Lack of disk space is usually the underlying cause of force merge failures.

Start migration:

POST _ultrawarm/migration/my-index/_warm

Check the migration's status:

GET _ultrawarm/migration/my-index/_status

{
  "migration_status": {
    "index": "my-index",
    "state": "RUNNING_SHARD_RELOCATION",
    "migration_type": "HOT_TO_WARM",
    "shard_level_status": {
      "running": 0,
      "total": 5,
      "pending": 3,
      "failed": 0,
      "succeeded": 2
    }
  }
}

If a migration is in the queue but has not yet started, it can be removed from the queue:

POST _ultrawarm/migration/_cancel/my-index

Return warm indexes to hot storage

Migrate them back to hot storage:

POST _ultrawarm/migration/my-index/_hot

There can be up to 10 queued migrations from warm to hot storage at a time.
Migrations requests are processed one at a time in the order they were queued.

Indexes return to hot storage with one replica.

Migrate indexes to Cold storage

As for UltraWarm storage, just change the endpoints accordingly:

POST _ultrawarm/migration/my-index/_cold
GET _ultrawarm/migration/my-index/_status
POST _ultrawarm/migration/_cancel/my-index

GET _cold/indices/_search

POST _cold/migration/_warm
GET _cold/migration/my-index/_status
POST _cold/migration/my-index/_cancel

Cost-saving measures

  • Choose good instance types and sizes.
    Leverage the ability to select them to tailor the service offering to one's needs.
  • Consider using reserved instances for long-term savings.
  • Enable index-level compression to save storage space and reduce I/O costs.
  • Use Index Lifecycle Management policies to move old data in lower storage tiers.
  • Consider using S3 as data store for infrequently accessed or archived data.
  • Consider adjusting the frequency and retention period of snapshots.
    By default, AWS OpenSearch takes daily snapshots and retains them for 14 days.
  • Enable autoscaling.
  • Optimize indexes' sharding and replication.
  • Optimize queries.
  • Optimize data ingestion.
  • Optimize indexes' mapping and settings.
  • Optimize the JVM heap size.
  • Summarize and compress historical data using Rollups.
  • Check out caches.
  • Reduce the number of requests using throttling and rate limiting.
  • Move to single-AZ deployments.
  • Leverage Spot Instances for data ingestion and processing.
  • Compress source data before sending it to OpenSearch to reduce the storage footprint and data transfer costs.
  • Share a single OpenSearch cluster with multiple accounts to reduce the overall number of instances and resources.

Further readings

Sources