AWS / ECS autoscaling configuration and guidance
We’re running Cribl in AWS, using ECS to manage docker containers on EC2 instances. We’re trying to tune our instance size and auto-scaling, and wondering if anyone else has any past experience on the matter.
For example, should we use memory reservations, and if so what are the best settings for soft and hard limits?
If we want to scale when under memory pressure, how do we get this information out?
I.e. to know when we’re under memory pressure we need to see heap usage of the worker threads, not just memory usage for the box (since the threads pre-allocate heap), what’s the best way to get this?
If anyone has anyone had any luck - good or bad - with this, I’d be keen to hear about your experiences!