Yeah.. Despite looking the same, not all "nodes" are created equal even if they look equal. Its just computers. There are so many variables at play you can't account for all of them. This is part of why "Chaos engineering" exists.
Archive: https://archive.today/Dgnpl
From the post:
>One thing we’ve started noticing while running repeated fresh deployments is that benchmark numbers can become weirdly convincing. Sometimes TOO convincing. A cluster will produce nearly identical CPU throughput across multiple fresh instances. Same storage behavior. Same latency profile. Same overall benchmark shape. At first glance it looks incredibly stable. We like that. Then one deployment suddenly starts failing under sustained runtime load. Um, why? That happened this week while repeatedly testing AWS t3.medium instances across multiple regions.
Yeah.. Despite looking the same, not all "nodes" are created equal even if they look equal. Its just computers. There are so many variables at play you can't account for all of them. This is part of why "Chaos engineering" exists.
Archive: https://archive.today/Dgnpl
From the post:
>>One thing we’ve started noticing while running repeated fresh deployments is that benchmark numbers can become weirdly convincing. Sometimes TOO convincing.
A cluster will produce nearly identical CPU throughput across multiple fresh instances. Same storage behavior. Same latency profile. Same overall benchmark shape. At first glance it looks incredibly stable. We like that.
Then one deployment suddenly starts failing under sustained runtime load. Um, why?
That happened this week while repeatedly testing AWS t3.medium instances across multiple regions.