Data & AI
Everything filed under data & ai.
- EMR clusters should die youngA long-running EMR cluster is a pet with a very large food bill. Treat clusters as disposable compute for a single job, and S3 as the only place data lives.
- Measure Kafka lag in seconds, not messagesA consumer group 50,000 messages behind might be fine or might be an hour stale. The number that matters to the business is how old the data is.