Trade-off between topics and partitions?
Having 1 million partitions is probably too many. You can still partition by user id with fewer partitions. Events for the same user are still kept in order. It's just that each partition will have...
View ArticleTrade-off between topics and partitions?
Deja vu! IMO, what you are describing is a database problem, even though you are talking/thinking about it as a queue problem. I'm sure you could construct something using Kafka (and Samza), but I...
View ArticleTrade-off between topics and partitions?
Hello, According to the Kafka FAQ "How do I choose the number of partitions for a topic", clusters with more than 10K partitions are not tested. I am looking for advice on how to scale the number of...
View ArticleTrade-off between topics and partitions?
Jun's advice is exactly what the FAQ says but maybe it should be rephrased because it is true this question comes up once in a while. Your requirement is: user must receive published messages in order....
View Article
