See https://github.com/spring-projects/spring-boot/commit/888a81beba23728078337ad91c27ca4abbcb189e#r38575845
Comment From: snicoll
paging @garyrussell
Comment From: garyrussell
return new KafkaClientMetrics(producerFactory.createProducer(), Collections.emptyList());
This would actually work for the simple use case because there is a single producer shared by all; however, there are other options - producer per thread and there can be multiple producers when transactions are in use; also if the shared producer gets a fatal error, it is refreshed.
@snicoll Maybe we need to add some hooks to the producer factory so you can get a notification whenever a new producer is created and removed?
We could do the same for consumers too. Normally, AdminClients are short lived, but spring cloud stream keeps some around for a long time.
Comment From: snicoll
This would actually work for the simple use case because there is a single producer shared by all;
Thanks for the feedback Gary. When I read the code, that's what I saw and decided to go with that approach. However, I am second guessing that decision now.
@snicoll Maybe we need to add some hooks to the producer factory so you can get a notification whenever a new producer is created and removed?
That sounds like what we should do for caches that are added on-the-fly. It's really late in the 2.3 cycle to try to fix this so I am tempted to revert to the deprecated JMX metrics for 2.3. I've flagged for team attention to get more feedback from the team.
Comment From: garyrussell
I have a solution for this; stay tuned for a spring-kafka PR later today.
Comment From: garyrussell
@snicoll FYI: https://docs.spring.io/spring-kafka/docs/2.5.0.BUILD-SNAPSHOT/reference/html/#micrometer-native
Comment From: snicoll
So we have a new API to add listeners in https://github.com/spring-projects/spring-kafka/pull/1464 - The plan is to add a customizer callback so that the factories that we auto-configure are configured with the metrics listener.