Pearson CCDAK New Attempt

The official Apache Kafka Connect documentation explains that connectors run within the same worker JVM, and all logs are written through the worker’s logging framework (Log4j or Log4j2). To isolate logs for a specific connector, Kafka Connect supports connector-specific logging contexts, which can be routed to separate log files using logging configuration.

By modifying the log4j.properties (or log4j2.xml) file to define a dedicated logger and appender for the connector’s package or class name, you can direct logs from that connector to a separate file. This is the recommended and documented approach for debugging individual connectors in a shared Connect cluster.

Option A is incorrect because there is no generic “enable connector context” switch. Option B may increase verbosity but does not isolate logs per connector. Option D is false; Kafka Connect explicitly supports connector-level log separation through logging configuration.

Therefore, the correct approach is to configure a dedicated log appender for the specific connector.

RetriableException is a subclass of KafkaException, indicating transient issues that might succeed if retried. Examples include:

LeaderNotAvailableException: Happens when metadata is not yet propagated or leader election is in progress.

NotEnoughReplicasException: Happens when the number of in-sync replicas is insufficient. This can be transient if replicas come back online.

From the Apache Kafka Java Client documentation:

"These exceptions (like LeaderNotAvailableException, NotEnoughReplicasException) are transient and the client will retry them automatically depending on retry configuration."

RecordTooLargeException and AuthorizationException are non-retriable as they indicate client-side or permission errors.

max.poll.interval.ms defines the maximum delay between invocations of poll() before the consumer is considered failed. It essentially gives consumers more time to process records before needing to poll again.

From Kafka Consumer Configuration Reference:

“max.poll.interval.ms: The maximum delay between invocations of poll() when using consumer group management. If poll() is not called before expiration, the consumer is considered failed.”

session.timeout.ms and heartbeat.interval.ms relate to group coordination and heartbeats.

fetch.max.wait.ms affects how long the broker waits to accumulate data before sending a fetch response.

According to the Apache Kafka producer documentation, serialization is configured independently for message keys and values. The key and value can use different serializers depending on their data types.

In this scenario, the message key is a string, so the producer must be configured with org.apache.kafka.common.serialization.StringSerializer for the key. The value contains JSON, so a suitable JSON serializer must be configured for the value, such as a custom JSON serializer, a library-based serializer, or a byte-array serializer after converting the JSON to bytes.

Option B correctly reflects this separation by specifying a string serializer for the key and a JSON serializer for the value. Kafka does not automatically infer serializers, and setting a serializer to null is invalid. Additionally, there is no built-in “JSON” serializer for keys, making Option D incorrect.

Therefore, the correct approach—fully aligned with Kafka’s producer configuration model—is to explicitly configure a StringSerializer for the key and a JSON-capable serializer for the value.

=========================