kafka exceptions

 When you attempt to produce to a Kafka topic and the broker is down or unreachable, the exact exception depends on your client library and configuration, but here's what typically happens:

Java/JVM Client (the reference client):

The producer operates asynchronously, so the behavior depends on timing:

• TimeoutException — This is the most common one. Specifically, if the producer can't get metadata or can't deliver the record within delivery.timeout.ms (default 120 seconds), you'll get an org.apache.kafka.common.errors.TimeoutException. The message is typically something like "Expiring X record(s) for topic-partition: delivery timeout".

• NetworkException — If there's an active connection that drops mid-send.

• DisconnectException — If the broker disconnects during the request.

• NotLeaderOrFollowerException — If the broker was up but the partition leader is unavailable.

How it surfaces:

• If you're using producer.send() with a callback, the exception is passed to the callback's onCompletion(metadata, exception) method.
• If you're calling producer.send().get() (synchronous), you'll get an ExecutionException wrapping the underlying cause (usually TimeoutException).

Key configs that control the behavior:

• delivery.timeout.ms (default 120s) — the upper bound on how long the producer will wait before giving up and throwing TimeoutException.
• request.timeout.ms (default 30s) — timeout for individual broker requests.
• retries / max.block.ms — control retry behavior and how long send() itself blocks waiting for metadata.

For other clients (Python's confluent-kafka, kafka-python, etc.), the pattern is similar — you'll generally see a delivery timeout or a KafkaError / NoBrokersAvailable exception if no brokers can be reached at all.

So the short answer: TimeoutException is the most common exception you'll encounter when a broker is down, after the delivery timeout period expires.

increase swap size

 sudo fallocate -l 4G /swapfile.new                                                                                                                                                                                                         
   sudo chmod 600 /swapfile.new                                                                                                                                                                                                               
   sudo mkswap /swapfile.new                                                                                                                                                                                                                  
   sudo swapoff /swapfile                                                                                                                                                                                                                     
   sudo swapon /swapfile.new                                                                                                                                                                                                                  
   sudo rm /swapfile                                                                                                                                                                                                                          
   sudo mv /swapfile.new /swapfile                                                                                                                                                                                                            
   free -h      

 

 

echo "username ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/username-temp