【问题】Failed to send producer request with correlation id 2 to broker 0 with

简介:

部署Flume在Window环境中,Kafka部署在Linux上,从Flume发送事件到Kafka始终有一下错误,

经过长时间在网上搜索终于把问题解决,

解决办法1:

修改kafka中配置项,

#advertised.host.name=<hostname routable by clients>

注释去掉,并配置上kafka所在linux的ip地址

advertised.host.name=192.168.10.10

重启kafka。


解决办法2(推荐):

将kafka所在linux的(ip,别名)配置到Window环境中hosts文件:

1
192.168.10.10  kafka.host

Flume配置文件consumer.properties中

1
producer.sinks.r.brokerList = kafka.host:9092





2016-04-16 16:43:34,069 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - kafka.utils.Logging$class.info(Logging.scala:68)] Connected to dx.zdp.ol:9092 for producing
2016-04-16 16:43:34,069 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - kafka.utils.Logging$class.info(Logging.scala:68)] Disconnecting from dx.zdp.ol:9092
2016-04-16 16:43:34,069 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN - kafka.utils.Logging$class.warn(Logging.scala:89)] Failed to send producer request with correlation id 2 to broker 0 with
 data for partitions [OtaAudit1,0]
java.nio.channels.ClosedChannelException
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:76)
        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:75)
        at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:106)
        at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:106)
        at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:106)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:105)
        at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:105)
        at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:105)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at kafka.producer.SyncProducer.send(SyncProducer.scala:104)
        at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:259)
        at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:110)
        at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:102)
        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
        at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:102)
        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:75)
        at kafka.producer.Producer.send(Producer.scala:77)
        at kafka.javaapi.producer.Producer.send(Producer.scala:42)
        at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:135)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
2016-04-16 16:43:34,079 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - kafka.utils.Logging$class.info(Logging.scala:68)] Back off for 1000 ms before retrying send. Remaining retries = 3
2016-04-16 16:43:34,522 (agent-shutdown-hook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:79)] Stopping lifecycle supervisor 12


     本文转自巧克力黒 51CTO博客,原文链接:http://blog.51cto.com/10120275/1764526,如需转载请自行联系原作者







相关文章
|
8月前
ES集群报错 failed to send join request to maste
ES集群报错 failed to send join request to maste
101 0
|
消息中间件 Kafka 流计算
Flink读取Kafka报Error sending fetch request
实时计算Flink读取消息队列Kafka,flink日志中出现Error sending fetch request (sessionId=1510763375, epoch=12890978) to node 103: {}. org.apache.flink.kafka.shaded.org.apache.kafka.common.errors.DisconnectException: null
9617 1
Flink读取Kafka报Error sending fetch request
|
18天前
|
消息中间件
PRECONDITION_FAILED - inequivalent arg ‘type‘ for exchange ‘exchange‘ in vhost ‘/‘: received ‘direct
PRECONDITION_FAILED - inequivalent arg ‘type‘ for exchange ‘exchange‘ in vhost ‘/‘: received ‘direct
9 0
|
2月前
|
消息中间件 Java
connection error;reply-code=503;unknown exchange type ‘x-delayed-message‘
connection error;reply-code=503;unknown exchange type ‘x-delayed-message‘
22 0
|
5月前
AMQPProtocolChannelException: PRECONDITION_FAILED - inequivalent arg ‘x-message-ttl‘ for queue ‘norm
AMQPProtocolChannelException: PRECONDITION_FAILED - inequivalent arg ‘x-message-ttl‘ for queue ‘norm
|
消息中间件 Dubbo 应用服务中间件
RocketMq-Request-Reply消息
RocketMq-Request-Reply消息
RocketMq-Request-Reply消息
|
11月前
|
域名解析 消息中间件 Kubernetes
【消息队列】解决ERR 1 [topic/channel] (: no such host
【消息队列】解决ERR 1 [topic/channel] (: no such host
236 0
|
消息中间件 网络架构
OpenStack报错:MessagingTimeout: Timed out waiting for a reply to message ID
OpenStack报错:MessagingTimeout: Timed out waiting for a reply to message ID
1098 0
|
存储 机器学习/深度学习 人工智能
Data topic details | Data
数据结构结构教程 李春葆(第五版)习题
513 0
Data topic details | Data
|
存储 机器学习/深度学习 人工智能
Data topic details 6 | Data
数据结构结构教程 李春葆(第五版)习题 第六章
146 0