阅读背景:

Flume与SparkStreaming集成

来源:互联网 

1、flume创立配置文件flume-spark-tail-conf.properties

# The configuration file needs to define the sources, 
# the channels and the sinks.
# Sources, channels and sinks are defined per agent, 
# in this case called "agent"

a2.sources = r2
a2.channels = c2
a2.sinks = k2

### define sources
a2.sources.r2.type = exec
a2.sources.r2.command = tail -F /opt/datas/spark_word_count.log
a2.sources.r2.shell = /bin/bash -c

### define channels
a2.channels.c2.type = memory
a2.channels.c2.capacity = 1000
a2.channels.c2.transactionCapacity = 100

### define sinks
a2.sinks.k2.type = avro
a2.sinks.k2.hostname = bigdata.eclipse.com
a2.sinks.k2.port = 9999

### bind the sources and sinks to the channels
a2.sources.r2.channels=c2
a2.sinks.k2.channel = c2
# 




你的当前访问异常,请进行认证后继续阅读剩余内容。

分享到: