We are running logfile parsing jobs in google dataflow using the Python SDK. Data is spread over several 100s of daily logs, which we read via file-pattern from Cloud Storage. Data volume for all files is about 5-8 GB (gz files) with 50-80 million lines in total.We are running logfile parsing jobs in google d
阅读背景：
Google Dataflow / Apache Beam Python - 来自PCollection的Side-Input会导致性能下降
来源：互联网
分享到：
