Applying Dataflow - Patterns and AntiPatterns

GDG Cloud Sydney
Tue, Mar 5, 2019, 5:45 PM (AEDT)

About this event

Stephan Meyn, Google Cloud Engineer will be talking to us about a range of considerations and issues you may be facing when trying to apply Dataflow to a real business problem and will also cover a few samples of what not to do.

The talk assumes you know Dataflow / Apache Beam and its concepts.

Doors Open 5:45 pm
Pizzas and beer from 5:45 pm
Presentation and Q&A 6 pm - 7 pm

Dataflow is a nice and simple paradigm: read data and apply a sequence of transform then output the result. Sound simple, right? However, trying to apply this to a real business problem can reveal unexpected pot holes on the way.

Some of the points to get the conversations going are as follows. Bring your questions!
1. Selected Anti Patterns
2. Coders and how to avoid making life too hard
3. Fusing - a boon and a curse
4. Templates and how to invoke them
5. What to do with all the files you just processed
6. CSV files and no headers
7. Java vs. Phython
8. RES calls during DF

So much to discuss, we may stop and reconvene!