Applying Dataflow - Patterns and AntiPatterns

Stephan Meyn, Google Cloud Engineer will be talking to us about a range of considerations and issues you may be facing when trying to apply Dataflow to a real business problem and will also cover a few samples of what not to do. The talk assumes you know Dataflow / Apache Beam and its concepts. Doors Open 5:45 pm Pizzas and beer from 5:45 pm Presentation and Q&A 6 pm - 7 pm Dataflow is a ni

Mar 5, 2019, 6:45 – 8:15 AM

RSVP'd

Key Themes

Data

About this event

Stephan Meyn, Google Cloud Engineer will be talking to us about a range of considerations and issues you may be facing when trying to apply Dataflow to a real business problem and will also cover a few samples of what not to do.

The talk assumes you know Dataflow / Apache Beam and its concepts.

Doors Open 5:45 pm
Pizzas and beer from 5:45 pm
Presentation and Q&A 6 pm - 7 pm

Dataflow is a nice and simple paradigm: read data and apply a sequence of transform then output the result. Sound simple, right? However, trying to apply this to a real business problem can reveal unexpected pot holes on the way.

Some of the points to get the conversations going are as follows. Bring your questions!
1. Selected Anti Patterns
2. Coders and how to avoid making life too hard
3. Fusing - a boon and a curse
4. Templates and how to invoke them
5. What to do with all the files you just processed
6. CSV files and no headers
7. Java vs. Phython
8. RES calls during DF

So much to discuss, we may stop and reconvene!

Organizers

  • Gary Lou

    Community organiser

  • Geshan Manandhar

    Simply Wall St.

    Community organiser

  • Roshan Ghale

    Practifi

    Community organiser

  • Katharine Russell

    Simply Wall St

    Community organiser

  • Adhish Panta

    Community organiser

Contact Us