This tech talk is one session at the beam summit. go to the event website for details and rsvp: https://www.aicamp.ai/event/eventdetails/W2022071808

Trustpilot is an e-commerce reviews platform delivering millions of new reviews to businesses each week. We are using Apache Beam on GCP Dataflow to deliver real-time streaming inferences with the latest NLP transformer models.

Our talk will touch on:

Infrastructure setup to enable Python Beam to interface with Kafka for streaming data

Taking advantage of Beam’s unified programming model to enable batch jobs for backfilling via BigQuery

Working with GPUs on Dataflow to speed up local model inference

MLOps: Using Dataflow as part of a continuous evaluation model monitoring setup