Data Science-AI-Cloud Sundays

GDG Cloud Kaduna

Join us for an exciting deep dive into "Cloud-Native Data Science for Beginners". In this interactive session, we’ll unr...

Nov 1, 6:00 – 8:00 PM (WAT)

5 RSVP'd

Key themes

AIGoogle CloudTech Talk / Meetup

About this event

✨ Turn Dark Data into Structured Gold: Unlocking Unstructured Assets with Multimodal AI & Cloud Pipelines

Welcome to this practical, hands-on engineering session! Across industries, organizations sit on petabytes of dark data—unstructured customer feedback, scanned receipts, contracts, support tickets, audio logs, and sensor PDFs—that are stored but never analyzed. Because traditional analytical databases only speak structured SQL, this valuable business context remains hidden, creating storage overhead without delivering insight.

In this workshop, we will bridge the gap between messy files and structured business value. You will learn how to build an end-to-end ingestion and transformation pipeline on Google Cloud. Using multimodal Gemini models, serverless processing on Cloud Run, and BigQuery, we will extract, validate, and normalize unstructured payloads into query-ready, structured tables that can power live dashboards, business intelligence, and AI agents.

🏗️ Today’s Technical Roadmap

We will step through each stage of designing and deploying an automated dark-data-to-insights pipeline:

  1. Deconstructing Dark Data & The Ingestion Layer:

    • Understand why traditional ETL/ELT pipelines fail when dealing with variable, multi-format payloads like messy invoices, handwritten notes, and mixed-language logs.

    • Configure Cloud Storage (GCS) landing buckets with automated event triggers via Pub/Sub to initiate processing whenever new unstructured assets land.

  2. Multimodal Extraction & Schema Enforcement with Gemini:

    • Leverage multimodal Gemini models via the Vertex AI API to read across text, layout geometry, tables, and images simultaneously.

    • Enforce deterministic output structures using Structured Outputs (JSON Schema enforcement) and Pydantic models to eliminate hallucinations and formatting errors.

    • Handle edge cases: extracting line items from skewed scans, normalizing irregular date formats, and computing confidence scores for human-in-the-loop validation.

  3. Serverless Transformation & Cloud Run Orchestration:

    • Deploy a lightweight, containerized Python parser to Google Cloud Run that processes events in parallel as files are uploaded.

    • Secure pipeline credentials and API keys using Secret Manager and least-privilege IAM service accounts.

  4. Loading & Querying Structured Gold in BigQuery:

    • Stream extracted records directly into partitioned and clustered BigQuery analytical tables.

    • Write standard SQL queries to uncover business trends that were previously trapped inside unreadable files—such as spend leakage across supplier invoices or customer sentiment trends from support tickets.

    • Register the new structured tables and data lineage inside Dataplex to ensure long-term governance, discoverability, and data quality.

🌟 Why This Session is Critical

  • Unlock Buried Business Value: Learn the exact techniques required to monetize and analyze data assets that most organizations ignore or write off as storage costs.

  • Eliminate Manual Data Entry: Replace brittle regex scripts and labor-intensive manual typing with resilient, AI-powered extraction pipelines.

  • Production-Grade Architecture: Move past one-off notebook scripts by implementing a scalable, serverless event-driven architecture that runs automatically in production.

  • High-Impact Portfolio Asset: Walk away with an end-to-end cloud pipeline repository demonstrating multimodal GenAI, serverless engineering, and enterprise data warehousing.

📝 What You Need to Bring

  • Your Laptop (fully charged).

  • A Google Cloud Project: With billing enabled to access Cloud Storage, Vertex AI / Gemini API, Cloud Run, and BigQuery.

  • A Web Browser & Code Editor: Ready to deploy pipeline code and inspect cloud resources.

  • A GitHub Account: Ready to commit and version-control your schema definitions and parser scripts.

📚 Essential Developer Resources

Host

  • Peter Okwukogu

    CoLab Innovation Hub

    Google Developer Expert (GDE) for Data Cloud

Organizers

  • Peter 'Pablo' Okwukogu

    Colab Innovation Hub

    Data Scientist & Community Lead

  • Robert John

    Data Team Lead

  • Asiya Amanda Pada

    CoLab Innovation Hub

    Aspiring AI Engineer