Join us for a hands-on session where you will learn how to create a simple yet powerful data pipeline using Google Cloud Platform's cost-effective services. This step-by-step guide will walk you through the process of setting up and automating a data pipeline that extracts data from an API, stores it in BigQuery, and visualises it with Looker Studio.

Detailed Agenda

1. Introduction and Setup (10 minutes)

• Brief overview of Google Cloud Platform services used in the pipeline.
• Setting up a Google Cloud Project.

2. Creating a Service Account for BigQuery (10 minutes)

• Step-by-step guide to creating and configuring a service account.
• Best practices for managing permissions and security.

3. Writing a Python Script for Data Extraction (20 minutes)

• Introduction to the OpenWeatherMap API for weather data.
• Live coding session to write a Python script that fetches data from the API and formats it for BigQuery.

4. Generating the requirements.txt File (5 minutes)

• Explanation of the importance of requirements.txt in managing dependencies.
• Demonstrate how to generate this file for your project.

5. Deploying the Python Script on Cloud Functions (25 minutes)

• Detailed instructions on creating and deploying Cloud Functions.
• Testing and verifying the deployment to ensure it works as expected.

6. Automating the Pipeline with Cloud Scheduler (15 minutes)

• Setting up Cloud Scheduler to trigger the Cloud Function on a daily basis.
• Tips on managing schedules and monitoring job executions.

7. Creating a Simple Looker Studio Report (25 minutes)

• Guide to connecting BigQuery to Looker Studio.
• Building a basic report to visualise the data collected by your pipeline.

8. Q&A and Troubleshooting (10 minutes)

• Open the floor for questions from attendees.
• Common issues and how to resolve them.

**What You'll Learn:**

✅ **Create Google Cloud Projects**: Learn how to set up and manage your projects efficiently.

✅ **Authenticate with Service Account**: Understand how to create and use service accounts for secure access.

✅ **Data Extraction with Python**: Write a Python script to fetch data from a public API.

✅ **Deploy on Cloud Functions**: Automate your Python script using Cloud Functions.

✅ **Automate with Cloud Scheduler**: Schedule your data pipeline to run automatically every day.

✅ **Visualize with Looker Studio**: Create simple yet effective reports to display your data.

**Prerequisites**

To make the most of this session, attendees should:

• Have a basic understanding of Python programming.
  
• Set up a Google Cloud account.
  

**This session is ideal for:**

👨‍💻 Developers and data enthusiasts looking to get hands-on experience with Google Cloud Platform.

👩‍💻 Anyone interested in building and automating data pipelines without incurring costs.

🤖 Professionals wanting to learn how to build time and cost-effective solutions on the cloud.