Google Professional-Data-Engineer Based on Real Exam Environment

Google Professional Data Engineer Exam Questions and Answers

Question 21

You have data pipelines running on BigQuery, Cloud Dataflow, and Cloud Dataproc. You need to perform health checks and monitor their behavior, and then notify the team managing the pipelines if they fail. You also need to be able to work across multiple projects. Your preference is to use managed products of features of the platform. What should you do?

Options:

Export the information to Cloud Stackdriver, and set up an Alerting policy

Run a Virtual Machine in Compute Engine with Airflow, and export the information to Stackdriver

Export the logs to BigQuery, and set up App Engine to read that information and send emails if you find a failure in the logs

Develop an App Engine application to consume logs using GCP API calls, and send emails if you find a failure in the logs

Question 22

Your company uses Looker Studio connected to BigQuery for reporting. Users are experiencing slow dashboard load times due to complex queries on a large table. The queries involve aggregations and filtering on several columns. You need to optimize query performance to decrease the dashboard load times. What should you do?

Options:

Configure Looker Studio to use a shorter data refresh interval to ensure fresh data is always displayed.

Create a materialized view in BigQuery that pre-calculates the aggregations and filters used in the Looker Studio dashboards.

Implement row-level security in BigQuery to restrict data access and reduce the amount of data processed by the queries.

Use BigQuery BI Engine to accelerate query performance by caching frequently accessed data.

Answer:

Explanation:

The scenario describes slow performance caused by complex queries with aggregations and filtering on a large table. The best way to optimize this type of workload in BigQuery for dashboarding is to pre-compute the needed data.

Materialized Views (MVs) are pre-computed views that cache the results of a query, including aggregations and filters. When a dashboard's query matches the MV's query (or a part of it), BigQuery can use the cached results, which is much faster than running the original complex query against the large raw table, directly improving dashboard load times. They are designed to improve performance and reduce costs for repeating, complex queries.

Correcting other options:

A (Shorter Refresh Interval): This would make the problem worse by triggering the slow, complex queries more frequently.

C (Row-Level Security): This is a security measure, not primarily a performance optimization. While it might slightly reduce the data scanned per user if the table is partitioned on the access column, it doesn't fundamentally speed up the complex aggregation and filtering logic which is the core problem.

D (BigQuery BI Engine): BI Engine is an in-memory analysis service for BigQuery that accelerates many SQL queries, and it is a good general option for BI. However, creating a Materialized View specifically pre-calculates the exact aggregations and filters needed for the slow dashboard, which provides a more targeted and often more dramatic performance improvement for known, complex, and recurring queries than a general-purpose caching service. The combination of MVs and BI Engine is a best practice, but the MV is the most targeted fix for pre-calculating the complex aggregations.

[Reference: Google Cloud Documentation on Materialized Views:, "In BigQuery, materialized views are pre-computed views that cache a query's results, enhancing performance and efficiency... They periodically refresh to capture changes from the underlying base tables, allowing BigQuery to read only the updated data. Materialized views improve query performance by storing precomputed results, which reduces the need to process raw data repeatedly. This caching mechanism speeds up retrieval times, especially for complex queries." (3Source: Optimizing Query Performance with BigQuery Materialized Views), "Smart tuning: BigQuery automatically rewrites queries to use materialized views whenever possible. Automatic rewriting improves query performance and reduces costs without changing query results." (Source: Use materialized views), ]

Question 23

You have a data pipeline with a Dataflow job that aggregates and writes time series metrics to Bigtable. You notice that data is slow to update in Bigtable. This data feeds a dashboard used by thousands of users across the organization. You need to support additional concurrent users and reduce the amount of time required to write the data. What should you do?

Choose 2 answers

Options:

Configure your Dataflow pipeline to use local execution.

Modify your Dataflow pipeline lo use the Flatten transform before writing to Bigtable.

Modify your Dataflow pipeline to use the CoGrcupByKey transform before writing to Bigtable.

Increase the maximum number of Dataflow workers by setting maxNumWorkers in PipelineOptions.

Increase the number of nodes in the Bigtable cluster.

Question 24

You have a data analyst team member who needs to analyze data by using BigQuery. The data analyst wants to create a data pipeline that would load 200 CSV files with an average size of 15MB from a Cloud Storage bucket into BigQuery daily. The data needs to be ingested and transformed before being accessed in BigQuery for analysis. You need to recommend a fully managed, no-code solution for the data analyst. What should you do?

Options:

Create a Cloud Run function and schedule it to run daily using Cloud Scheduler to load the data into BigQuery.

Use the BigQuery Data Transfer Service to load files from Cloud Storage to BigQuery, create a BigQuery job which transforms the data using BigQuery SQL and schedule it to run daily.

Build a custom Apache Beam pipeline and run it on Dataflow to load the file from Cloud Storage to BigQuery and schedule it to run daily using Cloud Composer.

Create a pipeline by using BigQuery pipelines and schedule it to load the data into BigQuery daily.

Answer:

Explanation:

The requirements are for a daily scheduled load, ingest, and transformation, and specifically a fully managed, no-code solution.

Ingest (Load): The BigQuery Data Transfer Service (DTS) is the fully managed, serverless, and no-code solution for batch loading files (including CSV from Cloud Storage) into BigQuery on a schedule. This is the "ingest" part.

Transform: After loading the raw data into a staging table using DTS, the transformation can be done using BigQuery SQL. This transformation query can then be automated using a Scheduled Query in BigQuery, which is also a fully managed and no-code feature that runs on a schedule.

Fully Managed & No-Code: Both DTS for Cloud Storage and Scheduled Queries are native BigQuery features that are fully managed and configured through the console without requiring code, directly meeting the constraints.

Correcting other options:

A (Cloud Run + Script): Cloud Run requires writing a custom Python script, which violates the no-code requirement.

C (Dataflow + Apache Beam + Cloud Composer): This is a powerful, highly scalable ETL solution, but it requires writing custom code (Apache Beam) and requires setting up and managing a workflow orchestrator (Cloud Composer/Airflow), which violates both the fully managed (Dataflow is serverless, but the code/pipeline itself is custom and needs maintenance) and no-code requirements.

D (BigQuery pipelines): "BigQuery pipelines" is not a distinct, official product name in the Google Cloud documentation that fulfills a no-code scheduled ETL. The closest product is the combination of DTS and Scheduled Queries, as described in option B.

[Reference: Google Cloud Documentation on BigQuery Data Transfer Service and Scheduled Queries:, "The BigQuery Data Transfer Service automates data movement into BigQuery on a scheduled, managed basis... The BigQuery Data Transfer Service supports loading data from Cloud Storage in one of the following formats: Comma-separated values (CSV)..." (Source: What is BigQuery Data Transfer Service? and Introduction to Cloud Storage transfers), "A scheduled query is a query that BigQuery automatically runs at regular intervals. When you configure a scheduled query, you specify the GoogleSQL SELECT statement to run, the destination table for the query results, and the frequency of the query." (Source: Scheduling queries), This combination delivers a fully managed, no-code ELT (Extract-Load-Transform) pipeline., ]

New Year Sale 70% Discount Offer - Ends in 0d 00h 00m 00s - Coupon code: save70

Google Professional-Data-Engineer Based on Real Exam Environment

Google Professional Data Engineer Exam Questions and Answers

Options:

Answer:

Options:

Answer:

Explanation:

Options:

Answer:

Explanation:

Options:

Answer:

Explanation:

CompTIA

Fortinet

Microsoft

Salesforce