Databricks Databricks-Certified-Data-Engineer-Associate Exam With Confidence Using Practice Dumps

The small country lookup DataFrame should be broadcast to the executors. A broadcast join copies the small side of the join to each worker, allowing every executor to join its partition of the billion-row events DataFrame locally. This avoids shuffling the very large DataFrame across the network and is effective when one relation is small enough to fit comfortably in executor memory. Broadcasting events_df would be impractical because it contains billions of rows. Caching or persisting both DataFrames can help when data is reused, but neither action by itself changes the join strategy or removes the expensive shuffle. Databricks exposes the PySpark broadcast function specifically to mark a DataFrame as suitable for a broadcast join. Therefore, option D is the correct optimization.

================

When migrating to Delta Live Tables (DLT) with a data pipeline that involves different programming languages across various data layers, the migration does not require unifying the pipeline into a single language. Delta Live Tables support multi-language pipelines, allowing data engineers and data analysts to work in their preferred languages, such as Python for data engineering tasks (raw, bronze, and silver layers) and SQL for data analytics tasks (gold layer). This capability is particularly beneficial in collaborative settings and leverages the strengths of each language for different stages of data processing.