Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Dumps : Databricks Certified Associate Developer for Apache Spark 3.5 – Python

dedup_df = iot_bronze_df.dropDuplicates(["event_ts", "sensor_id", "metric_value"])

dropDuplicates accepts a list of columns to use for deduplication.

This ensures only unique records based on the specified keys are retained.

To sort a PySpark DataFrame by multiple columns with mixed sort directions, the correct usage is:

python

CopyEdit

df.orderBy("age", "salary", ascending=[True, False])

age will be sorted in ascending order

salary will be sorted in descending order

The orderBy() and sort() methods in PySpark accept a list of booleans to specify the sort direction for each column.

Documentation Reference: PySpark API - DataFrame.orderBy

Spark supports multiple broadcast joins in a single query plan, as long as each broadcasted DataFrame is small enough to fit under the configured threshold.

Execution Plan:

Spark broadcasts df2 to all executors.

Joins df1 (big) with broadcasted df2.

Then broadcasts df3 and performs another join with the intermediate result.

The result is efficient and avoids shuffling large data.

Why the other options are incorrect:

B: Multiple broadcast joins are supported in Spark 3.x.

C: The join condition is correct since all use id as the key.

D: broadcast() can be used inline; it’s valid syntax.