Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Exam Dumps : Databricks Certified Associate Developer for Apache Spark 3.0 Exam

Explanation

The key here is to understand that columns that are passed to DataFrame.drop() are ignored if they do not exist in the DataFrame. So, passing column name associateId to transactionsDf.drop()

does not have any effect.

Passing a list to transactionsDf.drop() is not valid. The documentation (link below) shows the call structure as DataFrame.drop(*cols). The * means that all arguments that are passed to

DataFrame.drop() are read as columns. However, since a list of columns, for example ["predError", "productId", "associateId"] is not a column, Spark will run into an error.

More info: pyspark.sql.DataFrame.drop — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1, QUESTION NO: 50 (Databricks import instructions)

Explanation

transactionsDf.repartition(transactionsDf.rdd.getNumPartitions()+2)

Correct. The repartition operator is the correct one for increasing the number of partitions. calling getNumPartitions() on DataFrame.rdd returns the current number of partitions.

transactionsDf.coalesce(10)

No, after this command transactionsDf will continue to only have 8 partitions. This is because coalesce() can only decreast the amount of partitions, but not increase it.

transactionsDf.repartition(transactionsDf.getNumPartitions()+2)

Incorrect, there is no getNumPartitions() method for the DataFrame class.

transactionsDf.coalesce(transactionsDf.getNumPartitions()+2)

Wrong, coalesce() can only be used for reducing the number of partitions and there is no getNumPartitions() method for the DataFrame class.

transactionsDf.repartition(transactionsDf._partitions+2)

No, DataFrame has no _partitions attribute. You can find out the current number of partitions of a DataFrame with the DataFrame.rdd.getNumPartitions() method.

More info: pyspark.sql.DataFrame.repartition — PySpark 3.1.2 documentation, pyspark.RDD.getNumPartitions — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 3, QUESTION NO: 23 (Databricks import instructions)

Explanation

Correct code block:

itemsDf.filter(col('supplier').contains('et'))

A mixup can easily happen here between isin and contains. Since we want to check whether a column "contains" the values et, this is the operator we should use here. Note that both methods are

methods of Spark's Column object. See below for documentation links.

A specific Column object can be accessed through the col() method and not the Column() method or through col[], which is an essential thing to know here. In PySpark, Column references a generic

column object. To use it for queries, you need to link the generic column object to a specific DataFrame. This can be achieved, for example, through the col() method.

More info:

- isin documentation: pyspark.sql.Column.isin — PySpark 3.1.1 documentation

- contains documentation: pyspark.sql.Column.contains — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1, QUESTION NO: 51 (Databricks import instructions)