Databricks Databricks-Machine-Learning-Professional Dumps

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow

MLflow provides a module called mlflow.shap that can be used to automatically calculate and log a Shapley feature importance plot using the SHAP library. The mlflow.shap.log_explanation function takes a predict function and a sample of input data, and computes the SHAP values and base values for each feature. It then logs the following artifacts to the current run:

• Base values
• SHAP values (computed using shap.KernelExplainer)
• Summary bar plot (shows the average impact of each feature on model output) The mlflow.shap.log_explanation function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc1234 References:
• mlflow.shap — MLflow 2.9.1 documentation
• Extract feature importance from a mlflow 1.9 PyFuncModel model
• 9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning
• load MLFlow model and plot feature importance with feature names

A model signature is a specification of the input and output schema of a model, which defines the names, data types, and shapes of the model inputs and outputs. Logging a model signature with an MLflow model has several benefits, such as:

• The signature can be used to document what type of data is expected as input and output by the model, which can help users and developers understand how to use the model correctly.
• The signature can be used to enforce schema validation when serving models via REST API, which can help prevent errors and ensure data quality. For example, Azure Machine Learning enforces compliance with the model signature for MLflow models that contain a signature1.
• The signature can be used to enable automatic conversion of input data to the appropriate format for the model, such as pandas DataFrame, numpy array, or PyTorch tensor. This can simplify the data preprocessing and inference steps and reduce the code complexity.

References:

• MLflow Models Documentation, Model Signature and Input Example, p. 6-7
• mlflow.models.signature Documentation, p. 1-2
• Guidelines for deploying MLflow models - Azure Machine Learning, p. 1

To replace all data in a feature table with newly computed data, you can use the fs.write_table method with the mode argument set to “overwrite”. This will overwrite the existing data in the feature table with the data in the provided DataFrame. The other options are incorrect because:

• Option A: fs.create_table will create a new feature table, not replace the data in an existing one. If the feature table already exists, it will raise an exception.
• Option B: fs.write_table without specifying the mode argument will use the default mode of “merge”, which will merge the new data with the existing data, not replace it.
• Option C: fs.write table is not a valid method name, as it is missing an underscore. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data.
• Option E: fs.create_table with the mode argument of “merge” is not a valid combination, as the mode argument is only applicable for fs.write_table. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data. References: Databricks FeatureStoreClient, Work with features in Workspace Feature Store

An HTTP Webhook is a mechanism that allows you to register a callback that is triggered by an event, such as a model registry event. The callback is an HTTP request that is sent to a specified URL, which can invoke an action or a notification on another platform or service. An HTTP Webhook is required for use cases that involve integrating the model registry with external tools or workflows that are not supported by Databricks1.

Sending a message to a Slack channel when a model version transitions stages is a use case that requires the use of an HTTP Webhook. This is because Slack is an external platform that is not natively integrated with Databricks, and the model registry events are not directly accessible by Slack. Therefore, to send a message to a Slack channel, you need to register an HTTP Webhook that is triggered by the model registry event of interest, such as MODEL_VERSION_TRANSITIONED_STAGE. The HTTP Webhook then sends a request to the Slack API endpoint that corresponds to the channel and the message content2.

The other options are incorrect because:

• Option A: Starting a testing job when a new model is registered does not require the use of an HTTP Webhook, but rather a job registry webhook. A job registry webhook is a type of webhook that triggers a job in a Databricks workspace when a model registry event occurs. A job registry webhook can be created using the Databricks REST API or the Python client databricks-registry-webhooks on PyPI3.
• Option B: Updating data in a source table for a Databricks SQL dashboard when a model version transitions to the Production stage does not require the use of an HTTP Webhook, but rather a Databricks SQL trigger. A Databricks SQL trigger is a mechanism that allows you to execute a SQL query or a notebook when a specified condition is met, such as a time interval or a file arrival. A Databricks SQL trigger can be created using the Databricks SQL UI or the Databricks REST API4.
• Option C: Sending an email alert when an automated testing job fails does not require the use of an HTTP Webhook, but rather a job alert. A job alert is a feature that allows you to send an email notification when a job run meets a specifiedcondition, such as a failure, a timeout, or a success. A job alert can be created using the Jobs UI or the Databricks REST API5.
• Option D: None of these use cases require the use of an HTTP Webhook is incorrect, as option E does require the use of an HTTP Webhook. References: MLflow Model Registry Webhooks on Databricks, Streamline MLOps With MLflow Model Registry Webhooks, Databricks SQL Triggers, Job Alerts, [Slack API]

A pyfunc model is a generic Python model that can be loaded and used to make predictions using any Python ML library. By logging a custom model class that implements the preprocessing logic as a pyfunc model, the same logic will be applied when loading the model and calling the predict method on new data. This ensures consistency and reproducibility of the model predictions. The fit method is not relevant for loading a logged model, as it is only used for training the model. The pyfunc model does not affect the parallelizability or the need for pipeline-like objects of the model deployment. References: mlflow.pyfunc, Creating custom Pyfunc models

The MLflow Model Registry provides a centralized model store, set of APIs, and UI, to collaboratively manage the full lifecycle of an MLflow Model. It provides model lineage, model versioning, model aliasing, model tagging, and annotations. The Model Registry defines four model stages that represent the lifecycle of a model: None, Staging, Production, and Archived. Each model stage has a different meaning and purpose:

• None: This is the default stage when a new model version is registered. It indicates that the model version is not ready for any downstream use cases and needs further development or testing.
• Staging: This stage indicates that the model version is ready for testing or validation in a pre-production environment. It can be used for quality assurance, user acceptance testing, or performance evaluation before moving to production.
• Production: This stage indicates that the model version is ready for deployment in a production environment. It can be used for serving predictions, scoring data, or supporting business decisions. Only one model version per registered model can be in the production stage at a time.
• Archived: This stage indicates that the model version is no longer in active use and can be safely deleted or moved to a long-term storage. It can be used for historical or audit purposes, or for restoring a previous model version if needed.

References:

• MLflow Model Registry Documentation, p. 1-2
• Introducing the MLflow Model Registry - Databricks, p. 1-2
• Manage MLflow Models — Databricks Documentation, p. 1-2

Concept drift in machine learning and data mining refers to the change in the relationships between input and output data in the underlying problem over time. In other domains, this change maybe called “covariate shift,” “dataset shift,” or “nonstationarity.” Concept drift can affect the performance and accuracy of predictive models that assume a static relationship between input and output variables. Concept drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Concept drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc References:

• Concept drift - Wikipedia
• A Gentle Introduction to Concept Drift in Machine Learning
• Model Drift & Machine Learning: Concept Drift, Feature Drift, Etc.
• Data Drift vs. Concept Drift: What Is the Difference?

Containers are a type of software packaging that can assist in real-time deployments by packaging software with its own application, tools, and libraries. Containers are isolated and portable environments that run on a shared operating system, but have their ownfile system, dependencies, configuration, and resources. Containers can help developers and operators to build, test, and deploy software faster, easier, and more reliably, as they ensure that the software runs the same way in any environment, whether it is a developer’s laptop, a test server, or a production cluster. Containers can also help to achieve scalability, high availability, and load balancing, as they can be easily created, destroyed, replicated, and orchestrated using tools such as Kubernetes, Docker Swarm, or Amazon ECS. Containers can also integrate with other tools and services, such as MLflow, Databricks, AWS CodeDeploy, and Google Cloud Deployment Manager, to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance1234 References:

• How to Package Your Application’s Infrastructure with Docker
• 21 open source DevOps tools (and what they do)
• Microservices [What is it & Top Benefits]
• 9 Best Software Deployment Tools for 2023 (Paid & Free)

To log single values like the number of trees in a random forest, you can use the mlflow.log_param function. This function allows you to log a parameter (e.g. model hyperparameter) under the current run. Parameters can be of any type, and can be logged using the following syntax:

Python

mlflow.log_param("num_trees", 100)

AI-generated code. Review and use carefully. More info on FAQ.

MLflow also offers a convenient way to log multiple parameters by indicating all of them using a dictionary1. The other options are incorrect because:

• Option A: mlflow.log_artifact is used to log output files in any format, not single values2.
• Option B: mlflow.log_model is used to log an MLflow Model along with its artifacts, not single values3.
• Option C: mlflow.log_metric is used to log numeric metrics, not single values4.
• Option E: There is a way to store values like this using the mlflow.log_param function1. References: mlflow.log_param, mlflow.log_artifact, mlflow.log_model, mlflow.log_metric

Data type tests and checks are particularly important for streaming deployments because of the following reasons:

• Because the streaming deployment is always on, all types of data must be handled without producing an error. Streaming data may come from various sources and have different formats, schemas, or quality. If the streaming deployment cannot handle the data properly, it may result in errors, failures, or inaccurate outputs. Therefore, data type tests and checks can help validate the data and ensure the streaming deployment can process it correctly.
• Because the streaming deployment is always on, there is no practitioner to debug poor model performance. Streaming data may also change over time, affecting the model performance and accuracy. If the streaming deployment does not have a mechanism to monitor and evaluate the model performance, it may not be able to detect or correct any issues. Therefore, data type tests and checks can help identify any data drift or anomalies that may impact the model performance and trigger alerts or actions.
• Because the streaming deployment is always on, there is a need to confirm that the deployment can autoscale. Streaming data may have varying volumes and velocities, requiring the streaming deployment to scale up or down accordingly. If the streaming deployment does not have a way to adjust the resources and throughput based on the data demand, it may result in bottlenecks, delays, or inefficiencies. Therefore, data type tests and checks can help measure the data load and ensure the streaming deployment can autoscale effectively. References: Streaming Data, Data Validation, Model Monitoring, Autoscaling

To compute predictions for spark_df when the missing feature values can be found in the Feature Store by searching for features by customer_id, you can use the following code block:

Python

# Get the missing features from the Feature Store using the model URI and the customer_id column

df = fs.get_missing_features(spark_df, model_uri, lookup_key="customer_id")

# Score the DataFrame using the model URI and the Feature Store Client

fs.score_batch(model_uri, df)

AI-generated code. Review and use carefully. More info on FAQ.

The fs.get_missing_features method takes a Spark DataFrame, a model URI, and a lookup key as arguments. It returns a new Spark DataFrame that contains the originalcolumns plus the missing features that are required by the model. The missing features are retrieved from the Feature Store by joining the DataFrame with the feature tables using the lookup key. The lookup key must match the primary key of the feature tables. The model URI must point to a registered model that was trained using features from the Feature Store1.

The fs.score_batch method takes a model URI and a Spark DataFrame as arguments. It applies the model to the DataFrame and returns a new Spark DataFrame that contains the original columns plus a prediction column. The model URI must point to a registered model that was trained using features from the Feature Store2.

The other options are incorrect because:

• Option A: fs.score_model is not a valid method name, as it is missing an underscore. The correct method name is fs.score_batch2.
• Option B: fs.score_model without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.
• Option D: fs.score_batch without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.
• Option E: fs.score_batch without specifying the lookup key will not work, as the fs.get_missing_features method requires a lookup key to join the DataFrame with the feature tables. The correct way is to use fs.get_missing_features with the lookup key “customer_id” before fs.score_batch12. References: Get missing features, Score batch

To drop a column from a Delta table, you need to enable column mapping mode for the table and use the ALTER TABLE statement with the DROP COLUMN clause. Alternatively, you can use the spark.read.format(“delta”).table(path) method to read the Delta table as a DataFrame and then use the drop method to drop the column. This will return a new DataFrame with the column removed. You can then overwrite the originalDelta table with the new DataFrame using the write.format(“delta”).mode(“overwrite”) method. The other options are incorrect because:

• Option A: spark.read.format(“delta”).load(path) will not read the Delta table as a table, but as a directory of files. This will not preserve the schema and metadata of the table, and will not allow you to overwrite the table with the new DataFrame.
• Option C: Delta tables can be modified using the Delta Lake APIs or SQL commands. Delta Lake supports schema evolution and column mapping to enable dropping columns from Delta tables.
• Option D: spark.read.table(path) will not work for Delta tables, as it expects a table name, not a path. To read a Delta table by its path, you need to use the spark.read.format(“delta”).load(path) or spark.read.format(“delta”).table(path) methods.
• Option E: spark.sql(“SELECT * EXCEPT star_rating FROM path”) will not drop the column from the Delta table, but only return a new DataFrame with the column excluded. This will not affect the original Delta table, and will not allow you to overwrite the table with the new DataFrame. References: Drop columns, Delta Lake Quickstart, Delta Lake column mapping

Batch deployment is the most common paradigm for machine learning projects, as it accounts for 80-90% of deployments1. Batch deployment leverages databases and object storage to process a large batch of input data at once, rather than processing each input data point individually in real time. Batch deployment is well suited for large data volume scenarios that need efficient processing but response time isn’t critical. For example, you might use batch deployment to generate recommendations for customers based on their purchase history, or to detect anomalies in sensor data collected over a period of time. Batch deployment is less expensive and more efficient than real-time deployment, as the underlying compute runs only during the batch job and can be scaled up or down as needed. Batch deployment can be implemented using Databricks Machine Learning endpoints, which provide a unified interface to invoke and manage model deployments across compute types2. References:

• Databricks Machine Learning Professional Exam Guide, Section 3: Model Deployment, p. 4
• Databricks Machine Learning Documentation, Deploying ML Models, Batch Inference, p. 1
• Machine Learning in Production - Databricks, The Four Deployment Paradigms, p. 9
• Deploy machine learning models in production environments, Deployment methods, p. 1

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks
• [Continuous Machine Learning with Structured Streaming and MLflow - Databricks]

To restore the original scikit-learn model object, the correct method is mlflow.sklearn.load_model, which takes the model URI as an argument and returns a scikit-learn model. This method can load models with the sklearn flavor, which preserves the original attributes and methods of the model. The mlflow.load_model method is a generic method that can load models with any flavor, but it returns a pyfunc model, which does not have the feature_importances_ attribute. The client.list_artifacts method is used to list theartifacts associated with a run, not to load a model. The client.pyfunc.load_model method is similar to mlflow.load_model, but it takes a local file path instead of a model URI. The MLflow Experiments UI can show the logged parameters, metrics, and artifacts of a run, but it cannot show the model object itself. References:

• MLflow Models
• MLflow Python API

To log feature importance data from a CSV file as an artifact of an MLflow run, the correct method is mlflow.log_artifact, which takes the local file path and the artifact path as arguments.The mlflow.log_model method is used to log a model with a specific flavor, such as pyfunc or spark. The mlflow.log_data method does not exist. The mlflow.log_model_and_data method also does not exist, and it is not necessary to log both the model and the data in this case. References:

• MLflow Tracking API
• MLflow Models

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References: MLflow Model Registry Webhooks, MLflow Model Registry Events