Databricks data-engineer-professional

Choose your answer

• A

  Because the VACUUM command permanently deletes all files containing deleted records, deleted records may be accessible with time travel for around 24 hours.

• B

  Because the default data retention threshold is 24 hours, data files containing deleted records will be retained until the VACUUM job is run the following day.

• C

  Because Delta Lake time travel provides full access to the entire history of a table, deleted records can always be recreated by users with full admin privileges.

• D

  Because Delta Lake's delete statements have ACID guarantees, deleted records will be permanently purged from all storage systems as soon as a delete job completes.

• E

  Because the default data retention threshold is 7 days, data files containing deleted records will be retained until the VACUUM job is run 8 days later.

Pick an answer to check it, or press Show answer to see the correct one.

Choose your answer

• A

  Three new jobs named "Ingest new data" will be defined in the workspace, and they will each run once daily.

• B

  The logic defined in the referenced notebook will be executed three times on new clusters with the configurations of the provided cluster ID.

• C

  Three new jobs named "Ingest new data" will be defined in the workspace, but no jobs will be executed.

• D

  One new job named "Ingest new data" will be defined in the workspace, but it will not be executed.

• E

  The logic defined in the referenced notebook will be executed three times on the referenced existing all purpose cluster.

Pick an answer to check it, or press Show answer to see the correct one.

Choose your answer

• A

  Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.

• B

  Use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.

• C

  Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.

• D

  Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.

• E

  Ingest all log information into a bronze table; use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.

Pick an answer to check it, or press Show answer to see the correct one.

Choose your answer

• A

  Use Auto Loader to subscribe to new files in the account_history directory; configure a Structured Streaming trigger once job to batch update newly detected files into the account_current table.

• B

  Overwrite the account_current table with each batch using the results of a query against the account_history table grouping by user_id and filtering for the max value of last_updated.

• C

  Filter records in account_history using the last_updated field and the most recent hour processed, as well as the max last_iogin by user_id write a merge statement to update or insert the most recent value for each user_id.

• D

  Use Delta Lake version history to get the difference between the latest version of account_history and one version prior, then write these records to account_current.

• E

  Filter records in account_history using the last_updated field and the most recent hour processed, making sure to deduplicate on username; write a merge statement to update or insert the most recent value for each username.

Pick an answer to check it, or press Show answer to see the correct one.

Choose your answer

• A

  Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.

• B

  Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.

• C

  Calculate the difference between the previous model predictions and the current customer_churn_params on a key identifying unique customers before making new predictions; only make predictions on those customers not in the previous predictions.

• D

  Modify the overwrite logic to include a field populated by calling spark.sql.functions.current_timestamp() as data are being written; use this field to identify records written on a particular date.

• E

  Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.

Pick an answer to check it, or press Show answer to see the correct one.