Certified-Data-Engineer-Professional 무료 덤프문제 온라인 액세스

A healthcare analytics team is implementing a dimensional model in Delta Lake for patient care analysis. They have a date dimension table and are evaluating design options to ensure it supports a wide range of time-based analyses. Which design approach for the date dimension will support efficient time-based querying and aggregation?

• A.Store the date as string in ISO format (YYYY-MM-DD) for readability.
• B.Store only the date value and calculate all time attributes in queries.
• C.Create separate dimension tables for different calendar systems (fiscal, academic, etc.)
• D.Pre-calculate attributes like fiscal_period, quarter, month_name, day_of_week, and holiday.

While reviewing a query's execution in the Databricks Query Profiler, a data engineer observes that the Top Operators panel shows a Sort operator with high Time Spent and Memory Peak metrics. The Spark UI also reports frequent data spilling. How should the data engineer address this issue?

• A.Repartition the DataFrame to a single partition before sorting.
• B.Increase the number of shuffle partitions to better distribute data.
• C.Convert the sort operation to a filter operation.
• D.Switch to a broadcast join to reduce memory usage.

A data engineer is evaluating tools to build a production-grade data pipeline. The team must process change data from cloud object storage, filter out or isolate invalid records, and ensure the timely delivery of clean data to downstream consumers. The team is small, under tight deadlines, and wants to minimize operational overhead while keeping pipelines auditable and maintainable.
Which approach should the data engineer implement?

• A.Use a hybrid approach: Ingest with Auto Loader into Bronze tables, then process using SQL queries in Databricks Workflows to generate cleaned Silver and Gold tables on a schedule.
• B.Ingest data directly into Delta tables via Spark jobs, apply data quality filters using UDFs, and use LDP for creating Materialized Views.
• C.Use LDP to build declarative pipelines with Streaming Tables and Materialized Views, leveraging built-in support for data expectations and incremental processing.
• D.Implement ingestion using Auto Loader with Structured Streaming, and manage invalid data handling and table updates using checkpointing and merge logic.

A company has a task management system that tracks the most recent status of tasks. The system takes task events as input and processes events in near real-time using Lakeflow Declarative Pipelines. A new task event is ingested into the system when a task is created or the task status is changed. Lakeflow Declarative Pipelines provides a streaming table (tasks_status) for BI users to query.
The table represents the latest status of all tasks and includes 5 columns:
task_id (unique for each task)
task_name
task_owner
task_status
task_event_time
The table enables three properties: deletion vectors, row tracking, and change data feed (CDF).
A data engineer is asked to create a new Lakeflow Declarative Pipeline to enrich the tasks_status table in near real-time by adding one additional column representing task_owner's department, which can be looked up from a static dimension table (employee).
How should this enrichment be implemented?

• A.Create a new Lakeflow Declarative Pipeline: use readStream() function with option readChangeFeed to read tasks_status table CDF; enrich with the employee table; create a new streaming table as the result table and use apply_changes() function to process the changes from the enriched CDF.
• B.Create a new Lakeflow Declarative Pipeline: use the read() function to read tasks_status table; enrich with employee table; store the result in a materialized view.
• C.Create a new Lakeflow Declarative Pipeline: use the readStream() function with the option skipChangeCommits to read the tasks_status table; enrich with the employee table; store the result in a new streaming table.
• D.Create a new Lakeflow Declarative Pipeline: use the readStream() function to read tasks_status table; enrich with the employee table; store the result in a new streaming table.

A data engineer is creating a daily reporting job. There are two reporting notebooks--one for weekdays and one for weekends. An "if/else condition" task is configured as
{{job.start_time.is_weekday}} == true to route the job to either the weekday or weekend notebook tasks. The same job would be used across multiple time zones. Which action should a senior data engineer take upon reviewing the job to merge or reject the pull request?

• A.Reject, as the {{job.start_time.is_weekday}} is not a valid value reference.
• B.Merge, as the job configuration looks good.
• C.Reject, as the {{job.start_time.is_weekday}} is for the UTC timezone.
• D.Reject, as they should use {{job.trigger_time.is_weekday}} instead.