Prepstellar
Get the app

DP-700 · Getting started

21 cards

Fabric Data Engineering End-to-End Workflow

Swipe, scroll or use ← →

  1. Follow the data journey

    A Fabric data engineering solution begins with a source and ends with usable data or an insight. Knowing the shape of that journey is what turns a long feature list into a small number of decisions, each one taken at a known point.

    The broad lifecycle is to get data, store it, prepare and transform it, analyze it, and deliver or act on the result. External integration, governance, monitoring, and lifecycle management support that path rather than interrupting it.

    Stage What happens
    Get data Bring data in from many sources, in real time, on a schedule, by continuous replication, or by reference.
    Store data Persist it in the format that suits the workload.
    Prepare and transform Clean, reshape, and enrich it without moving it out of OneLake.
    Analyze and train Query it, build models, and explore it.
    Track and visualize Report on it, monitor it live, and trigger actions.
    External integration Connect to automation, governance, and developer tooling.
    1 / 21

  2. Follow the data journey

    OneLake is what keeps the stages connected. Because every Fabric workload reads from and writes to OneLake, data does not travel between engines: a dataset ingested by a pipeline, refined in a notebook, and visualized in a Power BI report stays in one place for the whole journey.

    That is why the workflow is best read as a sequence of responsibilities rather than a requirement to copy data into a new platform for each engine. The stages describe what has to happen, not how many systems have to hold the data.

    2 / 21

  3. Quick check

    Which sequence represents the core Fabric data lifecycle?

    1. AGet data, store it, transform it, analyze it, and deliver insights

      Right. The journey runs from acquisition and storage through preparation, analysis, and delivery or action.

    2. BDeploy reports, provision a separate lake per team, then export files

      Separate lakes and exported files are the fragmentation the shared platform is built to remove.

    3. CVisualize the data, delete its source, and then create the capacity

      Visualization comes after the data exists and is prepared; sources are not discarded to reach it.

    3 / 21

  4. Get data: match the method to how it arrives

    Ingestion is not one button. Choose it according to the source and the delivery pattern, because that choice sets the latency, the cost, and whether a second copy of the data exists at all.

    Pipelines move batch or scheduled data, eventstreams ingest and route real-time events, mirroring continuously replicates operational databases, and shortcuts reference external storage without copying it.

    Method Use it when
    Data pipeline Data moves on a schedule or in batches, through connectors to the source.
    Eventstream Events arrive continuously and must be captured, transformed, and routed.
    Mirroring An operational database must be replicated continuously, without a custom ETL pipeline.
    Shortcut Data already sits in external storage and should be reached in place, with no copy.
    OneLake data sharing A live, governed dataset must be reached across tenants without copying it.
    4 / 21

  5. Get data: match the method to how it arrives

    Two of these are easy to confuse because both avoid writing an ingestion job by hand.

    Mirroring replicates: it keeps producing an up-to-date copy inside Fabric from a supported operational database, continuously and without a custom pipeline. A shortcut virtualizes: it points at a location such as Azure Data Lake, Amazon S3, or Google Cloud Storage, and the files stay where they are.

    So the question to ask is whether the data has to be present in Fabric or merely reachable from it. A scheduled pipeline would answer neither case well here: it copies, but only when it runs.

    5 / 21

  6. Quick check

    Engineering data already sits in external object storage and must be reachable from Spark with no ETL and no duplicate files. Which method fits?

    1. AA scheduled data pipeline that copies the files nightly

      A pipeline copies on a schedule, so it both builds movement logic and leaves a second copy behind.

    2. BMirroring, which continuously replicates the source

      Mirroring is for continuous replication from an operational database, and it also produces a copy in Fabric.

    3. CA OneLake shortcut to the external location

      Right. A shortcut references the external file store in place, without copying the data or building a pipeline.

    6 / 21

  7. Store data: pick the item for the workload

    The destination reflects the way the data will be processed and consumed. Fabric offers a storage item per data shape and access pattern, and choosing the wrong one shows up later as a query that cannot be written in the language the team uses.

    Item Fits
    Lakehouse Flexible big-data storage combining files and managed Delta tables, with an automatic SQL endpoint.
    Warehouse Structured relational analytics with full T-SQL, stored procedures, and ACID transactions.
    Eventhouse Real-time analytics on streaming and telemetry data using Kusto Query Language (KQL).
    SQL database Transactional workloads and operational analytics.
    Semantic model Curated business logic, measures, and hierarchies that power reports and AI.
    7 / 21

  8. Store data: pick the item for the workload

    The distinctions are worth stating as sentences, because exam scenarios describe the need rather than naming the item.

    • A lakehouse combines files and managed Delta tables for flexible big-data processing.
    • A warehouse supports structured relational analytics with T-SQL, which is the store to reach for when the requirement is written in relational terms.
    • An eventhouse stores streaming and telemetry data for KQL analytics, so it pairs with an eventstream on the ingestion side.
    • A SQL database serves transactional workloads rather than large analytical scans.
    • A semantic model adds curated business logic for reporting; it is a modeling layer, not a landing zone.
    8 / 21

  9. Quick check

    A team needs structured relational analytics with full T-SQL support. Which item fits?

    1. AAn eventhouse built for real-time telemetry

      An eventhouse is the store for streaming and telemetry data queried with KQL, not with T-SQL.

    2. BA Fabric warehouse with full T-SQL support

      Right. The warehouse is the Fabric store for structured relational analytics with full T-SQL capabilities.

    3. CA semantic model holding curated logic

      A semantic model supplies measures and hierarchies to reports; it is not the relational analytics store.

    9 / 21

  10. Keep your progress in the app

    That’s 3 of 8 quick checks. In the app they stay answered, and every lesson remembers where you left off.

    Get the app Keep reading here

  11. Prepare and transform without moving the data

    Once data is in OneLake, it can be transformed with code-first engines or low-code tools, all inside Fabric and with no movement between engines. The choice is mostly about who does the work and how much control the job needs.

    Dataflow Gen2 provides low-code Power Query transformations, while notebooks provide a code-first environment for engineering work. Notebooks can ingest, prepare, transform, and analyze data using languages such as Python, T-SQL, and Scala.

    Tool Its style
    Dataflow Gen2 A visual Power Query interface for cleansing, transforming, and enriching data.
    Notebook Code-first authoring and execution for engineering and analysis.
    User data function Reusable custom Python logic that pipelines and notebooks can invoke.
    Spark job definition The parameters that let a batch or streaming Spark application be submitted.
    10 / 21

  12. Prepare and transform without moving the data

    Spark job definitions package parameters for batch or streaming Spark applications, which is what makes a heavy job repeatable rather than something an engineer re-enters by hand.

    None of these four is a storage item, and none of them is a schedule. That matters for the next decision: choosing a transformation tool does not yet say when the transformation runs.

    11 / 21

  13. Quick check

    Which tool supplies a low-code Power Query experience for cleansing and enriching data?

    1. AA notebook

      A notebook is the code-first environment, written in languages such as Python, T-SQL, and Scala.

    2. BDataflow Gen2

      Right. Dataflow Gen2 is the visual Power Query route for cleansing, transformation, and enrichment.

    3. CA semantic model

      A semantic model holds curated business logic for reporting rather than transformation steps.

    12 / 21

  14. Orchestrate the work you have written

    Pipelines coordinate the work. A pipeline can collect, process, and transform data through a series of steps, and it can run notebook activities as part of that series.

    The engineer therefore separates transformation logic from orchestration logic: a notebook or dataflow performs the transformations, while a pipeline controls when and in what sequence the activities run. Keeping the two apart is what lets the same notebook be developed interactively in the morning and scheduled unchanged that night.

    Neither object does the other's job. A notebook does not allocate capacity or create identities, and a pipeline does not define reporting measures or replace the storage layer.

    13 / 21

  15. Quick check

    How do a notebook and a pipeline divide the work?

    1. AThe notebook promotes content to production, and the pipeline defines the reporting measures

      Promotion between stages belongs to deployment pipelines, and measures belong to a semantic model.

    2. BThe notebook holds every organizational file, and the pipeline replaces the OneLake storage layer

      Files live in OneLake, which stays the storage layer no matter which item processes the data.

    3. CThe notebook performs the code-first data work, and the pipeline orchestrates when and in what order it runs

      Right. Transformation logic lives in the notebook, and the pipeline controls the sequence and timing of activities.

    14 / 21

  16. Analyze and deliver, in batch and in real time

    Prepared data can support SQL or KQL analysis, machine learning, semantic models, reports, dashboards, and automated actions. Power BI reports visualize governed data built on semantic models, real-time dashboards monitor streaming data with KQL queries, and Activator detects conditions in streams and triggers automated actions.

    Delivery also reaches outside Fabric. REST APIs and client libraries give programmatic access, Git integration with Azure DevOps and GitHub brings version control and CI/CD to Fabric items, and Purview supplies governance, cataloging, and compliance across the estate.

    15 / 21

  17. Analyze and deliver, in batch and in real time

    Two delivery paths are worth holding side by side, because scenarios usually describe one of them.

    Requirement Path
    Nightly copy, custom code transformations, interactive reports Pipeline ingestion, notebook transformation, semantic modeling, then Power BI reporting.
    Continuous telemetry, low latency, KQL analysis An eventstream ingests and routes the events into an eventhouse, which is then analyzed with KQL.

    A weekly batch pipeline into a warehouse would not meet a low-latency requirement, and no shortcut or semantic model can play the part of an event processor or a KQL store.

    16 / 21

  18. Quick check

    A source emits telemetry continuously and the team needs low-latency ingestion with KQL analysis rather than scheduled batches. Which path fits?

    1. AUse an eventstream to ingest and route the events into an eventhouse

      Right. Eventstreams handle continuous arrival and routing, and eventhouses store streaming data for KQL analytics.

    2. BUse a weekly batch pipeline that copies the events into a warehouse

      A weekly batch is the opposite of low latency, and a warehouse is queried with T-SQL rather than KQL.

    3. CUse mirroring to build the reports without an analytical data item

      Mirroring replicates operational databases, and analysis still needs a store designed for streaming data.

    17 / 21

  19. Keep it running after delivery

    The operational loop continues after delivery, and DP-700 treats it as core engineering work rather than an afterthought: securing and managing, monitoring, troubleshooting, and optimization all belong to the role.

    A complete workflow therefore includes access control, scheduled or event-driven execution, observation of runs, correction of failures, and performance improvement. Monitoring is not a substitute for ingestion planning, it is not confined to reports, and it is not something the platform leaves to someone else.

    18 / 21

  20. Quick check

    Where does monitoring and optimization belong in an end-to-end workflow?

    1. AOnly inside Power BI, once the engineering items have been removed

      Monitoring covers pipeline and Spark runs across the solution, not just the reporting layer.

    2. BBefore any source exists, in place of planning the ingestion

      Planning ingestion is a design step; monitoring observes what the running solution actually does.

    3. CAfter delivery and throughout operations, as a continuing loop

      Right. A production solution keeps observing runs, correcting failures, securing access, and improving performance.

    19 / 21

  21. Key takeaways

    • Sequence the core lifecycle as ingest, store, transform, orchestrate, analyze, and deliver, with OneLake keeping the stages connected.
    • Select pipelines, eventstreams, mirroring, or shortcuts according to how data should arrive — scheduled movement, continuous events, continuous replication, or no-copy reference.
    • Use a lakehouse, warehouse, eventhouse, SQL database, or semantic model according to the workload, because the store decides the query language and the access pattern.
    • Keep transformation tools distinct from the pipeline that orchestrates their execution, and treat security, monitoring, and optimization as part of the same workflow.
    20 / 21

  22. Quick check

    A team must copy data nightly, transform it with custom code, and publish it in interactive reports. Which sequence is appropriate?

    1. ANotebook reporting, semantic-model storage, pipeline visuals, then lakehouse ingestion

      The stages are in the wrong order, and none of these items performs the role assigned to it here.

    2. BPipeline ingestion, notebook transformation, semantic modeling, then Power BI reporting

      Right. Scheduled movement, code-first transformation, curated business logic, and finally visualization.

    3. CEventhouse ingestion, deployment-pipeline transformation, then Spark reports

      An eventhouse serves streaming data, deployment pipelines promote content, and Spark is not the reporting surface.

    21 / 21

  23. 8 quick checks · then the test

    In the app, finishing the quick checks opens this lesson’s 10-question test, and the ones you miss come back exactly when you’re about to forget them.

    Take this lesson’s test → Next lesson: DP-700 Exam OverviewGet the app
← Previous Keep your progress · Get the app Next card →

The whole course, on your phone

Lessons you can read, audio you can listen to on the way to work, and practice that remembers what you got wrong.

Get the app