Top 30 Azure Data Factory Interview Questions and Answers

These 30 questions cover what interviewers ask about Azure Data Factory (ADF), from the basics to real scenarios. Each answer is written so you can say it in an interview in under a minute. ADF questions are especially common in India for Azure data engineer roles, and most of the ideas also apply to Data Factory in Microsoft Fabric, which is covered by the DP-700 exam.

Question list

Show all 30 questions

1. What is Azure Data Factory?
2. What are pipelines, activities, datasets and linked services?
3. What is an integration runtime, and what types are there?
4. What does the Copy activity do?
5. What types of triggers does ADF have?
6. Pipeline parameters vs variables
7. What are the main control-flow activities?
8. What is the Lookup activity, and what are its limits?
9. What is the Get Metadata activity used for?
10. How do you monitor pipeline runs?
11. Mapping data flows vs the Copy activity
12. How do you build an incremental load with a watermark?
13. Schedule trigger vs tumbling window trigger
14. How do you parameterise linked services and datasets?
15. How does the ForEach activity work, and how do you control parallelism?
16. How do you handle errors and failures in a pipeline?
17. How do you store secrets and connect securely?
18. How does CI/CD work for ADF?
19. How do you copy data from an on-premises database?
20. How do you make the Copy activity faster?
21. Execute Pipeline activity: why split pipelines?
22. How do you run a pipeline when a file arrives?
23. Scenario: copy 200 tables from SQL Server to the lake with one pipeline
24. Scenario: a daily pipeline failed halfway. How do you rerun it safely?
25. Scenario: the Copy activity fails on a few bad rows
26. Scenario: process each file only once, even if the trigger fires twice
27. How do you reduce ADF costs?
28. ADF vs Azure Databricks: when do you use which?
29. ADF vs Data Factory in Microsoft Fabric
30. How would you design a metadata-driven framework in ADF?

Beginner Azure Data Factory interview questions (freshers)

1. What is Azure Data Factory?

Beginner · Very Common · Basics

Azure Data Factory (ADF) is Microsoft’s cloud service for moving and transforming data, and for orchestrating data pipelines. You build pipelines visually or as JSON, connect to more than 90 sources and destinations, copy data between them, and schedule or trigger the work. ADF mostly moves and orchestrates; heavy transformations usually run in other services it calls, such as Azure Databricks, Synapse or SQL. It is serverless: you pay for what runs, not for servers.

2. What are pipelines, activities, datasets and linked services?

Beginner · Very Common · Basics

A pipeline is a group of steps that does one job, such as loading sales data. Each step is an activity: copying data, running a notebook, looking up a value or looping. A linked service is a connection to a system, like a connection string, holding the server and how to log in. A dataset points to specific data inside that system, such as one table or a folder of files. So: activities use datasets, datasets use linked services, and pipelines hold activities.

3. What is an integration runtime, and what types are there?

Beginner · Very Common · Integration runtime

The integration runtime (IR) is the compute ADF uses to run activities. There are three types. The Azure IR is fully managed by Microsoft and is used for cloud sources and for mapping data flows. The self-hosted IR is software you install on your own machine or VM, so ADF can reach on-premises or private-network data. The Azure-SSIS IR runs existing SSIS packages in the cloud. Choosing the IR is really choosing where the work runs and what it can reach.

4. What does the Copy activity do?

Beginner · Very Common · Copy activity

The Copy activity reads data from a source and writes it to a sink, for example from SQL Server to Parquet files in Azure Data Lake Storage. Along the way it can map columns, convert file formats, compress data, and skip or log bad rows. It is the most used activity in ADF. It copies data with light changes only; for real transformations, such as joins and aggregations, use a mapping data flow or call Databricks or SQL.

5. What types of triggers does ADF have?

Beginner · Very Common · Triggers

There are four trigger types. A schedule trigger runs pipelines on a clock, such as every day at 2 AM. A tumbling window trigger runs once for each fixed, non-overlapping time window, keeps state, and supports backfills, retries and dependencies. A storage event trigger runs when a file is created or deleted in Blob or Data Lake Storage. A custom event trigger runs on events sent through Azure Event Grid. You can also run a pipeline manually or through the API.

6. Pipeline parameters vs variables

Beginner · Very Common · Parameters

Parameters are inputs given to a pipeline when it starts, such as a date or a table name; they cannot change during the run. Variables are values you can set and change during the run, with the Set Variable or Append Variable activities, for example a counter or a list you build up. Use parameters to make one pipeline reusable for many inputs, and variables for working values inside a run. Both are read with expressions such as @pipeline().parameters.tableName.

7. What are the main control-flow activities?

Beginner · Common · Control flow

Control-flow activities decide what runs and when. ForEach loops over a list. If Condition and Switch choose a branch. Until repeats until a condition is true. Wait pauses. Execute Pipeline calls another pipeline. Lookup and Get Metadata read values to drive the logic, and Set Variable stores them. Filter narrows a list, and Web calls an HTTP endpoint. Together they let one pipeline adapt to its input instead of being hard-coded.

8. What is the Lookup activity, and what are its limits?

Beginner · Common · Lookup

The Lookup activity reads a small amount of data, such as a config table or the last watermark value, and makes it available to later activities. It can return just the first row or all rows. Its output is limited: at most 5,000 rows and about 4 MB. So use it for control values and lists of things to process, not for moving real data. To process bigger lists, page through them or use a Copy activity instead.

9. What is the Get Metadata activity used for?

Beginner · Common · Get Metadata

Get Metadata returns information about a file, folder or table without reading the data itself: whether it exists, its size, its last modified time, its structure, or the list of files in a folder (childItems). It is often combined with ForEach to process every file in a folder, or with If Condition to check that a file exists before loading it. It is a cheap way to make a pipeline react to what is actually there.

10. How do you monitor pipeline runs?

Beginner · Common · Monitoring

The Monitor hub in ADF Studio shows every pipeline, activity and trigger run, with status, duration, inputs, outputs and error messages, and lets you rerun from the start or from a failed activity. Run history is kept for 45 days, so send diagnostic logs to Log Analytics for longer history and for queries across runs. Set up Azure Monitor alerts on failed runs, so people find out quickly, and add your own row counts or audit tables for data checks.

Intermediate Azure Data Factory interview questions

11. Mapping data flows vs the Copy activity

Intermediate · Very Common · Data flows

The Copy activity moves data with light changes, such as mapping columns or converting formats, and starts quickly. A mapping data flow is a visual transformation tool: joins, aggregations, derived columns, lookups, pivots and slowly changing dimensions, without writing code. Data flows run on a Spark cluster that ADF manages, so they take a few minutes to start unless you set a time-to-live to keep the cluster warm. Use Copy to move data and data flows, or Databricks, to transform it.

12. How do you build an incremental load with a watermark?

Intermediate · Very Common · Incremental load

Keep the last loaded value, such as the highest updated_at, in a watermark table. In the pipeline, use a Lookup to read the old watermark and another Lookup to get the current maximum from the source. A Copy activity then loads only rows between the two values. Finally, a Stored Procedure activity saves the new watermark, but only after the copy succeeds. Load into a staging table and MERGE into the target, so reruns do not create duplicates. See incremental loading and watermarks.

13. Schedule trigger vs tumbling window trigger

Intermediate · Very Common · Triggers

A schedule trigger just starts pipelines at set times and does not remember past runs. A tumbling window trigger runs once for each fixed time window, such as each hour, and passes the window’s start and end times to the pipeline. It keeps state: it can backfill past windows, retry a failed window, limit how many windows run at once, and wait for another tumbling window to finish first. Use it when each run must process exactly one time slice.

14. How do you parameterise linked services and datasets?

Intermediate · Common · Parameters

Add parameters to the linked service, such as server or database name, and to the dataset, such as schema, table or file path, then use them inside with expressions like @{linkedService().serverName} or @dataset().tableName. Pipelines pass the values in at run time. This lets one generic dataset and one linked service serve hundreds of tables or several environments, instead of creating a separate object for each. It is the basis of metadata-driven pipelines.

15. How does the ForEach activity work, and how do you control parallelism?

Intermediate · Common · ForEach

ForEach runs the activities inside it once for every item in a list, often a list from a Lookup or Get Metadata, and you read the current item with @item(). By default it runs items in parallel: the batch count is 20, and the maximum is 50. Turn on Sequential when the order matters or a system cannot handle many connections. Inside a ForEach you cannot nest another ForEach directly, so call a child pipeline with Execute Pipeline instead.

16. How do you handle errors and failures in a pipeline?

Intermediate · Very Common · Error handling

Every activity has retry settings: how many times and how long to wait between tries, which handles temporary errors. Connect activities with success, failure, completion or skipped paths, so a failure can trigger a logging step or an alert, for example a Web activity that posts to Teams. Log errors to a table using the activity’s output, @activity('Copy').error.message. Make steps idempotent, so a rerun is safe, and set up Azure Monitor alerts for failed pipeline runs.

17. How do you store secrets and connect securely?

Intermediate · Very Common · Security

Never put passwords in linked services or pipeline JSON. Store them in Azure Key Vault and reference them from linked services. Better still, use the data factory’s managed identity to log in to Azure services such as Storage, SQL and Key Vault, so there is no password at all. For private networks, use a managed virtual network with private endpoints, or a self-hosted integration runtime inside your network. Grant the factory only the permissions it needs.

18. How does CI/CD work for ADF?

Intermediate · Very Common · CI/CD

Connect only the development factory to Git (Azure DevOps or GitHub). Developers work in feature branches and merge to the main branch with pull requests. Publishing (or an automated build with the ADF npm package) produces ARM templates for the factory. A release pipeline deploys those templates to test and production factories, replacing environment values such as connection strings with parameters. Stop triggers before deploying and start them again afterwards, and never edit the test or production factories by hand.

19. How do you copy data from an on-premises database?

Intermediate · Common · Integration runtime

Install a self-hosted integration runtime on a machine that can reach the database, inside the company network. It connects out to Azure over HTTPS, so no inbound firewall ports are needed. Create a linked service that uses this runtime, then copy with the Copy activity as usual. For reliability and more throughput, register up to four machines in the same runtime. Keep the runtime software updated, and give its machine enough CPU and memory for the volume.

20. How do you make the Copy activity faster?

Intermediate · Common · Performance

First see where the time goes in the copy activity’s monitoring details: reading, writing or queueing. Then tune:

• Raise the data integration units (DIUs, from 4 up to 256) for cloud copies, and raise parallel copies.
• Read the source in parallel with physical or dynamic range partitions.
• Copy many small files with a wildcard instead of one by one.
• Use bulk options such as PolyBase or COPY into Synapse.
• Use staged copy through Blob storage when the sink loads faster from files. For self-hosted runtimes, add nodes or bigger machines.

21. Execute Pipeline activity: why split pipelines?

Intermediate · Common · Design

The Execute Pipeline activity calls another pipeline and can wait for it to finish. Splitting work into a parent pipeline that orchestrates and child pipelines that each do one job makes them easier to reuse, test and rerun. It also works around limits, such as no ForEach directly inside another ForEach, and keeps each pipeline small, because ADF limits how many activities one pipeline can hold. Pass parameters from parent to child to keep children generic.

22. How do you run a pipeline when a file arrives?

Intermediate · Common · Triggers

Use a storage event trigger on the Blob or Data Lake Storage container, filtered by folder path and file name ending, such as .csv. When a file is created, Azure Event Grid fires the trigger, and the pipeline receives the folder and file name through @triggerBody().folderPath and @triggerBody().fileName. Ignore empty files, and expect duplicate or late events, so make the pipeline record which files it has already processed. Writers should write a temp file and rename it, so the trigger never sees half-written files.

Advanced and scenario-based Azure Data Factory interview questions (experienced)

23. Scenario: copy 200 tables from SQL Server to the lake with one pipeline

Advanced · Very Common · Scenario

Build a metadata-driven pipeline instead of 200 copies. Keep a control table listing each table’s schema, name, load type (full or incremental), watermark column and target path. A Lookup reads the control table, and a ForEach, with a sensible batch count, runs a parameterised Copy activity for each row, using one generic source dataset and one generic sink dataset. Write Parquet files to the lake, update each table’s watermark after its copy succeeds, and log the row counts and status to an audit table.

24. Scenario: a daily pipeline failed halfway. How do you rerun it safely?

Advanced · Very Common · Scenario

First find the cause in the Monitor hub and fix it. Then rerun from the failed activity, which ADF supports, so successful steps are not repeated. This is only safe if every step is idempotent: each run processes a fixed date passed as a parameter, overwrites its own output folder or partition, and loads into tables with MERGE instead of plain inserts. Watermarks must only move after a successful load. With that design, rerunning the whole day is also safe.

25. Scenario: the Copy activity fails on a few bad rows

Advanced · Common · Scenario

Turn on fault tolerance in the Copy activity: it can skip incompatible rows, such as wrong data types or too-long values, instead of failing the whole copy. Turn on session logging, so the skipped rows and their errors are written to a storage account for review. Then add a check that fails the pipeline if the number of skipped rows passes a limit, so a broken source does not silently lose half its data. Fix the root cause with the source team.

26. Scenario: process each file only once, even if the trigger fires twice

Advanced · Common · Scenario

Event triggers can fire more than once or late, so the pipeline must protect itself. Keep a table of processed files, with the file path and its last modified time or ETag. At the start of the run, look the file up; if it is already there, end the run. After a successful load, insert the file into the table. Load the data with MERGE on a business key, so even a double run creates no duplicates, and move processed files to an archive folder.

27. How do you reduce ADF costs?

Advanced · Common · Cost

ADF charges for orchestration (activity runs), data movement (DIU-hours), data flow cluster time and the integration runtimes. To cut costs:

• Avoid thousands of tiny activity runs by batching work.
• Right-size DIUs instead of using the maximum.
• Use mapping data flows only for real transformations, and set a short time-to-live.
• Run copies on self-hosted runtimes where data is already on-premises.
• Stop unused Azure-SSIS runtimes. Check costs per pipeline in Azure Cost Management.

28. ADF vs Azure Databricks: when do you use which?

Advanced · Very Common · Architecture

They work best together. ADF is good at connecting to many systems, copying data and orchestrating steps on a schedule, with little code. Databricks is good at heavy transformations with Spark: complex joins, large aggregations, streaming and machine learning, written in Python or SQL. A common design: ADF copies raw data into the lake and calls Databricks notebooks or jobs for the transformations, or Databricks Lakeflow Jobs orchestrates everything itself when most of the work is in Databricks.

29. ADF vs Data Factory in Microsoft Fabric

Advanced · Common · Fabric

Data Factory in Microsoft Fabric is the next generation of the same idea, built into Fabric. Pipelines and many activities look familiar, but there are differences: Fabric uses connections instead of linked services and datasets, Dataflow Gen2 (Power Query) instead of mapping data flows, and it writes straight into OneLake lakehouses and warehouses. Billing comes from Fabric capacity. Existing ADF still works and is common in companies; new Microsoft-based platforms increasingly use Fabric, which is what the DP-700 exam covers.

30. How would you design a metadata-driven framework in ADF?

Advanced · Common · Design

Store all the variable parts in control tables: sources, tables, load type, watermark columns, target paths, schedules and active flags. Build a small number of generic, parameterised pipelines: a master pipeline that reads the control table, and child pipelines for full loads, incremental loads and transformations. Log every run, with start and end time, row counts, status and errors, to an audit table. Adding a new table then means inserting a row, not building a pipeline, and the same framework deploys to every environment through CI/CD.

How to prepare

Practise in a free Azure account: build a pipeline that copies a table from Azure SQL to Data Lake Storage, make it incremental with a watermark, and run it with a tumbling window trigger. For the general skills behind many of these answers, read idempotency in data pipelines, incremental loading and the Top 100 data engineering interview questions.