Fabric Workspace Settings

Choose the Spark pool for the workspace

Workspace settings decide what compute a notebook lands on before anyone writes a line of code, so the first decision is which Spark pool the workspace uses by default.

Every Fabric workspace is created with a Starter Pool already associated with it. Starter Pools are prehydrated, medium-size live pools, and their default configuration comes from the Fabric capacity SKU that was purchased. Node and machine sizing is handled for you behind the scenes, which is why an Apache Spark session commonly starts in five to ten seconds. That is a deliberate trade: you give up sizing control and get an immediate session.

To change any Spark setting in a workspace you need the Admin role in that workspace.

For workloads with specific compute requirements, a workspace admin can create a Custom Spark Pool instead, and size it around the job.

Two costs come with that flexibility. The capacity admin must leave the Customized workspace pools option enabled in the capacity's Spark Compute settings, and moving the workspace default from Starter Pool to a custom pool can push session start to around three minutes.

Decide how far item-level settings can go

A workspace admin can also decide whether individual items are allowed to depart from the workspace default at all.

Customize compute configuration for items is the switch. When it is on, users can adjust session-level properties such as driver and executor cores and driver and executor memory for individual notebooks and Spark job definitions through an Environment. When the workspace admin turns it off, the default pool and its compute configuration are used for every environment in the workspace — item-level overrides simply stop applying.

Environment is the object that carries those flexible configurations: compute properties, the Spark runtime to use, and library package dependencies for the workload.

In the Environment tab, a workspace admin can select an existing Environment as the workspace default, or create a new one from the dropdown. Choosing a default Environment fixes the Spark version and the library dependencies that Spark jobs in that workspace start from.

If the option to have a default Environment is disabled, the admin instead selects a Fabric runtime version from the available runtimes listed in the dropdown. Either way the workspace ends up with a defined runtime; the difference is whether library dependencies travel with it.

High concurrency mode is a separate setting: it lets several notebooks share a single Spark session instead of each holding its own.

Keep runaway Spark jobs inside a limit

Jobs settings control how Spark work is admitted and how long it may live, which is what stops one workload from holding a capacity indefinitely.

Every workspace is enabled with Optimistic Job Admission by default. Turning on Reserve maximum cores for active Spark jobs switches that approach off and reserves the maximum cores for the jobs instead. The Spark session timeout setting customizes session expiry for interactive notebook sessions; the default expiry for an interactive Spark session is twenty minutes.

Set maximum job lifetime is the guardrail for the opposite problem: a job that never ends. Turn it on, enter a value, choose a unit such as hours, and save, and Fabric automatically cancels any eligible job that runs longer than that.

The limit is deliberately selective. It applies only to user-submitted jobs — work a person explicitly starts, running under that person's identity.

System-managed jobs are triggered and managed by Fabric on your behalf, so they continue to run to completion. That is what keeps background maintenance healthy while user workloads stay under the guardrail. Set a limit generous enough for your longest legitimate user job, because reaching it means cancellation.

Group workspace content into domains

A domain is a way of logically grouping all the data in an organization that belongs to one area or field — most often a business department, so that department can manage its data under its own regulations and needs.

Data is grouped by associating workspaces with domains. When a workspace is associated with a domain, all the items in that workspace are associated with the domain too and receive a domain attribute as part of their metadata. A subdomain refines the grouping one level further, under a parent domain.

Two things follow from that association: users can filter content by domain in the OneLake catalog, and some tenant-level governance settings can be delegated to the domain level so each business unit sets its own rules.

What domain assignment does not do is just as examinable.

Domain assignment does not affect item visibility or accessibility for tenant users. Discovery, visibility, and access depend on the workspace role and item permissions, not on the domain. Every user in a tenant can also see every domain defined in the tenant, whatever their domain role: someone who is neither contributor nor admin of a "Finance" domain still sees it in the OneLake catalog domain filter.

A domain image is the cosmetic side of the same idea: pick an image or colour and it becomes part of the catalog theme while that domain is selected, so people can see which domain's items they are browsing.

Know who can assign a workspace, and how

Three roles share the work of creating and managing domains, and their limits are easy to confuse.

One condition sits on top of that table: to assign a workspace to a domain, a domain contributor must be a workspace admin — they must hold the Admin role in the workspace they are trying to associate.

Fabric admins and domain admins assign existing workspaces from the domain page in three ways.

• By workspace name, useful when naming conventions already encode the business context; several workspaces can be selected at once.
• By workspace admin, selecting users or security groups so that every workspace they administer joins the domain.
• By capacity, useful when each department has its own capacity.

The last two exclude "My workspaces" and affect existing workspaces only: workspaces created, or attached to those capacities, after the action are untouched.

A default domain covers that gap. Define a domain as the default for named users or security groups and Fabric scans the organization's workspaces: a workspace that already has a domain keeps it, an unassigned one is assigned to the default domain, and any new workspace those principals create afterwards is assigned too. Those principals generally become domain contributors of the workspaces assigned this way.

Tier cold data automatically with a lifecycle policy

OneLake storage tiers exist to lower retention cost: keep data that is read often in the hot tier, and move quieter data to cool or cold storage. A lifecycle management policy does that moving for you, based on when a file was created, last modified, or last accessed.

Typical policies read like this: after 30 days without modification move files to cool, and after 90 days without modification move them to cold; or, after 30 days without being accessed, move the files under one path to cool and bring a file back to hot if it is accessed again.

The scope of the arrangement is fixed: each workspace has a single lifecycle policy, and that policy holds up to ten rules. It affects all files in the workspace unless its rules are scoped to specific paths.

A rule is a JSON object with a name, an enabled flag, the type Lifecycle, and a definition built from four parts.

With no filter, a rule applies to every file in the workspace, and files cannot be excluded; when more than one filter is defined, a logical AND joins them. Each rule needs at least one action-condition pair, and if two actions land on the same file, the least expensive one wins — cold is considered less expensive than cool.

Work with access time, and update the policy

The last-access condition needs Fabric to know when files were read, and it arranges that itself. Apply a rule with the days after last access condition and OneLake automatically turns on access time tracking for the workspace and sets the last access time of every file to the current day. Remove the last rule that uses that condition and tracking is automatically turned off again.

Two details keep the tracking cheap and accurate. Only the first read in a twenty-four-hour period updates the last access time, so repeated reads do not add latency; and metadata operations, such as getting a file's properties, are not access operations and never move the timestamp. If a file is modified between reads, the last access time is the more recent of the two values.

Automatic return to hot is the companion action: available only alongside the last-access condition, it has no effect on files that were already cool before the rule existed, and it moves a file from cool back to hot only once in thirty days to avoid repeated early-deletion charges.

To manage the policy you must be a workspace admin. Two routes exist:

• The Fabric portal, under Workspace settings → OneLake → Lifecycle management, where rules are added, deleted, paused, and reactivated.
• The lifecycle policy APIs, where only full policy updates are supported: resubmit the entire policy through the import operation.

Timing matters when you promise results. Policies attempt to run once per day and may be delayed if the previous run has not finished, and new or updated rules take up to twenty-four hours to take effect. Cool storage has a minimum thirty-day retention and cold a minimum of ninety, so moving data earlier can bring early movement fees.

Size the Apache Airflow runtime

Apache Airflow Job workspace settings configure the default Airflow runtime for the workspace, and they offer the same two shapes: a Starter Pool that is configured by default, and Custom Pools you create. If customizing compute configurations for items is disabled, the Starter Pool is used for every environment in the workspace.

Starter Pools give an instant runtime that is deprovisioned when nobody is using it; Custom Pools trade that away for an always-on runtime and real sizing controls.

Creating a Custom Pool follows the workspace settings: open Data Factory → Apache Airflow Runtime Settings, expand Default Data Workflow Setting — which starts on Starter Pool — and choose New Pool.

Four properties are then yours to set: a name; a compute node size, where Large suits complex or production DAGs and Small suits simpler ones; enable autoscale, so the pool scales nodes up and down as needed; and extra nodes, which let the pool run more DAGs at the same time. Each extra node provides the capacity to run three more workers. Select Create to finish.

The settings can also be read programmatically. The Airflow workspace settings read operation is in beta, so the caller needs a viewer or higher workspace role, the Workspace.Read.All or Workspace.ReadWrite.All delegated scope, and the beta query parameter set to true. Its response returns the workspace's default pool template.

Key takeaways

• Spark compute: the Starter Pool is prehydrated from the capacity SKU and starts a session in 5-10 seconds; a Custom Spark Pool adds node size, autoscale, and dynamic executor allocation, at roughly three minutes to start.
• Item overrides: with item compute customization off, every environment uses the default pool and its configuration; the default Environment carries runtime and libraries, or the workspace picks a Fabric runtime version instead.
• Job guardrails: Optimistic Job Admission is the default, interactive sessions expire after twenty minutes, and a maximum job lifetime cancels user-submitted jobs while system-managed maintenance runs to completion.
• Domains: assignment tags every item in the workspace for discovery and delegated governance, never for access; a domain contributor must also be a workspace admin, and a default domain preserves existing assignments.
• OneLake lifecycle: one policy of up to ten rules per workspace, running daily and taking up to 24 hours to apply; the last-access condition switches access time tracking on automatically, and API updates resubmit the whole policy.
• Airflow: Starter Pool for development, Custom Pool for an always-on production runtime with node size, autoscale, and extra nodes.