You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[Shortcut for a lakehouse ](../../onelake/assign-variables-to-shortcuts.md)
77
-
- Notebook, through [NotebookUtils](../../data-engineering/notebook-utilities.md#variable-library-utilities) and [`%%configure`](../../data-engineering/author-execute-notebook.md#spark-session-configuration-magic-command)
77
+
- Notebook, through [NotebookUtils](../../data-engineering/notebook-utilities.md#variable-library-utilities) and [`%%configure`](../../data-engineering/author-execute-notebook.md#spark-session-configuration-magic-command)
78
78
-[Dataflow Gen 2](../../data-factory/dataflow-gen2-variable-library-integration.md)
79
79
-[Copy job](../../data-factory/cicd-copy-job.md)
80
+
-[User data functions](../../data-engineering/user-data-functions/connect-to-data-sources.md#get-variables-from-fabric-variable-libraries)
80
81
81
82
## Considerations and limitations
82
83
@@ -85,4 +86,4 @@ The following items support the variable library:
Copy file name to clipboardExpand all lines: docs/data-engineering/user-data-functions/connect-to-data-sources.md
+58-13Lines changed: 58 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,28 +5,29 @@ ms.author: eur
5
5
ms.reviewer: luisbosquez
6
6
author: eric-urban
7
7
ms.topic: overview
8
-
ms.date: 03/31/2025
9
-
ms.search.form: Add new data connections to user data functions items
8
+
ms.date: 11/10/2025
9
+
ms.search.form: Add new fabric item connections to user data functions items
10
10
---
11
11
12
-
# Connect to data sources from your Fabric User data functions item
12
+
# Connect to Fabric items from your Fabric User data functions item
13
13
14
-
Fabric User data functions provides native data source connections by using the Manage connections feature in the Fabric portal. This feature allows you to connect to your Fabric data sources without having to create connection strings or manage access credentials.
14
+
Fabric User data functions provide connections to supported fabric data sources and items by using the **Manage connections** feature in the Fabric portal. This feature allows you to connect to your Fabric data sources without having to create connection strings or manage access credentials. For fabric items that aren't a data source you can securely connect to those items within a function.
15
15
16
16
In this article, you learn how to:
17
17
18
18
- Create a new connection for your user data functions item.
19
19
- Use your new connection in your function code.
20
20
- Modify or delete your data connection.
21
21
22
-
## Supported data source connections in Fabric User data functions
22
+
## Supported items in Fabric User data functions
23
23
24
-
The following data sources are currently supported for Fabric User data functions:
24
+
The following items are currently supported for Fabric User data functions:
25
25
26
26
-[Fabric SQL databases](../../database/sql/overview.md) for read/write operations
27
27
-[Fabric warehouses](../../data-warehouse/create-warehouse.md) for read/write operations
28
28
-[Fabric lakehouses](../lakehouse-overview.md) for read/write operations for Lakehouse files and for read-only operations for the SQL Endpoint.
29
29
-[Fabric mirrored databases](../../mirroring/overview.md) for read-only operations
30
+
-[Fabric Variable library](../../cicd/variable-library/variable-library-overview.md) to define configuration settings as variables. [Learn more](./python-programming-model.md).
30
31
31
32
## Create a new data connection for your user data functions item
32
33
@@ -58,9 +59,9 @@ Once created, the new connection to the data source you selected is shown in the
58
59
:::image type="content" source="..\media\user-data-functions-manage-connections\manage-connections-4.png" alt-text="Screenshot of the connections side pane with a new data source connection created." lightbox="..\media\user-data-functions-manage-connections\manage-connections-4.png":::
59
60
60
61
### 3. Use your connection alias in your function code
61
-
Once you're back in the portal editor, you need to add the alias of the connection you created in the `Manage Connections` tab to your code. This alias is automatically created based on the name of the Fabric item you are connecting to.
62
+
Once you're back in the portal editor, you need to add the alias of the connection you created in the `Manage Connections` tab to your code. This alias is automatically created based on the name of the Fabric item you're connecting to.
62
63
63
-
In this case we will use a code sample called "Read data from a table in SQL Database". You can find this sample by clicking on the Edit tab, then clicking on the "Insert sample" button and navigating to "SQL Database".
64
+
In this case we'll use a code sample called "Read data from a table in SQL Database". You can find this sample by clicking on the Edit tab, then clicking on the "Insert sample" button and navigating to "SQL Database".
64
65
65
66
:::image type="content" source="..\media\user-data-functions-manage-connections\manage-connections-5-1.png" alt-text="Screenshot of Insert Sample data catalog with a list of data sources." lightbox="..\media\user-data-functions-manage-connections\manage-connections-5-1.png":::
> While this sample connects to a SQL Database, it does not need a schema or data in your database to run.
99
+
> While this sample connects to a SQL Database, it doesn't need a schema or data in your database to run.
99
100
100
101
To use the data connection you created, modify the following line in this sample: `@udf.connection(argName="sqlDB",alias="<alias for sql database>")` by replacing the value of the `alias` with the one you obtained from the `Manage Connections` menu. The following code shows this example with the value `ContosoSalesDat`:
After modifying the code, you can test your changes by using the [Test capability](./test-user-data-functions.md) in Develop mode. Once you are ready, you can publish your function using the Publish button in the toolbar. This operation may take a few minutes.
110
+
After modifying the code, you can test your changes by using the [Test capability](./test-user-data-functions.md) in Develop mode. Once you're ready, you can publish your function using the Publish button in the toolbar. This operation might take a few minutes.
110
111
111
112
:::image type="content" source="..\media\user-data-functions-manage-connections\manage-connections-6-1.png" alt-text="Screenshot of the 'Publish' button." lightbox="..\media\user-data-functions-manage-connections\manage-connections-6-1.png":::
112
113
113
-
Once the publishing is completed, you can run your function by hovering on its name in the Functions Explorer list, and clicking on the "Run" button in the side panel. The bottom panel "Output" will show the outcome of running the function.
114
+
Once the publishing is completed, you can run your function by hovering on its name in the Functions Explorer list, and clicking on the "Run" button in the side panel. The bottom panel "Output" shows the outcome of running the function.
114
115
115
116
:::image type="content" source="..\media\user-data-functions-manage-connections\manage-connections-7.png" alt-text="Screenshot of the side panel used to Run a function." lightbox="..\media\user-data-functions-manage-connections\manage-connections-7.png":::
116
117
117
118
And that's all you need to connect to a data source from your Fabric User Data Functions.
118
119
119
-
## Next steps
120
+
## Get variables from Fabric variable libraries
121
+
122
+
A [Fabric Variable Library](../../cicd/variable-library/tutorial-variable-library.md) in Microsoft Fabric is a centralized repository for managing variables that can be used across different items within a workspace. It allows developers to customize and share item configurations efficiently. Follow these steps to use Variable Libraries in your functions:
123
+
124
+
1. Add a connection to a variable library using **Manage connections** and get the **alias** for the variable library item.
125
+
1. Add a connection decorator for the variable library item. For example, `@udf.connection(argName="varLib", alias="<My Variable Library Alias>")` and replace alias to the newly added connection for the variable library item.
126
+
1. In the function definition, include an argument with type `fn.FabricVariablesClient`. This client provides methods you need to work with variables library item.
127
+
1. Use `getVariables()` method to get all the variables from the variable library.
128
+
1. To read the values of the variables use, either `["variable-name"]` or `.get("variable-name")`.
129
+
130
+
**Example**
131
+
In this example we simulate a configuration scenario for a production and a development environment. This function sets a storage path depending on the selected environment using a value retrieved from the Variable Library. The Variable Library contains a variable called `ENV` where users can set a value of `dev` or `prod`.
returnf"incorrect settings define for ENV variable"
162
+
```
163
+
164
+
## Related content
120
165
121
166
-[Create a Fabric User data functions item](./create-user-data-functions-portal.md) from within Fabric or [use the Visual Studio Code extension](./create-user-data-functions-vs-code.md)
122
-
-[Learn about the User data functions programming model](./python-programming-model.md)
167
+
-[Learn about the User data functions programming model](./python-programming-model.md)
Copy file name to clipboardExpand all lines: docs/data-engineering/user-data-functions/python-programming-model.md
+30-13Lines changed: 30 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ ms.author: eur
5
5
ms.reviewer: sumuth
6
6
author: eric-urban
7
7
ms.topic: overview
8
-
ms.date: 07/7/2025
8
+
ms.date: 11/10/2025
9
9
ms.search.form: Write new user data functions items
10
10
---
11
11
@@ -31,7 +31,7 @@ A user data functions item contains one or many functions you can invoke from th
31
31
udf = fn.UserDataFunctions()
32
32
```
33
33
34
-
- Every function is identified with a `@udf.function()` decorator. This decorator defines if your function can be invoked individually from the portal or an external invoker. Using this decorator will also require the function to have a return value. Functions with this decorator can access the connection objects denoted by the `@udf.connection` decorator.
34
+
- Every function is identified with a `@udf.function()` decorator. This decorator defines if your function can be invoked individually from the portal or an external invoker. Using this decorator also requires the function to have a return value. Functions with this decorator can access the connection objects denoted by the `@udf.connection` decorator.
35
35
36
36
**Invokable function example**
37
37
```python
@@ -118,7 +118,7 @@ The supported output data types are:
118
118
119
119
## How to write an async function
120
120
121
-
Add async decorator with your function definition in your code. With an `async` function you can improve responsiveness and efficiency of your application by handling multiple tasks at once. They are ideal for managing high volumes of I/O-bound operations. This example function reads a CSV file from a lakehouse using pandas. Function takes file name as an input parameter.
121
+
Add async decorator with your function definition in your code. With an `async` function you can improve responsiveness and efficiency of your application by handling multiple tasks at once. They're ideal for managing high volumes of I/O-bound operations. This example function reads a CSV file from a lakehouse using pandas. Function takes file name as an input parameter.
## Generic connections for Fabric items or Azure resources
191
191
192
-
Generic connections allow you to create connections to Fabric items or Azure resources using your User Data Functions item owner identity. This feature generates an Entra ID token with the item owner's identity and a provided audience type. This token is used to authenticate with Fabric items or Azure resources that support that audience type. This process will give you a similar programming experience to using managed connections objects from the [Manage Connections feature](./connect-to-data-sources.md) but only for the provided audience type in the connection.
192
+
Generic connections allow you to create connections to Fabric items or Azure resources using your User Data Functions item owner identity. This feature generates a Microsoft Entra ID token with the item owner's identity and a provided audience type. This token is used to authenticate with Fabric items or Azure resources that support that audience type. This process will give you a similar programming experience to using managed connections objects from the [Manage Connections feature](./connect-to-data-sources.md) but only for the provided audience type in the connection.
193
193
194
194
This feature uses the `@udf.generic_connection()` decorator with the following parameters:
195
195
@@ -258,18 +258,18 @@ You can connect to a [Fabric Cosmos DB item](../../database/cosmos-db/overview.m
258
258
### Connect to Azure Key Vault using a generic connection
259
259
Generic connections support connecting to an Azure Key Vault by using the `KeyVault` audience type. This type of connection requires that the Fabric User Data Functions owner has permissions to connect to the Azure Key Vault. You can use this connection to retrieve keys, secrets, or certificates by name.
260
260
261
-
You can connect to [Azure Key Vault](https://learn.microsoft.com/azure/key-vault/general/basic-concepts) to retrieve a client secret to call an API using a generic connection by following these steps:
261
+
You can connect to [Azure Key Vault](/azure/key-vault/general/basic-concepts) to retrieve a client secret to call an API using a generic connection by following these steps:
262
262
263
263
1. In your **Fabric User Data Functions item**, install the `requests`and the `azure-keyvault-secrets` libraries using the [Library Management experience](./how-to-manage-libraries.md).
264
264
265
265
1. Go to your **Azure Key Vault resource**and retrieve the `Vault URI`and the name of your key, secret or certificate.
266
266
267
267
:::image type="content" source="..\media\user-data-functions-python-programming-model\key-vault-connection-1.png" alt-text="Screenshot showing the Azure Key Vault endpoint URL and values." lightbox="..\media\user-data-functions-python-programming-model\key-vault-connection-1.png":::
268
268
269
-
1. Go back to your **Fabric User Data Functions item**and use this sample. In this sample, we will retrieve a secret from Azure Key Vault to connect to a public API. Replace the value of the following variables:
269
+
1. Go back to your **Fabric User Data Functions item**and use this sample. In this sample, we retrieve a secret from Azure Key Vault to connect to a public API. Replace the value of the following variables:
270
270
-`KEY_VAULT_URL`with the `Vault URI` you retrieved in the previous step.
271
271
-`KEY_VAULT_SECRET_NAME`with the name of your secret.
272
-
-`API_URL` variable with the URL of the API you'd like to connect to. This sample assumes that you are connecting to a public API that accepts GET requests and takes the following parameters `api-key` and `request-body`.
272
+
-`API_URL` variable with the URL of the API you'd like to connect to. This sample assumes that you're connecting to a public API that accepts GET requests and takes the following parameters `api-key`and`request-body`.
This `UserThrownError` method takes two parameters:
359
-
-`Message`: This string is returned as the error message to the application that is invoking this function.
360
-
- A dictionary of properties is returned to the application that is invoking this function.
361
-
362
-
359
+
-`Message`: This string is returned as the error message to the application that's invoking this function.
360
+
- A dictionary of properties is returned to the application that's invoking this function.
361
+
362
+
## Get variables from Fabric variable libraries
363
+
364
+
A [Fabric Variable Library](../../cicd/variable-library/tutorial-variable-library.md) in Microsoft Fabric is a centralized repository for managing variables that can be used across different items within a workspace. It allows developers to customize and share item configurations efficiently.
365
+
1. Add a connection to a variable library using **Manage connections**and get the **alias**for the variable library item.
366
+
1. Add a connection decorator, `@udf.connection(argName="varLib", alias="<My Variable Library Alias>")` to reference the alias of the variable library item.
367
+
1. In the function definition, include an argument withtype`fn.FabricVariablesClient`. This client provides methods you need to work with variables library item. For example, `def standardize_date(rawDate: str, varLib: fn.FabricVariablesClient) ->str:`
368
+
1. Use `getVariables()` method to get all the variables from the variable library.
369
+
```python
370
+
# Get all variables from the variable library item
371
+
variables = varLib.getVariables()
372
+
```
373
+
1 To read the values of the variables use, either `["variable-name"]`or`.get("variable-name")`.
374
+
```python
375
+
# Get desired format from environment or use default
376
+
date_format = variables["DATE_FORMAT"]
377
+
# Another way to get the variable
378
+
# date_format= variables.get("DATE_FORMAT")
379
+
```
363
380
364
-
## Next steps
381
+
## Related content
365
382
- [Reference API documentation](/python/api/fabric-user-data-functions/fabric.functions)
366
383
- [Create a Fabric User data functions item](./create-user-data-functions-portal.md)
367
-
- [User data functions samples](https://github.com/microsoft/fabric-user-data-functions-samples)
384
+
- [User data functions samples](https://github.com/microsoft/fabric-user-data-functions-samples)
Copy file name to clipboardExpand all lines: docs/data-factory/pricing-dataflows-gen2.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,8 +51,8 @@ At the end of each run, Dataflow Gen2 adds up the CU usage from each engine and
51
51
|---------|---------|---------|---------|
52
52
|Standard Compute (Dataflow Gen2 (CI/CD)) | Based on each mashup engine query execution duration in seconds. Standard Compute has two tier pricing depending on the query duration. | - For every second up to 10 minutes, 12 CU<br>- For every second beyond 10 minutes, 1.5 CU | Per Dataflow Gen2 item |
53
53
|Standard Compute (non CI/CD) | Based on each mashup engine query execution duration in seconds. | 16 CU| Per Dataflow Gen2 item |
54
-
|High Scale Dataflow Compute | Based on Lakehouse/Warehouse SQL engine execution (with staging enabled) duration in seconds. | 6 CU | Per workspace |
55
-
|Data movement | Based on Fast Copy run duration in hours and the used intelligent optimization throughput resources. | 1.5 CU | Per Dataflow Gen2 item |
54
+
|High Scale Dataflows Compute | Based on Lakehouse/Warehouse SQL engine execution (with staging enabled) duration in seconds. | 6 CU | Per workspace |
55
+
|Data movement | Based on Fast Copy run duration in seconds and the used intelligent optimization throughput resources. | 1.5 CU | Per Dataflow Gen2 item |
56
56
57
57
## Virtual Network Data Gateway Pricing with Dataflow Gen2
0 commit comments