Configure FTP Server Connections in NCC Portal¶
This article describes how to set up and manage FTP server data sources in the NCC Portal, following Microsoft Learn Fabric documentation style.
Prerequisites¶
Before you begin, gather the following information:
- Server path
- Username
- Password
Step 1: Create an FTP Connection¶
To connect your FTP server to your Fabric tenant:
- Navigate to Manage Connections and Gateways and select + New to create a new connection.
- If required, create a private endpoint.
- Name your connection using the CON_NCC prefix. This naming convention ensures the connection is visible and manageable within the NCC Portal.
- Select FTP as the Connection Type and enter the server and Authentication method to connect to your FTP server.
TIP
Using the CON_NCC naming convention helps you quickly identify and manage connections in the NCC Portal.
Step 2: Add a Data Source¶
- In the NCC Portal, go to Tenant Settings > Data Sources.
- Select Add DataSource.
- Complete the following fields:
| Field | Description | Example/Default Value |
|---|---|---|
| Name | Data source name in NCC | InSpark_Sales |
| Data Source Type | Type of data source | FTP |
| Namespace | Prefix for storing data in Lakehouses | InSpark_Sales |
| Code | Identifier for pipelines | FTP_01 |
| Description | Description of data source | FTP server connection to InSpark_Sales |
| Connection | Name of the connection in Fabric | NCC_FTP_LISTS_SALES |
| Environment | NCC environment for the data source | Development |
Step 3: Create a Landing Zone Entity¶
- Go to Landing Zone Entities.
- Select New Entity.
- Fill in the required details:
| Field | Description | Example/Default Value |
|---|---|---|
| Pipeline | Not used | PL_LDZ_COPY_FROM_FTP_01 |
| Data Source | Data source for connection | InSpark_Sales |
| Source schema | Path to the file you want to extract from the FTP server | home/sales |
| Source name | Name of the file to extract, including the file extension | total_sales.csv |
| Incremental | Extract data incrementally | False |
| Has encrypted columns | Check if table has sensitive data | False |
| Entity value | Optional. Entity values reference | |
| Lake house | Lakehouse for storing data | LH_Data_Landingzone |
| File path | File path for data storage | InSpark_Sales |
| File name | File name for data storage | total_sales |
| File type | Expected file type | Csv |
Step 4: Create a Bronze Zone Entity¶
- Go to Bronze Zone Entities.
- Select New Entity.
- Enter the following information:
| Field | Description | Example/Default Value |
|---|---|---|
| Pipeline | Orchestrator pipeline for parsing | PL_BRZ_COMMAND |
| Landing zone entity | Landing zone entity to be parsed | InSpark_Sales/total_sales |
| Entity value | Optional. Entity values reference | See example below |
| Column mappings | Optional. Column mapping info | |
| Lake house | Lakehouse for storing data | LH_Bronze_Layer |
| Schema | Schema for storing data | dbo |
| Name | Table name for storing data | total_sales |
| Primary keys | Unique identifier fields (Case sensitive) | id |
Example Entity value:
| Name | Value |
|---|---|
| ColumnDelimiter | , |
| CompressionType | none |
| Encoding | UTF-8 |
| EscapeCharacter | \ |
| FirstRowIsHeader | 1 |
| RowDelimiter | \r\n |
Step 5: Create a Silver Zone Entity¶
- Go to Silver Zone Entities.
- Select New Entity.
- Provide the following details:
| Field | Description | Example/Default Value |
|---|---|---|
| Pipeline | Orchestrator pipeline for parsing | PL_SLV_COMMAND |
| Bronze layer entity | Bronze layer entity to be parsed | dbo.total_sales |
| Entity value | Optional. Entity values reference | |
| Lake house | Lakehouse for storing data | LH_Silver_Layer |
| Schema | Schema for storing data | dbo |
| Name | Table name for storing data | total_sales |
| Columns to exclude | Comma-separated columns to exclude (Case sensitive) | |
| Columns to exclude from history | Comma-separated columns to exclude from compare (Case sensitive) | |