Skip to content

Configure SFTP Data Sources in NCC Portal

This article describes how to configure and manage SFTP server connections in the NCC Portal, following Microsoft Learn Fabric documentation style.

Prerequisites

Before you start, make sure you have:

  • Server path
  • Username
  • Password

Step 1: Create an SFTP Connection

To connect your SFTP server to your Fabric tenant:

  1. Go to Manage Connections and Gateways and select + New to create a new connection.
  2. If required, create a private endpoint.
  3. Name your connection using the CON_NCC prefix. This naming convention helps ensure visibility and manageability within the NCC Portal.
  4. Select SFTP as the connection type and provide the server and authentication method details.

TIP
Using the CON_NCC naming convention helps you quickly identify and manage connections in the NCC Portal.

Step 2: Add a Data Source

  1. In the NCC Portal, navigate to Tenant Settings > Data Sources.
  2. Select Add DataSource.
  3. Complete the following fields:
Field Description Example/Default Value
Name Data source name in NCC InSpark_Sales
Data Source Type Type of data source SFTP
Namespace Prefix for storing data in Lakehouses InSpark_Sales
Code Identifier for pipelines SFTP_01
Description Description of the data source SFTP server connection to InSpark_Sales
Connection Name of the connection in Fabric NCC_SFTP_LISTS_SALES
Environment NCC environment for the data source Development

Step 3: Create a Landing Zone Entity

  1. Go to Landing Zone Entities.
  2. Select New Entity.
  3. Fill in the required details:
Field Description Example/Default Value
Pipeline Not used PL_LDZ_COPY_FROM_SFTP_01
Data Source Data source for connection InSpark_Sales
Source schema Path to the file you want to extract from the SFTP server home/sales
Source name Name of the file to extract, including the file extension total_sales.csv
Incremental Extract data incrementally False
Has encrypted columns Indicates if the table contains sensitive data False
Entity value Optional. Entity values reference
Lake house Lakehouse for storing data LH_Data_Landingzone
File path File path for data storage InSpark_Sales
File name File name for data storage total_sales
File type Expected file type Csv

Step 4: Create a Bronze Zone Entity

  1. Go to Bronze Zone Entities.
  2. Select New Entity.
  3. Enter the following information:
Field Description Example/Default Value
Pipeline Orchestrator pipeline for parsing PL_BRZ_COMMAND
Landing zone entity Landing zone entity to be parsed InSpark_Sales/total_sales
Entity value Optional. Entity values reference See example below
Column mappings Optional. Column mapping info
Lake house Lakehouse for storing data LH_Bronze_Layer
Schema Schema for storing data dbo
Name Table name for storing data total_sales
Primary keys Unique identifier fields (Case sensitive) id

Example Entity value:

Name Value
ColumnDelimiter ,
CompressionType none
Encoding UTF-8
EscapeCharacter \
FirstRowIsHeader 1
RowDelimiter \r\n

Step 5: Create a Silver Zone Entity

  1. Go to Silver Zone Entities.
  2. Select New Entity.
  3. Provide the following details:
Field Description Example/Default Value
Pipeline Orchestrator pipeline for parsing PL_SLV_COMMAND
Bronze layer entity Bronze layer entity to be parsed dbo.total_sales
Entity value Optional. Entity values reference
Lake house Lakehouse for storing data LH_Silver_Layer
Schema Schema for storing data dbo
Name Table name for storing data total_sales
Columns to exclude Comma-separated columns to exclude (Case sensitive)
Columns to exclude from history Comma-separated columns to exclude from compare (Case sensitive)

Next steps