Configure MySQL Data Sources in NCC Portal¶

This article describes how to connect a MySQL Server to your NCC Portal, add it as a data source, and configure entities for data ingestion and transformation.

Prerequisites¶

Before you start, gather the following details:

Server name
Database name
Username
Password

Step 1: Connect to MySQL Server¶

To connect your MySQL Server to your Fabric tenant:

Visit Connect to MySQL server and follow the instructions to create a new connection.
If required, create a private endpoint.
Name your connection using the CON_NCC prefix. This naming convention helps you manage connections in the NCC Portal.

Step 2: Add a Data Source in NCC Portal¶

In NCC Portal, go to Tenant Settings > Data Sources.
Select Add DataSource.
Complete the following fields:

Field	Description	Example/Default Value
Name	Name of the source MySQL database
Data Source Type	Type of data source	`MySQL`
Namespace	Prefix for storing data in Lakehouses
Code	Identifier for pipelines	`MYSQL_01`
Description	Description of data source
Connection	Name of the connection in Fabric	- Set in previous step
Environment	NCC environment for the data source	`Development`

Step 3: Create a Landing Zone Entity¶

Go to Landing Zone Entities.
Select New Entity.
Fill in the required details:

Field	Description	Example/Default Value
Pipeline	Not used
Data Source	Data source for connection	- Set in previous step
Source schema	Schema name in the MySQL Server
Source name	Table or view name in the MySQL Server
Incremental	Extract data incrementally	`False`
Has encrypted columns	Indicate if the table contains sensitive data	`False`
Entity value	Optional. Entity values reference
Lake house	Lakehouse for storing data	`LH_Data_Landingzone`
File path	File path for data storage	Is filled automatically
File name	File name for data storage	Is filled automatically
File type	Expected file type	`Parquet`

TIP

Click here to see how to configure the source incrementally.

Click here to see how to apply data encryption to your sensitive data.

Step 4: Create a Bronze Zone Entity¶

Go to Bronze Zone Entities.
Select New Entity.
Enter the following information:

Field	Description	Example/Default Value
Pipeline	Orchestrator pipeline for parsing	`PL_BRZ_COMMAND`
Landing zone entity	Landing zone entity to be parsed	- Set in previous step
Entity value	Optional. Entity values reference
Column mappings	Optional. Column mapping info
Lake house	Lakehouse for storing data	`LH_Bronze_Layer`
Schema	Schema for storing data	`dbo`
Name	Table name for storing data	Is filled automatically
Primary keys	Unique identifier fields (Case sensitive)

Step 5: Create a Silver Zone Entity¶

Go to Silver Zone Entities.
Select New Entity.
Provide the following details:

Field	Description	Example/Default Value
Pipeline	Orchestrator pipeline for parsing	`PL_SLV_COMMAND`
Bronze layer entity	Bronze layer entity to be parsed	- Set in previous step
Entity value	Optional. Entity values reference
Lake house	Lakehouse for storing data	`LH_Silver_Layer`
Schema	Schema for storing data	`dbo`
Name	Table name for storing data	Is filled automatically
Columns to exclude	Comma-separated columns to exclude (Case sensitive)
Columns to exclude from history	Comma-separated columns to exclude from comparison (Case sensitive)

Example scenario¶

The company InSpark has a MySQL Server connection in Fabric named NCC_MYSQL_SALES to the database InSpark_Sales. In this database, there is a table total_sales in the schema dbo that the company wants to extract. The configuration is as follows:

Data Source¶

Field	Value
Name	`InSpark_Sales`
Data Source Type	`MySQL`
Namespace	`InSpark_Sales`
Code	`MYSQL_01`
Description	`MySQL Server connection to InSpark_Sales`
Connection	`NCC_MYSQL_SALES`
Environment	`Development`

Landing Zone Entity¶

Field	Value
Pipeline	`PL_LDZ_COPY_FROM_MYSQL_01`
Data Source	`InSpark_Sales`
Source schema	`dbo`
Source name	`total_sales`
Incremental	`False`
Entity value
Lake house	`LH_Data_Landingzone`
File path	`InSpark_Sales`
File name	`total_sales`
File type	`Parquet`

Bronze Zone Entity¶

Field	Value
Pipeline	`PL_BRZ_COMMAND`
Landing zone entity	`InSpark_Sales/total_sales`
Entity value
Column mappings
Lake house	`LH_Bronze_Layer`
Schema	`dbo`
Name	`total_sales`
Primary keys	`id`

Silver Zone Entity¶

Field	Value
Pipeline	`PL_SLV_COMMAND`
Bronze layer entity	`dbo.total_sales`
Entity value
Lake house	`LH_Silver_Layer`
Schema	`dbo`
Name	`total_sales`
Columns to exclude
Columns to exclude from history

Next steps¶

Add the source to a load planner group