Data Sources in NCC Portal¶
Connect your data to NCC Portal using Microsoft Fabric for secure, scalable integration. This process ensures sensitive credentials and authorization details are protected, while maintaining separation between metadata and access controls.
TIP
If you configure connections directly within a notebook, you may not need to set up a connection in Fabric.
What are Data Sources?¶
A data source in Microsoft Fabric provides a unified API endpoint for accessing your data. This standardization streamlines data discovery, querying, transformation, and analysis across your environment.
Prerequisites¶
Before you begin, ensure you have:
- Access to a Microsoft Fabric tenant.
- Permissions to create connections and data sources.
Supported Connection Types¶
NCC Portal supports a variety of integrated connection types for data ingestion and processing. Review the following table for available connectors and their capabilities. Select a connector name for configuration guidance and supported features.
| Connector | Extract data | Incremental extraction | Data privacy | Fabric connection needed |
|---|---|---|---|---|
| AFAS | ✓ | ✓ | ✓ | ✓ |
| Azure Blob Storage | ✓ | − | ✓ | ✓ |
| Azure Cosmos DB | ✓ | − | ✓ | ✓ |
| Azure Data Factory | ✓ | − | − | ✓ |
| Fabric OneLake | ✓ | − | − | - |
| FTP server | ✓ | − | − | ✓ |
| Microsoft Dynamics | ✓ | − | ✓ | ✓ |
| MySQL | ✓ | ✓ | ✓ | ✓ |
| Oracle | ✓ | ✓ | ✓ | ✓ |
| PostgreSQL | ✓ | ✓ | ✓ | ✓ |
| Pyspark Notebook | ✓ | ✓ | ✓ | − |
| Python Notebook | ✓ | ✓ | ✓ | − |
| REST API | ✓ | − | − | ✓ |
| SAP | ✓ | ✓ | ✓ | ✓ |
| SharePoint Lists | ✓ | − | ✓ | ✓ |
| SFTP | ✓ | − | − | ✓ |
| SQL Server | ✓ | ✓ | ✓ | ✓ |
NOTE
Additional connection types may be added in future releases. For the latest information, see the official documentation or release notes.
Incremental Extraction¶
Incremental extraction retrieves only new or changed data since the last extraction, optimizing resource usage and improving data ingestion efficiency.
To enable incremental extraction, configure the relevant settings in the Landing Zone Entity. The source system must provide a Date or DateTime column to track changes or new records.
NOTE
Incremental extraction is available only when the source system includes a suitable date or timestamp column.
Data Privacy¶
Data privacy features allow you to encrypt or anonymize sensitive data by specifying which columns require protection. Configure data privacy settings in the Landing Zone Entity. Protected data remains encrypted or anonymized from the Landing Zone through to the Gold Layer and is not automatically decrypted.
For more information, see Data privacy.
Fabric Connection¶
A Fabric connection indicates whether you must configure a Fabric data source connection in Microsoft Fabric before using the connector in NCC Portal.
If the Fabric connection column is marked with a check (✓), you need to set up the corresponding data source in Microsoft Fabric. This enables secure and managed connectivity between NCC Portal and your data source.
For guidance on adding and managing data sources in Fabric, see Add a data source in Microsoft Fabric.
How to Connect Your Data¶
To connect your data source to NCC Portal, follow these steps:
- Create a connection in Fabric.
- Create a data source in NCC Portal.
-
Create a landing zone entity in NCC Portal.
File type specific 4. Create a bronze entity in NCC Portal. 5. Create a silver entity in NCC Portal.
The first three steps are data source specific, while the last two depend on the extracted file type. For example, database sources may require different connections and configurations but typically result in Parquet files as the extracted file type.