Data Factory
MergedFacilitate data citizens / data engineers persons tasks on Microsoft FABRIC datafactory ==> ==> Create a rich templates around REST API ingestion use case
Bamakus Montreuil on 18 Apr 2024 13:54:07
Facilitate data citizens / data engineers persons tasks on Microsoft FABRIC datafactory
==> Create a rich templates around REST API ingestion use case
As a result, we are waiting for a full pipeline exposing tasks by tasks
1) Source setup : How to get the Oauth2 access token <== inside copy task or not
2) Source setup : Push this access token on the Rest API header bearer <== inside copy task or not
3) Source setup : declare body (post) or URL (get) parameters
4) Copy the JSON format result on an SQL table (with arrays/record columns type & arrays/record potential as children )
5) Target can be a Raw/Staging /Silver layer table ( or a Temporary dataset)
6) loop on this SQL table ( or a Temporary dataset) according to call a children REST API
, passing body or URL parameters
7) Copy children api json result on a new sql tables
8) Something very very performant as ingestion pattern (similar on python) , please