Skip to main content

Data Factory

Merged

Facilitate data citizens / data engineers persons tasks on Microsoft FABRIC datafactory ==> ==> Create a rich templates around REST API ingestion use case

Vote (1) Share
Bamakus Montreuil's profile image

Bamakus Montreuil on 18 Apr 2024 13:54:07

Facilitate data citizens / data engineers persons tasks on Microsoft FABRIC datafactory


==> Create a rich templates around REST API ingestion use case


As a result, we are waiting for a full pipeline exposing tasks by tasks


1) Source setup : How to get the Oauth2 access token <== inside copy task or not


2) Source setup : Push this access token on the Rest API header bearer <== inside copy task or not


3) Source setup : declare body (post) or URL (get) parameters


4) Copy the JSON format result on an SQL table (with arrays/record columns type & arrays/record potential as children )


5) Target can be a Raw/Staging /Silver layer table ( or a Temporary dataset)


6) loop on this SQL table ( or a Temporary dataset) according to call a children REST API

, passing body or URL parameters


7) Copy children api json result on a new sql tables


8) Something very very performant as ingestion pattern (similar on python) , please