Azure data factory services

Azure Data Factory is Azure’s ETL Cloud service that offers a code-free Graphical User Interface (GUI) for serverless data integration and data transformation. It is used to prepare data, construct ETL and ELT processes, and orchestrate and monitor pipelines code-free.

Azure Databricks, an Analytics platform opens a collaborative space for Data Engineers and Data Scientists, whereas, Azure Data Factory is primarily focused on Data integration and mapping data flows. Azure Data Factory known for its GUI to drag and drop data features in creating pipelines helps visualize data flows visually, whereas Databricks uses Python, Spark, R, Java, or SQL, therefore, requires a certain amount of coding knowledge.

Though Azure and SSIS are both ETL tools, Azure Data Factory, in addition to its native data factory functionality, allows for the creation of an SSIS runtime to store and execute SSIS packages. SSIS though it comes pre-built with SQL servers, it cannot connect with services like Azure Databricks, Azure Synapse, etc. Whereas with ADF, you can connect with SSIS, Powerquery, and other services along with designing both ETL and ELT flows.

The data-driven workflows in Azure Data Factory works in 4 steps which are: Collect, Transform, Publish, & Monitor. In the Collect phase, data is collected from multiple disparate sources into blobs or clusters. In the Transform phase, data is transformed and enriched with computing services like Hadoop, Data Lake Analytics, etc. In the Publish stage, the data in business-consumable format is loaded into the BI or Analytics tool. In the Monitor stage, pipeline monitoring activities and analytics deployment is done.