9,865 questions with Azure Data Factory tags

Sort by: Updated
1 answer One of the answers was accepted by the question author.

Synchronizing a Synapse table from an on-premise Oracle table

Hi, I've a Synapse table on a dedicated SQL pool and I need to update it reading from an Oracle table. I've already implemented a Self-hosted IR and I've already read from the Oracle table rightly with the Oracle connector. The requirement from my…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,540 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-06T15:45:42.77+00:00
pmscorca 857 Reputation points
accepted 2024-06-12T20:41:02.55+00:00
pmscorca 857 Reputation points
2 answers

I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.

I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-05-21T02:20:06.9066667+00:00
ram boyapati 15 Reputation points
commented 2024-06-12T19:52:02.8866667+00:00
Ivan Claudio-Galarza 6 Reputation points
0 answers

How to run a purge command in ADX using ADF?

I tried running the purge command but it keeps returning syntax error. Even a simple command like: .purge table MyTable in database DBName allrecords Gives this error: BadRequest_SyntaxError: Request is invalid and cannot be executed The command is…

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
496 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T18:37:13.4033333+00:00
Sailesh Choyal 0 Reputation points
1 answer

File movement from sharepoint Nested Subfolders using Microsoft Graph API to blob using ADF

I am trying to copy files from sharepoint to Azure Blob Storage using Microsoft Graph API in ADF. I am able to do it when the sharepoint folder structure is flat. I am facing issue when the sharepoint has nested subfolders. Is there a way to get…

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,538 questions
Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
11,021 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
9,990 questions
asked 2024-06-05T02:34:02.96+00:00
Jagdev Sinder 0 Reputation points
commented 2024-06-12T17:44:18.6766667+00:00
Amira Bedhiafi 17,706 Reputation points
1 answer

Copy activities from two difference self hosted integration runtime

Good afternoon. I have a task to prepare a pipeline for a copy activity from an Oracle database as the source to MS SQL as the sink. The Oracle database uses a different self-hosted integration runtime. After creating the copy activity, I received this…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T08:09:25.3533333+00:00
Eugeniusz Kamiński 0 Reputation points
commented 2024-06-12T17:33:54.4966667+00:00
Amira Bedhiafi 17,706 Reputation points
0 answers

Dynamic Content Option for Password Field in SFTP Connector in Azure Synapse Analytics

I am working on a feature in Azure Synapse Analytics that connects to different SFTP servers dynamically based on a meta-data table. As I proceeded through it, I found out that in SFTP Linked Service, the PASSWORD field does not allow dynamic content.…

Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,039 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,540 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T17:19:45.9533333+00:00
Khushank Madaan 0 Reputation points
edited the question 2024-06-12T17:21:44.0533333+00:00
Khushank Madaan 0 Reputation points
2 answers

Unable to create ADF Linked Service to Azure SQL database using Self-Hosted Integrated Runtime

Hi all, I am having trouble creating a Linked Service that connects to one of our Azure SQL databases. After entering all required Linked Service fields and testing the connection using our self-hosted integration runtime, I received the following error…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-07T21:59:28.4466667+00:00
Ricardo-2519 20 Reputation points
commented 2024-06-12T17:05:25.36+00:00
phemanth 7,190 Reputation points Microsoft Vendor
0 answers

Mapping individual array values to new values based on a lookup table

In Azure Data Factory, how can I perform a lookup on each item in a string array or line of delimited text to map each item to a new value? I have a set of codes in one column. I would like to map these codes to their descriptions and have the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-05T17:59:20.6233333+00:00
David Lang 40 Reputation points
edited a comment 2024-06-12T16:43:07.4533333+00:00
David Lang 40 Reputation points
0 answers

Error de Conexión REST: TrustFailure y Certificado Inválido en Azure Data Factory

Description: Hello everyone, I'm facing an issue while trying to connect a REST service in Azure Data Factory. The configuration is as follows: Base URL: https://yam.3psi.com.pe/wsextdatsee/api/operacioncliente?datpet=20383929050_1_Lgo2023$1 …

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T16:34:44.1533333+00:00
Yamaha_motors 0 Reputation points
1 answer

Mongo DB and Mongo DB Atlas connectors in ADF compatibility

Hello, Is it true that both connectors - the Mongo DB and the Mongo DB Atlas connector are incompatible with higher than 4.2 versions of Mongo DB? Or is it only the first one and the second is may be compatible with higher versions? If they both are…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-11T08:45:01.4633333+00:00
Martin 0 Reputation points
commented 2024-06-12T16:14:44.19+00:00
Harishga 4,875 Reputation points Microsoft Vendor
0 answers

Issue with Salesforcev2Connector

Hi team, I have a question regarding this article. First I have created a Linked Service, DataSet and copy activity in the pipeline which copy from SalesForce to Azure SQL database through…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T09:48:44.23+00:00
Michał Janczewski 0 Reputation points
commented 2024-06-12T15:38:17.6566667+00:00
BhargavaGunnam-MSFT 28,106 Reputation points Microsoft Employee
1 answer

What enhancements and fixes being done on V2 snowflake connector for ADF?

Hi Microsoft Support team, I would like to know when the following features will be available for the new snowflake connector as our code will have a huge impact if these are not enabled. Multi statement execution in the script activity. Script…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-07T16:02:25.2933333+00:00
Sivaram Yarru 0 Reputation points
commented 2024-06-12T15:05:34.5166667+00:00
BhargavaGunnam-MSFT 28,106 Reputation points Microsoft Employee
1 answer

ADF dynamic expression builder to keep trailing zeros on

I working with an API which accepts datetime in a particular format like yyyy-MM-ddThh:mm:ss.fffZ for example 2021-02-30T15:30:30.070Z when I write this back to cosmosDB to store this date for incremental load for next run, the training zero is not…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-06T23:36:44.9533333+00:00
Pavithra Udayakumar 0 Reputation points
commented 2024-06-12T13:42:23.8033333+00:00
Pavithra Udayakumar 0 Reputation points
0 answers

Get jsonlines response from Rest API and load into SQL DB using ADF pipeline

Hi Guys, Requirement is to call Rest APIs which will response data in jsonlines format (not Json array) and load data into table. Tried with Web activity and Set Variable to capture the response but not getting the response in expected format. Can you…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-04-15T10:19:39.51+00:00
SarmisthaS 0 Reputation points
edited a comment 2024-06-12T12:19:42.8866667+00:00
Sarmistha Sarkar 0 Reputation points
3 answers

Datafactory insertion leads PluginNotRegisteredException,Message=Invalid type 'SalesforceV2' is provided in 'linkedServices

Getting Plug in error while inserting data (Dataset from Cloud Salesforce ) to SQL database (Onprimissive). Unfortunately it leads the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-02-22T14:34:04.67+00:00
Chandran, Vinitha 40 Reputation points
answered 2024-06-12T09:45:57.1333333+00:00
Rohan Adwant 0 Reputation points
0 answers

ErrorCode=ODataFailedClientCreation

we would like transfer data about odata to sql. Our linked service is working. Only the ADF pipeline doesn't work with folling error... …

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T05:44:04.19+00:00
T1 ADM Sven *** 0 Reputation points
commented 2024-06-12T08:41:55.01+00:00
phemanth 7,190 Reputation points Microsoft Vendor
0 answers

ADF copy activity - How to preserve hierarchy?

Hi, I tried to config a copy activity on ADF so I can get the right folder hierarchy in the sink. Source is on an FTP server with following path: folder1/folder2/variable(path)/ folder1/folder2: names are fixed and not required in the sink. …

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-12T03:58:53.45+00:00
Hieu Ngo 0 Reputation points
edited a comment 2024-06-12T08:36:07.3533333+00:00
Harishga 4,875 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Creating ADF linked service to connect to a storage account on private endpoint using service principal

HI, Currently my storage account is open and allows public access. I disabled public network access to storage and followed this article for creating a secured SFTP connection through a firewall -…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,811 questions
Azure Firewall
Azure Firewall
An Azure network security service that is used to protect Azure Virtual Network resources.
589 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,228 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-10T10:28:27.29+00:00
Shashwat Tiwary 40 Reputation points
accepted 2024-06-12T08:31:44.7266667+00:00
Shashwat Tiwary 40 Reputation points
0 answers

ADF web activity payload is to big

We are using ADF now, there is a step need send Post to Remote Sever, but this payload is too big, far more than 896KB, it's already 15MB, so we try to store the request payload to blob and then use the Copy Activity, but we facing another issue is Copy…

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,471 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,538 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-07T03:06:18.0733333+00:00
Ricky chen (陈帅) 0 Reputation points
commented 2024-06-12T05:59:45.2866667+00:00
Ricky chen (陈帅) 0 Reputation points
0 answers

Dynamic mapping adf copy activity

Hi I am trying to do a dynamic mapping between file to SQL Table in ADF Copy activity. In the mapping section, I am adding the following dynamic content , for just one column for testing purpose. But it gives error "Expression of type: 'String'…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,865 questions
asked 2024-06-10T03:37:59.9866667+00:00
Krishnamohan Nadimpalli 401 Reputation points
commented 2024-06-12T05:59:32.34+00:00
phemanth 7,190 Reputation points Microsoft Vendor