Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,846 questions
1 answer
ADF Pipeline runs successfully but data doesn't show up
Hi team Using Azure Data Factory pipeline using two web activity Source web is reading data from log analytics workspace w1 table t1 and I am passing the result of source web activity as request body of destination web activity then my destination web…
asked 2024-05-16T17:50:19.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 39%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rishabh Srivastava
60
Reputation points Microsoft Employee
answered 2024-05-16T18:03:45.96+00:00
hossein jalilian
3,880
Reputation points
0 answers
Connect SharePoint Online List with Synapse Integrated Runtime Managed Virtual Network
Getting an error when attempting to create a Linked Service for SharePoint Online List in Synapse. I've followed the instructions and prerequisites on MS documentation, but getting the following error when testing the connection: An error occurred while…
asked 2024-05-16T14:01:45.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 56.00000000000001%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC2%3C/text%3E%3C/svg%3E)
CarmeloLoPresti-2973
41
Reputation points
commented 2024-05-16T17:55:14.45+00:00
CarmeloLoPresti-2973
41
Reputation points
1 answer
Using cached data from one data flow activity as the source for another data flow activity
On this reference page... https://learn.microsoft.com/en-us/azure/data-factory/data-flow-sink ...MS says, "Write to activity output The cached sink can optionally write your output data to the input of the next pipeline activity. This will allow you…
asked 2024-05-15T14:57:15.3366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 54%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDL%3C/text%3E%3C/svg%3E)
David Lang
20
Reputation points
commented 2024-05-16T17:51:00.2666667+00:00
David Lang
20
Reputation points
0 answers
How to store an array in a json file in ADF
I have a web activity with url drive/drive-id/items/folder-id/children then I have a filter activity that filters all folders and then I'm appending it in an array. Now what I want is to store all folder ids in a json file. and later I will load that…
asked 2024-05-16T17:48:41.5333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 35%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMY%3C/text%3E%3C/svg%3E)
Mansi Yadav
0
Reputation points
edited the question 2024-05-16T17:49:16.21+00:00
Mansi Yadav
0
Reputation points
0 answers
How do I call an API that redirects on the backend with a Copy Activity in Azure Data Factory without an authentication error?
I'm running into issues in Azure Data Factory (ADF) with an API I am working with. I am unfortunately not at liberty to share the API URL or much data on the call I am making, but I can say that I have been testing this API call in Postman and through…
asked 2024-05-16T14:25:53.8566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 10%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWJ%3C/text%3E%3C/svg%3E)
Westerfield, Jared T
0
Reputation points
commented 2024-05-16T17:45:23.0433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,761
Reputation points Microsoft Employee
1 answer
filter and lookup transformation
Hi, have 2 tables table1 .jobid 1 table2 tjob2.jobid1 and jobname='test' how to filter jobid1 where jobname<>'test'
asked 2024-05-16T10:11:47.8166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
185
Reputation points
answered 2024-05-16T17:27:34.21+00:00
AnnuKumari-MSFT
31,561
Reputation points Microsoft Employee
0 answers
Data factory Authentication With Azure SQL DB + Managed Identities + Federated Credential
I am facing ADF Auth issues with Azure SQL DB. ADF is in torus tenant and Azure SQL DB is in Microsoft tenant. Below are the steps i have followed to up FCI. Created User Assigned managed identity in Torus tenant. Created App registration in Torus…
asked 2024-05-14T04:01:25.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 28.000000000000004%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVB%3C/text%3E%3C/svg%3E)
Venkatesh Bandaru
61
Reputation points Microsoft Employee
commented 2024-05-16T17:24:35.3633333+00:00
Venkatesh Bandaru
61
Reputation points Microsoft Employee
0 answers
Copy activity throwing an error, when a auto incremented column is there on the Sink side
Hello Team, I am trying to copy data from Amazon S3 to Azure SQL database. On Azure SQL side I have set an autoincrement column(Row_Index) in a table, however the copy activity throws an error that "Column 'Row_Index' does not allow…
asked 2024-05-15T22:53:21.2033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 0%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESC%3C/text%3E%3C/svg%3E)
Shweta Chavan
40
Reputation points
commented 2024-05-16T17:20:13.5933333+00:00
Shweta Chavan
40
Reputation points
0 answers
Salesforce V2 Link Service doens't work. error message: a task was canceled Source=mscorlib
I am migrating the ADF salesforce connector to the new V2 connector and now testing it. When testing the connection for the link service, it is good. However, it failed in the copy activity. with message: Failure happened on 'Source' side.…
asked 2024-05-16T13:35:58.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 81%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECL%3C/text%3E%3C/svg%3E)
Choi, Liam (he/him/his)
0
Reputation points
commented 2024-05-16T16:45:34.02+00:00
BhargavaGunnam-MSFT
26,761
Reputation points Microsoft Employee
1 answer
How can I copy all files from sharepoint (nested folder structure) to ADLS container by using Azure data factory
I need help in building an iterative approach. Let's say we have used drives/drive-id/root/children and got initial level folders and file, files I will simply copy and if there is odata.nextLink that I can easily handle. Now for folders I will use for…
asked 2024-05-14T15:58:42.53+00:00
Mansi Yadav
0
Reputation points
commented 2024-05-16T16:41:16.3166667+00:00
AnnuKumari-MSFT
31,561
Reputation points Microsoft Employee
2 answers
Azure Data Flow Sink - Connection string is not valid or missing Server/User/Database properties
Could you please assist here to resolve this issue below ? We are trying to read Delta table and store it as table in Azure SQL Database as using ADF Data Flow. Source Delta table via inline dataset from ADLS Gen2 its working fine and slink is getting…
asked 2024-05-13T06:36:57.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 84%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Kashif Ahmed
20
Reputation points
commented 2024-05-16T16:35:57.3133333+00:00
AnnuKumari-MSFT
31,561
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
CDC Start button disabled. Cannot Start CDC in Azure data factory
I created a CDC in adf and made a Git configuration to connect to Azure Devops Repos with a new branch created. Now, I checked out to a new branch. In the new branch, CDC is also there but i cannot start the CDC . It says Please publish to enable the…
asked 2024-05-03T13:15:19.5066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 71%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Sudip Bala
0
Reputation points
accepted 2024-05-16T16:30:17.46+00:00
Sudip Bala
0
Reputation points
0 answers
how to retrieve the path to workspace_publish branch OR any branch in devops , use that path in azure powershell script to read file from that branch.
I want to retrieve path to the branch and folder inside it, basically i wan to read the jsons here and change the dev keyword with uat and save the file back to the sam folder in workspace_publish branch. I can change dev to uat, but am not able to…
asked 2024-05-15T12:58:55.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 24%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EW%3C/text%3E%3C/svg%3E)
Wasim
20
Reputation points
commented 2024-05-16T16:26:35.4466667+00:00
ShaikMaheer-MSFT
38,126
Reputation points Microsoft Employee
1 answer
ADF - CDC - enable and disable CDC tables during deployment
Hi, I have setup ADF pipeline for data sync of tables between Azure SQL Database to SQL Managed Instance databases. I am using Data Flow and CDC to track changes and sink activity to replicate it to target. However, when we deploy any table changes (add…
asked 2024-05-08T04:29:23.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 83%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ashwani SRIVASTAVA
40
Reputation points
commented 2024-05-16T15:02:29.5533333+00:00
BhargavaGunnam-MSFT
26,761
Reputation points Microsoft Employee
0 answers
Error: 11402 -The value of the property '' is invalid: 'The remote name could not be resolved:
I am setting up a Linked service for ODATA in ADF, for IntegrationRuntime I am selecting "AutoResolveIntegrationRuntime" but connection to ODATA resource is failing: The value of the property '' is invalid: 'The remote name could not be…
asked 2024-05-08T05:48:44.8233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 3%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Bhat, Asif
0
Reputation points
commented 2024-05-16T14:44:10.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 66%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Patil, Rohan
0
Reputation points
1 answer
REST API Connection through ADF with Auth Error
I am working on adding a data source through a rest api in a data factory data flow. The api requires an api key, which I have. I've configured the system with the base url, auth header, and with an anonymous auth type. When testing the connection in the…
asked 2021-09-27T17:32:29.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 22%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Amanda Darcangelo
1
Reputation point
commented 2024-05-16T14:22:02.6866667+00:00
Westerfield, Jared T
0
Reputation points
5 answers
One of the answers was accepted by the question author.
Copy Data Activity failes on datatype uniqueidentifier when Staging is enabled and no column mapping configured.
Using Azure Data Factory Copy Data Activity from Azure SQL Database to Azure Synapse Analytics. The Copy Data Activity failed with InvalidCastException error 'The given value of type String from the data source cannot be converted to type…
asked 2020-06-30T14:49:54.783+00:00
XchangeVisions
301
Reputation points
commented 2024-05-16T14:18:39.53+00:00
XchangeVisions
301
Reputation points
6 answers
How can I prevent Azure Data Factory (Web Activity) from handling Redirects?
We have a requirement to perform a series of callouts to get the report and we have built the pipeline and used the web activity to make the callouts. In that process, we’re getting the below error while making a get request using the web activity. …
asked 2022-11-29T15:00:41.277+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 42%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETB%3C/text%3E%3C/svg%3E)
Tim Batchelor
6
Reputation points
commented 2024-05-16T14:14:18.9533333+00:00
Westerfield, Jared T
0
Reputation points
2 answers
One of the answers was accepted by the question author.
In Azure Data Factory, is it possible to define link service and dataset with dynamic type?
In Data Factory, is it possible to define link service and dataset with dynamic type? Since the legacy Salesforce connector will soon be deprecated, I want to create a single link service and dataset to support both the legacy and new connector instead…
asked 2024-04-22T19:49:53.3066667+00:00
Choi, Liam (he/him/his)
0
Reputation points
accepted 2024-05-16T14:00:45.23+00:00
Choi, Liam (he/him/his)
0
Reputation points
0 answers
‘Failed to execute script. Exception: 'Odbc Operation Failed.’ Error in Script activity in ADF
Hi Team, We have few pipelines in ADF where we are fetching the Data from multiple API endpoints using Copy Activity. We are connecting to API by leveraging ‘Rest API’ linked service and Integration Runtime(IR) as ‘AutoResolveIntegrationRuntime’. We are…
asked 2024-05-16T10:51:36.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 31%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
Harshvardhan Medikonda
0
Reputation points
commented 2024-05-16T13:43:20.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 50%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Somi
0
Reputation points