I am using the azure policy to whitelist the domain for outbound connectivity from Azure Data Factory to other services. But facing issues in connectivity due to throttling applied on policy.

Harshwardhan Deshmukh 5

I am using the azure policy (https://learn.microsoft.com/en-us/azure/data-factory/configure-outbound-allow-list-azure-policy) which is applied at resource group level. This policy is working as expected and is only allowing outbound connectivity to the whitelisted linked services url's.

But I am facing some throttling issues as mentioned in the document, For an individual Azure Data factory: 1,000 requests / 5 minutes. Only 1,000 activity runs can be executed in a 5-minute period. Subsequent run requests fail once this limit is reached.

But even after lot of time with no requests, the connectivity always fails to the whitelisted linked service. Is it like once I exhaust the above limit then no further request will be successful and there is no reset time for the above applied throttling??
If from azure data factory studio I disable this policy, the connectivity again starts working as expected, enabling again results in same error.

Error ->
'["Connector or activity name: dfs_storage_in, connector or activity type: AzureBlobFS, error: Calling partner RP EvaluatePolicyAsync returned an invalid status code 'Forbidden', ReasonPhrase Forbidden","Connector or activity name: erds_sql, connector or activity type: AzureSqlDatabase, error: Calling partner RP EvaluatePolicyAsync returned an invalid status code 'Forbidden', ReasonPhrase Forbidden"]'

Also wanted to know when is the official release of this azure policy planned if any idea? As I am planning to use it in my project.

Harishga 4,485 Reputation points Microsoft Vendor

2023-11-28T13:04:32.3733333+00:00
Hi@Harshwardhan Deshmukh
Welcome to Microsoft Q&A platform and thanks for posting your question here.

Based on the error message you provided, it seems like the Azure Policy is blocking the outbound connectivity to the whitelisted linked services. The error message indicates that the policy is returning a 'Forbidden' status code, which means that the policy is blocking the request.

To resolve the issue, you can try the following steps:

Check if the policy is still applied to the resource group and the Azure Data Factory. You can do this by navigating to the Azure Policy blade in the Azure portal and checking the policy assignments.

Check if the whitelist URLs are correct and up to date. You can do this by navigating to the Linked Services blade in the Azure Data Factory and checking the URLs.

Check if the Azure Data Factory has the necessary permissions to access the whitelisted linked services. You can do this by checking the access policies for the linked services.

Regarding the throttling issue, once you exhaust the limit of 1,000 requests in a 5-minute period, subsequent requests will fail until the limit is reset. The reset time for the limit is not mentioned in the documentation, but it is recommended to wait for at least 5 minutes before making additional requests.

Regarding the official release of the Azure Policy, I do not have any information on that. However, you can keep an eye on the Azure Policy documentation for any updates on the release date.

Reference:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling
I hope this information helps you resolve the issue. Let me know if you have any further questions or concerns.
Harshwardhan Deshmukh 0 Reputation points

2023-11-28T14:36:13.47+00:00

Hello @Harishga thank you for your answer

The ADF and its connectivity to other whitelisted linked services URLs were working fine before the increase in workload with the policy enabled.
But after this workload increase / 5 mins, it gives the above-forbidden error every time, and now the connection to all of the linked service work shows the forbidden error message.
Regarding the reset time waiting before executing new requests, there has been no activity on the ADF for 2 days due to this failure, but this reset time for the limit doesn't seem to work for new requests and still shows the same Forbidden error.

The policy, whitelisted URL's, and the permissions to data factory are configured correctly
Harishga 4,485 Reputation points Microsoft Vendor

2023-11-29T17:19:14.6233333+00:00

Hi@Harshwardhan Deshmukh
Based on the documentation, the throttling limit for Azure Data Factory is 1,000 requests per 5 minutes. Once this limit is reached, subsequent run requests will fail until the 5-minute period is over. After the 5-minute period, the limit is reset, and new requests can be executed. However, if still experiencing the same Forbidden error after 2 days of no activity, it is possible that there may be other factors causing the issue. It is recommended to check the Azure portal for any alerts or notifications related to the Azure Data Factory or the linked services.
Harishga 4,485 Reputation points Microsoft Vendor

2023-11-30T16:13:57.0433333+00:00

Hi@Harshwardhan Deshmukh
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Harshwardhan Deshmukh 5 Reputation points

2023-12-01T05:13:51.79+00:00
Hello @Harishga
Nope, I haven't found any resolution yet.

List of activities tried from our end

Checked and verified the linked services url’s which are whitelisted in the preview Azure Policy applied at Resource Group Level.

Rbac roles to linked services is configured properly as when we disable the policy from Azure Data factory Studio (image below shows the place where we disable the preview policy feature) and test connection to linked services, everything is successful.

Tried redeploying the ADF via terraform and still facing the same issue which is Forbidden issue with the Fresh Data Factory we deployed, which don't have any calls made till now.

Disabling the Outbound Network Rule Policy temporarily resolves the issue. However, this workaround exposes our ADF compute instances to unrestricted outbound traffic, which is a security risk.
Harishga 4,485 Reputation points Microsoft Vendor

2023-12-06T06:36:39.59+00:00

Hi@Harshwardhan Deshmukh
Thank you for getting back to us and providing more information about the issue you are facing with Azure Policy and connectivity issues with whitelisted linked services.

Based on the information you have provided, it seems that the issue might be related to the Azure Policy configuration. Since disabling the policy temporarily resolves the issue, it's possible that the policy is blocking the outbound traffic to the whitelisted linked services.

We suggest that you review the Azure Policy configuration and check if there are any misconfigurations or conflicts that might be causing the issue. You can also try creating a new policy and test it to see if it resolves the issue.

If you are still facing the same issue, we recommend that you contact Azure Support for further assistance. They can help you troubleshoot the issue and provide you with a solution.
Shahzad 0 Reputation points

2024-05-06T22:12:53.91+00:00

Was trying to solve the issue, from last 4 hours. thanks man.

Share via

I am using the azure policy to whitelist the domain for outbound connectivity from Azure Data Factory to other services. But facing issues in connectivity due to throttling applied on policy.