Share via

Why do bulk download requests from OneDrive fail when using the M365 graph API?

Matthew Lee 0 Reputation points
2024-05-03T16:08:46.0666667+00:00

Not sure if this is a code related issue, or a network issue. or node related issue. It is hard to pinpoint any root cause as the data received is variable and inconsistent. When requesting downloads in bulk from the Graph API using Fetch and node JS there appears to be an intermittent problem.

At first it was assumed that node and fetch were struggling with the number of requests being made for example we would run an API request to gather info on 1000 files and for each file request a download using the methods provided by the Microsoft JavaScript SDK for the M365 API.

 

What was initially discovered was that when 2000+ files were requested multiple downloads would fail. For example, only 700 of these files would land on disk. In circumstances where downloads were failing there appear to have been no retry attempts.

 

The flows were checked for event loop lag and the CPU for excessive usage. Neither were apparent as issues. What was indicated in a error response was that fetch was just failing. It was not receiving a response.

 

A timer was added to create a space between individual requests, this worked and did mediate well.

 

However, it was also apparent that download requests were failing when smaller amounts of files were requested, for example between 400 and 500. It is conceivable that this should not be a problem for an API to manage.

 

I am just wondering if anyone else has encountered similar problems.

Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
10,772 questions
OneDrive
OneDrive
A Microsoft file hosting and synchronization service.
847 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Said A 791 Reputation points
    2024-05-03T19:06:52.2133333+00:00

    Hey there,

    You might need to audit how many units your App is consuming, at the User level and tenant level (if the app is being used by other users, or if there are other apps using the same API)

    Consult this page: https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online

    Each operations has a cost in units and based on your tenant and license count your get a threshold.

    You also have other recommendations to apply to your app to make it more "Attractive" for the API.

    Feel free to provide more details,

    Good luck!

    Regards,

    Said

    0 comments