Dependabot is timing out

[MikeAlhayek]

The Dependabot action runs as schedule. It times out after running for 60 mins.

I believe that this issue happened after CentralPackageTransitivePinningEnabled was enabled. #16566

Last successful run was August 13th.

[Piedone]

The last couple of runs failed with various different errors, as visible in the output of the workflow run. One error that appeared thrice in slightly different variations, including the most recent run is this:

2024-09-16T19:42:14.1794297Z Dependabot encountered '6' error(s) during execution, please check the logs for more details.
2024-09-16T19:42:14.1795324Z +-----------------------------------------------------------------+
2024-09-16T19:42:14.1796052Z |                  Dependencies failed to update                  |
2024-09-16T19:42:14.1796863Z +-------------------------------------------------+---------------+
2024-09-16T19:42:14.1797843Z | GraphQL                                         | unknown_error |
2024-09-16T19:42:14.1798654Z | GraphQL.MicrosoftDI                             | unknown_error |
2024-09-16T19:42:14.1799521Z | GraphQL.SystemTextJson                          | unknown_error |
2024-09-16T19:42:14.1800459Z | Microsoft.IdentityModel.Protocols.OpenIdConnect | unknown_error |
2024-09-16T19:42:14.1801442Z | OpenIddict.Validation.SystemNetHttp             | unknown_error |
2024-09-16T19:42:14.1802348Z | MessagePack                                     | unknown_error |
2024-09-16T19:42:14.1803160Z +-------------------------------------------------+---------------+
2024-09-16T19:42:14.6885640Z Failure running container 50880fd4a02906f4326c9766bee5e9f11d2b7da4ddbe9dae0e91bd24fd28bd7d
2024-09-16T19:42:16.4978038Z Cleaned up container 50880fd4a02906f4326c9766bee5e9f11d2b7da4ddbe9dae0e91bd24fd28bd7d
2024-09-16T19:42:16.5307419Z   proxy | 2024/09/16 19:42:16 264/2078 calls cached (12%)
2024-09-16T19:42:16.5308361Z 2024/09/16 19:42:16 Posting metrics to remote API endpoint
2024-09-16T19:42:16.5598325Z   proxy | 2024/09/16 19:42:16 Successfully posted metrics data via api client
2024-09-16T19:42:17.6763512Z ##[error]Dependabot encountered an error performing the update

Error: The updater encountered one or more errors.

For more information see: https://github.com/OrchardCMS/OrchardCore/network/updates/885578226 (write access to the repository is required to view the log)

The more info link is a circular reference that just eventually brings you back to the workflow run.

If you look for e.g. MessagePack in the log, this is what you get:

2024-09-16T19:41:29.7691127Z updater | 2024/09/16 19:41:29 INFO <job_885578226> Checking if MessagePack 2.2.60 needs updating
2024-09-16T19:41:29.8655727Z   proxy | 2024/09/16 19:41:29 [500] GET https://api.nuget.org:443/v3/registration5-gz-semver2/messagepack/index.json
2024-09-16T19:41:29.8913078Z   proxy | 2024/09/16 19:41:29 [500] 200 https://api.nuget.org:443/v3/registration5-gz-semver2/messagepack/index.json
2024-09-16T19:41:29.9046322Z updater | 2024/09/16 19:41:29 INFO <job_885578226> Filtered out 29 pre-release versions
2024-09-16T19:41:30.0017308Z   proxy | 2024/09/16 19:41:30 [502] GET https://api.nuget.org:443/v3-flatcontainer/messagepack/2.2.60/messagepack.nuspec
2024-09-16T19:41:30.0038706Z   proxy | 2024/09/16 19:41:30 [502] 200 https://api.nuget.org:443/v3-flatcontainer/messagepack/2.2.60/messagepack.nuspec
2024-09-16T19:41:30.0088939Z updater | 2024/09/16 19:41:30 INFO <job_885578226> Latest version is 2.5.172
2024-09-16T19:41:30.0092307Z updater | 2024/09/16 19:41:30 INFO <job_885578226> Requirements to unlock all
2024-09-16T19:41:30.0093019Z 2024/09/16 19:41:30 INFO <job_885578226> Requirements update strategy 
2024-09-16T19:41:30.0093592Z updater | Finding updated dependencies for MessagePack.
2024-09-16T19:41:30.1055766Z   proxy | 2024/09/16 19:41:30 [504] GET https://api.nuget.org:443/v3-flatcontainer/messagepack/2.5.172/messagepack.nuspec
2024-09-16T19:41:30.1075099Z   proxy | 2024/09/16 19:41:30 [504] 200 https://api.nuget.org:443/v3-flatcontainer/messagepack/2.5.172/messagepack.nuspec
2024-09-16T19:41:30.2096633Z   proxy | 2024/09/16 19:41:30 [506] GET https://api.nuget.org:443/v3-flatcontainer/messagepack.annotations/2.5.172/messagepack.annotations.nuspec
2024-09-16T19:41:30.2120579Z   proxy | 2024/09/16 19:41:30 [506] 200 https://api.nuget.org:443/v3-flatcontainer/messagepack.annotations/2.5.172/messagepack.annotations.nuspec
2024-09-16T19:41:30.3137128Z   proxy | 2024/09/16 19:41:30 [508] GET https://api.nuget.org:443/v3-flatcontainer/microsoft.net.stringtools/17.6.3/microsoft.net.stringtools.nuspec
2024-09-16T19:41:30.3155672Z   proxy | 2024/09/16 19:41:30 [508] 200 https://api.nuget.org:443/v3-flatcontainer/microsoft.net.stringtools/17.6.3/microsoft.net.stringtools.nuspec
2024-09-16T19:41:30.4217707Z   proxy | 2024/09/16 19:41:30 [510] GET https://api.nuget.org:443/v3-flatcontainer/system.collections.immutable/6.0.0/system.collections.immutable.nuspec
2024-09-16T19:41:30.4261155Z   proxy | 2024/09/16 19:41:30 [510] 200 https://api.nuget.org:443/v3-flatcontainer/system.collections.immutable/6.0.0/system.collections.immutable.nuspec
2024-09-16T19:41:30.5297824Z   proxy | 2024/09/16 19:41:30 [512] GET https://api.nuget.org:443/v3-flatcontainer/system.reflection.emit.lightweight/4.7.0/system.reflection.emit.lightweight.nuspec
2024-09-16T19:41:30.5471419Z   proxy | 2024/09/16 19:41:30 [512] 200 https://api.nuget.org:443/v3-flatcontainer/system.reflection.emit.lightweight/4.7.0/system.reflection.emit.lightweight.nuspec
2024-09-16T19:41:30.5565570Z updater | 2024/09/16 19:41:30 INFO <job_885578226> Updating MessagePack from 2.2.60 to 2.5.172
2024-09-16T19:41:31.0060621Z   proxy | 2024/09/16 19:41:31 [514] POST /update_jobs/885578226/record_update_job_unknown_error
2024-09-16T19:41:31.0641532Z   proxy | 2024/09/16 19:41:31 [514] 204 /update_jobs/885578226/record_update_job_unknown_error
2024-09-16T19:41:31.1095191Z   proxy | 2024/09/16 19:41:31 [516] POST /update_jobs/885578226/record_update_job_error
2024-09-16T19:41:31.1096652Z   proxy | 2024/09/16 19:41:31 [516] 204 /update_jobs/885578226/record_update_job_error
2024-09-16T19:41:31.1577255Z   proxy | 2024/09/16 19:41:31 [518] POST /update_jobs/885578226/increment_metric
2024-09-16T19:41:31.1578694Z 2024/09/16 19:41:31 [518] 204 /update_jobs/885578226/increment_metric
2024-09-16T19:41:31.1629892Z   proxy | 2024/09/16 19:41:31 [520] POST /update_jobs/885578226/record_update_job_unknown_error
2024-09-16T19:41:31.1631266Z   proxy | 2024/09/16 19:41:31 [520] 204 /update_jobs/885578226/record_update_job_unknown_error
2024-09-16T19:41:31.1641654Z updater | 2024/09/16 19:41:31 ERROR <job_885578226> Error processing MessagePack (Dependabot::DependabotError)

So, not too helpful with all these unknowns, and the listed packages don't seem to be related to the changes under #16566.

MessagePack is NOT a package we directly reference, BTW.

The last PR that Dependabot opened was #16549 on 12 August, which interestingly originates from a failing run (its successful rerun didn't actually do anything since "Dependabot workflows cannot be re-run. Retrigger this update via Dependabot instead.").

My guess is that Dependabot is simply choking on the update due to us having a huge solution with an extreme amount of packages being referenced, and we occasionally have to do manual updates. I've updated a GitHub support request about this.

[MikeAlhayek]

this is one fixed.

[Piedone]

GitHub support only pointed to the docs for now BTW, what I've previously read but didn't help.

[Piedone]

Unfortunately, this isn't fixed. Dependabot has a limitation of 150 manifests per repo: https://docs.github.com/en/enterprise-cloud@latest/code-security/supply-chain-security/understanding-your-software-supply-chain/troubleshooting-the-dependency-graph#are-there-limits-which-affect-the-dependency-graph-data According to GitHub support, we have 190 (we have 217 projects in the solution but apparently, not all of those have NuGet dependencies).

We could perhaps batch updates by having multiple updates, scheduled hours apart, in dependabot.yml, with different directory patterns to match a roughly even number of projects (or at least less than 150) each. Or a similar strategy but with dependency name patterns.