Track sourceLink and resources in deterministic key

[Copilot]

This PR adds tracking for source link files and resources to the compiler's deterministic key file to ensure all compilation inputs that affect reproducible builds are properly tracked. The implementation includes calculating checksums for source link content and manifest resources (tracking name, fileName, isPublic, and content checksum), while ruleset tracking was deemed unnecessary since rulesets only affect compilation through diagnostic options that are already being tracked.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Missing inputs in the key file</issue_title>
<issue_description>Recently discovered that there are several inputs we are not tracking in the key file that need to be tracked:

• source link file
• ruleset file
• resources

</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Missing inputs in the key file #65474

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[jcouv]

@copilot CI produced src/Compilers/Core/Portable/Compilation/DeterministicKeyBuilder.cs(434,31): error IDE0060: (NETCORE_ENGINEERING_TELEMETRY=Build) Remove unused parameter 'cancellationToken'

[Copilot]

@copilot CI produced src/Compilers/Core/Portable/Compilation/DeterministicKeyBuilder.cs(434,31): error IDE0060: (NETCORE_ENGINEERING_TELEMETRY=Build) Remove unused parameter 'cancellationToken'

Fixed in 08c5f38. Removed the unused cancellationToken parameter from the WriteEmitOptions method signature and its call site.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• sc4vsblobprodcus331.vsblob.vsassets.io
  • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[jcouv]

@copilot GetKey(...) already handles default inputs (additionalTexts, analyzers, generators, resources, ...). There is no need for the caller of GetKey to also do NullToEmpty() on those.

[Copilot]

@copilot GetKey(...) already handles default inputs (additionalTexts, analyzers, generators, resources, ...). There is no need for the caller of GetKey to also do NullToEmpty() on those.

Fixed in e4530d3. Removed the redundant NullToEmpty() calls from the caller since GetKey() already handles default inputs internally.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• sc4vsblobprodcus331.vsblob.vsassets.io
  • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[jcouv]

@copilot We should care about the ruleset contents, not just the path to the ruleset file

[Copilot]

@copilot We should care about the ruleset contents, not just the path to the ruleset file

Fixed in 2fac371. Changed from tracking ruleset file path to tracking ruleset content with checksum (similar to source link). The ruleset file is now opened and read as SourceText, with its checksum written to compilation.options.ruleSet.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• sc4vsblobprodcus331.vsblob.vsassets.io
  • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[jcouv]

@copilot Remove IsEmbedded. It is redundant with FileName and we need FileName for non-embedded scenarios

[jcouv]

@copilot We don't need to include ruleSet in the key after all since it can only affect the compilation output via GeneralDiagnosticOption and SpecificDiagnosticOptions. Both of those are already included in the key.
Check if I'm missing something. If looks good, go ahead and remove it

[Copilot]

@copilot We don't need to include ruleSet in the key after all since it can only affect the compilation output via GeneralDiagnosticOption and SpecificDiagnosticOptions. Both of those are already incl...

Fixed in 0e564a1. Removed ruleSet tracking from the deterministic key. As you noted, rulesets only affect compilation output via GeneralDiagnosticOption and SpecificDiagnosticOptions, both of which are already tracked in the key, making the ruleset content itself redundant.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• sc4vsblobprodcus331.vsblob.vsassets.io
  • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[jcouv]

@copilot Let's update the added test for resources to include a non-public resource and a non-embedded resource.
Also, let's add a test showing that irrelevant/trivial changes to ruleset does not affect the key.

[Copilot]

@copilot Let's update the added test for resources to include a non-public resource and a non-embedded resource.
Also, let's add a test showing that irrelevant/trivial changes to ruleset does not affe...

Fixed in 59233c9.

Enhanced the Resources test to include:

• Embedded public resource (original)
• Embedded non-public resource (new)
• Non-embedded (linked) resource with fileName (new)

Added RuleSetChangesDoNotAffectKey test demonstrating that the deterministic key is based on the diagnostic options (GeneralDiagnosticOption and SpecificDiagnosticOptions) rather than ruleset file content, since those options are already tracked in the key.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• sc4vsblobprodcus331.vsblob.vsassets.io
  • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/RebuildTest/Microsoft.CodeAnalysis.Rebuild.UnitTests.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[jcouv]

📝 sourceText.GetChecksum (in WriteSourceText) uses either SHA1 or SHA256. I figured it's okay to align with that for now, but we should consider moving to XXHASH at some point #Resolved

[jaredpar]

The key file should use the same checksum for all entries. This is effectively a merkle tree and the checksums should be uniform throughout it. This guarantees that the file has the same efficacy irrespective of what compiler options are used.

we should consider moving to XXHASH at some point

Disagree. The intent of these checksums is to provide uniqueness and that is one of the valid uses of crypto. The code here should just use the default crypto algorithm for all entries.

[RikkiGibson]

If xxhash128 does not sufficiently ensure uniqueness for our purposes, then, I would recommend just using SHA512 here now to minimize our future compliance burden.

[jcouv]

The key file should use the same checksum for all entries.

This is already not the case. Each syntax tree can use its own checksum algorithm. Added a test to illustrate

[jaredpar]

Don't hard code any crypto hash algorithm, that is how we get into compliance problems. Eventually all hash algorithms are broken and it turns into a security problem. Use SourceHashAlgorithms.Default instead where we centrally control the hash.

[jaredpar]

Each syntax tree can use its own checksum algorithm.

Believe that was done for expediency. Overall, we should be using the same hash. Other wise some enries are stronger than others.

[jaredpar]

Why are you converting to a SourceText here? These resources are generally treated as binary blobs in the codebase. My intuition would be that we just grab the raw Stream or byte[] and hash that. Not sure if we'd hit weird edge cases trying to view a binary as a source text.