Preserve enum type in custom attribute argument of type obj (#995)

[edgarfgp]

Problem

Assigning an enum to a custom attribute argument of type obj loses the enum type. The value is stored in metadata as the underlying int32, so reading the attribute back gives a plain int rather than the enum. C# stores the same thing as the enum, so this is also a cross-language inconsistency.

Fixes #995.

Before

open System

type MyAttribute() =
    inherit Attribute()
    let mutable prop : obj = null
    member _.Prop with get () : obj = prop and set (v: obj) = prop <- v

type MyEnum = A = 1 | B = 2

[<My(Prop = MyEnum.B)>]
type C = class end

let v = (typeof<C>.GetCustomAttributes(false)[0] :?> MyAttribute).Prop
v.GetType()                // System.Int32
Convert.ToString v         // "2"

After

v.GetType()                // MyEnum
Convert.ToString v         // "B"

The enum type is now written into the attribute blob using the ECMA-335 enum tag (0x55 + type name), the same encoding C# produces, so it round-trips as the enum. Reading that encoding back (which previously threw) is also handled, and decode/encode are kept symmetric so static linking preserves it.

Quotations

The original triage worried this needed quotation/TAST changes. It doesn't: the fix only reads the enum type (already on the Const node) when emitting the attribute. Quotations are untouched and already keep the enum type added a test to confirm.

[github-actions]

❗ Release notes required

You can open this PR in browser to add release notes: open in github.dev

---

✅ Found changes and release notes in following paths:

Change path	Release notes path	Description
src/Compiler	docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

[github-actions]

🔍 Tooling Safety Check — Affects-Compiler-Output
Affects-Compiler-Output: changes custom attribute blob encoding (enum 0x55 tag in il.fs/IlxGen.fs)

Generated by PR Tooling Safety Check · opus46 3.2M · ◷

[T-Gro]

🤖 This review was generated by AI (@expert-reviewer agent). Findings may contain inaccuracies — please verify independently.

Nice, targeted fix for #995 — the ILAttribElem.Enum carrier plus the symmetric encode tag (0x55 + type name) are the right model, and consolidating the 0x55 decode into decodeCustomAttrElemType removes the duplicated named-arg branch. Two correctness concerns about the decode side, both in decodeILAttribData/parseVal in src/Compiler/AbstractIL/il.fs — see inline comments.

Summary:

1. The encode path now writes the enum's true underlying width, but the decode fallback still reads a fixed int32, so boxed enums whose underlying type isn't int32 (e.g. the LongEnum/int64 case this PR adds) don't round-trip through decodeILAttribData and can desync the blob pointer for any following named args. The runtime test passes only because the CLR — not decodeILAttribData — reads the metadata.
2. The new ILType.Value _ -> ILAttribElem.Enum(ty, v) wrap in the System.Object branch also fires for boxed primitives (int/bool/char/byte…), not just enums, mislabeling them as Enum on decode.

[T-Gro]

When is this code path exercised?
Pls demonstrate with a test cases that uses long-backed enum and also has value(s) which would not fit an int32.

[T-Gro]

Don't we have a function for checking ilArgTy being object instead of a vanilla name check?