Doc updates and refactoring

docfx/.editorconfig

@@ -32,3 +32,17 @@ tab_width = 2
 # requires: https://marketplace.visualstudio.com/items?itemName=PaulHarrington.EditorGuidelinesPreview
 guidelines = 92
 
+# Settings for https://marketplace.visualstudio.com/items?itemName=MadsKristensen.MarkdownLint
+# Sadly, that is dragged in by https://marketplace.visualstudio.com/items?itemName=MadsKristensen.MarkdownEditor2
+# while there is a built-in version that doesn't support mermaid diagrams without all the CoPilot cruft.
+#    see: https://developercommunity.visualstudio.com/t/Mermaid-rendering-does-not-work/10968192#T-N10986717
+# so the "third party" (not really) one is used.
+# The linter is NOT used in this repo as it is not flexible enough and inconsistently complains about perfectly valid markdown as not consistent.
+# sadly, the mdlint package doesn't have configuration for the `lines_above` or 'lines_below` parameters so disable the rule (MD022)
+#md_blanks_around_headings = false
+#md_ul_indent = 4
+#md_fenced_code_language = false # fenced "code" may just be text and no language exists
+#md_code_fence_style = backtick
+#md_emphasis_style = asterisk
+#md_strong_style = asterisk
+

docfx/CommandLine/Diagnostics/UNC000.md

@@ -1,4 +1,4 @@
-# UNC0000: An internal analyzer exception occurred.
+# UNC0000: An internal analyzer exception occurred
 An internal error occurred in the analyzer. Please [report](https://github.com/UbiquityDotNET/Ubiquity.NET.Utils/issues)
 the issue with as much detail as possible, ideally with a small repro to help identify the
 problem. Please include the full stack of the exception as shown in the message details.

docfx/CommandLine/Diagnostics/UNC001.md

@@ -4,6 +4,7 @@ type does not contain an attribute that designates it as a command. The generato
 ***ignore*** the property and the ***entire*** class.
 
 ## Example:
+
 ``` C#
 using System.IO;
 
@@ -19,6 +20,7 @@ internal class testInput1
 ```
 
 ## Fix:
+
 ``` C#
 using System.IO;
 

docfx/CommandLine/Diagnostics/UNC002.md

@@ -27,6 +27,7 @@ internal class testInput1
 ```
 
 ## Fix
+
 ``` C#
 using System.IO;
 

docfx/CommandLine/Diagnostics/UNC003.md

@@ -6,6 +6,7 @@ incorrect type is used. If this is ignored or suppressed the property is ignored
 generation.
 
 ## Example
+
 ``` C#
 using System.IO;
 

docfx/CommandLine/Diagnostics/UNC004.md

@@ -1,13 +1,13 @@
-# UNC004 : Property type is nullable but marked as required.
+# UNC004 : Property type is nullable but marked as required
 This diagnostic is reported when a nullable type is marked as `Required`. This usually
 indicates an error in the source applying the attributes. An explicitly annotated nullable
 type has a legit value of null, therefore marking it as `Required` makes no sense. Required
 means that it is validated as specified on the command line, this validation only occurs at
 the time of ***invoking*** the command. (If a different command is parsed from the command
 line arguments then no validation occurs).
 
-
 ## Example
+
 ``` C#
 using Ubiquity.NET.CommandLine.GeneratorAttributes;
 

docfx/CommandLine/Diagnostics/UNC005.md

@@ -1,14 +1,17 @@
-# UNC005 : Arity specified for property type is invalid.
+# UNC005 : Arity specified for property type is invalid
 This diagnostic indicates that the arity specified in an attribute does not match the type
 of value for that property^1^. The default arity is usually enough but it is sometimes valid
 to limit the "or more" default to a max value. In particular with collections there may be
-a limit to the maximum number of values allowed so the arity specifies that. Not that the
-arity applies to the ***values*** of a property. That is:
-`--foo true` is ONLY allowed if the minimum arity is > 1, otherwise only the option itself
-is allowed (for example: `--foo`). Setting the arity to a maximum that is > 1 requires a
-collection type to bind the parsed values to. Setting a minimum > 0 makes it required to
-specify a value for the command. That is, with a minimum arity of 1 `--foo` is an error.
+a limit to the maximum number of values allowed so the arity specifies that.
 
+> [!IMPORTANT]
+> The arity applies to the ***values*** of a property. That is, `--foo true` is ONLY allowed
+> if the minimum arity is > 1, otherwise only the option itself is allowed (for example:
+> `--foo`). Setting the arity to a ***maximum*** that is > 1 requires a collection type to
+> bind the parsed values to. Setting a ***minimum*** > 0 makes it required to specify a
+> value for the option IFF the option itself is provided. That is, with a minimum arity of 1
+> `--foo` is an error. But not specifying the option is not (unless it is marked as
+> "Required")
 
 ---
 ^1^ see the [System.CommandLine docs](https://learn.microsoft.com/en-us/dotnet/standard/commandline/syntax#argument-arity)

docfx/CommandLine/index.md

@@ -7,6 +7,7 @@ parsing and binding. Additionally an analyzer is provided to aid in identifying
 with usage of the attributes for generation.
 
 ## Analyzer Diagnostics
+
 Rule ID | Title |
 --------|-------|
 [UNC000](https://ubiquitydotnet.github.io/Ubiquity.NET.Utils/CommandLine/diagnostics/UNC000.html) | An internal analyzer exception occurred. |

docfx/IgnoredWords.dic

@@ -1,2 +1,12 @@
+antlr
 arity
+bool
+initializer
+interop
+marshalling
+namespace
 nullable
+runtimes
+src
+Theming
+utils

src/.editorconfig

@@ -32,6 +32,20 @@ tab_width = 2
 # requires: https://marketplace.visualstudio.com/items?itemName=PaulHarrington.EditorGuidelinesPreview
 guidelines = 92
 
+# Settings for https://marketplace.visualstudio.com/items?itemName=MadsKristensen.MarkdownLint
+# Sadly, that is dragged in by https://marketplace.visualstudio.com/items?itemName=MadsKristensen.MarkdownEditor2
+# while there is a built-in version that doesn't support mermaid diagrams without all the CoPilot cruft.
+#    see: https://developercommunity.visualstudio.com/t/Mermaid-rendering-does-not-work/10968192#T-N10986717
+# so the "third party" (not really) one is used.
+# The linter is NOT used in this repo as it is not flexible enough and inconsistently complains about perfectly valid markdown as not consistent.
+# sadly, the mdlint package doesn't have configuration for the `lines_above` or 'lines_below` parameters so disable the rule (MD022)
+#md_blanks_around_headings = false
+#md_ul_indent = 4
+#md_fenced_code_language = false # fenced "code" may just be text and no language exists
+#md_code_fence_style = backtick
+#md_emphasis_style = asterisk
+#md_strong_style = asterisk
+
 # match ISO standard requirement for C/C++
 [*.c,*.h,*.cpp]
 insert_final_newline = true

src/Ubiquity.NET.CommandLine.SrcGen/CommandGenerator.cs

@@ -31,21 +31,12 @@ var optionClasses
                         .Select( static (m, ct) => (RootCommandInfo)m! ) // convert nullable type to non null as preceding where clause filters out null values
                         .WithTrackingName( TrackingNames.CommandClass );
 
-            context.RegisterSourceOutput( optionClasses, Execute );
+            context.RegisterSourceOutput( optionClasses, Generate );
         }
 
         private static RootCommandInfo? CollectCommandAttributeData( GeneratorAttributeSyntaxContext context )
         {
-            // Do nothing if the target doesn't support what the generated code needs or something is wrong.
-            // Errors are detected by a distinct analyzer; code generators just NOP as fast as possible.
-            // see: https://csharp-evolution.com/guides/language-by-platform
-            var compilation = context.SemanticModel.Compilation;
-            if( context.Attributes.Length != 1 // Multiple instances not allowed and 0 is just broken.
-                || compilation.Language != "C#"
-                || !compilation.HasLanguageVersionAtLeastEqualTo( LanguageVersion.CSharp12 ) // C# 12 => .NET 8.0 => supported until 2026-11-10 (LTS)
-                || context.TargetSymbol is not INamedTypeSymbol namedTypeSymbol
-                || context.TargetNode is not ClassDeclarationSyntax commandClass
-            )
+            if(!TryGetNamedTypeSymbol(context, out INamedTypeSymbol? namedTypeSymbol))
             {
                 return null;
             }
@@ -61,9 +52,7 @@ var optionClasses
             foreach(ISymbol member in members)
             {
                 // filter to referenceable properties
-                // ignore nullable value types as that would produce an error
-                // (Can't use nullable annotations for generic types
-                if(member.CanBeReferencedByName && member is IPropertySymbol propSym /*&& !propSym.Type.IsNullableValueType()*/)
+                if(member.CanBeReferencedByName && member is IPropertySymbol propSym)
                 {
                     propertyInfoBuilder.Add( new PropertyInfo( propSym, Constants.GeneratingAttributeNames ) );
                 }
@@ -76,12 +65,33 @@ var optionClasses
                    );
         }
 
-        private static void Execute( SourceProductionContext context, RootCommandInfo source )
+        private static void Generate( SourceProductionContext context, RootCommandInfo source )
         {
             var template = new Templates.RootCommandClassTemplate(source);
             var generatedSource = template.GenerateText();
             string hintPath = $"{source.TargetName:R}.g.cs";
             context.AddSource( hintPath, generatedSource );
         }
+
+        // Do nothing if the target doesn't support what the generated code needs or something is wrong.
+        // Errors are detected by a distinct analyzer; code generators just NOP as fast as possible.
+        // see: https://csharp-evolution.com/guides/language-by-platform
+        private static bool TryGetNamedTypeSymbol( GeneratorAttributeSyntaxContext context, [MaybeNullWhen(false)] out INamedTypeSymbol nts)
+        {
+            var compilation = context.SemanticModel.Compilation;
+            if(context.Attributes.Length != 1 // Multiple instances not allowed and 0 is just broken.
+             || compilation.Language != LanguageNames.CSharp
+             || !compilation.HasLanguageVersionAtLeastEqualTo( LanguageVersion.CSharp12 ) // C# 12 => .NET 8.0 => supported until 2026-11-10 (LTS)
+             || context.TargetSymbol is not INamedTypeSymbol namedTypeSymbol
+             || context.TargetNode is not ClassDeclarationSyntax
+            )
+            {
+                nts = null;
+                return false;
+            }
+
+            nts = namedTypeSymbol;
+            return true;
+        }
     }
 }

src/Ubiquity.NET.SrcGeneration.UT/CSharp/LanguageTests.cs

@@ -0,0 +1,113 @@
+// Copyright (c) Ubiquity.NET Contributors. All rights reserved.
+// Licensed under the Apache-2.0 WITH LLVM-exception license. See the LICENSE.md file in the project root for full license information.
+
+namespace Ubiquity.NET.SrcGeneration.UT.CSharp
+{
+    [TestClass]
+    public class LanguageTests
+    {
+        [TestMethod]
+        public void Literal_values_are_correct( )
+        {
+            Assert.AreEqual( "{", CSharpLanguage.ScopeOpen );
+            Assert.AreEqual( "}", CSharpLanguage.ScopeClose );
+
+            Assert.HasCount( ExpectedKeywords.Length, CSharpLanguage.KeyWords );
+            Assert.IsTrue( ExpectedKeywords.SequenceEqual( CSharpLanguage.KeyWords ) );
+
+            Assert.AreEqual( "true", CSharpLanguage.AsLiteral( true ) );
+            Assert.AreEqual( "false", CSharpLanguage.AsLiteral( false ) );
+
+            // Escaping of strings is... challenging. The escaped literal form
+            // includes the escaping characters so testing for that requires escaping the escape chars...
+            Assert.AreEqual( "\"this is a \\\"test\\\"\"", CSharpLanguage.AsLiteral( "this is a \"test\"" ) );
+
+            Assert.AreEqual( "'\u0124'", CSharpLanguage.AsLiteral( '\u0124' ) );
+            Assert.AreEqual( "'\\u2028'", CSharpLanguage.AsLiteral( '\u2028' ) );
+
+            Assert.AreEqual( $"@{ExpectedKeywords[ 0 ]}", CSharpLanguage.MakeIdentifier( ExpectedKeywords[ 0 ] ) );
+            Assert.AreEqual( "foo_bar", CSharpLanguage.MakeIdentifier( "foo bar" ) );
+        }
+
+        private readonly ImmutableArray<string> ExpectedKeywords
+            = [ // Source: Language spec. §6.4.4 Keywords
+                "abstract",
+                "as",
+                "base",
+                "bool",
+                "break",
+                "byte",
+                "case",
+                "catch",
+                "char",
+                "checked",
+                "class",
+                "const",
+                "continue",
+                "decimal",
+                "default",
+                "delegate",
+                "do",
+                "double",
+                "else",
+                "enum",
+                "event",
+                "explicit",
+                "extern",
+                "false",
+                "finally",
+                "fixed",
+                "float",
+                "for",
+                "foreach",
+                "goto",
+                "if",
+                "implicit",
+                "in",
+                "int",
+                "interface",
+                "internal",
+                "is",
+                "lock",
+                "long",
+                "namespace",
+                "new",
+                "null",
+                "object",
+                "operator",
+                "out",
+                "override",
+                "params",
+                "private",
+                "protected",
+                "public",
+                "readonly",
+                "ref",
+                "return",
+                "sbyte",
+                "sealed",
+                "short",
+                "sizeof",
+                "stackalloc",
+                "static",
+                "string",
+                "struct",
+                "switch",
+                "this",
+                "throw",
+                "true",
+                "try",
+                "typeof",
+                "uint",
+                "ulong",
+                "unchecked",
+                "unsafe",
+                "ushort",
+                "using",
+                "virtual",
+                "void",
+                "volatile",
+                "while"
+              ];
+    }
+}

src/Ubiquity.NET.SrcGeneration.UT/GlobalNamespaceImports.cs

@@ -15,9 +15,11 @@ set of namespaces that is NOT consistent or controlled by the developer. THAT is
 global using System;
 global using System.CodeDom.Compiler;
 global using System.Collections.Generic;
+global using System.Collections.Immutable;
 global using System.Diagnostics.CodeAnalysis;
 global using System.Globalization;
 global using System.IO;
+global using System.Linq;
 
 global using Microsoft.VisualStudio.TestTools.UnitTesting;
 

src/Ubiquity.NET.SrcGeneration/CSharp/CSharpLanguage.cs

@@ -1,6 +1,8 @@
 // Copyright (c) Ubiquity.NET Contributors. All rights reserved.
 // Licensed under the Apache-2.0 WITH LLVM-exception license. See the LICENSE.md file in the project root for full license information.
 
+using Microsoft.CodeAnalysis.CSharp;
+
 namespace Ubiquity.NET.SrcGeneration.CSharp
 {
     /// <summary>Support for generating source files in the C# language</summary>
@@ -17,20 +19,26 @@ public static class CSharpLanguage
         /// <returns>literal string suitable for output to a writer for the C# language as-is</returns>
         public static string AsLiteral(bool value)
         {
-            return value ? "true" : "false";
+            return SymbolDisplay.FormatPrimitive( value, quoteStrings: false, useHexadecimalNumbers: false )
+                ?? throw new NotSupportedException("Internal error, bool formatting is not supported!?");
         }
 
         /// <summary>Gets a literal value for a <see cref="string"/> that is specific to the C# language</summary>
         /// <param name="value">value to get as a literal</param>
         /// <returns>literal string suitable for output to a writer for the C# language as-is</returns>
-        /// <remarks>This, basically, surrounds <paramref name="value"/> with quotes</remarks>
+        /// <remarks>This, basically, surrounds <paramref name="value"/> with quotes, while handling any escape characters etc...</remarks>
         public static string AsLiteral( string value )
         {
-            return $"\"{value}\"";
+            return SymbolDisplay.FormatLiteral( value, true );
         }
 
-        // TODO: char value
-        //       This requires either simple single quotes OR, it needs conversion to a hex representation if not printable
+        /// <summary>Generates a literal for the character, which may require escaping to form a proper literal</summary>
+        /// <param name="value">Character value to make as a literal</param>
+        /// <returns>Literal as a string</returns>
+        public static string AsLiteral( char value )
+        {
+            return SymbolDisplay.FormatLiteral( value, true );
+        }
 
         /// <summary>Gets the language keywords</summary>
         /// <remarks>

src/Ubiquity.NET.SrcGeneration/ReadMe.md

@@ -8,6 +8,7 @@ things get complicated and there are lots of "decisions" to make based on the in
 get downright unruly.
 
 ## Support includes
+
 * `StringExtensions` to support manipulations of strings commonly used by source generators
     * Method to split a string into lines fit for use in XML doc comments
     * Method to escape processing for a single string for comments
@@ -31,8 +32,10 @@ get downright unruly.
         [additional lines of text]
         <Scope End - emmitted on Dispose of return (RAII pattern)>
         ```
+
 ### C# target language specific support
 While other languages are possible this is the only one currently "built-in".
+
 * `CSharpLanguage` contains constants and statics for generating C# source
     * Constants for the open/close of a scope ("{","}")
     * Array of known keywords to allow escaping text that uses them