Core migration

[NimZwei]

• Migrated all project to new SDK-style .csproj files.
• Target framework net8.0-windows.
• No separate ext-libs solution, everything in one solution with subfolders.
  • All project build in their respective bin/obj folders.
  • To create a bin folder including all projects, use dotnet publish
• No more .frame files
  • For setting version information at the builderver, use GitVersion or similar tools.
• No more config-merging
  • Single app.config in HeuristicLab project.

Known issues:

• Some older plugins are currently disabled (external-evaluation, ...)
• Script editor does not work.
• No working communication with HeuristicLab-Hive (which is still .net framework 4.6.1).

ToDos:

• Split plugin infrastructure into 2 assemblies (logic & ui) to have a non-windows-target plugin infrastructure.
  • Remove windows target all non-ui projcts.
• App-domain related code was commented out for proof of concept, but this needs to be properly addressed.
• The HeuristicLab project contains references to all other projects, which is not very plugin-stylish.

[Shabbafru]

Further Todos as discussed:

• Switch Hive Drones from AppDomains to seperate processes (needed for plugin loading)
• Refactor PluginInfrastructure PluginManager / ApplicationManager code (needs to be done together with Hive Drone refactoring)
• Do we still need CommandLineArgumentHandling? I think this is all related to the StarterForm and Applications
• Backup OKB DB from services server
• Remove Plugin Update mechanism
• Remove Application stuff and starter form
• Try update Avalalon Edit and get rid of autocompletion
• Remove external evaluation problems for matlab and scilab
• Remove OKB client + services
• Delete ExtLibs solution
• Delete Tests solution
• Delete FxCop files
• Remove signing, this was primarily needed for AppDomains, we don't have AppDomains anymore (see https://github.com/dotnet/runtime/blob/main/docs/project/strong-name-signing.md )

[gkronber]

* Figure out if we can / should remove Application stuff and starter form

I would tend to say yes. The main reason for the starter is to be able to check plugin descriptions and dependencies before loading the Optimizer. Another reason was to provide the possibility of multiple apps. However, I would say we can just provide them as separate executable instead of via the common starter.

* Figure out if its ok to remove external evaluation problems for matlab and scilab

Ok, can later be re-added as a plugin (outside of HL main repo).

[Shabbafru]

Topic for discusion: When extracting all UI parts from the PluginInfrastructure project into a separate project to have a UI-less PluginInfrastructure, the ErrorHandling class does not split well. It provides a ShowErrorDialog method, which obviously requires WinForms, so it goes into the new PluginInfrastructure.Views project. However, there are some cases, where non-UI projects show an error dialog, such as the VRP:
...

I'm also not quite sure about this, but I think in general (and in all other places in HL its like this) an exception is thrown in the non-gui code and the gui catches the error and displays the dialog. So I would also go with exceptions.

But, if we really need an alternative, we could also do something like the following in PluginInfrastructure:

public static void ShowError(string message, Exception exception) {
  Type guiErrorHandling = Type.GetType("HeuristicLab.PluginInfrastructure.Views.ErrorHandling");

  if (guiErrorHandling == null) {    
     throw exception;
  } else {        
    guiErrorHandling.GetMethod("ShowErrorDialog", [typeof(string), typeof(Exception)]).Invoke(null, [message, exception]);
  }
}

[Shabbafru]

Please note that cbd3aec breaks reading/writing HL files (and therefore also the StartPage) until heal-research/HEAL.Attic#53 is done

[NimZwei]

We have encountered a performance issue in certain cases when using .NET 8, which is resolved by upgrading to .NET 9. The issue arises due to a bug (?) in the .NET 8 runtime related to reflection on system types with self-referencing generic constraints.

Details:

When loading a Problem that implements IProblemInstanceConsumer<TData>, we instantiate all corresponding IProblemInstanceProvider<TData> to display them in the UI. This is done via the ApplicationManager.Manager.GetInstances(type) method, which iterates over all currently loaded types to find those that are instantiable and implement the specified type.

For non-generic types, this process is straightforward. However, for generic types, we rely on TypeExtensions.BuildType to handle the generic arguments. For example, when querying all instantiable types for IProblemInstanceConsumer<TData>, we must check if TSPData can be used as a valid generic argument for types with a single generic parameter.

Rather than calling type.MakeGenericType and catching potential exceptions, HeuristicLab proactively filters out types that would cause an exception. This is done to avoid the overhead of handling unnecessary exceptions when iterating through the types. For instance, if a generic parameter has a type constraint (where T : SomeType), HeuristicLab discards any types that don't fulfill these constraints.

In .NET 8, several generic interfaces with self-referencing generic argument constraints were introduced, such as public interface IAdditiveIdentity<TSelf, TResult> where TSelf : IAdditiveIdentity<TSelf, TResult>?. The problem is that calling Type.GetGenericParameterConstraints on the TSelf argument returns an empty list, effectively losing the constraint information. This behavior is peculiar because it works as expected on user-defined types (see example below).

Due to the missing self-referencing constraint, HeuristicLab fails to filter out incompatible types before attempting to build a generic type, leading to exceptions. Since .NET 8 introduced many types with self-referencing generic constraints, this results in a significant increase in exceptions.

I could not find any issues, pull requests, or Stack Overflow discussions addressing this behavior, but it appears to be resolved in .NET 9. Therefore, for HeuristicLab, we should either upgrade to .NET 9 as soon as possible or live with the performance regression for such cases.

using System;
using System.Linq;

var tselfCustomType = typeof(ISelfReferencing<>).GetGenericArguments()[0];
var tselfSystemType = typeof(IUtf8SpanParsable<>).GetGenericArguments()[0];

Console.WriteLine(tselfCustomType.GetGenericParameterConstraints().Count()); // 1
Console.WriteLine(tselfSystemType.GetGenericParameterConstraints().Count()); // 0

public interface ISelfReferencing<TSelf> where TSelf : ISelfReferencing<TSelf>? { }

[NimZwei]

It seems that the discussed issues that cause some exceptions for the PluginInfrastucture TypeExtensions (see #3186 (comment)), stems from a runtime bug, not so much a .net 8/9 bug.

When using explicit runtime versions for a self-contained deployment (to be independent from the installed runtimes), it seems that the bug is present in version 8.0.8 and resolved in the runtime version 8.0.10.

This can be reproduced by the following code:

var tselfCustomType = typeof(ISelfReferencing<>).GetGenericArguments()[0];
var tselfSystemType = typeof(IUtf8SpanParsable<>).GetGenericArguments()[0];

Console.WriteLine(Environment.Version);
Console.WriteLine(tselfCustomType.GetGenericParameterConstraints().Count());
Console.WriteLine(tselfSystemType.GetGenericParameterConstraints().Count());

public interface ISelfReferencing<TSelf> where TSelf : ISelfReferencing<TSelf>? { }

global.json

{
  "sdk": {
    "version": "8.0.304", // or 8.0.403
    "rollForward": "disable"
  }
}

When publishing using two different global.json files to configure the runtime, we get to different outputs

dotnet publish '.\HeuristicLab\3.3\HeuristicLab-3.3.csproj' --outputo bin --self-contained --framework net8.0-windows`

sdk	runtime	output
8.0.304	8.0.8	8.0.8 1 0
8.0.403	8.0.10	8.0.10 1 1