5x Lessons Learned from Migrating a Large Legacy to .NET 5/6

In January 2020 I wrote the post Not planning now to migrate your .NET 4.8 legacy, is certainly a mistake. Hopefully we followed our own advice and have been migrated most of our non-UI code. This way latest NDepend version 2021.2 can now run analysis, reporting, power tools and API against .NET 5 on Windows, Linux and MacOS.

We learn a few things during this migration journey. Let me expose those in five points:

1) .NET Standard is not dead

Instead of having multiple redistributables – one for each OS – we kept a single one. Having a single redistributable is making things easier for users and easier for us to maintain. Thus we needed a way to use the same binaries in Visual Studio .NET 4.x context and .NET 5 context. The answer was .NET Standard 2.0. With .NET Standard 2.0 the same DLL can be executed both :

• from within a Windows .NET Framework 4.7.2 (or 4.8) process,
• and from within a Windows/Linux/MacOS .NET Core 2.0, 3.0, 3.1, .NET 5 or .NET 6 process.

This is pretty cool IMHO. Notice that .NET Framework doesn’t support .NET Standard 2.1 thus .NET Standard 2.0 is the right version if .NET Fx / .NET Core compatibility is important for you.

We didn’t migrate straight to .NET 5. Instead 99% of the migrated code is now compiling against .NET Standard 2.0. Only the assembly .\net5.0\NDepend.Console.MultiOS.dll is compiled against .NET 5.0. This lightweight executable assembly just contains a class to invoke the entire analysis/reporting processing implemented in .NET Standard 2.0 assemblies. Its role is just to let dotnet.exe bootstrap the .NET 5.0 runtime.

As a cons of favoring .NET Standard 2.0 we are obviously limited within .NET Standard 2.0 API boundaries. Many new APIs have been added to .NET 5 and .NET 6 but invoking them now would break the requirement to support .NET Framework 4.x, which I guess will hold for quite a few more years. However:

1. The .NET Standard 2.0 code executed in a .NET 5 process still benefits from .NET 5 performance improvements.
2. If a new API is really needed, say DateOnly for example, we can still include its source code (as long as it doesn’t mean including tons of other classes). By doing so, when time will come we’ll be ready. We already relied on this trick with success for example for the interface System.Collections.Generic.IReadOnlyList<T> that appeared with .NET Fx 4.5 when our code was still compiling against .NET Fx 4.0.
3. We can still build some interfaces to inject some new API implementations within .NET Standard 2.0 code. This means that we’ll need to handle those API somehow in .NET Framework 4.x.

So far we stick with .NET Standard 2.0 and didn’t need to harness 2) or 3). .NET is converging years after years to one .NET but .NET Standard 2.0 is the least common denominator of past, present and future .NET runtimes. Thus abiding now most of your code by .NET Standard 2.0 offers a lot of guarantee for the future. For example we can now easily plan .NET 6.0 support in a 2022 release with a single-class-assembly .\net6.0\NDepend.Console.MultiOS.dll. Moreover if one day it makes business sense to run in browser on Blazor we’ll be quickly ready for that. Our .NET Standard 2.0 assemblies 7-zipped (+ referenced assemblies) only weight 7MB so this could become a realistic scenario.

2) Expect pain points

Microsoft did an awesome job with .NET Standard and .NET Core at presenting and supporting most APIs commonly used in .NET Fx. However expect some unsupported API that will cause workaround and refactoring headaches. Here are some such pain points we stumbled upon:

• The good old Delegate.BeginInvoke() API is not supported in .NET Core, .NET 5/6… I wrote a blog post to explain in details how we got around this limitation: Migrating Delegate.BeginInvoke Calls to .NET Core, .NET 5 and .NET 6
• The Thread.Abort() badass API is proposed by .NET Standard2.0 and .NET 5 but throws a PlatformNotSupportedException in this context. Yes this API was dangerous but when used carefully and properly it helped a lot. Here also I wrote a blog post to explain in details how we got around this: On replacing Thread.Abort() in .NET 6, .NET 5 and .NET Core
• CSharpCodeProvider.CompileAssemblyFromSource() CodeDOM API is not supported in .NET Core, .NET 5/6…. We use this API to compile on-the-fly CQLinq queries. We now use Rosyln instead. Since the .NET 5 runtime doesn’t embed Roslyn, we had to embed Roslyn in our redistributable. This means 3MB extra and our redistributable weight 13MB instead of 10MB (more on redistributable size later in the last section).
• The class AppDomain is proposed by .NET 5 but it’s actually just an empty facade. There is a single AppDomain at .NET Core runtime. We use AppDomain to unload CQLinq queries compiled assemblies. However unloading is needed only in the UI part wish is not yet migrated. Thus at .NET 5 analysis time we don’t use anymore AppDomain and in the future, during UI migration, we plan to rely on AssemblyLoadContext to unload compiled assemblies.
• The Rijndael implementation now only supports 128 bits block size as shown in the open sourced .NET Core implementation. When the Rijndael encryption was adopted as AES the block size restriction was added. This limitation was quite annoying because we used this algo so far with 256 bits block size for all our licensing stuff. BouncyCastle.NetCore provides a 256 bits .NET Core impl but we preferred to comply with the standard. As a drawback a license activated with a previous NDepend version on a Windows machine needs to be re-installed if the user plans to use the new NDepend .NET 5 impl on the same Windows machine.
• Usual UI concepts like color, font style, image… are not proposed by .NET Standard nor .NET Core implementations. Thus we defined our own abstractions and provides some facilities through NDepend.API to convert them to the UI framework used. See for example the struct NdpColor returned for example by Debtrating.ToBackColor() .that can be easily converted to a color object on any UI framework thanks to the Alpha, R, G B properties.
• To build NDepend reports, for more than a decade we generated with the tool xsltc.exe the assembly NDepend.ReportXsl.dll from a large 158KB XSL document. The assembly generated is a .NET Framework 2.0 assembly and it references the assembly System.Data.SqlXml.dll not supported in .NET Core. xsltc.exe hasn’t been modernized. Fortunately on-the-fly XSL compilation through XslCompiledTransform.Load(string) is much faster than expected: it loads and compiles our 158KB XSL document in 70 milliseconds! To our surprise this great performance relieved the need for pre-compiled XSL in the 202KB assembly NDepend.ReportXsl.dll.

Those were our main pain points but expect more depending on which .NET Framework APIs you use.

3) Tooling can help anticipate most migration headaches

With its API, its code querying engine and all its features that interact, NDepend is quite a flexible product . As a consequence many accidental features popped-up. By accidental features I mean valuable features that were not planned initially but that can be achieved quickly with a few steps. One such accidental feature is explained in this blog post Quickly assess your .NET code compliance with .NET Standard.

This feature proved to be very useful during our large-scale migration. It made easy to list up-front non .NET Standard compliant code of core assemblies we had to migrate. Also for each non-compliant class and method matched, the list of non-supported classes, methods and fields consumed is provided.

NDepend code non-compliant with .NET standard

For a more intuitive assessment we used the NDepend metric view, that highlighted the classes and methods matched by the non-compliant code query edited above. With this information it has been much easier to anticipate hot-spots. When more than 500 classes are involved in a migration, begin able to anticipate and plan upfront is not a luxury but a necessity.

.NET migration: Non compliant code highlighted vs. compliant code

Notice that an alternative tool exists to list non-compliant API: Porting Assistant for .NET. I am not sure how it does compare with what has been explained.

4) Caring for code maintainability can end up being a matter of survival

I am in the code quality / code maintainability / technical-debt business for more than 15 years and the post Case Study: 2 Simple Principles to achieve High Code Maintainability summarizes well my position on how to achieve high code maintainability. The two principles are:

• Layered Architecture: it prevents entangled code, the well know spaghetti code phenomenon. Dependencies get mastered and when it is time for the code to evolve it is straightforward to add new classes and interfaces that naturally integrate with existing ones.
• High Test Coverage Ratio: its consequence is that when code covered by tests gets refactored, existing tests get impacted. With not much efforts the developer discovers regression problems and fix them before they reach production and become bugs to fix. The more code is covered by tests the more you’ll benefit from this shield.

Code maintainability efforts especially pays off when the entire legacy gets shacked-up: Will the migration be a matter of months or more than a year? The answer of this question can determine if a project will survive or die.

Concerning the High Test Coverage Ratio discipline we abided by for a long time there is not much to say. We have 14K unit tests and a ratio of 86% coverage on a 4.500 classes application. We have a lot of confidence in our test suites to catch early accidental regressions. As a matter of fact it saved us dozens of time during this migration. It is not just about the manual testing time saved, the friction saved by releasing product with few bugs, it is also about the satisfaction of working with thousands of gentle gardians that watch daily over our code.

Concerning the Layered Architecture it really helped because the large set of classes and methods using a UI framework to refactor or move out from .NET Standard assemblies was de-facto layered. We listed those classes and methods with a code query and exported this set to the dependency graph. The query can be easily tweaked to list from high-level to low-level. But the graphical representation of dependencies was convenient to help understanding at each step what was going on and what should be done.

Here is the code query if you’d like to adapt it for your own usage:

Notice the coreMethodsUser property in the result. For each core method M depending on UI to bubble up in a .NET Framework assembly, coreMethodsUser lists core methods that call M and that don’t depend on UI. Each of these methods represent a friction point that must be carefully handled: should we keep such caller method in the .NET Standard boundaries and use the Dependency Inversion Principle (DIP) to inject the UI impl? should we instead refactor the method and its surrounding code? Having this information and being able to generate the appropriate calling graph each time was an eye-opener to take the right decisions.

Of course core code should have been separated from UI code in the first place. But after 16 years of active development we were not there. Better late than never! This is great that now our core domain code is implemented against .NET Standard 2.0 and can be run pretty much everywhere 🙂

---

I recently came across the ScreenToGif OSS project. I enjoy using this popular tool when I need it and I was curious to have a look at its source code. Its 700+ classes are all entangled: from any two namespaces A and B there is a path from A to B and a path from B to A. I had a chance to discuss this with Nicke Manarin the developer of ScreenToGif that wrote:

“I’m aware of the issues and I have been working on to improve the app codebase, but unfortunately it takes time as I can only invest a few hours per week. Also, I’m migrating the whole project to .NET 5, so that’s also taking some time to finish.”

The cost to migrate is aggravated by the fact that the architecture is entangled.

5) Third-Party libraries referenced must be modular, lightweight and open-sourced

Time and experience leads developer to look at third-party libraries differently:

• Junior developers tend to reference plenty of libraries because it is quite attractive to avoid reinventing the wheel and get job done asap.
• Senior developers knows that each referenced library is a potential burden for the future.

Each reference is a trade-off between (effort to do it yourself + maintain it) vs. (effort to integrate the library + effort to update and test new versions + amount of problems it can potentially cause in the future).

We had such a library that we considered as a burden: DevExpress DXperience. Don’t take me wrong, DevExpress is doing a great job at providing a stable library with tons of shiny controls. The thing is that when we initially referenced DevExpress DXperience (in 2008!) the goal was to quickly provide a Visual Studio theming and docking look and feel to our product. We were not even using 3% of the API surface but this library being not modular we had to embed 20MB of binaries.

On the other hand our own binaries don’t even weight 15MB! With time we could have used more controls from this library but because it was so weighty we knew that at a point we would get rid of it.

Migrating toward .NET Core was the right time to tame all our references and we replaced DXperience with the great OSS project DockPanelSuite. We had to fix a few bugs but it was worth it to reference a much lighter library (434KB vs. 20MB). Our redistributable is now significantly smaller: 13MB vs. 18MB. We mentioned earlier that we had to append 3MB of Roslyn else it would have weighted 10MB. More importantly at runtime 25MB to 30MB of fresh RAM is now saved (DevExpres DLLs + its JITed code).

Too often I see products weighting far too much like 10MB for some smartphone flashlight gadget. To me, such inflation is an indicator of low-quality product. The same remark applies to web applications. Nowadays page speed (and hence page size) is a core web vital Google SEO indicator. Pages larger than 1.5MB tend to be penalized, especially on their mobile versions. Most complex pages can be implemented within 1.5MB of HTML, CSS, JS, resources… but it demands more efforts like: custom JS, minified code, optimized resources, lazy loading… Relying on a CDN (Content Delivery Network) helps but cannot magically transform a garbage page into clean one.

Apart the DXperience library, the other few libraries we are referencing are all open-sourced, lightweight and can be compiled de-facto against .NET Standard 2.0. Those are a solid guarantees for the future.

Conclusion

I hope our detailed migration journey can help other teams tackle their legacies. As explained it can be seen as an opportunity to reimburse quite a lot of technical-debt and also to put your hands into some modern APIs and tooling. Running our code on Linux and MacOS was like science-fiction a few years ago and now we are there. Exciting time 🙂