Join the DZone community and get the full member experience.

It always bothered me that in order to implement horizontal scaling in Magic, I had to arguably kill its most important feature; Its dynamic Hyperlambda file system. Sure, I could duplicate the same Hyperlambda files, create a Docker image, and spawn up 50 Kubernetes PODs having an identical "modules" folder - But then I would lose Magic's primary feature, being its ability to dynamically change modules on the run, without interrupting normal usage, having the same changes duplicated automatically to all 50 PODs.

Well, as of version 10.0.5 of Magic, this is no longer a problem - At least in theory. Simply because now Magic no longer directly reads files from the file system, but exclusively consumes dynamic "files" through four interfaces, completely abstracting away all IO of Hyperlambda files, and any other dynamic files may I add. These interfaces are as follows.

• IFileService - Manipulates "files"
• IFolderService - Manipulates "folders"
• IRootResolver - "Namespace" resolver for files and folders
• IStreamService - Manipulates streams

Of course the idea being that you can now implement your own implementations of the above interfaces, wire up your IoC container to use your custom implementation, and catching! No more file system dependencies. This of course allows you to create a "virtual" file system that exclusively reads files from for instance ScyllaDB, Redis or something similar.

Doing such a thing allows you to 100% transparently use dynamic Hyperlambda files, with the "feeling" of that it's a simple static file system, while actually storing your files in some super scalable storage, scaled out automatically to in theory millions of servers, duplicating your Hyperlambda files unto every single Kubernetes POD you've got in your Kubernetes cluster.

Then combining the above with for instance Kubeless, results in that you can spawn up identical Magic servers on the fly, on a per need basis, effectively giving you a 100% perfectly working DYI "serverless" architecture.

In later articles I might create for instance a Redis backed file system storage to illustrate the point - But for now, at least you can do it yourself if you wish, effectively allowing you to "build Facebook or Twitter" using Hyperlambda, by which I mean monster Kubernetes clusters, serving millions of users simultaneously - While everything of course feels like a child's play, due to the simplicity that Hyperlambda provides you with, making everything feel almost like LEGO as you do.

Simplicity does not imply compromising scalability

This allows Magic users to login to their Magic Dashboard, use Hyper IDE to edit a file, as if it was persisted in the local file system, save that file, and have the changes propagated to thousands of PODs in your production Kubernetes cluster 100% automatically, without any compilation, interpreting, or deployment what so ever occurring in the process - Effectively never interrupting production while still updating thousands of servers in your production cluster simultaneously. Arguably making Magic "more cloud" than AWS and Azure combined ... ;)

The point being that clicking the above "Save" button would simultaneously update a million live production servers at the exact same time, with zero interference towards your production environment, while having multiple "namespaces" within each of your "virtualised Magic servers",  completely isolating each of your virtual file systems, using some sort of intelligent implementation on your IRootResolver, possibly namespacing your virtual file systems by for instance the URL or something. The advantage of Hyperlambda here becomes obvious, since for instance other serverless technologies such as Microsoft Azure implies "cold starts" if your application hasn't been accessed for some time, while with Hyperlambda you could share the same physical cluster between millions of different apps, implying you'd never have to experience a cold start, simply because the binaries running on each of your PODs are the exact same, while only the Hyperlambda parts changes on a "per app level", without the capacity to see files not within your "namespace". Pretty cool I'd say ^_^

Notice, you'll need to actually implement the above 4 interfaces to actually take advantage of this. But that should really be a no-brainer for an average C# senior software developer ...

• Download Magic to try it out for yourself