Bringing .NET to the Edge: How to Run Your CLI Applications on Cloudflare Workers with WASI

A Walkthrough of Adapting Complex .NET CLI Applications for Cloudflare Workers and the Browser Using WASI

A few months ago, we embarked on a journey to enhance the documentation for Bebop, our efficient schema language. This endeavor led us to confront a significant challenge: bebopc, our compiler, wasn't compatible with Stackblitz. The core issue? Stackblitz can't run native binaries, effectively meaning bebopc couldn’t be used for interactive documentation.

This roadblock sparked our exploration into WebAssembly System Interface (WASI) as a potential solution. If we could enable bebopc to operate seamlessly in the browser, mirroring its native functionality, it opened up new areas for how we can build documentation and further enhance developer experience.

At this time, we discovered that Steve Sanderson was already pioneering efforts to compile .NET projects into standalone, WASI-compliant modules. His work, while groundbreaking, had its limitations. However, it provided an invaluable foundation for our own development.

Our breakthrough came with the recent release of .NET 8. It introduced the "wasi-experimental" workload, a significant advancement over the previous Wasi.Sdk. This development marks a pivotal step toward integrating native, built-in support for WebAssembly scenarios in both server-side and browser contexts within .NET.

What About Blazor?

It's important to address the elephant in the room: Blazor. Yes, Blazor does enable .NET code to run in the browser. However, it's not without its caveats. Implementing Blazor means essentially embedding the entire .NET runtime into the browser. This approach, while functional, is far from ideal. It's cumbersome, not standard, and poses significant challenges when attempting to use a Blazor module as a generic library in various applications. Moreover, the size constraints render it impractical for edge computing environments.

We do utilize Blazor for our bebopc online REPL, but it's not the solution we're looking for in crafting versatile developer experiences. Blazor, for all its merits, doesn't align with our vision of a lean, efficient, and universally adaptable bebopc.

Some Ceremony

Before diving into the specifics of porting your .NET CLI application to WASI, it's crucial to address some preliminary setup steps. While I won't detail the process of setting up the environment – such as installing the wasi-sdk or the .NET wasi-experimental workload – you can find automated scripts for these tasks on our GitHub at bebop repository or check out Steve’s introduction video.

Step 1: Setting Up AssemblyInfo.cs

Begin by navigating to your project's Properties directory. Here, you'll either edit an existing AssemblyInfo.cs file or create a new one. Your first task is to insert the following line that informs the compiler that your assembly is compatible with WASI.:

[assembly:System.Runtime.Versioning.SupportedOSPlatform("wasi")]

Step 2: Modifying the .csproj File

Next, head over to your .csproj file. You'll need to add a specific PropertyGroup. This group is vital for the build process, and we'll delve into its details later in this post:

<PropertyGroup Condition="'$(RuntimeIdentifier)' == 'wasi-wasm'">
   <DefineConstants>$(DefineConstants);WASI_WASM_BUILD</DefineConstants>
</PropertyGroup>

Step 3: Addressing Bugs

Currently, the .NET compile target for WASI has some instabilities, including a notable bug where the build step ignores the 'InvariantGlobalization' setting. This bug, tracked in dotnet/runtime#94407, can cause your WASI program to crash upon launch if it uses culture-specific code, such as int.Parse.

To circumvent this, you'll need to modify the entry point of your application. Add the following line as the very first line of your application, before any other calls:

if (RuntimeInformation.OSArchitecture is Architecture.Wasm) {
Environment.SetEnvironmentVariable("DOTNET_SYSTEM_GLOBALIZATION_INVARIANT", "1");
}

This line effectively enables InvariantMode at runtime, mitigating the issue with culture-specific code.

If You Build It, They Will Run

Embarking on the journey of compiling your .NET project into a WASI module may seem daunting, but it's simpler than you might think. Let's dive into the process:

Building the WASI Module

To build your project as a WASI module, run the following command:

dotnet build "$project_dir" -c Release \
    /p:RuntimeIdentifier=wasi-wasm \
    /p:PublishSingleFile=false \
    /p:WasmSingleFileBundle=true \
    /p:WASI_SDK_PATH="$WASI_SDK_PATH"

If the build succeeds, head over to the AppBundle folder in your project's output directory. The command above compiles your project into a single .wasm file. You can run this file using wasmtime:

wasmtime run bebopc.wasm $*

Testing and Debugging

At this stage, it's crucial to test the basic functionality of your application. In our experience with porting bebopc, we encountered issues with Spectre.Console and errata. The Console class had unsupported features under WASI, leading us to develop a VirtualTerminal (see implementation) to maintain consistent logging with our native binary.

Addressing the File Size

You'll likely notice that the .wasm file is substantial in size; bebopc, for instance, was around 31 MB. This size is impractical, especially for edge environments. To tackle this, we turn to trimming and feature disabling:

dotnet publish "$compiler_dir" -c Release \
    /p:RuntimeIdentifier=wasi-wasm \
    /p:PublishSingleFile=false \
    /p:WasmSingleFileBundle=true \
    /p:WASI_SDK_PATH="$WASI_SDK_PATH" \
    /p:InvariantGlobalization=true \
    /p:TrimMode=full \
    /p:DebuggerSupport=false \
    /p:EventSourceSupport=false \
    /p:StackTraceSupport=false \
    /p:UseSystemResourceKeys=true \
    /p:NativeDebugSymbols=false

Note: It's essential to use publish instead of build here, as trimming only occurs with publish.

This approach reduced bebopc to 11 MB - a notable improvement, but still larger than ideal.

Further Trimming and Conditional Compilation

To further shrink the binary size, we need to guide the trimmer in identifying unused code. In our case, our language server, heavily dependent on OmniSharp, was a significant contributor to the size. Since it's invoked dynamically, the trimmer keeps it in the final binary. This is where our earlier-added property group becomes critical:

#if !WASI_WASM_BUILD
    if (_flags.LanguageServer)
    {
        await BebopLangServer.RunAsync();
        return BebopCompiler.Ok;
    }
#endif

Using this directive, we successfully trimmed all language server-related code, reducing our module size to a more manageable 5 MB.

A Note on Trimming and Reflection

One important thing to keep in mind: trimming will disable reflection by default, which will break System.Text.Json at runtime. To circumvent this, consider using source generation (learn more) and custom converters, as demonstrated in our implementation (see example).

.NET on the Edge

With a functional WASI module in hand, it's time to deploy it on the edge. In our example, we've used Cloudflare Workers to turn bebopc into an HTTP-queryable service, leveraging Cloudflare's extensive global network. The complete code for this implementation is available at bebopc-worker repository. This section will concentrate on the crucial steps to initialize and run your .NET application WASM module within a Cloudflare Worker JavaScript environment.

Handling Arguments

In the Cloudflare Worker environment, argument handling for the .NET module needs special attention. .NET expects the first argument in Environment.GetCommandLineArgs to be the process name. Therefore, when constructing an args array to pass to WASI, it should be structured as follows:

const args = ['bebopc', `--${body.generator}`, `${body.outFile}`, '--in', '--out'];

Importantly, individual arguments should not contain spaces, and each array element should represent a distinct flag or value. For instance, to add an argument with a value, you might do:

if (body.namespace) {
    args.push('--namespace', body.namespace);
}

Stubbing Imports

The next step involves stubbing certain imports. The .NET workload creates a module with imports that are not available or used in Cloudflare's WASI runtime, leading to initialization failure if not handled. To stub these imports, create a new import object:

// Custom WASI import, including handling for the 'sock_accept' function.
const wasiImport = {
    ...wasi.wasiImport,
    // Stub 'sock_accept' as it's added by the .NET 8 wasi workflow but not used.
    sock_accept: () => {
        console.log('sock_accept called');
        throw new Error('sock_accept not implemented');
    },
};

Instantiating the WASM Module

After setting up your imports, you can instantiate the WebAssembly module:

// Instantiate the WebAssembly module with the configured WASI import.
const instance = new WebAssembly.Instance(mywasm, {
    wasi_snapshot_preview1: wasiImport,
});

// Ensure the worker remains active until the WASM execution completes.
ctx.waitUntil(wasi.start(instance));

Returning the HTTP Response

Finally, the output written to standard output can be sent back as an HTTP response:

return new Response(stdout.readable, {
    status: 200,
    headers: {
        'Content-Type': 'application/json; charset=utf-8',
    },
});

Combining all these steps, you can now make an HTTP request to your worker, run the .NET CLI application on the edge, and receive a response in return.

---

In Closing

The successful porting of a .NET CLI application to run on Cloudflare Workers via WASI is a testament to the evolving versatility of .NET. This accomplishment highlights not just a technical possibility but also a new dimension in which .NET applications can operate. By stepping into the realm of WASI and edge computing, we're showcasing the adaptability and potential of .NET beyond its traditional boundaries.

At Betwixt Labs, this development aligns with our commitment to pushing the limits of technology and improving developer experiences. While our immediate plans don't include a broad implementation of Cloudflare Workers (as we are still preparing to deploy a offline web-based version of bebopc for use on websites), this project serves as a proof of concept for the flexibility and capabilities of WASI in conjunction with .NET. It’s an example of how we can leverage modern technologies to enhance the functionality and reach of applications like Bebop.

Looking ahead, we're excited to delve into more innovative uses of WASI. Our next exploration involves building a plugin system for .NET applications, allowing for plugins written in various programming languages. This initiative underscores our dedication to exploring new horizons in software development and enriching the ecosystem around Bebop.

Stay tuned for more updates, and for a deeper dive into how we're using Bebop to streamline the API development process, visit Betwixt.ai. Don't forget to subscribe to our updates – there's always something new on the horizon at Betwixt Labs!