How to load a Zig project into the website using WASM (with build.zig)

I have a tool written in Zig and I wanted to put the program online for everyone to try it. The tool has a very simple interface, but very computationally heavy operations. It accepts a string and returns True or False. Personally, I think WASM is a perfect solution for this use case because JavaScript does not care what happens inside the tool, and the tool is independent from the environment. I only need to glue them together by:

taking the input from a <textarea> and passing it into the tool, and
logging errors into a console.log.

Therefore, I decided to compile the tool into WASM. However, I could find only one complete example online of how to do this without using external libraries (emscripten). Check out https://blog.mjgrzymek.com/blog/zigwasm. However their explanations rely on prior knowledge of WASM, so I did not really understand how memory works in their example. I also wanted to replace their ugly long build command zig build-exe add.zig -target wasm32-freestanding -fno-entry --export=add -O ReleaseFast with a nice build script generated from build.zig.

Therefore, I decided to share how I integrated my Zig project into the web page. My main goals are:

Building .wasm file using build.zig, instead of build-exe command, and
Passing strings between the Zig program and JavaScript.

Note that this post is done for Zig 0.13. Because Zig is not really stable, this information might be outdated.

WASM Outline

First, we need to build our project into .wasm file. It is important to make sure that you remove all your dependencies on functionality that is unavailable for WASM. For me, I removed every line that uses:

std.log / std.debug
GeneralPurposeAllocator
std.time I am pretty sure it is possible to exclude these lines from the compilation using some comptime but I did not have time to dive into this right now.

Let us show how we can integrate the whole Zig project into the website. We will start with outlining the whole work with some simple code to give a nice understanding of core principles used.

Building WASM

First, you need a file that will contain the logic of your WASM interactions. For me, it is simply main.zig. Let us write some starting code we can experiment with in main.zig:

// main.zig

extern fn printInt(a: i32) void;

pub export fn add(a: i32, b: i32) void {
    printInt(a + b);
}

Now, we can build main.zig into WASM file. The build.zig looks something like this:

// build.zig
const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.resolveTargetQuery(.{ .cpu_arch = .wasm32, .os_tag = .freestanding });
    const optimize = b.standardOptimizeOption(.{});

    var exe_mod = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
    });
    exe_mod.export_symbol_names = &.{ "add" };

    var exe = b.addExecutable(.{
        .name = "your_project",
        .root_module = exe_mod,
    });
    exe.entry = .disabled;
    exe.import_symbols = true;
    exe.export_table = true;

    b.installArtifact(exe);
}

The point of this script is to replace zig build-exe main.zig -target wasm32-freestanding -fno-entry --export=add -O ReleaseFast with just zig build install. Important points here:

You can obtain the WASM target by b.resolveTargetQuery(.{ .cpu_arch = .wasm32, .os_tag = .freestanding }); This removes the need for -target wasm32-freestanding
You set exe.entry = .disabled; instead of passing -fno-entry
You export your functions with exe_mod.export_symbol_names = &.{ "add" }; to avoid --export=add

With this, you can run zig build install and it will build your WASM file.

Loading WASM from JavaScript

Some resource refer you to some libraries that automate the interaction with WASM, like emscripten, saying that that writing interaction by hand is too tedious. Don’t believe them, it is really easy if you have ever programmed in Zig. I only needed one static .js file with 30 lines of code.

const program = {
    obj: null,
    printInt(a) {
        console.log(a)
    }
}

WebAssembly.instantiateStreaming(fetch('your_project.wasm'), {
    env: {
        printInt: (a) => program.printInt(a),
    }
}).then((obj) => {
    program.obj = obj
    obj.instance.exports.add(1, 2)
})

After loading this js into your website, you should see 3 logged into your console. Here, the line obj.instance.exports.add(1, 2) corresponds to calling the Zig function add(1, 2). And since we have bound the symbol printInt to (a) => program.printInt(a), the function add prints the sum of 1 and 2 into the console. The program constant with its parameter obj seem useless right now because we could write printInt: (a) => console.log(a) directly, but further I will show why I needed it.

By far, this post covered what was already shown by other guides. However, when you want to do something more complex (passing strings, structs into WASM), the internet just suggests you to give up. However, it is really easy when you understand how memory works for WASM.

How WebAssembly works with memory

From now on, I will refer as WebAssembly to the thing in your browser that executes .wasm files (or to the WebAssembly package for JS). On the other hand, the output of our build script, i.e. the .wasm file itself and its internal execution, will be called WASM. This distinction is important because WebAssembly and WASM have two completely different perspectives. WebAssembly is a kind of an emulator that runs WASM, which means that WebAssembly has full control of what WASM does. On the other hand, WASM sees itself isolated from the world, and it is never aware of the WebAssembly. Therefore, WASM can work only with its own memory.

To pass data into WASM, WebAssembly needs to access the current memory of WASM and write the data into some address. Then, WebAssembly can pass the pointer to the data into WASM.

We do not want to come up with all these pointers by ourselves, so we can pass this job to Zig. It is nicely done by exposing heap allocation function. Zig will allocate the memory and return the address to WebAssembly. Then, WebAssembly will put some data into the address, and pass this address as an argument to other functions.

Implementing String Communication between WASM and JS

For example, let me show how I pass strings into the Zig program. First, let us show the extended version of main.zig:

// main.zig
extern fn printStr(msg: [*]const u8, len: usize) void;

pub export fn allocStr(len: usize) ?[*]const u8 {
    const sl = std.heap.wasm_allocator.alloc(u8, len) catch return null;
    return sl.ptr;
}

pub export fn freeStr(ptr: [*]const u8, len: usize) void {
    std.heap.wasm_allocator.free(ptr[0..len]);
}

fn print(str: []const u8) void {
    printStr(str.ptr, str.len);
}

pub export fn splitHalf(str: [*]const u8, len: usize) i32 {
    print(str[0..len]);
    print(str[0..(len / 2)]);
    print(str[(len / 2)..len]);
}

The allocStr function allocates the slice of bytes, the freeStr deallocates the slice, the print function prints the string by calling printStr function provided by JavaScript, and splitHalf function prints three versions of some string. Here, the argument of splitHalf is an example of passing strings from JS into WASM, and the print/printStr function is an example of passing strings from Zig into JavaScript.

Since we export more functions to JS, here is our updated build.zig:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.resolveTargetQuery(.{ .cpu_arch = .wasm32, .os_tag = .freestanding });
    const optimize = b.standardOptimizeOption(.{});

    var exe_mod = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
    });
    // Only this line has changed
    exe_mod.export_symbol_names = &.{ "allocStr", "freeStr", "splitHalf" }; // <- New functions here!

    var exe = b.addExecutable(.{
        .name = "your_project",
        .root_module = exe_mod,
    });
    exe.entry = .disabled;
    exe.import_symbols = true;
    exe.export_table = true;

    b.installArtifact(exe);
}

And our JavaScript file will look like this:

const program = {
    obj: null,
    split(str) {
        let str_addr = this.obj.instance.exports.alloc(str.length)
        let str_arr = new Int8Array(this.obj.instance.exports.memory.buffer, str_addr, str.length)
        str_arr.set((new TextEncoder()).encode(str))

        this.obj.instance.exports.splitHalf(str_addr, str.length)
    },
    printStr(ptr, len) {
        const str_arr = new Int8Array(this.obj.instance.exports.memory.buffer, ptr, len)
        console.log((new TextDecoder()).decode(myArray))
    },
}

WebAssembly.instantiateStreaming(fetch('your_project.wasm'), {
    env: {
        printStr: (str, len) => program.printStr(str, len),
    }
}).then((obj) => {
    program.obj = obj
})

Here, you can see why I needed the program.obj. The reason is that, printStr function accepts an address and length of a string to be printed. But it also needs the bytes located at that address! We can access these bytes by reading the obj.instance.exports.memory as an array of bytes and calling (new TextDecoder()).decode(myArray) to obtain a JavaScript string. So this is how you pass strings from Zig to WebAssembly.

On the other hand, let us look at the function split, which passes a JavaScript string str into the Zig function splitHalf. first, we obtain an address inside WASM by calling str_addr = this.obj.instance.exports.alloc(str.length). Then, we obtain an array of bytes let str_arr = new Int8Array(this.obj.instance.exports.memory.buffer, str_addr, str.length) that is mapped to the memory of WASM, starts at address str_addr and is str.length bytes long. Next, we encode our string using TextEncoder and modifying the memory of WASM by modifying the mapped array: str_arr.set((new TextEncoder()).encode(str)). Thus, in WASM, there is an address str_addr that contains the sequence of bytes encoded from str. Now we can call our function this.obj.instance.exports.splitHalf(str_addr, str.length), which will output three strings into the console.

You can try opening the browser console and print something like program.split('abcdef'). The console will then respond with:

> abcdef
> abc
> def

Conculsion

Although there are many different tools to evade this “cumbersome” process, using the WebAssembly interface directly gives you full control and understanding of what your browser does.

This was a short description of my approach to working with WebAssembly Memory model and I hope this post will make someone’s life a little bit easier.

References

[1] Zig documentation of WebAssemly
[2] Mozilla documentation for loading WASM file into .js
[3] Guide on how to use string with C -> WASM
[4] Documentation on memory model used by WebAssembly package

Olzhas Zhangeldinov - Personal Web Page