To this day, I still have my reservations, though my position on IP law as a whole is relatively conflicted as I’m both opposed to its existence but also opposed to how it’s been trampled to hurt artists.

Anyway, that’s a topic for another post. I’m going to need to simmer for longer on that one.

Something strange happened last year. As people around me started seemingly using these systems effectively - something I didn’t know how they could even do - I was interested in learning more about how to use these systems for myself, so that I properly understood how people can evangelise these things so thoroughly, when from my perspective, they were mostly just semi-useful stochastic parrots.

So over the last week, I invested about $200 USD into creating a reference Vulkan renderer for Cavey, my voxel game I’m working on for fun. There’s going to be no hyperbole and no glaze in this post; I sunk my own earned money into this thing, and in exchange for that money I expect results.

Let’s see how Claude did on my personal, army-of-one project. Interspersed between the anecdotes, I’ll share my thoughts on various parts of the process.

Baby steps: porting a WebGPU renderer to Vulkan

If you know me, you know exactly how disillusioned I am about WebGPU. It’s a great idea, and certainly good enough for many projects, but it’s a very slow-moving graphics library with lowest-common-denominator features and a few painfully design-by-committee choices, like the endless annoyance of not using an industry-standard shader format - for strangely secretive reasons?

I had had enough. WebGPU wasn’t going to cut it for me or for this project, so it was time to move on. The obvious choice for my target spec device - the Steam Deck OLED - would be Vulkan, which natively exposes all of the features of that device (including HDR rendering, which I was particularly excited about). It’s more general while still being widely supported, but unfortunately also more complex as it foists more responsibility onto the programmer (with dubious payoff).

To start weaning Cavey off of WebGPU and towards Vulkan, I had built version 3 of Cavey’s high-level rendering abstractions to be completely opaque. Anything built on this renderer wouldn’t know what graphics library it was interfacing with. I had already set up some basic-enough plumbing to get a splash screen rendering under WebGPU, but nothing more:

Under the hood, the frontend API was simple and declarative; it’s a static render-graph-style API, just minimally flexible enough for a renderer that’d be GPU driven. Very much YAGNI-style:

pub fn new(
	frontend: &mut RenderFrontend<ChosenBackend>,
	window_size: PhysicalSize<u32>,
) -> Result<Self> {
	profiler_span!("SplashScreenRenderer::new");
	let mut g = RenderGraph::empty();

	let sync_logo = SyncSource::new(Self::load_splash_logo()?);
	let logo = g.texture(Texture::Uninit {
		label: "Splash Screen Logo".to_string(),
		size: TextureSize::D2 { size: Self::SPLASH_SIZE.as_u16vec2(), array: 0 },
		data_type: TextureDataType::Unorm8x4
	});
	g.act(Act::SyncTexture { texture: logo, source: sync_logo });

	let sync_uniforms = SyncSource::new(Self::create_uniforms(window_size));
	let uniforms = g.buffer(Buffer {
		label: "Splash Screen Uniforms".to_string(),
		initial_content: BufferData::zeroed::<SplashScreenUniforms>()
	});
	g.act(Act::SyncBuffer { buffer: uniforms, source: sync_uniforms.clone() });

	let surface = g.texture(Texture::Surface);
	g.act(Act::Raster {
		label: "Splash Screen".to_string(),
		binds: hashmap! {
			(0, 0) => RasterBind::UniformBuffer { buffer_id: uniforms },
			(0, 1) => RasterBind::SampledTexture { texture_id: logo }
		},
		push: None,
		vertices: None,
		assembly: RasterAssembly::Triangles,
		cull_back: false,
		shader: RasterShader::bundled("splash_screen::vertex", Some("splash_screen::fragment")),
		colour: Some(RasterColour {
			target: surface,
			blend: None
		}),
		depth: None,
		draw: RasterDraw::FullScreenQuad
	});
	let act_present = g.executable_act(Act::Present { surface });

	let proc = frontend.compile(g)?;
	Ok(Self { proc, sync_uniforms, act_present })
}

Claude’s job here was to write a second backend for this API that would do exactly the same thing that the WebGPU backend did. This would not be a 1:1 translation however; Claude would have to reason about more than what the WebGPU code provided, for example handling the synchronisation between host and device, managing the swapchain itself, and handling resource state transitions, to name a few things.

It started on the 13th of March with a technical doc, written as a collaboration between myself and Claude on my laptop. I didn’t have a working desk setup as I had only just moved across the Atlantic:

By the 25th of March, I had my desktop in basic working order and could hit the ground running. I put Claude in the driver’s seat and paid a finite amount of attention to what it was doing. My job was to specify what Claude should do and to direct high-level design decisions; I wasn’t particularly interested in specifics of how Claude chose to set up the device or other minutiae - those aren’t the bits worth caring about. Essentially, I was a principal engineer pair programming with a digital intern.

The next day, we together had successfully burned through a massive number of the tasks, and by the end of the day, had recreated the splash screen fully, with some back-and-forth to resolve some of the bugs and edge cases that cropped up. All of our plans were documented in Markdown files and we ticked off each set of steps as we completed them.

For context: this Vulkan transition is a leap I had put off for months, not just because I had to move countries, but because it was a mentally tiring / overwhelming task. I knew the outline of what needed to be done. I could do it if I bothered to, but I didn’t have the executive function to do it when there was always something else to be thinking about.

That said, working it out with Claude made all the difference, because of the way you have to use these bots to get value out of them.

Claude is not boundlessly superintelligent. I’d compare it to an adequately capable university student. If you ask Claude to synthesise whole codebases from nothing, it’ll do a serviceable job, but probably not a good job. Tick the boxes, hand in the assignment, leave it to bitrot.

Whenever I’ve worked with Claude at my day job, I’ve figured out that really, you’re still the software architect, because Claude can’t be. At the end of the day, the latent space of possible code generations is far more populated with bad solutions than good ones as a pure statistical fact (there are more ways to screw up code than there are ways to write it correctly), so your job is to bias the code generation in a direction that lands you in a good part of the latent space, for what your definition of “good code” is.

So, I’ve learned to essentially “code in English” by precisely and comprehensively specifying what the software should do, chunking it down into tiny bite-sized steps, and getting Claude to just do them, with no grand architecture or plan in mind - essentially, doing Semantic Compression but with a robot.

It turns out this approach is good for both humans and for AI. By breaking down the problem into manageable chunks, I could better understand what steps needed to be taken, and Claude could expose areas where I didn’t specify enough at the desired level of detail. For those gaps, I would go out and find lectures on YouTube, or blog posts on the internet, and synthesise a strategy to come back to Claude with and finish the plan.

It’s a kind of synergy where Claude points out the problems, I figure out the solutions, and Claude fixes the problem with my solution.

It worked really well. I got the thing ported in a day. I would happily use Claude for this again.

Extending further: building a Slang voxel ray tracer

I wasn’t about to build out the whole render graph API with no project to apply it to. Instead, I was fully intending to build the railroad as the train was moving.

So, I dove straight into Claude’s main assignment: building out a new reference ray-tracing renderer for my voxel game.

I took a similar approach; asymmetric pair programming with comprehensively planned documents prepared upfront. However, I did something slightly different this time; I also got Claude to do a round of research specifically into 64-trees and fast voxel ray tracing, so that I could come up with a good acceleration structure using them. I specifically asked for evidence to back each technique and approach it researched, which yielded good results out of the gate:

By April Fool’s Day, we had set up a simple ray-traced scene, ready for integrating these ideas into. I didn’t have to recall any of my memorised equations or look up the Ray Tracing books online; Claude is one of the best math equation recallers in the world, even if it struggles to actually do the math. (Just like a uni student, one might say…)

The next day, I had it implement a routine for tracing a single 64-bit 4x4x4 chunk, and then had it extend to a whole contiguous world. At this stage, it was still completely separate from the rest of the game, so this terrain wasn’t “real”, but it was good enough to test the ray tracing approaches we had co-designed. Performance was great:

Runs at 90fps on Deck, pretty goooood

[image or embed]

— Daniel P H Fox (@phfox.net) 2 April 2026 at 22:26

By the very next day, we had planned and executed on the steps necessary to wire in actual voxel data from the rest of the game. At this point, I identified some latent bugs in the code and struggled a little bit to get Claude to fix them.

Here’s where we talk about Claude’s tendency to go off the rails sometimes. Much like a naïve uni student, it will often try and hack its way around a problem rather than taking the principled path, and oftentimes will run with a completely unjustified diagnosis that can often end up being completely wrong.

That is to say, you should not trust these things to be correct. They’re very productive idiots.

Throughout this process, I have spent a good amount of time intervening in what Claude is doing. I’m quick to interrupt and quick to correct, and if I feel like something has gone far off the tracks, I know how to rewind to an earlier point in the conversation and nuke the code changes.

I’ve often had to remind Claude that we’re allowed to build out the rengraph API, or that things are meant to happen on the GPU and should not be synced back to the CPU, or that recompiling the graph every frame defeats the point of having a static compiled graph and that it should prefer declarative solutions.

Some of that is wisdom, some of that is taste. Claude needs it to stay on the rails.

This is why I say that you can’t let Claude be completely in the driver’s seat. Left alone, it will make counterproductive choices that don’t fit into a larger vision of the system, especially across chat compaction boundaries or across sessions. Pretty much: enjoy Claude’s memory of what it did while it remembers because it will be gone in an hour’s time.

End game: a beautiful reference renderer

Anyhow, with the core ray tracing routine set up, it was time to start decorating the voxel scene and working towards something with the graphical fidelity needed to produce reference renders for assisting in future rendering decisions.

On the 4th of April, we started experimenting with porting the old renderer’s UV mapping code to render albedo colours. There were bumps in the road because Claude hadn’t kept track of the different coordinate spaces, so for a while, the UVs were all camera relative:

The bug was quickly identified and fixed, and not long after, we had incorporated the normal and AO maps alongside albedo to get a more complete-looking scene with very little difficulty.

Now rendering albedo, normal and combined AO/emissive maps in the ray tracing path! Still no water pass as that's more complex to pull off. Next up: going to start setting up a proper tonemapping pipeline and get some antialiasing in there.

[image or embed]

— Daniel P H Fox (@phfox.net) 4 April 2026 at 19:49

I also got Claude to port over the “smart bevels” code that would bend the normals of texels that lie along the exposed edges of blocks. This worked flawlessly first try with no planning needed.

From there, I turned my attention to tonemapping. I had Claude do a bunch of research into which swapchain format should be used for HDR, and how to query parameters like peak display brightness.

I often find myself having extended conversations with Claude about technical decisions like these. At first, Claude advocated for the use of PQ with physically-based luminance values, which I respected, but ultimately didn’t see much of a point in pursuing as I had less use for declaring absolute nit values for my renderer. Instead, I pushed back and advocated to Claude that the scene should internally be rendered with physically sensible lighting intensities, but that the output doesn’t have to be calibrated for a reference display; it just needs to look good on people’s real-world devices, which meant having a properly calibrated paper-white in the scene.

We ended up settling on scRGB with the Uchimura tonemapping curve applied per-channel for simplicity, parameterised by the properties of the display so that HDR-capable devices like the Steam Deck could punch above paper-white while SDR displays would look reasonably consistent, minus the shine.

Simultaneously, I took the opportunity to tell Claude to implement a jittered accumulation buffer for some antialiasing, as an input to the tonemapping pipeline:

Accumulating multiple samples per pixel in the ray traced path now. (Not TAA, no reprojection - just to get cleaner reference stills.)

[image or embed]

— Daniel P H Fox (@phfox.net) 5 April 2026 at 12:04

Up until this point, it had all been smooth sailing. Now for the biggest challenge so far; building out a physically based lighting model.

My end goal here was not necessarily photorealism, but instead physical plausibility. I wanted to avoid lighting “hacks” as much as possible in my renderer, so that it would be easy to progressively enhance or swap the lighting techniques later down the line without interfering with content authored for older lighting systems. Having a physically plausible renderer would also aid in handing off assets between programs; for example, allowing me to author the PBR textures externally.

The goal then was to implement a Monte Carlo integrator that would sample the whole hemisphere of lighting, combine it with a physically based material to attenuate incoming light rays, then customise it to taste with a variety of stylistic tweaks, most notably a pixelation effect to match the art style of the game.

I started by getting Claude to implement random hemisphere sampling to capture the effect of sky light, plus a sample towards the sun for clear directional shadows. When discussing the physical correctness of this, Claude was able to justify it as a kind of Next-Event Estimation. At this point, we were firmly leaving the realm of full-framerate rendering and moving towards “interactive offline rendering”.

Once combined with a physically based material, this already produced some very pleasant effects; for example, by lowering material roughness, you could get nice wet-looking surfaces with smooth specular effects. Claude handled the microfacet equations here; I asked it to review the old PBR implementation and it pointed out some deficiencies that could then be trivially fixed. The results look good to my eyes, but I didn’t inspect quite so carefully.

At this point, I started hitting my stride with Claude, going through the plan and execute loop at great speed (and expense). I regularly hit the daily limit, and just started spending into extra usage to keep the velocity up. I think it’s worth talking about what happened there.

When it comes to all of these lighting concepts, I’m more than familiar with them. I’ve spent a very long time thinking about lighting models for Cavey, and more generally have been interested in light transport for a long time now; coming up on half a decade.

However, I always found that ray traced lighting techniques quickly escape from the realm of easy intuiting into the realm of abstract maths and more difficult reasoning. You quickly start running into integrals and clever sampling/reuse tricks that ultimately suck a lot of the accessibility out of these lighting methods, except to those who sink a lot of time attempting to implement them and get familiar.

This is where I found Claude really helps. Because Claude has a bunch of knowledge baked into itself, it can easily execute on any lighting technique you know to ask about, and since you can ask Claude clarifying questions, it can also patiently explain the mechanics of how they work. I found this invaluable.

What’s more, since there’s pretty reasonably clear answers on how to build up these kinds of “physically correct” lighting models, Claude got a whole lot better at doing what it was supposed to do first try, so long as I kept it on the rails architecture-wise. The region of latent space associated with malfunctioning PBR equations seems to be smaller than the region associated with badly-structured code.

Back to the results. By sampling the block hit by the hemisphere ray for emissive colour and doing a secondary sunlight check, a single bounce of sunlight could be trivially added, which helped to fill in a lot of missing ambient light. Claude did this on my behalf with no trouble.

I then told Claude to port over the physically-based Hillaire sky model I had previously implemented for the old renderer, but with a twist; it should adjust the constants so that they match the new physically-correct lighting values in this renderer. It got it right on its second try; the first try made the sky a few hundred times too bright, which I had to report to it diligently.

the new reference renderer is so good

[image or embed]

— Daniel P H Fox (@phfox.net) 5 April 2026 at 22:57

That pixelated sun effect was the result of me and Claude going back-and-forth for about an hour debating what the best way of pixelating the sun disc would be. The general idea was established from the start; sample at multiple points, tonemap to predict what the on-screen colour would be, average to get antialiasing, then inverse tonemap to return back to physically-accurate values. Claude went down a rabbit hole with it and started introducing overcomplex tuning factors, but eventually I just told it to stop and reason from first principles. That pretty much directly led to the current zero-parameter solution, which looks the best and preserves its antialiased look no matter how intense the sun is.

I also briefly mentioned that sunsets were too bright, at which point it was able to immediately identify that the Mie scattering coefficient was too high. With approximately no effort on my part, sunsets immediately skyrocketed in visual quality to now looking the best they ever have. This is code that I will for-sure be reusing for the full-performance renderer.

Auto-exposure and physically based bloom were next in line. These required a lot of manual tweaking by myself in order to get right, and will still require yet more tweaking over time. However, Claude was able to get functioning versions of these right off the bat, even if they were not initially tuned well.

And finally, right as I was approaching my monthly spend limit I had set for this project, I implemented atmospheric fog, which was the icing on the cake. Claude executed on this flawlessly, and I did yet more manual tuning after the fact just to tone down the effect to my liking.

As a small disclaimer; since the purpose of this project was to produce a reference renderer, these last steps (especially volumetrics) really sent the performance over the edge for the Steam Deck. Even my 4090 in my desktop could only muster around 20 frames a second here, so I wouldn’t take this as Claude revolutionising real-time graphics or anything. However, for the purposes which I set out to utilise it for, this is now a highly competent and tuned reference renderer which I can use to set up further visual testing.

Conclusions

My feelings on AI coding agents is no longer that they are incapable of producing good work. My questions are now primarily about what exactly the speed-up is, practically, for realistic teams.

For individuals like myself with clear direction and autonomy, it seems pretty clear that these tools let you work at the speed of design, rather than the speed of code. For these kinds of small teams without much bureaucracy, I have no doubt that code is some degree of bottleneck, and these tools do realistically offer good speedups, even if you don’t succumb to the “vibe-coding exponentials” and instead pursue a pair programming approach.

That said, I’m still unconvinced that code is truly the bottleneck at many organisations. I can see value in Claude being able to deliver cheaper prototypes more quickly as discussion artifacts when getting alignment across teams, but I’m not sure how much of this translates to faster release processes. How does Claude speed up a sign-off from Legal?

Overall though, this experience has shown me that I need to use the tools that I have an opinion about. There’s two groups of people who I think are about to be badly hurt by the progression of this technology. One of those groups is obviously the investor-hype types that grandiosely overstate what this tech can do. But the other group belong to the staunchly moralistic anti-AI school of thought, denying outright that this tech can do anything useful at all.

I understand the moral, ethical and legal reservations, and do still share them. I’m not running to this tech, exactly. That said, I don’t see any fundamental reason why there couldn’t be a morally, ethically and legally legitimate version of this tech if it were developed and trained by authors that properly (and auditably) handled training data, and which developed it as a public good rather than as a tool of profit.

This is not technology I would bet against. Say what you want about the companies (and I have a lot of choice words for them), but there is some sense in which coding has changed. Just maybe don’t listen to the hype lords, either - go use this tech for yourself and find your own voice.

VDOM

For a decade now, I haven’t been a fan of UI approaches that involve comparing trees of elements. Often, these are referred to as “virtual DOMs” or VDOMs (this is terminology inherited from the web).

My main criticism of these approaches is that they throw away granularity information that exists at the time of building the VDOM. By design, the VDOM assumes that everything can change between re-renders of UI, and so must compare everything to retrieve the actual minimal set of changes desired.

If the original builder of the VDOM specified the minimal change they wanted, this work is totally avoidable. In its absence, even a simple property change can balloon into a lot of work done by the CPU just to re-infer the information.

Fine-grained reactivity

My goal when building Fusion (my fine-grained reactivity framework) five years ago was to eliminate this re-inference step. Instead of comparing trees of elements, my idea was to explicitly allow bindings to be made between properties of elements and objects in the code. If the object receives a change, the property is updated.

The key innovation of the system was to allow objects to bind to other objects. You could define computations that take the current state of some objects and returns a newly bindable state. When used in a UI, this established a granular chain of cause-and-effect through the UI, and allowed for changes to be applied directly without re-inferring anything.

However, this still didn’t satisfy me completely. The most obvious issue is a colouring problem; code that is written “reactively” has more ceremony than code which is written “traditionally”. The cost to the programmer of managing a fleet of objects required supporting concepts in memory management and helper utilities for improving ergonomics.

There’s also a more subtle cost in frameworks like this; they tend to over-memoise information by their nature. Since every object is a binding, every object has a state that needs to be synchronised to its users. This typically means the object is a state container. As a result, you effectively memoise every causal node in the system, which often means memoising every piece of dynamic state, even trivially inferrable ones. The cost of this memory has been shown to be notable in practice - it would be more ideal to create a scheme that more naturally encourages recomputation of trivial states rather than universal memoisation.

The principle of information loss

From my experience, using fine-grained reactivity has led to almost universally more predictable and favourable performance compared to VDOM systems. Many of the highest-performance frameworks today adopt similar principles.

But why is it better? In my eyes, it’s because it moves memoisation work closer to the subject being memoised.

Since the memoisation done by the reactive objects can take advantage of information available in the local context, they’re able to track changes granularly without re-inferring any missing information. The bindings and objects involved are directly and trivially known.

Attempting to apply this principle, I would extrapolate that the best memoisation techniques are likely ones which deduplicate calculations “as they happen” - parts of the computation which wish to reuse historical data can elect to do so on their own, in the same path of computation that returns the result of the computation. To the end user, the computation simply provides a result - caching details are encapsulated in some kind of context side-effect.

That’s what I’ve been exploring next, as part of a reductive approach I call “flip caches”.

Flip caches

Let’s consider an example of a gallery app. We want to show a bunch of photos with different widths and heights in a list view, similar to Google Images.

Our state may look like this:

struct Photo {
	width: f32,
	height: f32,
	image_content: Texture
}

struct UIState {
	photos: Vec<Photo>
}

Let’s write a naive immediate-mode function that implements a wrapping layout.

fn build(state: &mut UIState, renderer: &mut Renderer) {
	const MIN_ROW_HEIGHT: f32 = 100.0;
	const GAP: f32 = 4.0;

	 // Break photos onto rows bounded by screen width
	let photo_rows = {
		let mut x = 0;
		let mut index = 0;
		let mut photo_row = vec![];
		let mut photo_rows = vec![];

		for photo in state.photos {
			let height = MIN_ROW_HEIGHT;
			let width = (photo.width / photo.height) * height;
			if x + width > imgui.screen_width {
				photo_rows.push(photo_row);
				photo_row = vec![];
			}
			photo_row.push(photo);
			x += width + GAP;
		}
		if !photo_row.is_empty() {
			photo_rows.push(photo_row);
		}
		photo_rows
	}

	// Render rows of photos scaled to fill screen width
	let mut y = 0.0;
	for photo_row in photo_rows {
		let total_gap_widths = (photo_row.len() - 1) as f32 * GAP;
		let total_photo_widths = photo_row.iter().map(|p| p.width).sum();
		let scale_factor = (imgui.screen_width - total_gap_widths) / total_photo_widths;
		let height = MIN_ROW_HEIGHT * scale_factor;
		let mut x = 0.0;
		for photo in photo_row {
			let width = photo.width * scale_factor;
			renderer.draw(Draw::Texture { x, y, width, height, texture: photo.image_content });
			x += width + GAP;
		}
		y += height;
	}
}

Every redraw event, we compute the layout in full, then directly send instructions to the renderer to draw textures where we want them. This is stateless and does not memoise, which makes it very easy to work with, but discards a lot of information. That’s especially bad when working with expensive layouts that involve many elements.

Let’s assume for demonstration that the photo_rows computation is a hotspot (perhaps it’s allocating too much, don’t ask!). Instead of immediately reaching for a VDOM or fine-grained reactivity, we could instead keep a cache between rebuilds. Maybe it looks like this:

struct PhotoRows(Vec<Vec<Photo>>);

fn build(state: &mut UIState, renderer: &mut Renderer, cache: &mut Cache) {
	const MIN_ROW_HEIGHT: f32 = 100.0;
	const GAP: f32 = 4.0;

	 // Break photos onto rows bounded by screen width
	let photo_rows = cache.get_or_init::<PhotoRows>(|| {
		let mut x = 0;
		let mut index = 0;
		let mut photo_row = vec![];
		let mut photo_rows = vec![];
		for photo in state.photos {
			let height = MIN_ROW_HEIGHT;
			let width = (photo.width / photo.height) * height;
			if x + width > imgui.screen_width {
				photo_rows.push(photo_row);
				photo_row = vec![];
			}
			photo_row.push(photo);
			x += width + GAP;
		}
		if !photo_row.is_empty() {
			photo_rows.push(photo_row);
		}
		PhotoRows(photo_rows)
	});

	// Render rows of photos scaled to fill screen width
	let mut y = 0.0;
	for photo_row in photo_rows {
		let total_gap_widths = (photo_row.len() - 1) as f32 * GAP;
		let total_photo_widths = photo_row.iter().map(|p| p.width).sum();
		let scale_factor = (imgui.screen_width - total_gap_widths) / total_photo_widths;
		let height = MIN_ROW_HEIGHT * scale_factor;
		let mut x = 0.0;
		for photo in photo_row {
			let width = photo.width * scale_factor;
			renderer.draw(Draw::Texture { x, y, width, height, texture: photo.image_content });
			x += width + GAP;
		}
		y += height;
	}
}

The idea here is that cache.get_or_init::<T> only runs the inner closure once, then it saves the returned T and uses it for all subsequent calls. (The type exists to allow for multiple different pieces of data to be independently cached in a type-inferable way.)

This cache solves our memoisation problem for this expensive layout computation without requiring a deep structural change to any code. Since the mechanism is generic, it can encapsulate as much or as little of the code as profiling indicates is necessary.

For example, if constructing commands for the renderer was also expensive, that could be memoised instead:

struct PhotoDrawInstructions(Vec<Draw>);

fn build(state: &mut UIState, renderer: &mut Renderer, cache: &mut Cache) {
	const MIN_ROW_HEIGHT: f32 = 100.0;
	const GAP: f32 = 4.0;

	let instructions = cache.get_or_init::<PhotoDrawInstructions>(|| {
		// Break photos onto rows bounded by screen width
		let photo_rows = {
			let mut x = 0;
			let mut index = 0;
			let mut photo_row = vec![];
			let mut photo_rows = vec![];

			for photo in state.photos {
				let height = MIN_ROW_HEIGHT;
				let width = (photo.width / photo.height) * height;
				if x + width > imgui.screen_width {
					photo_rows.push(photo_row);
					photo_row = vec![];
				}
				photo_row.push(photo);
				x += width + GAP;
			}
			if !photo_row.is_empty() {
				photo_rows.push(photo_row);
			}
			photo_rows
		}

		// Render rows of photos scaled to fill screen width
		let mut instructions = vec![];
		let mut y = 0.0;
		for photo_row in photo_rows {
			let total_gap_widths = (photo_row.len() - 1) as f32 * GAP;
			let total_photo_widths = photo_row.iter().map(|p| p.width).sum();
			let scale_factor = (imgui.screen_width - total_gap_widths) / total_photo_widths;
			let height = MIN_ROW_HEIGHT * scale_factor;
			let mut x = 0.0;
			for photo in photo_row {
				let width = photo.width * scale_factor;
				instructions.push(Draw::Texture { x, y, width, height, texture: photo.image_content });
				x += width + GAP;
			}
			y += height;
		}

		PhotoDrawInstructions(instructions)
	});

	for inst in instructions {
		renderer.draw(inst);
	}
}

The desirable property of this solution is that it obeys Tennant’s Correspondence Principle: memoising a passage of code does not require restructuring as fine-grained reactivity does. The default assumption made is that computations are trivial (which they often are).

Of course, the problem now is cache invalidation, but we have existing techniques to address this.

impl UIState {
	fn set_photos(&mut self, photos: Vec<Photo>) {
		self.photos = photos;
		self.photos_changed = true;
	}
}

// ... later ...

if self.photos_changed {
	cache.clear::<PhotoDrawInstructions>();
}

To implement such a cache, I would use a “flipping” method to allow read access to the previous version of the cache while allowing write access to a fresh version of the cache. This is why I use “flip caches” to describe this technique; it only requires space for two T allocations, plus a little book-keeping in the cache object itself, and flips between these allocations for predictable performance.

Why I think this could be useful

I haven’t yet implemented a fuller UI system using flip caches, but I like the solution because it essentially filters rebuild events for closures which still rebuild in an immediate-mode fashion. There is no colouring problem like fine-grained reactive signals have.

Additionally, flip caches avoid the information loss problem of VDOM techniques; since the caching happens “inline” in the build function, diffing is unnecessary - the renderer receives the draw instructions it needs directly.

Of course, there’s one caveat to all of this, which is that re-constructing trees of components every rebuild could become expensive; a forest of small components can add up in a way that isn’t true of fine-grained reactive systems, which construct their trees once.

I’m interested to explore this system further to see whether higher-level abstractions can be built from this, to reach similar ergonomics to fine-grained reactive systems. In particular, I’m interested in orthogonal techniques to do with cache invalidation patterns, and how these caches could perhaps carry incidental UI state such as animation timelines.

I recommend you go watch this revelatory talk by Ryan Fleury. The chain of logic they describe in that talk reflects pretty much my entire history with memory management up to today. I’ll continue this post assuming you know what I’m talking about.

The joy of arenas

Back in 2023, while working on Fusion, I rediscovered arena allocation independently in a GC language. Without anyone else ever having explained the concept to me, I immediately saw just how much more powerful structured memory management is, and how it leads to programs that are both trivial to reason about, and also fully deterministic.

It’s so utterly powerful that I managed to implement (what Fusion calls) time-travelling checks to predict use-after-frees, which have been a revelation for catching edge cases that I otherwise would have missed.

It turns out these ideas are not original in the slightest! Ryan Fleury, Casey Muratori, Johnathan Blow and the rest of the gang have been out there for years singing the praises on arena allocators, and my lived experience dealing with all of this is all the proof I need that these ideas are every bit as good as each one of them said they were.

I love arena allocators. They’re everything I’ve loved about “maids” in the Roblox ecosystem for years, and they’re almost word-for-word identical to “scopes” (which are really just properly-done arenas that run destructors instead of freeing memory).

The rub

There’s only one problem with arena allocators. It’s not even really that much of a problem, but it’s still worth mentioning IMO.

Arenas are still visible in your code. (Boo hoo, right? Unusable.)

When I designed “scopes” for Fusion, that was a feature - it declared a code path’s intent to allocate, and allowed you to declare their lifetime explicitly in your code, preventing whole classes of bugs. For any language that exists today, I think it’s the right approach.

But really, I’ve been looking for a way to make code look every bit as clean as garbage collection, while using a deterministic structured approach to memory management that can be analysed ahead of time and trivially checked for safety. I don’t know if today’s languages are up to that task without doing some gross things that I’d never recommend over simply writing the damn thing.

Wait, I’m writing a language already

Yeah, turns out I’m the benevolent dictator of my own still-in-flux language. And, turns out I’ve already been stewing on this problem in a different context.

One of the top features I’ve wanted in Wolf (my language) are implicit function parameters, aka context, so that you don’t have to do “prop drilling” to transiently pass relevant wide-area information down to deeply nested functions. I got the idea from my frontend UI experience, ironically enough - it’s a feature I implemented and loved in Fusion, which itself was somewhat stolen from React. (Hey - React’s useful for something!)

My original line of thinking here was that it’d be really nice if you could contextually define an arena allocator to use inside of a function, a bit like how Jai (Johnathan Blow’s language) lets you do the same thing. That would make it a lot more ergonomic to have dynamically-sized literal constructors in Wolf - no need for any special syntax to tie them to a specific arena allocator. It also makes all functions generic over allocation strategy by default, which is rad.

However, I realised something pretty nice about Wolf. Since it’s a highly declarative programming language without things like mutation, pointers or references baked deeply into it, everything is lexically scoped. In theory, that makes the whole language pretty compatible with stack allocation, as you might find by default in C. It’s not a 100% solution, but a good 80% solution.

I was… half satisfied. Then I thought a little harder about what stack allocation really is.

Oops! All arenas

Yeah, turns out stack allocators are just mini little arena allocators, aren’t they?

This is about when I realised that Wolf could have a really nice generic allocation strategy that does the same thing as a stack allocator, but doesn’t prescribe use of a stack per se. You just implicitly create a scratch allocator for each block, and if it passes any values out of itself, then you implicitly parameterise the block with an allocator to use for those values.

-- Pseudocode
fn [.return_alloc : allocator] (
	let local_alloc = allocator
	-- Allocated temporarily for the duration of the block
	let needs_memory = malloc [.name "steve", .age 27] in local_alloc
	-- Important: use the return allocator here
	let will_return_this = malloc [.name "baxter", .age 5] in return_alloc
	free local_alloc -> will_return_this
)

Via induction, since all Wolf programs are composed from blocks, you can now guarantee that all memory is allocated and deallocated correctly (if sometimes conservatively, and with some extra nuances/implementation details that aren’t relevant here).

There’s even more nice things where that came from, too. You don’t strictly have to use one allocator per block; you could use as many as you like, wherever you like, because none of those details actually appear anywhere in the syntax.

For example, you can determine when the last time a value from an allocator is used in a block, and at that point, you can choose to drop the allocation right away. Alternatively, you could choose not to if you locally know it’ll get dropped by an allocator in a higher lexical scope. In other words, you can fit your arena allocator lifetimes more tightly or loosely to your program depending on context, all completely automatically.

What’s more, you could decide to use a right-sized allocator just for the parts of the block where the sizes of values are statically known, and use a more flexible allocator for the unbounded dynamically-sized things like user strings. (The fact that Wolf intentionally tries to be as compile-time-analysable as possible helps massively here.)

Oh right, flexible allocators for dynamically sized things… shoot.

Who manages the manager?

This model of memory management in Wolf works perfectly well in theory, when you have arenas that are assumed to be growable and flexible, and when you have infinite memory space.

In reality, you get a finite amount of virtual memory address space, and of that, a dismal fraction that’s available as physical memory.

At this point, I was thinking about allocation strategies for arena allocators that needing to be dynamically growable, or perhaps having a dedicated allocator just for growable arrays of some kind, which could then be used as a springboard for implementing dynamically sized arena allocators.

I was decently happy with that idea. That’s about when I discovered Ryan’s talk.

There’s hardware for that, dummy

I had overcomplicated the whole thing because I wasn’t thinking about how the OS actually manages virtual address space. You can use the hardware’s MMU to just… reserve tons of contiguous memory and forget about it. It’s not like it’s going to be backed by any physical memory until you actually use it, so it’s free real estate.

So, the way Ryan describes the technique: make every arena allocator gigantic in virtual memory, then just forget about it. Sure, you’ll have a maximum arena size, but it’ll be something like 64 GB. No need to sweat over it when it’s orders of magnitude larger than the data your program is dealing with, and when you have 256 TB of virtual address space to carve up for arenas. Plus, this is where statically knowing the size of things can really come in clutch.

Ryan says that this gives you ~O(1) allocation and deallocation, all super close to the metal. I’m inclined to believe it, though I’m not about to repeat it without citing Ryan as I haven’t run through the whole analysis myself.

Zero overhead allocation and deallocation with fully implicit memory management that’s as clean as garbage collected code. I’m dreaming, right?

Remaining considerations

So obviously, I’m super jazzed about this. From my very brief searches online, I haven’t really found a language that does all of this together, in the way I’m describing here. Wolf may well be (one of) the first languages setting foot in this territory.

However, I still have a few lingering thoughts on my mind that I haven’t yet cleared up:

• I suspect that I shouldn’t fully rely on the MMU to reserve gobs of virtual address space. I might implement a hybrid solution that chains large-enough blocks of contiguous space instead (something like 1GB chunks?) which might play better on platforms that don’t have as many bits of virtual address space.
• I’m also not entirely convinced yet that I can escape doing allocator management this way. I’m still thinking about ways of potentially carving up the address space in a way that can be used for allocating arenas of various sizes, just in case. I don’t really know.
• I’m also clear-eyed that people may still want to implement more specialised allocation techniques. My idea is that arena allocators are a perfectly good implicit language feature, and that through arena allocation, it should be trivial to manage the lifetime of custom allocators and their reserved memory. So I’m not thinking about them too hard right now.

Overall though? I might have just solved memory management.

I’ve defeated that damn garbage collector.

The golden rule

Simply: I don’t get to control what you think. I don’t get to make demands of you. If it is within the law, you are allowed to do that thing if you really want to. I won’t stop you if you’re determined. You are a human with autonomy and reasons for being the way you are.

Will I still have an opinion on you doing certain things? Sure thing. It’s not illegal to think about whether I would do what other people are doing. But you don’t have to ask for permission from me, you live your best free life and I’ll sit over here allowing you to do that.

I grade people primarily on one axis only: whether they treat others as humans or not. I want to live in a world where we see the humanity in each other.

Why this is my rule

I’m a highly non-prescriptive person in general. I don’t want to be involved in what people around me do in their private lives or their private time, unless we agree to be involved in something. If all parties are consenting, it’s not my business, and I don’t feel inclined to know. That’s one reason.

But more broadly - I understand that we have a process for outlining things that shouldn’t be done, and that process is democracy. If I wanted to stop people from doing something because I was concerned about its impact, that’s the path I would pursue.

Systemic problems demand systemic solutions, not individual ones. The idea that a society of individual choices are responsible for the world’s largest problems isn’t sound to me; governments are bottlenecks through which all legal behaviour must pass. It’s the clear place to target when trying to solve problems, especially ones where responsibility is amorphous and hard to pin down. Make it blameless instead - institute a system that outlines what shouldn’t be done. If you fail to do that, then find out how to do it next time.

I ultimately think the behaviour of individuals should be driven by what they think they should do, not what they’re told to do by others. If I want to reduce the amount of meat in my diet, that’s wonderful and I should go do that. But there’s no logical reason why it’s the solution to a systemic problem; people don’t work like that.

So, if I want others to eat less meat, I can offer a recommendation or a reason, but if they say no, then that’s their choice and it would be improper to override it. It would be especially improper to treat them badly over it, publicly shame them, make people feel bad for associating with them… you get the idea.

It’s almost always the case that there’s a reason people are compelled to be a certain way. Their reasoning may not be sound or rational, but people don’t generally get out of bed in the morning to make the world a more miserable place. I often say that there’s a nugget of truth in every opinion, you just have to work through the dressing to find it.

This generally leads towards my big idea of how we should make change in the world. It’s not by dictating to others what is good for them, regardless of how obvious I think it is. It is instead by listening first, understanding what they think and believe, identifying those nuggets of truth, and figuring out where to go from there. That’s how you build a base around an idea and transform that into change.

That’s true even of the people I disagree with the most vigorously today. I don’t begrudge anyone, because people are complex mosaics of attributes, not single dimensional beings.

Why I’m concerned

It feels like a fair few people don’t do this anymore because they don’t believe in these ideas anymore. It’s mistaken for ideas that aren’t helpful, like centrism or conformism. I think that’s a dying shame that the difference isn’t known better.

It often feels like people demand others to do things for them, or be cast out. Believe in the same ideas we do. Conform to what we tell you to do. They attend gatherings where important speakers present romantic visions of a world where dissent is quieted. To some small extent, perhaps they even pride themselves on how they treat some others badly.

That’s not a phenomenon that’s boxed into a certain sect of society, by the way. There’s endless talks about extremes on a spectrum, but society is far more higher-dimensional than that. It’s not about simple politics or social causes, it runs deep into even the most niche topics where there isn’t unanimity. I’ve seen it in enthusiast spaces where the stakes are supposedly zero.

We have forgotten how to talk about our significant differences in any space on any topic. The moment I realised this, I saw it everywhere. It changed the way I view the world.

What I’m doing

As you may have seen, I’m not on social media, except for YouTube. I don’t think I need to explain how it relates to this concept; it’s far too easy for people to walk in the door and shit on your plate. I don’t want to be part of the global town square if that’s what’s on offer.

I much prefer tighter, closer connection. When I get to know you, and we become friends, and we do more things together, I think that motivates people to place more of a value on humanity. When my friends come out with things I can’t accept, being close to them places a cost on rejection and encourages me towards listening. The same logic works when my friends disagree with me, too.

In general, this has made my life much more pleasant. By taking this stance, I haven’t compromised on how I think about the world at all. Instead, the opposite has happened: I feel like I have more conviction in my view of the world, as I understand more about the other people who live in it. I’ve become softer and less demanding of others, and it’s brought me closer to a lot of people in my life.

It feels like it’s been a long road of personal development to understand this. There was a time where I was insufferable and made myself miserable for no reason.

Writing this is a bit like documenting what I’ve learned since then, I suppose. It turns out life can be sunnier if you find the humanity in others. It’s not hard, you just have to believe a little.

You’ll likely start with a very valid question…

Why?

I don’t believe we’re at the end of programming language history. We haven’t found the perfect language yet (does that even exist?) so why not spend time exploring more of the space? Sure, I’m not an academic, and absolutely daft, but I figure there’s still fun to be had in the process of exploring the possibility space.

In particular, there’s a few ideas that excite me terribly:

True consistency

Most programming languages are at least influenced by the syntax and semantics of C.

I think that C is a perfectly fine programming language! Basic perhaps, but somewhat fine. However, I think that most languages designed in C-style aren’t really consistently designed languages. Things that are ostensibly similar look different on the page, and things that are ostensibly different look similar on the page.

After a decade+ of working with the Lua language, I know how much I love a simple mental model. I wonder what that could look like when applied to a language’s syntax.

I want a language where you can point to language features visually, by pattern matching with your eyes. Instead of building towering complexes of features, you should be able to hold a model of the whole language in your head, and it should map 1:1 with each syntax item.

Incremental recomputation

This is probably the headline thing I think about these days. After years working on reactive signal libraries, I’m desperately tired of all the extra syntax cruft.

I often think about immediate-mode programming. Conceptually, it’s far and away the best choice for expressing logic clearly. The only drawback of it is that blowing away a bunch of computation just to do it all again can be incredibly wasteful, especially when you need to perform more complex layout reflow for example.

Why can’t a compiler optimise that?

Of course, the answer defies simple explanation - side effects, mutation, blah blah. But that’s just why we can’t do it in existing languages. What would a language built for that look like?

Imagine you didn’t just run through a whole function every time you needed to update the result of a previous calculation. What if you could just re-run the part that’s actually relevant?

Right now, I fully believe that something like this is the bridge we need between immediate-mode and retained-mode logic.

Memory management

After implementing scopes in Fusion, I’m pretty furious we ever spent so much time on garbage collection. Structured memory management is so obviously superior it’s not even funny.

The problem is that doing structured memory management in Lua is also so obviously against the grain. Lua has no facilities for doing it ergonomically, because it’s built on the philosophy that garbage collection will handle almost everything for you.

Simultaneously, I’m a pretty happy Rust user, but I’m not about to defend Rust’s memory management approach either. Lifetimes are hell, and the language as a whole is confusing and unintuitive to learn if you’re not familiar with its concepts.

When I listen to Graydon Hoare and Casey Muratori, I wonder just how many trivial memory management techniques we’re missing when we think in terms of RAII and lifetimes.

Could you make a language that trivialises memory management by taking a broader view of the problem?

Open objects

It seems really common these days to define objects as rigid nominal types with only a defined set of data on them.

Perhaps you’re making a 3D modelling program and you want to store some vertex data:

struct Mesh {
	vertices: Vec<Vertex>
}

struct Vertex {
	position: Vec3,
	normal: Vec3,
	uv: Vec2
}

And then perhaps, the tool has a vertex painting mode, where you want each vertex to additionally have a vertex colour and alpha.

These things logically live on the vertex, so you could put them there:

struct Vertex {
	position: Vec3,
	normal: Vec3,
	uv: Vec2,
	colour: Vec3,
	alpha: f32
}

But what if you wanted that vertex colour and alpha to come from an interchangeable “vertex colour map”? You’d have to store them externally:

struct VertexColourMap {
	vertices: Vec<VertexColour>
}

struct VertexColour {
	colour: Vec3,
	alpha: f32
}

I’m going to apply a somewhat-arbitrary transformation to this code. I’m going to switch the data around to be stored in SOA format, rather than AOS.

struct Mesh {
	positions: Vec<Vec3>
	normals: Vec<Vec3>,
	uvs: Vec<Vec2>
}

struct VertexColourMap {
	colours: Vec<Vec3>,
	alphas: Vec<f32>
}

Notice that, by performing this switch-around, we’ve actually flattened down one of the dimensions of this problem. No matter whether we store the data in Mesh or VertexColourMap, we do not need to change the data structure at all:

// The data is stored the same way (!)
struct Mesh {
	positions: Vec<Vec3>
	normals: Vec<Vec3>,
	uvs: Vec<Vec2>,
	colours: Vec<Vec3>,
	alphas: Vec<f32>
}

This led me to a really cool insight - what if you could build objects by joining together fields like this?

Instead of having a “mesh struct” with a rigid structure, what if an object could be formed from any fields you wanted to view together? In a system like that, you could easily define extra fields for an object in your own namespace, and transparently associate them with data that comes from elsewhere in the codebase.

Instead of “closed objects” that can’t be extended by others, you’d have “open objects” that are just syntax sugar for array accesses anywhere in memory.

I view this as the ultimate expression of the open-closed principle, or perhaps just the logical conclusion of entities from ECS. Objects become mere IDs - things you use to access fields.

What would a language built around “open objects” look like? Could open objects be made efficient? Where would they fall short?

Introducing Wolf

To pursue these questions, late last year I decided to pull the trigger and push the first commit to a repository called wolf.

Back then, I didn’t intend to actually implement the language I was designing. All I wanted was to explore what me and my good friend Trey thought “our ideal language” could look like.

In that time though, I’ve ruthlessly scrapped and rethought a ton of ideas. In the time that some people may have completed a whole traditional language, I’ve just barely figured out how the basic constructs of the language should even work, and even then I’m not entirely sure.

That said, beyond the pages, there’s a few truths I seem to have landed on in my head:

Wolf works like maths

Wolf is designed along the grain of mathematics. Equals is equals - there’s no assignment or mutation by default.

In that sense, Wolf is firmly placed in the camp of immutable functional languages built atop the lambda calculus, but not for any of the traditional reasons - I didn’t even know what the lambda calculus was before I started.

I don’t necessarily agree that mutation and assignment are fundamental operations that should appear in normal logic. I think most logic should be stateless, and when you’re writing stateless logic, you shouldn’t need mutation in theory. (The lambda calculus backs this up!)

The real reason I think we reach for mutation is because functional syntax sucks. Recursion is not an ergonomic replacement for a for loop in many cases. Why haven’t we tried to close that gap?

So, to the maximal extent possible, Wolf should let you express stateless functions the same way you express maths - as substitutable expressions.

Wolf mutates, but carefully

The last section may have given you the impression that I’m a Haskell enjoyer (more of an F# guy tbh), but I’ve omitted most of the story.

You see, I don’t think mutation is something to be avoided. After all, our computers are basically just overgrown Turing machines. To deny mutation in a programming language is to invent a new alternate reality.

Instead, I simply view things in reverse to most languages. In my head, mathematics is the base of logic, and mutation/imperative style is an extension to basic logic that allows it to interact with an event-based model of the outside world.

From that angle, I think it is perfectly rational for Wolf to support mutable dynamic objects, because they’re a reality of programming a Turing machine. The only key difference is that I think it should be tightly managed, so that both humans and machines can figure out how these dynamic objects and mutation events affect the execution of the program.

One particular mistake I’m not keen to re-enact is the idea of allowing unmanaged memory allocations and frees. I see no reason for there to be a “global allocator”, and have no desire to implement things like borrow checking or RAII.

Why are we trying so hard to make objects manage themselves internally, when we could just manage them better from the outside? We already know about concepts like arenas, object pools and slab allocators. These things trivialise memory management, but I don’t blame people for not using them when many vogue languages don’t elevate them to a prime position.

In Wolf, I want all dynamic features to route through structured memory management techniques like these, rather than random vibes-based resource allocations or garbage collectors.

But importantly, it shouldn’t feel like you’re managing memory at all - it should be transparent, almost implicit in what you’re writing. After all, it’s maths, right?

Wolf is ruthlessly smart

I think static analysis is non-negotiable in this day and age for good autocomplete and program optimisation. I don’t want another dynamic scripting language.

So, Wolf is designed to be 100% statically analysed, type-checked and optimised. The language avoids features that would require dynamic or run-time analysis to check the correctness of, or which lead to optimisation footguns.

Instead of methods, Wolf prefers pipelining and freestanding functions. When you can pipe an object into a function, you don’t need methods anymore. I already write most of my code using freestanding functions today in all programming languages I work in, and it makes modularity so much easier to achieve because it obeys the open-closed principle naturally.

Instead of polymorphism, Wolf prefers switching based on statically-available type information. This sort of thing should work a lot better in Wolf, because it’s built without a lot of the concepts that make indirection necessary in other languages. We know that random memory access is especially expensive when factoring in things like cache misses, so why depend so heavily on it?

Instead of dynamic types, Wolf prefers inferred types. If you define a minimum amount of information about your interfaces, that the compiler should be able to propagate through your implementations. Inference should be a forwards process, since code is written forwards, and that unlocks the best possible autocomplete and error messages for unfinished code.

Conclusions

Wolf is still extremely early in its life. It doesn’t even have a reference implementation, or even a completed design. Mostly, it’s just ideas, but I have some conviction that these ideas are good to pursue further.

Long-term, I want to bridge the gap between lightweight scripting languages and powerful systems languages - a kind of systems-language Lua that threads the needle between expressivity and performance.

I don’t make any promises that this project goes anywhere, but if you see me making commits, then you know what I’m up to. Wish me luck on my misguided programming language adventures :)

This is a strange post to write - Roblox tool development has served me incredibly well. I’ve been lucky enough to make a great thing out of plugin and package development; it’s covered my rent every month and it’s been consistent for me. If I were a ruthless capitalist, perhaps I’d ride out the income until it dried up.

But I respect you more than that.

The truth of the matter is that, since starting my day job last summer, I’ve found myself immersed in Roblox day in and day out. It’s great work! I enjoy it massively. But I don’t wish to take it back home with me.

Instead, I’ve found greater inspiration in the world of things that live outside of the engine. Whether that’s learning about voxel engines with Daydream, or thinking about the future of UI and declarative programming with Fusion, I’ve realised that’s what I want to spend my time on.

So I’m going to quit while I’m ahead. My Roblox projects continue to be massively successful, but I’m happy with where they’ve gone and what they’ve inspired in others. I like to say that “enough” is the most powerful word in the world; I think I have found “enough”.

What happens next

I plan to wind down Studio Elttob, which has been the brand under which all of my Roblox-only projects have lived.

This will involve opening up most of its products, cancelling in-progress projects, and rehousing some others.

• Cancelled: The Elttob Atmos and Elttob InCommand plugins have been cancelled.
  • There was so much in the in-progress builds of Atmos that I was genuinely proud of! - but I don’t feel compelled to work on these projects at this stage.
• Cancelled: The VOLUMIKA 2 project has been cancelled.
  • While this was already somewhat functional, I struggled with performance issues for a long time, and those haven’t been resolved.
• Opened up: The following paid products will enjoy extended support until 1 September 2025, at which point they’ll be opened up to the community so that everyone can benefit from them. This includes:
  • Elttob Relight
  • Elttob Reclass
  • All Classic Suite plugins
  • VOLUMIKA
• Continuing: Fusion will continue development, but:
  • Studio Elttob branding will be removed from the project.
  • The FusionKit for Roblox project has been cancelled.
  • Fusion’s Roblox APIs will continue to be supported, but will be modularised over time to allow Fusion to be used for general purpose Luau development.
• Opened up: Vanilla 4 for Roblox Studio will be opened up:
  • Vanilla 4 for Roblox Studio will be opened up to the community, but new icons won’t be designed.
  • The Vanilla compiler, previously internal and proprietary, will be open sourced.
• Going private: Vanilla itself will be taken private:
  • Plans for a commercial Vanilla icon set have been dropped.
  • The Vanilla icons will return to being a personal icon set for my own projects.

My new priorities

With Studio Elttob going away, I can sharpen my focus on my remaining projects.

Front burner: Daydream

Daydream can be traced all the way back to some of my earliest Roblox projects back in 2012/13. Since then, it’s enjoyed multiple transformations:

• Becoming the first infinite voxel engine on Roblox capable of Minecraft-scale worlds in 2016
• Having its technology repurposed for the monumentally successful Blox in 2019
• Developing into a highly-optimised engine for Blox Survival in 2021
• Being spun off of Roblox into the standalone Project Stockholm, now called Daydream, where it is today!

In many ways, Daydream is the one project I’ve wanted to work on for the longest. It’s also the most educational project I’ve ever worked on; originally used to learn about the basics of voxel engines, then teaching me so much more about high-performance Luau and game optimisation, and now serving as a vehicle for me to learn computer graphics, GPU programming, and low-level engine development.

I’m more motivated to work on Daydream now than I’ve ever been before. I love it so much.

So, I’m now focusing my efforts on Daydream first and foremost, to give it the time of day that it deserves. I’ll continue to work on the Daydream YouTube series so that I can educate more people about the myriad of interesting computer science problems that arise from it. Perhaps one day, it’ll even be something you can play! But I’m not thinking that far ahead yet.

Back burner: Fusion

I still believe in Fusion as the ultimate companion for Luau development and remain committed to providing support for it. I also still intend to take Fusion further than where it is today, though I can’t say that I’ll do so on an expedited schedule.

I think Fusion is overall in a good place after the push to get 0.3 out the door. From here, things look to mostly be performance work and ergonomic tweaks, and some project restructuring.

As a little treat; I currently plan to use Fusion and Luau inside of Daydream! I want to build a complete non-Roblox UI using it. I’ve even been floating the idea of rebuilding the whole documentation website in Fusion one day, but that’s more of a wishful thought.

And that’s it

All of my other existing projects aren’t going to be actively developed. That doesn’t mean they’re over, and it doesn’t mean I won’t start more of them - it just means I won’t touch any particular project unless I have a motivation to.

I’m making a point of paring down my active projects to exactly two, because that’s where I think I’m happiest right now. If I don’t make progress on other projects, I’m fine with that. Daydream and Fusion are where I want to push right now.

This doesn’t mean I’m leaving the Roblox community, by the way. I have a whole lot of Roblox knowledge and I’m incredibly interested in the future of the platform. I want to help build Roblox up to be the best place it can be, so you’ll still see my username around.

It’s just that the user in question will be a hobbyist, not a full time creator :)

The primary reason is that it’s hard to properly read frame rates, because they are actually a reciprocal unit; to double your framerate, you must halve the amount of time taken to process a frame.

This can be fine in isolation - defining a “target framerate” is still a valid concept, because it’s reasonable to want to serve someone, say, 120 frames per second.

But when you try and compare frame rates with each other, or track frame rate through development, you’ll quickly find that framerate can be misleading. Targeting 240 FPS, but your prototype is hitting 2000 FPS? Plenty of headroom, surely - but actually, you’re already using 12% of the frame!

That’s why I’m moving away from the use of framerate as a debug metric. It just isn’t relevant during development. Instead, I’m measuring what percentage of the frame budget is being used, which I’ve set at 240 FPS for Daydream (since I have a good graphics card, and I feel like it should be possible).

Having a linear, easy-to-interpret percentage on-screen at all times makes performance usage visible and understandable in a way that - personally - I find framerate doesn’t offer. I derived the idea from a friend that used to work at Unity and Halfbrick; when they talk about performance, they always point to two things: set a budget early, and make expensive things visibly expensive.

A primer on the garbage collection problem

Imagine you have an object constructor like this:

local function make_person(
	name: string
)
	local self = {
		name = name,
		age = math.random(5, 16)
	}

	local function clean_up_person()
		print(`{self.name} (aged {self.age}) has been cleaned up`)
	end

	return self
end

We would like to run clean_up_person when the user of make_person has stopped using the returned table in all of their code.

do
	local person = make_person("Allison")
	task.wait(10)
	print("I am", person.name, "and I am", person.age)
end
-- call clean_up_person() here automatically....

The idiomatic way to do this would be via the __gc metamethod, but that’s disabled in Luau for good reason. Instead, we can use a weak table to detect when the object has been garbage collected from the outside, and run our cleanup code then:

local function run_on_gc(
	object: unknown, 
	callback: () -> ()
)
	local alive_test = setmetatable({object}, {__mode = "v"})
	task.spawn(function()
		repeat task.wait(1) until alive_test[1] ~= object
		callback()
	end)
end

local function make_person(
	name: string
)
	local self = {
		name = name,
		age = math.random(5, 16)
	}

	local function clean_up_person()
		print(`{self.name} (aged {self.age}) has been cleaned up`)
	end

	run_on_gc(self, clean_up_person)

	return self
end

The only problem is that this doesn’t work, because make_person still contains a strong reference to self, which we either must change to a weak reference or set to nil. However, both of those options means we can’t access self.name and self.age in clean_up_person, because the object would no longer be accessible to our code.

This has been a fundamental thorn in the side of Luau library developers for a very long time, requiring the use of manual memory management methods like :destroy() methods, or structured memory management frameworks like scopes or maids.

Enter: garbage collected reference counting.

The core idea of GCRC is simple; strongly store the object, and give out weak handles that reference back to the object. The handles can be destroyed freely without destroying any of the object’s information. When no handles remain, run the cleanup code manually, just like any other reference counting technique.

Here’s what that looks like for a simplified example with only one handle:

local function run_on_gc(
	object: unknown, 
	callback: () -> ()
)
	local alive_test = setmetatable({object}, {__mode = "v"})
	task.spawn(function()
		repeat task.wait(1) until alive_test[1] ~= object
		callback()
	end)
end

local function ref_counter<T>(
	callback: () -> ()
): {}
	local handle = table.freeze {}
	run_on_gc(handle, callback)
	return handle
end

local function make_person(
	name: string
)
	local self = {
		name = name,
		age = math.random(5, 16)
	}

	local function clean_up_person()
		print(`{self.name} (aged {self.age}) has been cleaned up`)
	end

	local handle = ref_counter(clean_up_person)
	handle.value = self
	
	return handle
end

The user of the code will have to access the data through the handle:

do
	local person = make_person("Allison")
	task.wait(10)
	print("I am", person.value.name, "and I am", person.value.age)
end
-- clean_up_person() runs here!

However, they should not hold onto the data strongly themselves, because the handle would be dropped immediately:

do
	local person = make_person("Allison").value
	task.wait(10) -- clean_up_person() could run here!
	print("I am", person.name, "and I am", person.age) -- kaboom
end

This can be prevented by changing make_person to provide indirect methods rather than direct data access:

local function make_person(
	name: string
)
	local self = {
		name = name,
		age = math.random(5, 16)
	}

	local function clean_up_person()
		print(`{self.name} (aged {self.age}) has been cleaned up`)
	end

	local handle = ref_counter(clean_up_person)
	function handle.name()
		return self.name
	end
	function handle.age()
		return self.age
	end
	
	return handle
end

Now, users are forced to intermediate all data access with the handle, so it can’t be accidentally dropped without also dropping access to the data:

do
	local person = make_person("Allison")
	task.wait(10)
	print("I am", person.name(), "and I am", person.age())
end

This “private data” concept maps well to local variables in closure-based OOP:

local function make_person(
	name: string
)
	local age = math.random(5, 16)

	local function clean_up_person()
		print(`{name} (aged {age}) has been cleaned up`)
	end

	local handle = ref_counter(clean_up_person)
	function handle.name()
		return name
	end
	function handle.age()
		return age
	end
	
	return handle
end

The technique can be generalised to support multiple handles being created at once, but that isn’t done in these examples for brevity.

While GCRC makes the resource management concerns invisible to the end user, they do necessitate a polling loop, which could be undesirable and may introduce extra latency between the time an object is dropped by user code, and the time the cleanup code actually runs, potentially making it unsuitable for external IO like file locks or network request closing.

I’ll be tackling this post from a technical perspective, but I’ve written a few years ago about even more problems with these frameworks from a low-code/designer perspective too.

UI is not a unique problem

We developed UI frameworks to allow us to define a simple source of truth, and derive a more complex set of consequences from it. Fundamentally, this is all that React, Fusion, and other frameworks try to do. Yet, it’s hardly the first time we’ve been confronted with this problem in the engine.

Consider the example of reusable model packages. Say I want to distribute a car on the Creator Store as a 3D model, built of multiple meshes. I want to give users an artist-friendly “paint colour” attribute so that they can configure the colour of the car’s meshes without reaching inside and making invasive modifications to each mesh.

Other benefits include (non-exhaustively):

• Efficient replication - a minimal amount of state can be driven from the server, and clients can infer the rest, reducing bandwidth usage
• Frictionless updates - the attributes can be configured externally without touching any internal instances, so there are no conflicting changes when upgrading to a new package version
• Fluent controls - a low-code user of this package can easily discover all of the axes it can be customised along, and can immediately tweak and see the effect of those controls, improving iteration times and encouraging asset variation

I’d feel pretty confident in saying that packages would benefit from being able to derive at least some state from a simpler set of inputs. The car would obviously still have other internal state, such as the velocity of the different assemblies, but as the user, we don’t really need to concern ourselves with that.

If this sounds familiar, it’s because I believe that packages with attributes are isomorphic to modern-day UI components - it’s almost exactly the same state derivation problem, and is amenable to equivalent solutions.

UI is not special - it’s merely the same problem at scale. Any well-done solution that solves one of these problems should solve the other, but no attention has been given to this at all.

Luau is fundamentally compromised

One of the most difficult problems in Fusion’s history has been the memory management problem; how can we make sure that resources allocated in state objects get destroyed properly when they’re recomputed, without destroying resources that are still in use? In particular, this is relevant for resources like instances which require explicit manual cleanup (e.g. unparenting or destruction).

Since the __gc metamethod is absent, the only thing Luau can do is respond to the absence of a resource handle (e.g. by polling a weak table), at which point it’s too late to perform logic involving that resource. You could log a generic message, perhaps, but the data stored inside that handle is gone, often making it impossible to take a new reference to whatever was on the other side (with the notable exception of instances, where the data model allows fresh handles to be obtained if you’re really persistent).

This always-early-or-late problem haunted Fusion for a very long time. No matter how hard I looked, I couldn’t find a way to square this circle. It’s a fundamental compromise of Luau, and I have left many notes on this blog over the years to warn you that you won’t solve it either.

In the end, I worked around it by introducing structured object lifetimes to Fusion, in the form of scopes. Famously, scopes are very easy to teach and nobody asks questions about them.

But all I can think of when I reflect on this problem, is just how entirely unnecessary it is. It’s an invented problem caused by a limited API surface. I have good reason to believe that we could offer a simpler and more understandable solution to this kind of problem if only the implementation lived lower down the tech stack, where the resources actually live.

This is just one of the downsides of building complex UI frameworks in user space, but it’s especially salient whenever people talk about post-React framework design. I don’t think it’s possible to combine granular reactivity with managed resources like instances without running headfirst into exactly this problem, and we will only be left with unsatisfying solutions.

Manual replication is un-Robloxian

We appropriated today’s attitude towards building Roblox UI directly from web conventions around single-page app (SPA) architecture. As the web world has learned over time though, there are tangible benefits to leveraging the server side more fully, especially when dealing with state that needs to be protected by server authority, and there is a lot of room to make the client/server divide more transparent and easier to work with.

UI shouldn’t require a special kind of script to be properly constructed. I think Roblox had something good going with server-side UI control, because it naturally allows the game state to live on the server, where there is one authoritative world view, making it easy to integrate with other game state and easy to control securely. It avoids the mental overhead of having to manage a client/server divide, and it’s my belief that we missed out on a better API design by heavily restricting the server side from being involved in user input and rendering.

The only reason we need client-side UI in 90% of use cases is to improve responsiveness in certain cases; for example low-latency responses to user input or smooth animation sequences. My view is that those are solvable design problems; even without adopting some advanced server authority system, you could at least replicate animations more smartly than how TweenService does things today, and you’d see meaningful payoff even if more complex actions would still have high latency.

Ultimately, I just think the engine needs to get smart about scheduling work to be done on clients at a fundamental level. Remotes and local scripts were a mistake in my eyes - sure, they’re a good power user tool for those who really need them, but we shouldn’t have ever shuttled novice developers through these mechanisms by force.

Would this require abandoning some purity around consistent worldviews? Would it mean that clients could get out of sync because they played animations at slightly different times, or processed inputs slightly differently from the server? Sure! But let’s not pretend like we ever had that - the speed of light already guarantees that clients have inconsistent worldviews due to networking. The sooner we stop trying to be precious about something we don’t have, and just implement proper rollback and server authority mechanisms, the better for the user.

Roblox UI should be following the example of the web and better learn how to leverage Roblox’s always-available server authority and cloud compute. We have a great replication and networking model with the potential to make server authority easy to reason about. Let’s lean into that rather than building yet another framework for SPAs with a complex manual networking layer.

It is time for a better way

It’s now my full belief that we should be spending less time ogling at Luau frameworks and more time investing into the fundamental design of the Roblox engine. That’s not to say that Luau frameworks have no place in a future Roblox, but they should not be a prerequisite tool to achieve reasonable UI goals.

We should take what we’ve learned about the way we want to build UI, and apply it by introducing key engine features to make the fundamental experience of building UI simpler. We are long overdue proper tools for componentisation and for managing the client/server boundary.

It’s also my belief that many of the modern paradigms we want to adopt, such as granular reactivity, would benefit from having lower-level implementations - not merely for some mythical language speedup (runtime complexity matters more), but because they would benefit from proper engine coordination in a way that would meaningfully reduce user-space mental overhead.

I’m thinking about following up this post with some thoughts about a mental model for UI that extends the skeuomorphic idea of bindings into a proper engine-level primitive, in a way that might even be compatible with actor-style parallelism. I’ll probably stew on it some more though, because I’m torn on a few design problems at the moment.

I believe that passthrough VR might not be so far off what we end up doing with AR glasses in the future, to address one of their fundamental weaknesses.

Let me explain.

Normal optics

I won’t spend too long on this part, because it’s well-trodden territory. Let’s start with a normal VR device, and look at how it gets light into your eyes.

In front of each of your eyes, you have a tiny display, which emits light to form an image, just like any other display you’ve ever used on a computer.

However, the display will be much too close to your eyes for you to get a clear image on your retina, because your eyes expect light to be coming in from much further away. So, a lens is used to shape the light rays so that the image is clear when it hits your retina.

That’s it… minus all the tiny details required to land you a nice job as a hardware designer.

Fundamentally, every headset on the market - including the Vision Pro - work this way. There’s no way for you to look “through” the headset at all; you are looking at a display.

So how come you can see the world outside of your Vision Pro?

How passthrough works

The answer is a technology called “passthrough”. The idea is simple; use cameras to show what the outside world looks like on the displays. If you do it right, then the light going to your eyes should look the same as the normal light that’d have come from the room.

This is how the Vision Pro works. It uses a bunch of camera feeds to figure out what light would normally be reaching your eyes. Then, it uses some reverse logic to figure out what part of the display should emit that light. When the display shines that light, it hits your eye from the correct angle and makes it look like you’re staring at your room, rather than a headset.

This approach has many upsides - mainly, you get full control over what light goes to your eyes. Because the light is blocked by the headset, it’s up to the display to recreate all the light that should be coming in. But of course, that also means you can modify the light, painting on your own objects and scenes to make them look real.

The Vision Pro decides to paint on a bunch of glassy apps and 3D objects to make it look like you have multiple large displays floating in your room.

Still with me so far?

How AR glasses work

Let’s now talk about how Meta’s AR glasses work. Meta would like you to think that nothing else like them exists, but there’s a few companies making AR glasses today.

Here’s my favourite pair, the XReal Air 2 Pro glasses.

These are actually based on a similar concept to VR headsets. The goal is to shine light from a display into your eyes, with the light coming in at just the right angle to look like it’s coming from inside of the room.

If you look at what’s behind the front glass, you’ll see a pair of transparent glass prisms. These perform a similar task to the VR headset lenses from before; they shape the light coming into your eyes from a display.

A display? What display?

With enough effort, you can find it. Look up under the glasses while you’re sending an image to them, and you’ll see a pair of tiny OLED displays embedded in the top of the frame, seen through the prism.

These displays shine downwards, reflecting off of the slanted glass prism and shooting off towards your eyes, with the same carefully-crafted shape you’d get from a VR headset.

If you get up close with a camera, you can see just how small these displays really are - yet they’re 1080p!

This approach has many downsides, which I won’t elaborate on for now. However, it has one very large upside - there is nothing in front of you except for a transparent glass prism, so you can let in real light. You can see the room you’re in as well as the light coming from the displays!

The occlusion problem

This setup works marvellously, except for one slight issue.

Because we are letting light in from the room, that light is mixing with the light from the displays. More specifically, the display can only add light on top - there’s no way to block any of the light that’s coming in from the room. This means that everything you show on the display will look “ghostly”.

As an example, here’s what I see when I open up XReal’s app.

Look behind the large tiles, and you’ll notice that you can see my lights and wall decorations through them. Everything is slightly semi-transparent, because the light from the display is mixing with the light from the room, a bit like when you look through a window and see your reflection mix with the room on the other side of the window.

This is what I call the “occlusion problem” - you can’t block out any of the light from the room. In particular, this means that optical AR tends to struggle outdoors in sunlight or in bright rooms.

Currently, XReal’s solution to this is to use electrochromic dimming on the front glass, blocking the light to create a dark backdrop so you can isolate the digital content. This kills the utility of these things as AR glasses, but it means you can see your content clearly. It’s basically a VR mode.

Would it be possible to dim only parts of the glass, perhaps like an LCD display? Yes, but there’s another problem at play here - the glass is so close to you that it will be out of focus.

Your eyes will be focused on the contents of the room (and the light coming from the display as if it was in the room), so the glasses themselves will appear blurry. That includes the front glass that’ll be doing the dimming.

This means, at best, you’ll only be able to do very approximate local dimming. This might be enough for improving legibility, but it won’t give you any crisp edges on objects. Everything would either look like it has an aura of darkness, or it’d have a soft, ghostly faded edge.

Solving this problem likely requires somehow blocking or interfering with the incoming light in some really precise, tightly-calibrated way. Nobody really knows how to do this yet.

And yet, If we don’t solve this problem, then the quality of optical AR will never match the experience of a headset using passthrough.

So, what to do?

Solving the occlusion problem

High-end AR glasses, like the more modern XReal Air 2 Ultra glasses, now come with integrated cameras for position and hand tracking. It’s not hard to imagine that these cameras could one day capture the appearance of your surroundings; after all, we have great cameras in our phones right now, and we’re surprisingly good at miniaturising technology.

So, my proposal is this: when we have the technology to provide a large-FOV pair of glasses with a dimmable front surface, and when we have the technology to capture what the environment looks like, what’s stopping us from doing passthrough like VR headsets do?

To me, blending “natural light” and “display light” is a pragmatic way to solve the problem. While you would need well-calibrated and decent-looking cameras, and you’d need to make sure that things align decently well between the two sources of light, this could end up being an interesting solution to the occlusion problem.

AR passthrough is strong because software is strong. With passthrough VR, we have experience stitching together images on-the-fly. The Vision Pro already shows good competency with image processing here.

Passthrough also avoids the trap of special hardware. While it may be possible to solve this problem with novel methods, those solutions will tend to be expensive and bespoke. Meanwhile, camera systems and dimmable glass are both transferable across product categories, and likely will be included with the product anyway. You don’t need specialised support beyond perhaps having a better array of cameras.

The best part is that the problems with approximate local dimming can be compensated for. Instead of having a shadowy aura around virtual objects, input from the cameras can be used to compensate for the decrease in brightness around objects. The end result is a crisp, sharp edge.

Admittedly, I’m biased as a computer graphics guy, but having thought about this for a long time, I don’t see any simpler solution without inventing wild new hardware.

Conclusion

So, was Apple wrong to invest so much into the Vision Pro? Surprisingly, I think it was a good idea.

By going with a passthrough system from the beginning, Apple can now invest in their passthrough processing pipeline and refine the quality of the output. Simultaneously, they can move towards smaller and smaller camera hardware. And when they do release glasses, they will have a head start on solving the occlusion problem.

There are some caveats to this idea, of course. For one, it would require that you can display a high-FOV image, which seems to be within the realm of possibility at this point given the recent advances by Meta.

It’d also require good quality camera feeds, which is both a hardware and image processing challenge. It’s possible that dedicated image processing silicon could make the leap to AR glasses, but it’s not a certainty given the space restraints, even if/when a computing puck is introduced.

And of course, there’s always the possibility that something genuinely new comes along and makes this all possible. Optics isn’t a solved field, after all.

But overall? I would not be surprised if passthrough AR was the direction things went, even for optical AR glasses. At least, that’s my take on it as a nerd.

I’m eager to hear other ideas.