We have a pretty neat system for Benchmarking Rust.

• Every commit triggers an automated build
• The build is uploaded to Steam
• Our benchmark PCs constantly update from Steam
• ..and run a bunch of benchmark tests

This means that for most changesets in our source control we have a benchmark that ran. The big idea being that we can track performance between changesets automatically.

Results View

I made a front end for displaying the results in Blazor. The TLDR on Blazor is VueJs in C#. The site is at stats.facepunch.com.

You can select the test in the panel on the bottom left, the graph shows the results per changeset. If you click on a column you can see the changeset comment along with the previous ones.

This lets us see historically where performance was affected. For example, using this tool diogo noticed a frame drop from this commit, and fixed it in this commit.

Data Interpretation

It isn’t as straight forward as see a spike and reverse that commit. There’s other factors.

For example, some hardware might have fucked up on the test machine and we had to replace it. There might have been some other Windows bullshit taking resources, or somehow two versions of Rust were running. We might have fucked up a commit where nothing was renderered – so performance would be super high but obviously irrelevant.

So what I’m saying is it needs to be taken with a pinch of salt. Adding events/comments to it would probably be a good idea.

Improvements

We should probably add more test machines, and they should be near identical. This would mean more runs per changeset.

Our Low machine is so slow at loading that they can miss running changesets. In an ideal world we’d queue up each changeset for 10 runs or something, to make sure everything gets a turn.

I haven’t put that much work into the running procedure here. It’s just a bat file in a loop that downloads and runs as much as it can. It would be ideal if steamcmd let us download a specific version based on the changeset number, right now we only grab the latest version.

Our Medium machine keeps freezing up, so there’s really limit data there.

Blazor

I love Blazor. It’s so much nicer to be able to use Linq and all the .net string manipulation functions instead of trying to re-remember javascript for it all.

For the graph I started looking at the existing solutions. I tried syncfusion’s stuff but it was way too slow. It seems that almost every chart solution interops to an existing javascript charting library. It seemed like a waste of time using Blazor and Webassembly if we were going to do that.

Making charts is pretty easy with the svg elements so I made a custom thing. This worked out especially good because I could be really specific in what I wanted to happen.

But it was still kind of slow. When there’s a callback Blazor’s default behaviour is to re-render the whole component. So doing MouseOver on the bars on the chart was recreating the chart every mouseover. I’m guessing this isn’t usually a problem but with 300 changesets it performed terribly.

Luckily I found that Blazor components have a ShouldRender method you can override. So I made a new component for everything I didn’t want to redraw on events and made ShouldRender only return true if it was really dirty.