Quarto website preview server hacks
Do you want to build an enormous quarto website? Strap in for a bumpy ride. Here, I have a seatbelt for you.
October 7, 2024 — October 30, 2024
quarto preview: Serving web.
Don’t use quarto preview for large sites. Use caddy instead.
I constantly recommend the excellent quarto system for all manner of research tasks, including writing papers, building websites and making presentations. My recommendation is not because of the preview server, which is the worst bit of the messiest part, at least for my daily use case.
The blog you are reading right now, which is itself a quarto website, is probably much bigger than anything that the developers of quarto experience in general and as such it exposes many deficiencies in the preview server that teensy little websites might not.
I spent a long time fighting with the preview server (i.e. the one that spins up when we invoke quarto preview) for quarto websites and eventually the list of information and hacks grew to deserve a page of its own.
Fear not! There are hacks to make it work better. Let me share the ones I know.
This page is dismissive of the quarto preview server, so you might think I am not a massive fan of quarto.
I am a massive fan of quarto insofar as I am a fan of anything, which is to say, grudgingly. It is an incredible bit of infrastructure that makes my life better. It is with deep love, and respect borne of my excellent experience, that I say: Quarto is so good that I will keep on using it despite how much the preview server sucks.
Further, maybe the quarto preview server will not suck for you. My use case is unusual: maintaining a blog with over 1000 posts is not a normal thing to do. As such it likely would not make sense for the developers of quarto to spend time on my use case; they will help more people by fixing other things.
In a world where I had leisure to contribute to open source, I would probably try to fix or replace the preview server, but that is not what I am being paid for and not what I wish to do in my scanty free time, i.e. it would not make sense for me to fix the quarto server either.
So! For now, here are easy, pragmatic workarounds that are pretty good and much easier than touching that giant and baffling web-app.
1 quarto preview is too fancy
First caveat: The quarto preview server invoked by quarto preview, is, per default, slightly too clever for my taste.
Excessive cleverness # 1: It tries to do something fancy with process management. I am not sure what the nature of the fanciness is, but the upshot is that the server is a mediocre citizen of the command-line environment. If I run it in the background it magically daemonises or something, which makes it hard to kill. If I run it in the foreground, it is reluctant to die when I press ctrl-c. This is especially annoying because sometimes the build process will hang and cannot be quit from the CLI. One reason this seems to happen is if a template pops the EJS stack, because I am building a custom listing or something. The server process is a deno executable, so the following will salvage the situation:
However, if I am running other deno processes on my computer, this will kill those too. I do not otherwise use deno, so I leave off my problem-solving there.
OTOH, if I run the preview server at the same time as a render process, it will die spontaneously sometimes.
Excessive cleverness 2: I am discombobulated when the quarto server tries to persuade my browser to switch to the “latest” updated page, since I am usually editing a few pages at once, and do not enjoy having my 7 open tabs suddenly decide to show me the same thing, instead of the 7 different things I wished to see. Infuriatingly, the back button does not work to undo this. Avoid this behaviour with
Excessive cleverness 3: Quarto chooses a new random port for the server each time, which is cute, but makes those 7 preview tabs impossible to bookmark and terrible for my browser workflow. I guess it is trying to optimise the possibility that if I run lots of servers, they will work? But I don’t have the many, many gigabytes of RAM and CPUs that would be required to run multiple copies of this app, so that is no use to me. I fix a predictable port thusly:
Putting these together, my invocation for a preview is
We should be equivalently able to encode that in a project setting via _quarto.yml:
However, that does not work for me; I find I need to set the CLI flags instead.
2 quarto preview server is broken don’t use **{#quarto-preview-broken}
tl;dr Efficient, reliable, convenient: choose none.
quarto previewuses colossal amounts of RAM for my site; I guess it is serving the site from memory?
Does 2.1GB seem like a lot of memory to serve small static files to a single user? Despite that, the quarto server itself is not that fast at serving files. It seems to block a lot and take ages to serve me a page, even if it has already rendered the page. A performance profile in the browser shows my HTML is being served at approximately 400 bytes/second, which is faster than typing, but not by much.
36.7s to serve 15KB quarto previewalso burns a surprising amount of compute. It tanks my battery if I try to edit my blog on the road, much more rapidly than, for example, running a full-featured real-time audio workstation with live effects processing.quarto previewsometimes serves stale versions of the content I am working on. I might see an old version of some page, even if I have seen it tantalisingly serve the updated version momentarily, before it reverts to some older version. This annoyance is worse for me than it might sound, because I waste attention each time it happens trying to work out if the problem is quarto or me. For something I do hundreds of times per day like edit my blog, it adds up to lots of time spent swearing and debugging, rather than writing.If the preview server is not spending all that RAM on keeping the content current, and all that compute time on wondering whether the content is current, what is it actually doing?
The precise degree of slowness is particularly corrosive to my personal attention span. 30 seconds is long enough to drag productivity to a halt, but not long enough to let me go away and do something else.
A full re-render via
quarto renderwhile also running the preview server behaves unpredictably. Sometimes it crashes the preview server, or leaves detritus lying around. So the server is not set-and-forget, but rather a thing I need to babysit, and make sure it does not clash with a full re-render.Sometimes, and I do not know why, the server decides a full re-render of the site is necessitated, although I have done nothing special, and then previewing that next one-line change needs a
1217 minute wait.
2.1 Workaround 1: continually restart
My fix for (some of) the weird bugs and misfeatures in quarto preview is to continually turn the preview server off and on again, which seems to keep memory use under control and keeps the pages more current. This is best done manually, by running it from the shell and doing a ^C to kill it.
I attempted to define a helper function which kills the quarto process and restarts it automatically, but the wily quarto process seems to evade my attempts to kill it by spawning detached subprocesses or something, so it doesn’t work. This is how far I got.
However, that does not work. quarto preview seems to detach if I run it in the background, and I do not know how to find the correct PID to kill it. Suggestions welcome.
Seeing this, I became enlightened. The best way to stop the quarto preview server is to never start it in the first place. Read on.
2.2 Workaround 2: use a different web server
You know what? I don’t even know why I spent time trying to get quarto’s preview server to work. HTTP servers are solved. We don’t need to wring bonus performance out of quarto’s cockamamie home-grown solution, because we do not need to use it. I use caddy because I am familiar with it and it is small and reliable. As with probably every web server on that list, it is much faster and more reliable than quarto preview.
While that guy is running in the background, I manually render the files I need from the CLI.
The VS Code quarto extension has a keyboard shortcut to do that too, which works ok, but using that brings up a modal dialog box in which I need to confirm that I wish to render HTML every time, so the CLI is quicker.
I then manually reload my browser to see the changes.
This is not quite smooth, but it is smoother than the alternative. That friction is a small price to pay for actually seeing the changes, not some arbitrarily stale content, and moreover, seeing them in under 17 minutes, which is how long it took the quarto preview server to serve me a single page after a full re-render last time I tried.
After rendering individual pages using quarto render my_page.qd, it seems to be necessary to do a full site re-render to reliably publish the site, otherwise most of it is missing.
There exists a file-watching, fancy-pants server for VS code, Live Server. I do not use that because it does not like serving files that I have hidden from the file explorer list, and I like hiding the HTML site output from the file explorer list to keep things tidy. YMMV.
Do not wish to install caddy? No problem, if you have quarto, you already have deno, so the built-in deno server probably works:
The deno path in fish shell is (dirname (realpath (which quarto)))/tools/aarch64/deno for ARM architectures and (dirname (realpath (which quarto)))/tools/x86_64//deno for x86_64 architectures.