In theory, the disk cache uses spare memory that applications are not using, so it is not guaranteed to be present if I am hosting it on a server with little-to-no free memory. However, if an application contains the cache then it will not be eliminated in a memory-starved environment.
In practice, this will probably rarely happen on my server.
On a memory-constrained system, forcing files into memory will increase memory pressure. You have roughly the same amount of cacheable data either way, but many VPS providers don't configure a swap partition by default, so the kernel can't page out your idle memory to make room for something else and you can run into OOM problems.
If you're not hitting the bounds of your machine's memory, the kernel disk cache will do a pretty good job. If you are, then you're more likely to cause problems by forcing more active data into memory.
For what it's worth, the kernel is free to evict your unused memory pages to disk (on a system with swap), which is almost an identical access model to disk caching:
Virtual memory: We need this previously-idle section of memory, so let's restore it from disk until it becomes idle again.
Disk caching: We need this previously-idle section of disk, so let's cache it in memory until it becomes idle again.
So if I am understanding this right, to really get any appreciable cache improvements would I need to pre-execute the templates once and store that generated output in a temporary file to leverage the disk cache and repeatedly fetch (cache hit) that file upon request?
The basic statement I was trying to make was "due to disk caching, putting your cache in memory or disk is roughly equivalent, except it's easier for the kernel to make decisions about an on-disk cache."
As far as re-executing templates, let's break down some hypothetical costs, including pretty optimistic network latency.
1. Accept, read, and parse a request from a socket: >50ms
2. Read a template from (solid-state) disk: 16µs
3. Parse and execute a template: 100µs
4. Serve the response over TCP: >20ms
If we replace (2) with a memory cache (whether it's your app or the kernel), the cost goes down to maybe 1-5µs.
Your time-to-response is effectively the same with template caching. So what's the point of a generated static site? Reducing CPU and memory pressure. Your page load times are almost completely bottlenecked on network, so you can optimize this using a CDN/external caching service (like Cloudflare) to bring the content closer to the user, or you can optimize by removing CPU cycles per page load to increase the number of concurrent connections you can handle.
There's a point where the userspace to kernel latency in serving a file is also a bottleneck, but nginx can use the sendfile() [1] system call to copy a file directly to a socket (from the kernel disk cache) without ever touching userspace, which is much faster than anything you're going to do with in-app caching, unless you also put the TCP stack in-app.
To clarify: do you watch filesystem for template changes, then store the parsed template in memory, or do you execute the template every time?
I'd be curious to see benchmarks against nginx with the equivalent html files already generated.
> The droplet hosting the former site only accepted its key, so losing the hard drive meant I lost access to maintain the site.
Droplet as in DigitalOcean? As long as you still have access to your DigitalOcean account, you can "Reset Root Password" and use "Console Access" to re-establish ssh.
> To clarify: do you watch filesystem for template changes, then store the parsed template in memory, or do you execute the template every time?
I watch the filesystem for template changes and re-read the template into memory. However, upon serving a request the template is re-executed. The rationale is to, in the future, expose the data portion of Go's template.ExecuteTemplate so a third-party content manager can pass in information such as date of publication, published by whom, tags, etc.
> As long as you still have access to your DigitalOcean account...
I had no idea that was possible, but now I am kicking myself. Fortunately, my previous site had very little. Thanks for pointing that out!
8 comments
[ 4.3 ms ] story [ 29.2 ms ] threadIn theory, the disk cache uses spare memory that applications are not using, so it is not guaranteed to be present if I am hosting it on a server with little-to-no free memory. However, if an application contains the cache then it will not be eliminated in a memory-starved environment.
In practice, this will probably rarely happen on my server.
Edited to add the "in practice" bit.
If you're not hitting the bounds of your machine's memory, the kernel disk cache will do a pretty good job. If you are, then you're more likely to cause problems by forcing more active data into memory.
For what it's worth, the kernel is free to evict your unused memory pages to disk (on a system with swap), which is almost an identical access model to disk caching:
Virtual memory: We need this previously-idle section of memory, so let's restore it from disk until it becomes idle again.
Disk caching: We need this previously-idle section of disk, so let's cache it in memory until it becomes idle again.
As far as re-executing templates, let's break down some hypothetical costs, including pretty optimistic network latency.
1. Accept, read, and parse a request from a socket: >50ms
2. Read a template from (solid-state) disk: 16µs
3. Parse and execute a template: 100µs
4. Serve the response over TCP: >20ms
If we replace (2) with a memory cache (whether it's your app or the kernel), the cost goes down to maybe 1-5µs.
Your time-to-response is effectively the same with template caching. So what's the point of a generated static site? Reducing CPU and memory pressure. Your page load times are almost completely bottlenecked on network, so you can optimize this using a CDN/external caching service (like Cloudflare) to bring the content closer to the user, or you can optimize by removing CPU cycles per page load to increase the number of concurrent connections you can handle.
There's a point where the userspace to kernel latency in serving a file is also a bottleneck, but nginx can use the sendfile() [1] system call to copy a file directly to a socket (from the kernel disk cache) without ever touching userspace, which is much faster than anything you're going to do with in-app caching, unless you also put the TCP stack in-app.
[1] http://linux.die.net/man/2/sendfile
I'd be curious to see benchmarks against nginx with the equivalent html files already generated.
> The droplet hosting the former site only accepted its key, so losing the hard drive meant I lost access to maintain the site.
Droplet as in DigitalOcean? As long as you still have access to your DigitalOcean account, you can "Reset Root Password" and use "Console Access" to re-establish ssh.
I watch the filesystem for template changes and re-read the template into memory. However, upon serving a request the template is re-executed. The rationale is to, in the future, expose the data portion of Go's template.ExecuteTemplate so a third-party content manager can pass in information such as date of publication, published by whom, tags, etc.
> As long as you still have access to your DigitalOcean account...
I had no idea that was possible, but now I am kicking myself. Fortunately, my previous site had very little. Thanks for pointing that out!