Neos content cache

Neos in in version 1.2 comes with a powerful content caching systems that might be a little tricky to understand at first, but used correctly it solved a lot of caching related issues often seen with content management systems. 
This post will try to shed some light on how the caching systems can be used for automatically managing content cache for a typical site that uses lists of nodes. ie a news-based site where an update to a news-node requires cache-purging on list pages as well.

Some background information

The background for the systems is very common use case. We have a specific nodetype  that we want to create a list of. This could be a news-nodetype that we want to create lists of latest news, og show news-items tagged with a certain tag. In this case, the nodetype is a "Service" that we wish to create lists of, but the specific type is of no importance, only the fact that when the specific nodetype changes, we need to update the generated from the nodetypes. 

Below is an example of a Page nodetype called Acme.Site:Service which contains a headline property that is used for rendering lists of services.

'Acme.Site:Service':
  superTypes: ['TYPO3.Neos.NodeTypes:Page']
  ui:
    label: 'Service'
    icon: 'icon-th-large'
    group: 'acme'
    inspector:
      groups:
        teaser:
          label: 'Teaser'
          position: 2
        options:
          label: 'Options'
          position: 1
  properties:
    headline:
      type: string
      defaultValue: ''
      ui:
        label: 'Headline'
        inspector:
          group: 'teaser'
          editorOptions:
            placeholder: 'Enter a headline'

To render the lists of services, here called a "service overview", we use a content nodetype called service view, here is the TypoScript used for rendering this. Nothing fancy going on here, we simply define a serviceTeasers block that is a Collection of alle Acme.Site:Service nodes in the site. Then we render each of these using a TYPO3.TypoScript:Template. There i a specific reason for why we do the looping over found service nodes in TypoScript instead of in the fluid template. 

prototype(Acme.Site:ServiceOverview) >
prototype(Acme.Site:ServiceOverview) < prototype(TYPO3.TypoScript:Array) {
        serviceTeasers = TYPO3.TypoScript:Collection {
                collection = ${q(site).find('[instanceof Acme.Site:Service]').get()}
                itemName = 'serviceNode'
                itemRenderer = TYPO3.TypoScript:Template {
                        templatePath = 'resource:/Acme.Site/Private/Templates/NodeTypes/ServiceTeaser.html'
                        service = ${serviceNode}
                }
                @process.wrap = ${'<div class="services">' + value + '</div>'}
        }
}

Using this setup, we can now render a list of all services, but the problem is, that the Neos content cache will cache the HTML generated on each page. The caching framework is smart enough to know that when the Service node itself changes, the content cache for this particular servicepage is cleared, so the change becomes visible. But it does not know, that the Service overview page which lists the services should also be cleared since the service might have change information that is used in rendering the service. So the effect is, that if the service changes title, the title is not updated on the service overview!

What to do?

The easy solution

One simple solution, is to tell Neos, that the service overview page (or at least the section that show the services) should not be cached. This might be perfectly valid solutions for many. Lets see how this is done. 

prototype(Acme.Site:ServiceOverview) >
prototype(Acme.Site:ServiceOverview) < prototype(TYPO3.TypoScript:Array) {
  serviceTeasers = TYPO3.TypoScript:Collection {
  collection = ${q(site).find('[instanceof CSIS.Site:Service]').get()}
    itemName = 'serviceNode'
    itemRenderer = TYPO3.TypoScript:Template {
      templatePath = 'resource://Acme.Site/Private/Templates/NodeTypes/ServiceTeaser.html'
      service = ${serviceNode}
      @cache {
        mode = 'uncached'
        context {
          1 = 'site'
          2 = 'node'
          3 = 'documentNode'
        }
      }
    }
    @process.wrap = ${'<div class="services">' + value + '</div>'}
  }
}

The only thing that is changed is the new @cache section. We have now set the mode'uncached' for the rendering of this particular content element (the service overview). The default value of this is "embed" which means that the content is embedded in its parent cache. This means that if we do not specify anything, the serviceoverview is embedded in the cache of the content collection it is inserted into, which is again cached as part of the Document node on which it is rendered.

Setting mode=''uncached" means, that when Neos generate the page where we show the section overview, it will make sure that the part where the list of services is generate is re-calculated every time, which solved the problem of services changing title and not being updated on the overview page.

However, the rest of the page is still cached, so menu-generation, other elements on the page are still perfectly cached.

The next time the page is rendered, only the service overview is re-rendered, the rest is fetched from cache, but the generation of the serviceOverview requires some knowledge of the context, specifically which site, node and documentNode we are rendered on. Hence we tell Neos to store the state of these three elements along with the parent cache, so we can use it when generating the service overview. 

For those familiar with TYPO3 CMS, this methods of doing it corresponds to having a plugin that is a USER_INT. 

The slightly more advanced way

If the service overview is a page that is shown a lot, we might want to make this page cached for performance reasons(performance is always good right?). Here the power of the caching framework comes into play. 

What we can do, is to tell Neos to cache each entry in the service list individually. This is easyily done by changing the @cache.mode from 'uncached' to 'cached'' (and removing the context part).
Remember the default was 'embed' which make the content part of the parent cache-entry. 'cache' means make a separate cacheentry for this item.

A little more configuration is required to make Neos clear these cache entries when the actual service node is changed. The resulting configuration is 

prototype(Acme.Site:ServiceOverview) >
prototype(Acme.Site:ServiceOverview) < prototype(TYPO3.TypoScript:Array) {
  serviceTeasers = TYPO3.TypoScript:Collection {
  collection = ${q(site).find('[instanceof CSIS.Site:Service]').get()}
    itemName = 'serviceNode'
    itemRenderer = TYPO3.TypoScript:Template {
      templatePath = 'resource://Acme.Site/Private/Templates/NodeTypes/ServiceTeaser.html'
      service = ${serviceNode}
      @cache {
        mode = 'cached'
        maximumLifetime = 86400
        entryTags {
          3 = ${'Node_' + this.service.identifier}
        }
      }
    }
    @process.wrap = ${'<div class="services">' + value + '</div>'}
  }
}

MaximumLige tells Neos how long to cache this particular cache-entry. Here it is set to 1 day (86400 seconds).

The interesting part is the entryTags. Here we have added a new entry tag that is constructed of the string 'Node_' and then the identifer (uuid) of the actual service being rendered. 

Neos caching framework works with tags. This means that when adding a cache-entry to the framework, a list of tags is added as well. This allows Neos (or us as developers) to purge cache-entries based on their tags.

Now, Neos already uses this. When a Document node is rendered, it is added to the content-cache, and is added with a tag 'Node_<IDENTIFIER>'. So whenever the node is cleared, it purges all cache entries having this tag. 

This can be seen by opening inspecting the TypoScript for TYPO3.Neos:Page) which included the typoscript

        @cache {
                mode = 'cached'
                entryIdentifier {
                        documentNode = ${node}
                }
                entryTags {
                        1 = ${'Node_' + node.identifier}
                }
        }


With the above configuration, we also add this tag to the cache-entries of the service-overview list items, which gives us the desired behaviour: When a service node changes, we also purge relevant (and only relevant) cache-entries used in list view.

Update for Neos 2.0

In Neos 2.0, there is a convenient Eel cache helper for generating these tags. So instead of writin ${'Node_' + node.identifier} we can use the syntax ${Neos.Caching.nodeTag(node)}. There is also a descendantOf view helper.

More information os read the docs

Under the hood

Lets try to take a look under the hood, to see the actual cache identifier made by Neos. I have found, that the easiest way to do this, is to change the caching framework to use database as caching backend. This makes it much easier to see what is actually being rendered in the content cache. But for this example, we simply used the default file-based cache. The cache identifier are placed under the Data/Temporary/Development/Cache/Cache/TYPO3_Neos_Content folder. 

To see what is going on, start by deleting all entries, and change the setting to the default, where the serviceoverview is cached using the default "embed" type.

Then hit the service overview page in the frontend to see whats is being generated. On my installation, this is what is rendered.

@todo: See content of cache entry

Integrating with Varnish cache

One reason for why it is much better to use the type='cached' methods than to use the 'uncached' fix described above, is when using Varnish to cache you site. Setting Varnish in front of your Neos websites, make view cached pages extremely effecient.