Ticket #12198 (closed PLIP: fixed)

Opened 5 years ago

Last modified 2 years ago

Ship Chamelon (2.x)/five.pt with Plone by default

Reported by: runyaga Owned by: runyaga
Priority: major Milestone: 5.0
Component: Templates/CSS Version:
Keywords: chamelon, zpt, template, optimization Cc: malthe

Description

Motivation

The current template engine, zope.pagetemplate engine (ZPTE), is inefficient. Plone 4.1 can gain a 20%+ performance increase by substituting the ZPTE with Chameleon pagetemplate engine (CPTE). By viewing profiling reports one can see a immediate decrease in function calls by 20-35k. The reduction in noise-to-signal alone is a significant improvement. As of Chameleon 2.0 and five.pt > 2.1 TTW customizations of page templates will continue to work. As of Chameleon 2.0 the parser is less strict and will work with non-XML. Chameleon has been used successfully in numerous production installtions.

NOTE: Chameleon 1.0 required strict XML templates. Compliance the Plone community and its 3rd parties would find very difficult to keep compliance.

Assumptions

Chameleon and most importantly the compatibility layer CMF/Plone would use, five.pt, should have some production usage by the time this PLIP is candidate for merging. Pyramid, a genetic cousin of Zope, ships with Chameleon as its OOTB template engine. The error reporting by Chameleon/fivt.pt is similar/equal in value to the existing ZPTE error reporting.

Proposal & Implementation

  • Possibly remove zope.pagetemplate, tal, metal, etc from buildout. This would reduce the amount of code shipping with Plone while taking on new code from Chameleon.
  • Refactor some of the more exotic usage of ZPTE in Plone.
  • Figure out how to run Plone with CPTE and ensure tests still pass (doctests may break).
  • Provide a documented way to “back out” of CPTE in case customer wants to revert to ZPTE; which should not be officially supported but comforting escape button.

Deliverables

  • Deliver stable packages of five.pt and its dependents
  • Run Plone test suite with five.pt enabled to ensure no breakage of integration test suite
  • Write migration document which sets expectations of 3rd party add-on developers that they may bump into edge cases which require new releases of their packages.

Risks

Chameleon has few “core developers”. The code is sophisticated (mitigated by test suite, malthe to provide info) but so is zope.pagetemplate (TAL/METAL/etc) and its dependencies. ZPTE specification has stalled and is under-specified. Some exotic statements may work in ZPTE but may not work in CPTE. 99.99% of the time this is due to exploitation of TAL/METAL in pagetemplates. PTs should be simple with the BrowserView providing all logic and pre-computation. Currently (as of Plone 4.1) there are still “Controller PageTemplates” and “Zope PageTemplates’ which do not have bound BrowserViews. Creating BrowserViews for all of these templates and refactoring them is out of scope. HINT: Removal of such legacy should be parallel PLIP.

Chameleon’s website is not well developed. Maybe even copying the TAL/METAL specification to the website would be useful. Since Chameleon uses that specification. There is not clear (on one page) explanation of z3c.pt, five.pt, chameleon intersection or what each package does.

Should CMF incorporate Chameleon first? This is more appropriate since Plone sites on CMF. This could warrant a CMF release?

Tracebacks will look differently than what people are used to. While the tb will be more concise it will refer to transient python modules (compiled templates). This is a risk in the community supporting wider Plone. Meaning the main supporters will need to have experience with chameleon before the community, as a whole, come on board w/ the technology.

Participants

  • Enfold Systems

Change History

comment:1 Changed 5 years ago by runyaga

Today, once can get this behavior by simply adding to their buildout (or setup.py)

via setuptools (the preferred way):

$package/setup.py install_requires=[five.pt]

via buildout.cfg

eggs = five.pt

comment:3 Changed 5 years ago by rossp

  • Milestone changed from 4.3 to Future
  • Summary changed from Ship Chamelon (2.x)/five.pt with Plone 4.3 by default to Ship Chamelon (2.x)/five.pt with Plone by default

Most of the FWT felt this is inappropriate for a minor release and should be postponed and considered for Plone 5.0. The possibility of breaking add-ons may be too much for a minor release.

We'd also like to see some more current performance tests to validate the claim of ~20% improvement. Can you document your method and then run some new performance tests of a few different pages on two different Plone sites: one OOTB without any additional content, and one content heavy Plone site with lots of content objects?

comment:4 Changed 4 years ago by kleist

  • Status changed from new to confirmed
  • Keywords chamelonzpttemplateoptimization added; chamelon zpt template optimization removed
  • Version set to 4.1

Confirming, just to clean up the report "Untriaged Tickets".

comment:5 Changed 4 years ago by eleddy

For future reference, this bug will affect any future merge and should be fixed before including by default:  https://github.com/malthe/chameleon/issues/88

comment:6 Changed 3 years ago by thet

chameleons tracebacks are indeed not always very helpful.

you often have to throw a pdb into the compiled files.

this, on the other hand, is also an advantage, because you have python access to the runtime data of the templates.

comment:7 Changed 3 years ago by rossp

I'm still -0 because I've not really seen performance numbers of the improvements for actual Plone pages under different real-world usage scenarios. If I saw a significant improvement on Plone overall across deployments sizes and scenarios I'd be a strong +1. If I saw significant degredation in one or more of our target deployment sizes or scenarios I'd be a regretful -1. Because I don't know I'm a -0, without being really clear what the benefits are, it's hard to feel right about deviating from the reference implementation.

comment:8 Changed 3 years ago by davisagli

  • Priority changed from minor to major
  • Version 4.1 deleted
  • Milestone changed from Future to 5.0

The framework team has approved this for Plone 5.0.

comment:9 Changed 3 years ago by kleist

  • Keywords chamelon, zpt, template, optimization added; chamelonzpttemplateoptimization removed

[just restored keywords garbled by Trac]

comment:10 Changed 3 years ago by MatthewWilkes

I'm going to say I'm +0 on this, I really like chameleon but I'd a plan in place to do a full security audit of how this interacts with AccessControl, as well as what extra attack surface is exposed by having a compiled python template.

comment:11 Changed 3 years ago by rochael

A security audit is a good idea in any case, but to add a note about how security works in CPTE/five.pt, for RestrictedPython, the same expression compiler as used in the native ZPTE is used so, in theory, the security profile of a site shouldn't change with the installation of CPTE.

comment:12 Changed 3 years ago by esteele

The Framework Team has decided to move on to Plone 5. Updating milestones accordingly.

comment:13 Changed 2 years ago by davisagli

  • Status changed from confirmed to closed
  • Resolution set to fixed

This has been merged for Plone 5.

Note: See TracTickets for help on using tickets.