Ticket #13561 (confirmed PLIP)

Opened 3 years ago

Last modified 3 years ago

Collections: folderish plus migration from Topic to Collection

Reported by: maurits Owned by:
Priority: minor Milestone: 5.0
Component: General Version: 4.3
Keywords: collection Cc:

Description (last modified by maurits) (diff)

Proposer: Maurits van Rees

Seconder: TODO

Motivation

In Plone 4.2 the new-style plone.app.collections were introduced. To improve their value as replacements for the old-style Topics, they should be folderish and there should be migration from old to new style.

Assumptions

A best-effort migration is good enough. Not all criteria from old-style Topics can be migrated exactly.

Nesting Collections should be possible, but discouraged.

The master branch of plone.app.collection is based on dexterity, but there is room for an intermediate 1.2.x release still based on Archetypes.

A new plone.app.collections.release preferably also works in Plone 4.2.x. Currently, this is the case.

Proposal & Implementation

  • Make the new-style Collections folderish, with migration code.
  • By default do not allow adding anything in a Collection. Managers or integrators can simply change the allowed_content_types in the portal_types tool.
  • Make migration from old-style Topics to new-style Collections an upgrade step.

The migration and folderish parts could be made into two plips if really wanted, but migrating a nested old-style Topic would then not work.

Deliverables

Most of the changes will be in plone.app.collection.

For proper support of folderish collections, changes are needed in archetypes.querywidget and plone.app.querystring.

Progress

The necessary changes have been made on branches of the affected packages. The beginning of the migration code was happily taken from the upgradepath branch started by Roel Bruggink two years ago.

Note that there are two migrations:

  1. from non-folderish Collections to folderish Collections
  1. from old-style Topics to new-style folderish Collections

Both are done with an upgrade step. The order should not matter.

The changes to archetypes.querywidget and plone.app.querystring may be safe enough for an earlier release as they should not affect the current non-folderish use case. But better safe than sorry.

Note that currently types/Collection.xml allows adding nested Collections. This may need to be disallowed by default, though I don't mind.

Risks

Some criteria cannot be migrated exactly. The code tries to be forgiving and log a warning and do something similar:

  • The new-style selection criterion does not support the 'and' operator. The migration makes it an 'or' operator.
  • The new-style (relative) path criterion does not support non-recursive search (depth=0). The migration makes it recursive (return any items in the path or in a subfolder of that path).
  • The new-style path criterion does not support multiple paths. The migration takes the first one.
  • The new-style integer criterion does not support a range of integers (the min:max direction). The migration logs an error and ignores this criterion.

We can make the code less forgiving and throw an exception more often if wanted. Search for TODO in upgrade.py for a few spots.

Non-standard criteria, so criteria with a class defined in an add-on product, will cause an exception. It should be reasonably simple for an add-on author to add support for this, by importing the plone.app.collection.upgrade.CONVERTERS dictionary and adding a converter to it.

Maybe not everyone wants to use the new-style collections. In that case we may want to move this from an upgrade step to a browser view that a manager can call. Also, the upgrade step is currently not run when installing plone.app.collection for the first time, so a user would need to go to portal_setup to run it. Some guidance on the best approach here would be good. Probably the best is to do the same as in plone.app.contenttypes.

Participants

Maurits van Rees

Change History

comment:1 Changed 3 years ago by maurits

  • Type changed from Bug to PLIP

comment:3 Changed 3 years ago by maurits

Added plip config in the Plone coredev 4.4 buildout: plip13561-collections-migration-folderish.cfg

The tests utterly fail during test setup though. Works fine in Plone 4.3.

comment:4 Changed 3 years ago by maurits

  • Description modified (diff)

comment:5 Changed 3 years ago by kleist

  • Keywords collection added
  • Status changed from new to confirmed
  • Component changed from Unknown to General

comment:6 Changed 3 years ago by frapell

Big +1 on this, thanks!

As for the final thought, i believe new-style Collection is going to be the default, so in my opinion, there should be an upgrade step that runs automatically on migration

comment:7 Changed 3 years ago by thet

from my point of view, if we cannot map all criterions to ones, which are fully supported by plone.app.collection, we cannot force integrators to upgrade to the new style collections. this will definitely produce wrong and thus unusable result sets in a lot of cases.

whether we can fix the criterions to support everything Topics did or we go for the proposed solution, that we provide a browser view for the migration, including some warnings.

until a decision on how to do, i'm -1. please note, i'm soon +1 when this issue is resolved.

comment:8 Changed 3 years ago by esteele

  • Milestone changed from 4.x to 5.0

The Framework Team has decided to move on to Plone 5. Updating milestones accordingly.

comment:9 Changed 3 years ago by thet

I revert to +1, since my concerns are a implementation detail, which can be discussed again later on.

Note: See TracTickets for help on using tickets.