Ticket #20144 (new PLIP)

Opened 17 months ago

Last modified 15 months ago

Make content-types folderish by default

Reported by: pbauer Owned by:
Priority: minor Milestone: 5.0
Component: Backend (Python) Version:
Keywords: Cc:

Description (last modified by pbauer) (diff)

Proposer: Philip Bauer
Seconder: Johannes Raggam

Abstract

Make Collection, Event, Document, News Items and Link folderish by default by inheriting from plone.dexterity.content.Container. Image and File stay itemish unless we find a use-case for those.

Motivation

This PLIP would:

  • Remove a inconsistency between the default-types and types that are generated ttw (they are folderish).
  • Allow developers and users to easily make the types folderish by changing the values for filter_content_types and/or allowed_content_types
  • This way editors can create site-structures without using the default-page feature.
  • Prepare for a possible removal of the default_page-feature

Proposal & Implementation

We would simply change the base-class of said types to Container. For new sites no other changes are necessary.

Objects in existing sites need to be updated. That upgrade-step runs when upgrading from Plone 5.0a3 to the next version and is also available as a upgrade-step in plone.app.contenttypes (e.g. if you use p.a.c in Plone 4). From a usability-perspective nothing changes for normal users. Since the FTI's are configured like this: <property name="allowed_content_types"/> you cannot add any types. Only if this setting is changed content can be added to these types.

Deliverables

  • Change the base-class (DONE)
  • Upgrade-Step to change existing objects (DONE)
  • Don't show the menu-item to add content to a folderish default_page if no content can be added to it (DONE)
  • Show folderish types in @@select_default_page (DONE)
  • Tests (DONE)
  • Documentation (TODO)
  • Fix test-failures in other packages (TODO)

Risks

  • Perfomance implications: All tests (including robot-tests) ran in same time.
  • Memory-footprint implications: We need to find out if the switch significanlty increases the memory-use of types.
  • UI-Problems:
    • Adding content to folderish default_pages. This generally works since there is the link "Add item to default page" but it's use could be irritating. By default the link would never show up with the changes in this PLIP since you cannot add content to these types unless you add some types to allowed_content_types for a FTI. If this is the case the type would no longer be selectable as a default_page. For now you can only override this by changing default_page_types in site_properties. This only makes sense in cases where you don't create a structure using folderish documents but if you store images and files inside the document.
    • There is also a js-problem with the link "Add item to default page" when @@folder-factories is opened in a popup and you select a type to be added you are redirected to the container instead of to the ++add++ view.
  • The new folder_contents shows the add-menu for folderish types even if allowed_content_types is empty.

Patricipants

Philip Bauer <pbauer>

Progress

Work in plone.app.contenttypes is done in the branch See  https://github.com/plone/plone.app.contenttypes/pull/196

  • Change to folderish: DONE
  • Upgrade-step: DONE
  • Show folderish types in @@select_default_page. The folderish types weere skipped, now they are only skipped if if users can add content. DONE in  https://github.com/plone/plone.app.content/commit/f745a13e2d2dd89e2dfcf51cb5d00acc05b29c80. I already commited this since it is also a problem for custom folderish types (like those that are generated ttw).
  • Don't show the menu-item to add content to a folderish default_page if no content can be added to it. DONE in  https://github.com/plone/plone.app.contentmenu/pull/6. I already did this since it is also a problem for custom folderish types (like those that are generated ttw).
  • Test for the upgrade-step: DONE
  • Test for folderish types: DONE

Alternatives

  1. A optional folderish profile to allow users and developers to have a supported way to get folderish default-types is in the pull-request  https://github.com/plone/plone.app.contenttypes/pull/195.
  1. collective.folderishtypes provides folderish types (for AT and DX) as an addon:  https://github.com/collective/collective.folderishtypes

Change History

comment:1 Changed 17 months ago by pbauer

  • Description modified (diff)

comment:2 Changed 17 months ago by pbauer

  • Description modified (diff)

comment:3 Changed 17 months ago by pbauer

  • Description modified (diff)

comment:4 Changed 16 months ago by pbauer

  • Description modified (diff)

comment:5 Changed 16 months ago by pbauer

  • Description modified (diff)

comment:6 Changed 16 months ago by pbauer

  • Description modified (diff)

comment:7 Changed 16 months ago by djay

A few notes (based on usecase analysis that went into  https://docs.google.com/a/pretaweb.com/document/d/1jJ7765rW-CpdOhKR6o4hFaOzQk6xmuePFZd-zTzxwFE/edit#heading=h.5ok1uk1g797w).

  • need to explicit if both folder and documents will remain and if folder types will have a rich text field.
  • since documents will be folderish and people will be using them to replace default pages, you will need to support all the ways default pages are used. you will need a way to delegate editing/review permissions (sharing) for the content part of the document, without giving it to any sub content like you can with default pages. The same applies with apply a portlet to a single document and not all sub documents. One way to support this is to change the UI allow each assignment to be either "this page only", "page and subcontent", "subcontent only"
  • can folderish types be made default pages of other folderish types? -if so where will new content be added?
  • default pages allow you to set a title and description used in navigation which is different than what's displayed on the page. Probably the best way to support this is to have an additional navigation title and description fields which if used are used for navigation only.
  • if Documents can have folder views, then something will have to change about caching rules since a document can now be in two caching categories. Should caching be based on view used?

comment:8 Changed 16 months ago by jaroel

I think this is how this should work:

  1. 'Folder' will remain, and will not have a bodytext.
  2. Delegating is not needed, because the document is editable. One can still set a document as a default view. So no change there.
  3. New content will be added to the 'parent object', not the 'default view object'. This should not be any different to the current situation.
  4. This will function like it did before.
  5. Yes, this is something that needs to be changed.

@djay Different navigation title/display title sounds like a nice PLIP. Would you draft one for us?

@pbauer I actually see no reason not to make every content type folderish use a folderish base-class. This would make life a lot easier when /someone/ has a usecase ;)

comment:9 Changed 16 months ago by jaroel

Also: 'content as default view' is not going anywhere. We need it for the Plone site root, because that is a Products.CMFDefault.Portal.CMFSite.

comment:10 Changed 16 months ago by djay

I believe the intention of this PLIP is that default view is likely to disappear in some future Plone release (not plone 5.0) and part of this PLIP is to prepare for that in a sensible way that doesn't leave the user experience in a worse state. Dealing with how CMFSite is handled would be issue of a future PLIP such as the draft one I linked to above.

With (2). If the goal is to ensure our UX is still good, and lets assume that people will try building sites using folderish documents and not use folders and default pages. At some point they may want to the fine grained control of placing a portlet just on the current page and not having it propogate. We can say at this point the solution is "oh thats not supported with folderish documents, you will have to rearrange things and use folders and default pages to get that affect. It's not a great UX. If you do support like I did, I'd rather turn off folderish documents than have to deal with those kind of calls.

As to if these suggestions should be done in another PLIP. Thats a procedural issue but my opinion is that we should approach a PLIP always with UX in mind. If we do something, we do it nice and not leaving a more confusing UX than we started with. Don't allow one more use case but then as a result add caveats to a few other usescases. So to me, if we go folderish, then the above suggestions are UX bugs that need to be fixed before delivery, rather than extra PLIPs.

comment:11 Changed 16 months ago by jaroel

I cannot predict what the future holds, but this PLIP's only intention is to have folderish contenttypes, so normal endusers don't have to use a folder + a document. The rest should stay the same, afaik. Removal of 'content as default view' should be PLIPed and will be discussed when and only when it is PLIPed.

With (2). It should be the same, but it'lll be a document that is the default view of a different document. It should supported for folderish document. And you can always turn this off by clearing 'Allowed content types' in portal_types. As for the 'ensure our UX is still good' part, we used to get loads of support calls because people don't understand 'content items as default view'. Back in the day we used to create a folder+document by default because people always wanted to add documents to document later on. Now we just ship folderish contenttypes and no support calls.

As for the 'Different navigation title/display title sounds like a nice PLIP.': this sounds like a great feature wether or not we'll have folderish contenttypes.

comment:12 Changed 16 months ago by djay

You need to read what I wrote more carefully. I'm not going to explain them a third time. There are UX bugs with this current proposal that will need to be addressed to prevent confusion when using sharing, portlets and navigation titles. There is no point introducing a feature if it makes the product more complicated. Please refer to recent UI team discussion on this PLIP.

comment:13 Changed 16 months ago by thet

  • Description modified (diff)

comment:14 Changed 16 months ago by thet

At the FWT meeting from 2014-11-25 we discussed this PLIP. We came up with a lot of pro's and con's, but the bottomline is:

  • We want this functionality as most thinks, it simplifies content management a lot,
  • We think it's too dangerous to implement it for Plone 5, which should soon go to beta state.

Philip, is it acceptable for you to target Plone 5.1 with this PLIP?

Refs (FWT meeting protocoll):  https://docs.google.com/document/d/1Ky3LQ9H7o8I2Izyy2YX8qd1NV1ZG9aEJ3SBugTeh6AA/edit#heading=h.2a47mepb0edx

comment:15 Changed 16 months ago by pbauer

Some numbers on weight and speed:

create and publish 1000 non-folderish documents with plone.api:
3:15 min
DB after packing: 11.5M = 9.3 MB

create and publish 1000 folderish documents with plone.api
3:33 min
DB after packing: 12.2M = 10 MB

comment:16 Changed 16 months ago by pbauer

Some more on weight and speed-implications (all tests were done in Plone 5 coredev).

AT: Documents
Create 4000 Documents: 844 seconds
DB size after packing: 51.9M

DX: Documents (itemish)
Create 4000 Documents: 1092 seconds
DB size after packing: 39.7M

DX: Documents (folderish)
Create 4000 Documents: 1160 seconds
DB size after packing: 42.4M

Summary:

  1. DX objects are slower but take up much less DB-space than Archetypes.
  2. The difference between itemish and folderish DX-types in speed and weight is measurable but almost irrelevant.

comment:17 Changed 16 months ago by pbauer

I talked to Timo and did the follwing:

There is now a pull-request for this PLIP:  https://github.com/plone/plone.app.contenttypes/pull/196

I removed the optional folderish profile from master and made a PR from it:  https://github.com/plone/plone.app.contenttypes/pull/195.

I cherry-picked the baseclass migration-method and -form to master since they are great to have disregarding this PLIP.

There are no solutions yet for the issues Dylan mentioned when people stop using folders and instead only use documents:

  • You can no longer have Portlets only on the "frontpage" of a folder. We could add a feature to portlets that allows a portlets to be inherited to its children.
  • Same goes for sharing: You can only delegate permissions for the objects and its children, not only the object if the object is page used as a folder.
  • Folderish types that you can add content to cannot be made into default pages. This can only be overridden by adding types to the default_page_types in site_properties.

I agree that it would be ok to wait with this PLIP until we have an idea how to solve these issues.

comment:18 Changed 16 months ago by pbauer

  • Description modified (diff)

comment:19 Changed 15 months ago by timo

A note regarding folderish collections: Folderish/nested collections have been removed from the Plone UI in Plone 4 and have been entirely removed in Plone 4.2.

 http://plone.293351.n2.nabble.com/Nested-collection-removal-in-Plone-4-0-4-2-td7556157.html

The reason was that it wasn't possible to create a good and intuitive UI for folderish collections. A collection is basically a persistent search. A collection child-element is then a sub-query that needs to inherit the query params from its parent (otherwise they do not make any sense at all). We neither have a UI for that, nor do I think that we really want to implement such a complex use case and expose it to our site administrators.

I had the use case for folderish collections in a project back then and it was super easy to just implement it with a custom base class. If we are serious about making things easier and having a cleaner code base, we have to live with the fact that we can not support 100% of the use cases out of the box.

comment:20 Changed 15 months ago by djay

based on the discussion in the UI team I'm putting forward some solutions to the problems I outlined factoring in that folderish might be an optional profile.

Navigation titles and description

  • It's kind of a hack that you have to use default pages to achieve different titles and descriptions on the navigation anyway. The proposal would be to add a navigation title and navigation description as additional fields on the settings tab for all types. These would be used if not empty. They would take priority over using the default pages title and description (or the default pages navigation title and description).

Caching rules

  • Its kind of a hack that type is used so heavily in the caching rules. The intent is that really how dynamic a page is and how much it depends on lots of other items. Since the proposal will be to include a page view without a listing then perhaps the rules should be based around which display view is currently set? So instead of mapping Content type to rule set, it should be a mapping of contenttype-viewtemplate to rule set.

Portlet assignment

  • Again achieving portlets just at a single level by using the default page is currently not very intuitive anyway. So instead we change the portlet manager UI to allow setting a "scope" when adding a portlet. The scope would either be "just this page", "this page and all subpages", "just subpages". Adjust the scope after adding would also need to be supported. In the case where a default page has a portlet and the parent has a "just this page" portlet then both would be shown.

Sharing assignment

  • This would mean there would need to be a permission separation between editing the content of a page and adding a removing items from inside a folderish page as you might want to give people access to add/remove news items but not change the collection or page view for example. One way to do this is add a scope similar to portlet assignment of "just this page", "this page and all subpages", "just subpages", however that as to be done for each kind of role. Some options for UI are:
    • This could be done by changing the checkbox to a drop down however this could get very cluttered.
    • Another way would be to switch to a UI thats more like content rule assignment. You add a sharing rule with a given user/group, scope and set of roles. Multiple rules can be added. e.g. Administrators - can "View" and "Edit" for "This page and all sub items". [edit] [remove]. One advantage of switching to this UI is it could allow more sophisticated sharing rules such as per type, ie allowing someone to only add News items.

Some or all of these improvements could be added without folderish content types and I think they would help users by making many use cases more obvious. All of those can be added in conjunction with adding folderish content. If we choose not to fix any of these issues and add folderish content types and keep default pages, there is a work around using default pages for each case. However its not a very nice work around as the user has to realise first and potentially restructure their content or even reimplement custom content types in order to gain back functionality they now find they need (such as adding a portlet just at the page level etc). This could be very frustrating. Potentially that could be worked around by documentation warning not to use folderish if you need these missing features.

comment:21 Changed 15 months ago by pbauer

Timo asked about the impact on the number of objects in the DB. Here are some new stats created using todays coredev using plone.app.contenttypes (branches master and folderish_plip) and ATContentTypes:

Creating 1000 documents using collective.loremipsum (master since the released version does not work in Plone 5)

DX Itemish:
Duration: 390 sec
Size in DB: 19.4 MB
Objects: 39099
DB-Objects per Document: 39

DX Folderish:
Duration: 420sec
Size in DB: 20.2 MB
Objects: 45187
DB-Objects per Document: 45

AT Itemish:
Duration: 237 sec
Size in DB: 15.5 MB
Objects: 25557
DB-Objects per Document: 26

The main difference to the previous stats is that collective.loremipsum populates all fields. It seems that the richtext-objects stored in the field 'text' are heavier than the old AT-method of simply storing the string.

comment:22 Changed 15 months ago by thet

philip, could you re-run the tests on buildout.coredev with the new pone.dexterity/master checked out (buildout.coredev 5.0 already includes it in checkouts)?

jens klein did some refactoring on plone.dexterity including better caching of schematas. i'm curious, if there is some performance improvement over archetypes creation.

comment:23 Changed 15 months ago by thet

philip,2: i'm using collective.folderishtypes since i've created it for all my projects. one UI related regression, which always bothered me is, that when you create or edit folderish item, you're redirected to the item's detail view. when you add another item, you are creating that one within the previous one. without folderish items, you're creating it in the parent folder - which is, what people normally expect.

i fixed this in current collective.folderishtypes:  https://github.com/collective/collective.folderishtypes/commit/e1b30194f1e022645bd745dbb0d51a351857b859

now, after creation/editing of items, you're redirected to the parent folder. maybe it's better to redirect only for creation. however, working with content feels much better now.

maybe this can also be used in the folderish approach of plone.app.contenttypes?

Last edited 15 months ago by thet (previous) (diff)

comment:24 Changed 15 months ago by thet

ok, maybe it's not that perfect. i'm already getting annoyed about the parent-folder-redirection.

however, i think this ui prob has to get thought about a bit more.

Note: See TracTickets for help on using tickets.