Ticket #9316 (closed PLIP: fixed)

Opened 7 years ago

Last modified 6 years ago

Unify folder implementations

Reported by: smcmahon Owned by: witsch
Priority: n/a Milestone: 4.0
Component: General Version:
Keywords: Cc: plip-advisories@…, dukebody

Description (last modified by witsch) (diff)

Copied from  PLIP #191 in the roadmap:

Unify folder implementations

We currently have "Folder" and "Large Plone Folder" implementations.
There should be only one.

Proposed by
Martin Aspeli
Seconded by
Andreas Zeidler
Proposal type
Architecture
Repository branch
plone.folder, plone.app.folder

Motivation

Shipping with two folder types is unnecessary for several reasons:

  • It forces the user to make an a-priori choice about the number of objects they plan to put into a folder
  • We ship with the "Large" type disabled by default to avoid UI confusion
  • We don't have a proper search-based UI for large folders anyway

Also the standard Folder type stores attributes, and has a single list _objects tuple which keeps the list of objects and order. This is prone to ConflictErrors and is slower. In simple benchmarks, a BTree-based folder performs orders of magnitude faster than a basic folder.

Proposal

Have a single folder implementation.

  • The internal storage is BTrees
  • It still supports ordering, by storing a separate sort order list/tree
  • It has at least two views - one search-based for "large" folders, one batch-based for "small" folders. This is either just a "display" menu choice, or a choice in the object's schema. Explicit sorting may be turned off for "large" folders.

Implementation

The package plone.folder in the Plone SVN provides a base implementation of a folder base class, which is not Archetypes specific, based on BTreeFolder2, but adding ordering. The exact ordering implementation is left up to an adapter, with a default providing explicit ordering. This allows other implementations, such as auto-sorting based on some key.

The diagram below shows the folderish base- and mix-in classes used in OFS, CMF, Archetypes and Plone. Count 'em:

http://plone.org/products/plone/roadmap/191/Folder%20mess.png

The new base class from plone.folder, i.e. OrderedBTreeFolderBase, is used by plone.app.folder to provide two folderish classes, one targeted to Archetypes (BaseBTreeFolder) as well as one for ATContentTypes (ATFolder). Both add relatively little extra code and setup in order to make them fully compatible with the original, to be replaced classes.

The package also provides a GenericSetup profile replacing the standard "Folder" content type with the new one. In-place migration will convert the internal data-structures when upgrading to use the new folders. Such migration code (including thorough tests) already exists in a project-specific package, and just needs to be moved into plone.folder itself. The migration runs relatively fast — in several performance tests about 13,000 folders holding some 200,000 items in total were migrated in about 5 minutes.

The Container type in plone.app.content does not use BTrees, nor does it support ordering. Rather than changing this class and providing migration, we could add a new type, e.g. called OrderedContainer or just Folder (to signify closer resemblance to the standard folder behaviour) that mixes in the new class instead of PortalFolder.

In addition, plone.folder also provides an ordering adapter, which only considers certain content to be orderable (implemented via marker interfaces). This allows for ordering of content that requires it, for example navigational items, without any added performance penalties if those share a folder with other, non-orderable content in large quantities.

Deliverables

  • New folderish base classes that are BTree-based and support large content sets as well as ordering.
  • Improved Products.ATContentTypes.content.ATFolder and Products.ATContentTypes.content.ATBTreeFolder versions which use BTrees and supports ordering.
  • New class plone.app.content.container.Folder (?) that uses BTrees and supports ordering.
  • The ability to switch to a folder view/behaviour that's optimised for "large" content sets.
  • In-place Migration for all existing sites.

Risks

  • There may be migration issues involved for derived folderish types with additional data-structures regarding containment and/or ordering.
  • This vision makes the existing BaseBTreeFolder in Archetypes orderable (though in some UI configurations you may not see the ordering, to prevent the user invoking slow operations), which may be unexpected.

Progress

Complete:

  • Create a base folder implementation - OrderedBTreeFolderBase
  • Hook this into BaseBTreeFolder in Archetypes
  • Fix CMFPlone so that it detects the order support based on a Zope 3 interface rather than a Zope 2 one
  • Test that order support works on the new folder
  • Update FTIs and portal_type's so that the default "Folder" portal_type is a btree-based folder
  • Enable next/previous navigation for this folder type
  • Enable photo-album support for this folder type
  • Enable archiving support for this folder type
  • Enable ordering policy to be specified by adapter

Releases:

  • 1.0b3 of plone.folder has been released in May and is in Dexterity.
  • 1.0a1 of plone.app.folder has been released in May in order to factor out all code specific to ATContentTypes
  • Both releases have been heavily tested in a large-scale Jarn project and will go into production next week.

To-do - ATContentTypes:

  • Merge in-place migration to new folder types
  • Write migrations for FTI and portal_type changes
  • Ensure compatibility with Plone 4.0, i.e. Python 2.6 & Zope 2.12

To-do - plone.app.content:

  • Create new Folder base class, using the new mixin, with tests

Benchmark results

The following two graphs show some simple benchmarks using the various types of folders (regular = ATFolder, large = ATBTreeFolder, ordered = ATBTreeFolder w/ order support) with 50 and 500 items, respectively. The tests conducted where creating content, retrieving a batch of items from the folder, getting all items from the folder as well as random accessing the items. The benchmark tests used can be found in the tests/ folder.

http://plone.org/products/plone/roadmap/191/folder_benchmarks_50_items.png http://plone.org/products/plone/roadmap/191/folder_benchmarks_500_items.png

A more realistic scenario was provided by some benchmarks based on  JMeter (available in jmeter):

http://plone.org/products/plone/roadmap/191/jmeter_performance_tests.png

The graphs show the time it took to create a folder with 500 news items both using a standard folder and the new btree-based folder with order support. Also, and that's probably the most interesting figure so far, the time for repeatedly browsing the "folder contents" view is decreased to slightly less than 75% by using the btree folder.

Another interesting thing to note is that the "content creation" tests show that btree-based folders are faster than the currently used implementation (from ATFolder) even for very small numbers of contained objects. The tests still run faster when adding 500, 50, 10, 5, 2 or even just a single object to the folder.

Participants

  • Martin Aspeli (IRC nickname: <optilude>)
  • Andreas Zeidler (IRC nickname: <witsch>)
  • Martijn Pieters (IRC nickname: <MJ>)
  • Alec Mitchell (IRC nickname: <alecm>)

Change History

comment:1 Changed 7 years ago by witsch

  • Status changed from new to assigned
  • Owner set to witsch
  • Description modified (diff)

many thanks steve & limi for copying this over in time before the deadline!! :) i've just fixed some of the remaining formatting glitches and will update the status soon...

comment:2 follow-up: ↓ 4 Changed 7 years ago by pupq

+1. And if it could sneak in a body field so that default pages can be a far-less-required feature, people will really, really love these new folders. That's the most common thing people ask about folders -- "can I add some text above the listing of the content items?"

comment:3 Changed 7 years ago by limi

  • Priority changed from minor to n/a

comment:4 in reply to: ↑ 2 Changed 7 years ago by witsch

Replying to pupq:

+1. And if it could sneak in a body field so that default pages can be a far-less-required feature, people will really, really love these new folders. That's the most common thing people ask about folders -- "can I add some text above the listing of the content items?"

while i can certainly see your point, i don't think this can be part of this PLIP. it'll require discussion and an official proposal — sneaking things in is not what we want to do...

comment:5 follow-up: ↓ 7 Changed 7 years ago by erikrose

  • Owner witsch deleted

Clearing Owner field of 4.0 PLIPs so we can use it to mean "implementor". (Many of these owners were automatically assigned from choosing a Component that had a default owner.)

comment:6 follow-up: ↓ 8 Changed 7 years ago by alecm

  • Owner set to witsch
  • Status changed from assigned to new

+1

This is important stuff, plus it makes Plone faster. Not sure how I became a participant though.

comment:7 in reply to: ↑ 5 Changed 7 years ago by witsch

Replying to erikrose:

Clearing Owner field of 4.0 PLIPs so we can use it to mean "implementor". (Many of these owners were automatically assigned from choosing a Component that had a default owner.)

in this case i think the owner was already assigned correctly... ;)

comment:8 in reply to: ↑ 6 Changed 7 years ago by witsch

Replying to alecm:

Not sure how I became a participant though.

:) i guess martin added you to the list when the  PLIP was originally put up. interestingly, however, it was only  created after we started the package back at the copenhagen performance sprint. so perhaps he wanted you to participate, and afair you did so in the discussion, no?

anyway, feel free to remove yourself from the list if you don't want to be blamed for the bugs later on... ;)

comment:9 Changed 7 years ago by witsch

  • Status changed from new to assigned
  • Cc optilude, mj, alecm, witsch added

adding the participants and taking it on (if somebody else wants to be responsible, please tell me! :))

comment:10 Changed 7 years ago by smcmahon

  • Cc plip-advisories@… added; optilude, mj, alecm, witsch removed

comment:11 Changed 7 years ago by dukebody

  • Cc dukebody added

comment:12 Changed 7 years ago by MatthewWilkes

FWT Vote: +1

comment:13 Changed 7 years ago by rossp

FWT vote: +1

comment:14 Changed 7 years ago by davisagli

FWT vote: +1.

Please include documentation for product developers on how to migrate existing subclasses of ATFolder.

What's the size of a BTree-based folder pickle vs. the current ATFolder?

comment:15 Changed 7 years ago by calvinhp

FWT Vote: +1

comment:16 Changed 7 years ago by raphael

FWT Vote: +1

comment:17 Changed 7 years ago by esteele

(In [27781]) Update to use the new PLIP number. refs #9316.

comment:18 Changed 7 years ago by esteele

Approved by FWT vote.

comment:19 Changed 7 years ago by witsch

  • Description modified (diff)

more formatting fixes to finally match the state of the  original plip

comment:20 Changed 7 years ago by witsch

  • Description modified (diff)

update ticket to reflect the current status

comment:21 Changed 7 years ago by witsch

  • Description modified (diff)

currently there are 0 test failures when running tests against Products.CMFPlone, Products.ATContentTypes and Products.Archetypes as of Plone 3.3rc5. the same tests still have to be conducted against Plone 4.0, though...

comment:22 Changed 7 years ago by witsch

  • Description modified (diff)

this is not ReSt, so unicode characters can be included directly... yay! :)

comment:23 Changed 7 years ago by witsch

(In [29060]) add  plone.app.folder as a dependency to Plone & update the FTI for "Folder" content to use the new implementation (refs #9316)

comment:24 Changed 7 years ago by witsch

(In [29061]) add review notes for PLIP #9316 (refs #9316)

comment:25 follow-up: ↓ 29 Changed 7 years ago by davisagli

(In [29513]) add FWT review, refs #9316

comment:26 Changed 7 years ago by esteele

Your PLIP has passed the Framework team's initial review. Feel free to discuss any suggested changes either here in the PLIP ticket or on the mailing lists. Final deadline for this PLIP is set for September 30.

comment:27 Changed 6 years ago by witsch

(In [30089]) fix the mess left behind after the too late at night r29060: of course the product can remain as ATContentTypes after the original ATFolder  was replaced with the btree-based implementation anyway. also loading plone.app.folder's zcml isn't necessary as the content type does still live in ATCT and the extra profile isn't needed either. so the only thing left is to load zcml for plone.folder.

this allows to actually run tests against in plip buildout again. (refs #9316)

comment:28 Changed 6 years ago by witsch

(In [30097]) partly apply current set of test patches for CMFPlone, resolving the test failures (refs #9316)

comment:29 in reply to: ↑ 25 ; follow-up: ↓ 30 Changed 6 years ago by witsch

Replying to davisagli:

(In [29513]) add FWT review, refs #9316

quoting parts of david's review to be able to provide comments inline:

Notes and observations

  • The original PLIP discussed the creation of a special view optimized for viewing folders with many items. I presume this was dropped due to the separate PLIP for revamping the collection UI? That's probably fine.

well, it wasn't really dropped because of that other PLIP, to be honest. it was just that when martin & i started working on this back in copenhagen, i never thought of a UI part, and i only read PLIP 191 again after steve had already used it as the base for this one... :)

iow, it was never my intention to provide an update folder_contents view for this, even though we have of course discussed the implications and possible solutions for quite a bit. we've also created a slightly adjusted version in one of our projects, but that's tailored towards the partial ordering adapter.

in any case, i won't have time to come up and implement such a view as part of this work. and i also think that it needs more discussion first. sean upton's work (as part of this year's gsoc) seems promising, and i'd very much like to see a PLIP for this for 4.1 or 4.2. it won't necessarily me who's driving this, though... :)

  • The original PLIP also discussed some migration code for converting existing the internal data structures of existing folders. Does this code exist where where it can be reviewed?

not yet, it still lives inside a migration package of the project it was developed for. i will merge it into plone.app.folder next thing, though.

It's crucial that we get this right, of course, and that it can be made to work for custom subclasses of ATFolder as well. Assuming this concern is addressed, it looks like custom subclasses will simply pick up the new mixin and "just work" as long as they aren't overriding the old ordering implementation.

right. the migration has proven to work for our project: some 15+k folders were migrated a while ago and are now used in production by several thousand users every (week)day. so far we haven't seen any problems with neither the migration nor the new folder implementation. however, i should mention that this project is using the partial ordering adapter instead of the default one.

  • The extension profile in plone.app.folder appears to merely duplicate the change that was made in CMFPlone's Folder.xml.

right, and the change to the default Folder.xml was bad anyway, see r30089. the same commit also keeps the extra profile from being loaded, fixing this issue. you are right, though, the profile is indeed still needed for plone 3.x.

  • bin/alltests shows erroring tests in a bunch of packages, mostly with a traceback and error similar to the following:

this was also fixed by r30089 — the problem was that the factory (in plone.app.folder) hadn't been properly initialized yet by the time PTC was trying to create a folder during site setup. as ATFolder itself is now patched (by this PLIP, as opposed to the stand-alone plone.app.folder for 3.x), the factory can simply remain to be the old one.

ValueError: Product factory for Folder was invalid

on a side node, i only realized last night that i left the PLIP in a completely unfinished state the night i was trying to wrap it up for the first round of reviews. i found a yet uncommitted change and also remembered having seen that error before. however, i also remember constantly dozing off over the keyboard that night — no real surprise at 3am, i guess...

at some point i think i gave up on the above mentioned test setup problem, deciding i would look into it in the morning. however, the funny thing to me is that apparently i was so tired that i didn't even remember something still wasn't quite working the next day. judging from my time tracking i must have thought everything was fine and never looked back... so again, sorry about the mess. :)

There are also a lot of test issues in Archetypes.

right, there are indeed a number of test issues left. i've just updated the branches and checked a couple of packages:

  • Product.Archetypes currently has 5 failures, 9 errors (these are very likely all because the sample type SimpleBTreeFolder can for some reason not be adapted to IOrdering anymore)
  • Product.ATContenTypes has 1 failure in traversal.txt
  • Product.CMFPlone has 0 failures
  • plone.folder has 0 failures (needs to be run using bin/instance test -s plone.folder for some reason)
  • plone.app.folder (which also needs to be run using bin/instance test ...) has 0 failures, but a problem with importing test_integration.py (i suspect a circular import or something)

i will investigate these asap, but none of them should be show-stoppers. the same tests all pass on plone 3.x, and i'm pretty sure the failures are only triggered because of the integrational changes in Archetypes and ATContentTypes.

I assume that these are mostly due to the switch to Plone 4 when this package was developed for Plone 3, but come on, these really need to be fixed for this PLIP to be considered seriously!

you're right of course. sorry about turning this in without properly finishing up first. but like i said, i thought i had... %)

  • I did not have time to verify the benchmarks included in the PLIP overview, but don't have reason to doubt them.

actually, it's pretty easy to run them — see the comments in benchmark.py. the jmeter tests will be trickier to set up, of course.

  • Any sense of how this affects pickle size and transaction size for operations on different sizes of folders?

no, i haven't checked yet. would you have any suggestions on what to test here in particular?

comment:30 in reply to: ↑ 29 Changed 6 years ago by witsch

Replying to witsch:

Replying to davisagli:

  • I did not have time to verify the benchmarks included in the PLIP overview, but don't have reason to doubt them.

actually, it's pretty easy to run them — see the comments in benchmark.py. the jmeter tests will be trickier to set up, of course.

well, except that i just noticed they're currently broken, because the old folder has no FTI anymore. i'll look into that, but not before the other issues are fixed...

comment:31 Changed 6 years ago by witsch

(In [30103]) add in-place migration code (refs #9316)

comment:32 Changed 6 years ago by witsch

(In [30105]) avoid circular imports when using plone 4.0 (refs #9316)

comment:33 Changed 6 years ago by witsch

(In [30120]) fix (most of) the migration tests — plip #9316 changes the original ATFolder class to use an implementation based on btrees, so it can no longer be used for testing the migration. hence a dummy content type much like the old class is needed... this leaves us with 1 failure and 1 error (refs #9316)

comment:34 Changed 6 years ago by witsch

(In [30122]) fix the remaining test issue — a side-effect of the added (old-style) test-folder class is that we now have the same behaviour for unmigrated btree-folders again, i.e. independently from which adapter is active (refs #9316)

comment:35 Changed 6 years ago by witsch

(In [30123]) heh, the "analysis" in r30122 was not quite correct: i still had the partial ordering adapter activated for testing purposes. guess my cold makes me forgetful... :) (refs #9316)

comment:36 follow-ups: ↓ 37 ↓ 38 Changed 6 years ago by davisagli

The test situation is looking much better. I'm still seeing the issues you mentioned in the ATContentTypes and Archetypes tests, as well as some in the CMFEditions tests. I think it's probably okay for these to be put off and fixed as part of the merge, as long as it gets done. ;)

The migration looks good...shouldn't we have this get called from an upgrade step in plone.app.upgrade though?

+1 for merge.

comment:37 in reply to: ↑ 36 Changed 6 years ago by witsch

Replying to davisagli:

I'm still seeing the issues you mentioned in the ATContentTypes and Archetypes tests,

yes, i didn't have time to look into them yet.

as well as some in the CMFEditions tests.

these errors like

Error in test test07_prepareObjectWithReferences (Products.CMFEditions.tests.test_ArchivistTool.TestArchivistToolZStorage)
Traceback (most recent call last):
  File ".../lib/python2.6/unittest.py", line 279, in run
    testMethod()
  File ".../Products.CMFEditions/Products/CMFEditions/tests/test_ArchivistTool.py", line 247, in test07_prepareObjectWithReferences
    for sub in cloneValues:
  File ".../Products/ZCatalog/Lazy.py", line 158, in __getitem__
    data.append(func(s[ind]))
  File ".../plone.folder/src/plone/folder/ordered.py", line 44, in _getOb
    return super(OrderedBTreeFolderBase, self)._getOb(id, default)
  File ".../Products/BTreeFolder2/BTreeFolder2.py", line 221, in _getOb
    return ob.__of__(self)
AttributeError: __of__

also occur in plone 4.0 and 3.3 with the following patch

  • src/Products.CMFEditions/Products/CMFEditions/tests/test_ArchivistTool.py

     
    4747        self.portal.acl_users.userFolderAddUser('reviewer', 'reviewer', 
    4848                                                ['Manager'], '') 
    4949        self.portal.invokeFactory('Document', 'doc') 
    50         self.portal.invokeFactory('Folder', 'fol') 
     50        self.portal.portal_types.getTypeInfo('Large Plone Folder').global_allow = True 
     51        self.portal.invokeFactory('Large Plone Folder', 'fol') 
    5152        self.portal.fol.invokeFactory('Document', 'doc1_inside') 
    5253        self.portal.fol.invokeFactory('Document', 'doc2_inside') 
    5354        self.portal.fol.invokeFactory('Document', 'doc3_outside') 

iow, btree folders don't seem to be compatible with versioning atm. my guess is that perhaps nobody really tried to use this combination before, especially when taking into consideration that "large folders" have so far been mostly neglected in plone...

i'll try to have a look into this, even though it's strictly not a related problem. however, if we want to migrate to btree folders, the issue must be addressed, of course...

I think it's probably okay for these to be put off and fixed as part of the merge, as long as it gets done. ;)

yes, it needs to get done, of course, but i'm also planning on it... :)

The migration looks good...shouldn't we have this get called from an upgrade step in plone.app.upgrade though?

we should — i'll add it to the list...

comment:38 in reply to: ↑ 36 Changed 6 years ago by witsch

Replying to davisagli:

+1 for merge.

hmm, i suppose i never "officially" stated that this was ready for a second round of reviews, which is probably also why it's missing from  the list of to be voted on PLIPs...

iow, could it please be added? :)

comment:39 Changed 6 years ago by rossp

FWT vote: +1 for merge

comment:40 Changed 6 years ago by esteele

This PLIP has been accepted for merging into Plone 4.0

The final vote was: Alec Mitchell +1 David Glick +1 Erik Rose +1 Laurence Rowe +1 Matthew Wilkes - Ross Patterson +1

Please merge your branches into the Plone 4.0 head by end-of-day Friday Oct 16. If you need assistance with merging, please contact me.

We'll be assigning a documentation ticket to this PLIP shortly. Please assist the docs team in documenting the changes and new features that this PLIP introduces.

comment:41 Changed 6 years ago by esteele

Please assist the doc team in creating/updating documentation relating to this PLIP. See #9615.

comment:42 Changed 6 years ago by witsch

(In [30519]) aha, all new packages need to be added to [test] in order to make $ bin/test ... work (refs #7822, #9316)

comment:43 Changed 6 years ago by witsch

(In [30537]) merge branch for plip9316 into 4.0@30536 (refs #9316)

comment:44 Changed 6 years ago by witsch

comment:45 Changed 6 years ago by witsch

  • Status changed from assigned to closed
  • Resolution set to fixed

the PLIP has been merged, and even though there are still things left to wrap up & polish this ticket can be closed...

comment:46 Changed 6 years ago by witsch

(In [30897]) fix benchmark tests using the testing content type from r30120 (refs #9316)

comment:47 Changed 6 years ago by witsch

(In [30898]) add results of running the benchmark tests with different numbers of content items (5, 50, 500, 5000) and a helper script for converting them into something mainly intended to be imported into a spreadsheet app, but also slightly more human-readable (refs #9316)

to run the benchmarks yourself, you might use it like so::

$ bin/instance test -s plone.app.folder --tests-pattern=benchmarks 2>&1 | benchmarks/convert.py

comment:48 Changed 6 years ago by witsch

(In [30929]) add migration step for unified folders (refs #9316)

comment:49 Changed 6 years ago by davisagli

(In [32097]) create ATFolders instead of Large Plone Folders for the initial example content, refs #9316, refs #9791. we still need to figure out what to do re migrating existing Large Plone Folders when upgrading to Plone 4.

comment:50 Changed 6 years ago by witsch

(In [33786]) remove getObjPositionInParent catalog index and use a sort index based on the folder's order information instead, thereby allowing to skip the (partial) reindexing of all items in the folder when renaming or moving items (refs #9316)

comment:51 Changed 6 years ago by witsch

(In [33788]) with r33786 the getObjPositionInParent catalog index can be removed and reindexOnReorder made a stub, which should improve renaming and reordering performance (refs #9316)

comment:52 Changed 6 years ago by witsch

(In [33789]) add buildout to remove the getObjPositionInParent catalog index and use a sort index based on the folder's order information instead (refs #9316)

comment:53 Changed 6 years ago by witsch

(In [33790]) eek, that change ended up on the wrong (local git) branch — the monkey needs to be applied unconditionally, since p.a.folder doesn't get "initialized" in plone 4 (refs #9316)

comment:54 Changed 6 years ago by witsch

(In [33791]) update "nogopip" buildout so it can be run directly from the plone 4 coredev checkout:

$ bin/buildout buildout:directory=$PWD -c experimental/nogopip.cfg

while still retaining the ability to

$ bin/develop status

etc... (refs #9316)

comment:55 Changed 6 years ago by witsch

(In [33792]) only do the extra sanity checks when running tests or in debug-mode (refs #9316)

comment:56 Changed 6 years ago by witsch

(In [33870]) simplify calculation of the mapping from rid to position-in-parent & make it work for results from different folders at the same time. the tests now work both with and without the "nogopip" patch (refs #9316)

comment:57 Changed 6 years ago by witsch

(In [33924]) add __getitem__ support to the default ordering adapter for optimized previous/next support in plone.app.folder (refs #9316)

comment:58 Changed 6 years ago by witsch

(In [33931]) add adapter for previous/next support that doesn't need the catalog, or the getObjPositionInParent index for that matter (refs #9316)

comment:59 Changed 6 years ago by witsch

(In [34078]) use our own zcml for testing & also register the partial ordering adapter now that they get along with each other (refs #9316)

comment:60 Changed 6 years ago by witsch

(In [34080]) omg, how pythonic! (refs #9316)

comment:61 Changed 6 years ago by witsch

(In [34154]) all those calls to getObjectPosition use adapter lookups etc, so introducing an optimization for the usual "all results are from one folder" case make sense (refs #9316)

comment:62 Changed 6 years ago by witsch

(In [34156]) pull in the "nogopip" changes from  plone.app.folder, which remove the getObjPositionInParent catalog index while keeping the possibility to sort by folder position (refs #9316)

comment:63 Changed 6 years ago by witsch

(In [34191]) replace monkey patch for Catalog._getSortIndex with a fake index that can sort search results according to their position in the container (refs #9316)

comment:64 Changed 6 years ago by witsch

(In [34194]) replace getObjPositionInParent with stub index capable of sorting search results according to their position in the container, a.k.a. "nogopip" (refs #9316)

comment:65 Changed 6 years ago by witsch

(In [34196]) only apply monkey patch for reindexOnReorder on plone 3.x & shortcut indexing completely if the fake index has been installed (refs #9316)

comment:66 Changed 6 years ago by witsch

(In [34220]) migrate getObjPositionInParent to stub index capable of sorting search results according to their position in the container, a.k.a. "nogopip" (refs #9316)

comment:67 Changed 6 years ago by witsch

(In [34432]) partly revert r26681 & r26682, thereby removing support for setting __parent__ and __name__ for content providing IContained as it can cause  severe performance issues when used on Zope 2.x (refs #9316)

comment:68 Changed 4 years ago by davisagli

  • Component changed from Infrastructure to General
Note: See TracTickets for help on using tickets.