Ticket #13866 (new Bug)

Opened 2 years ago

Last modified 2 years ago

WorkflowTool updateRoleMappings performance

Reported by: thepjot Owned by:
Priority: minor Milestone: 4.x
Component: Unknown Version: 4.3
Keywords: workflow, migration, performance Cc:

Description

I've only tested this with Plone 4.0.9 so far, but I assume this will apply to later versions as well. Source of the problem is in Products.CMFCore (2.2.4) tho. But could be fixed with an override in CMFPlone if need be.

Background

When upgrading plone, one of the steps is to introduce the "Site Administrator" role with a given set of permissions. Following this addition, the workflowtool is triggered to 'updateRoleMappings'.

Problem

For our customer, I noticed this took over an hour (7370 portal catalog records, 9.4GB of blobstorage). I also noticed that the blobcache was being filled. (Which can be countered with 'zeo-client-blob-cache-size')

The culprint appears to be the 'reindexObject' method, or more exactly, the metadata. In the workflow tool we explicitly reindex the 'allowedRolesAndUsers' index. But without explicit mention, this also means the entire metadata sheet is re-made. Which causes the whole file to be loaded to fill in the required fields.

Fix

This can be averted by adding a 'update_metadata=0' keyword to the reindexObject in the '_recursiveUpdateRoleMappings' function. Which speeds up the performace significantly. Added a file with the function as I made it. Only added the update_metadata keyword.

Use case

1) Change workflow for Image/File to 'one_state_workflow'
2) Update security settings
3) Change workflow for Image/File to 'intranet_workflow'
4) Update security settings (I time this)

I did this twice, once with the default behaviour. Once with a minor change.

Test 1: TO 'intranet_workflow' WITH metadata (default)

1515 object(s) updated. updateRoleMappings function took 79624.944 ms

Test 2: TO 'intranet_workflow' WITHOUT metadata (change)

1515 object(s) updated. updateRoleMappings function took 17730.919 ms

Conclusion

This is quite a significant change. To my knowledge, there is nothing in the metadata set that is used for workflow, security or permission purpose.

Attachments

recursiveUpdateRoleMappings.txt Download (1.3 KB) - added by thepjot 2 years ago.
fixed function

Change History

Changed 2 years ago by thepjot

fixed function

comment:1 Changed 2 years ago by thepjot

Forgot to mention, this also occurs in Plone 4.3.2 since I upgraded our customer that version.

comment:2 Changed 2 years ago by thepjot

  • Version changed from 4.0 to 4.3

comment:3 follow-up: ↓ 4 Changed 2 years ago by maartenkling

i like, can you make a pull request for this?

comment:4 in reply to: ↑ 3 Changed 2 years ago by thepjot

Replying to maartenkling:

i like, can you make a pull request for this?

Override function in CMFPlone?

comment:5 Changed 2 years ago by maartenkling

Yes, if you make a pull request in Products.CMFPlone it can be reviewed and merged if okay

comment:6 Changed 2 years ago by thepjot

Hi Maarten,

I made a pull request;  https://github.com/plone/Products.CMFPlone/pull/186 But think I picked the wrong branch, figure I need to go for 4.3.X ?

comment:7 Changed 2 years ago by maartenkling

to both master and 4.3.x

comment:8 Changed 2 years ago by thepjot

Update, made a new pull request for the right branch:  https://github.com/plone/Products.CMFPlone/pull/187

comment:10 follow-up: ↓ 11 Changed 2 years ago by thepjot

Seems I was wrong. There is a 'allowedUsersAndGroups' in the metadata, which is used in various places. Making it required that that field is updated..

comment:11 in reply to: ↑ 10 Changed 2 years ago by stevenlooman

Replying to thepjot:

Seems I was wrong. There is a 'allowedUsersAndGroups' in the metadata, which is used in various places. Making it required that that field is updated..

Can you show me where allowedUsersAndGroups is referenced?

comment:12 Changed 2 years ago by thepjot

I just noticed today that that was in custom code I made for a client. So my fix is still valid, haven't had time to redo the pull.

comment:13 Changed 2 years ago by stevenlooman

Great!

comment:14 Changed 2 years ago by thepjot

Note: See TracTickets for help on using tickets.