Ticket #13260 (confirmed PLIP)
Remove portal_skins from Products.CMFPlone
Reported by: | esteele | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | 5.0 |
Component: | Backend (Python) | Version: | |
Keywords: | Cc: | eleddy, maurits, encolpe, phgross, marciomazza, gforcada, buchi, tschanzt, yurj, lukasg, maethu, sureshvv, piv, jcbrand, saily |
Description (last modified by saily) (diff)
Proposer: Elizabeth Leddy
Seconder: Eric Steele
Motivation
- CMFForm controller has not been recommended for many years now.
- There are several bugs in trac that are stuck because of issues with cpy redirects and munging the request and they can not be fixed until we move on
- It is yet another deprecated technology that new plone users have to learn.
- Plone can not be installed on several hosting platforms because cpy is not real python.
- Basic forms like login have to be customized in skins because they can not be extended
- New features and UI upgrades can not be accomplished [easily] without this upgrade.
.metadata is dead, long live .metadata!
Assumptions
Plone 5.
Proposal & Implementation
Summary: Get rid of /skins in Products.CMFPlone
- Move all cpy scripts and validators to z3c.form when and where its possible and makes sense. Not all forms will get rewritten but we can do our best. In some cases its easier to rewrite than to translate and vice versa. rewriting the forms that are direct translated can happen later.
- Move all cpt files to browser views
- Merge Products.PasswordResetTool into CMFPlone
- Migrate old test cases from PloneTestCase to plone.app.testing.
- Rewrite login logic. There is no possible way to sanely migrate it
- Add new unit tests to most things, especially login
If time:
- Dig through the bug tracker and kill myriads of login bugs
Deliverables
- Sanity
- Examples on how to customize key forms (login)
- Documentation on how to restore old templates for those that need it. This assumes we will still ship with CMFFormController (I think we have to for other reasons) and that the deprecated folder will remain. Open to suggestions on this one.
- Documentation on all of the things that were removed for the upgrade notes
Risks
- This is very, very old stuff.
- This is a huge undertaking that will require a decent amount of effort from a lot of people.
- Many people customize several of these forms. Documentation on how to extend z3c for specific use cases will help a lot. Will solicit feedback from the community on what these major use cases should be.
- Other plips that will affect this include: #13283, #13270 - could be a nasty merge but already talking with davisagli about how to make it easier.
Requires #10888
Participants
- Encolpe Degoute
- Lukas Graf
- Gil Forcada
- Marcio Mazza
- Maurits van Rees
- Thomas Buchberger
- Timon Tschanz
- Suresh V
- Daniel Widerin
Progress
Work has started on this plip. To get started, check out the plip-13260-step-by-step branch and run
bin/buildout -c plips/plip13260-step-by-step.cfg
The old branches were just too old to migrate, to much changed in the last year. So we decided not to contiue work in old plip buildout: plips/plip13260-cpy-removal.cfg.
As you migrate something, please move it into plone_deprecated. Make sure to update docs/GOODBYE.rst so that writing docs in the end isn't aweful.
I am not sure the best way to organize all these tasks yet so I'll start detailing here. The repo at the moment should be the best tell of whats been migrated.
In general, everyone is welcome to grab the branch and hack it. For sanity:
- enclope will be in charge of code reviews.
- eleddy will test user interaction and try to herd cats
- we need someone to liason with the testing team and be a test czar.
- help with documentation later would be great! putting this in CHANGES.txt only will be impossible to document. Adding GOODBYE.rst to the branch for now, which will be converted to real documentation later. If people already want to
Still to be done, by folder in skins (yell if I'm wrong on any of this). Please place your name next to something if you are tackling it!::
- /plone_3rdParty: this is deleted by #12453. MatthewWilkes will handle repackaging in #13270 of presentation view stuff.
- /plone_ecmascript: this is a bit of a hole. some of this will be deprecated (e.g. kss) but in general it should all move to /browser/javascript with a proper profile and upgrade step.
- /plone_form_scripts:
- Migrating site_feedback to z3c.form would be a big win for integrators. NOTE: Already migrated to browserviews during Wine sprint (but not yet z3c.form).
- delete_confirmation.cpy[.metadata] is confirmation from deleting an item. Migrate. Also migrate the pts from plone_forms.
- selectViewTemplate.cpy[.metadata] needs migrated
- validate_*: 1 by 1 needs a lookup but these should all be artifacts of other form controller shit. Migrating the rest then seeing whats here and *NOT* being invoked from plone_deprecated should turn up 0. Still need to check.
- /plone_forms:
- delete_confirmation_page.cpt[.metadata] - migrate with stuff from plone_form_scripts (above)
- folder_rename_form.pt: direct translation. still used in many places
- member_search_*: migrate.
- undo_form: migrate
- /plone_images: oh boy. this is for someone with a stomach. I am not even sure if this is possible or right for this plip... BUT.... There are still clients and places in core which rely on acquisition to get to images. This is a performance mess and bad practice. It's been slowly cleaned up. Validate that all references to icons go through ++resources++ from portal root in core. garbas is working on replacing these with css icons in mockup/p.a.toolbar and working with him seems like a great start here.
- /plone_login: this will be rewritten. direct translation is insane. eleddy working on this as an experimental add on to start.
- /plone_prefs: migrate to standard control panel
- /plone_styles: move to browser/stylesheets, upgrade step to fix location if needed
- /plone_templates: everything in here is a simple translation to a browser view.
- /plone_wysiwyg: This is still required for tinymce. need to make sure the translation doesn't break that. I'm surprised this isn't done by adapter or something.
Last but not least, unregister all plone_skins directories that were affected. Make sure there are instructions on how to quickly re-register.
Major Note: I'm not hell bent on actually removing the "skins" folder if moving things into browser doesn't make sense. However, I think the standard for most addons and projects is things like images/js/css should be in the browser folder and thats where people look for it. If we don't have to explain skins anymore to newbies all the better combined with the fact of having CMFPlone itself be a good example of how to make a plone egg would be great but is not required. Open to arguments on leaving css/js/images there. Either way, they should take lesser priority than actual cpt/cpy conversion.
Change History
comment:3 Changed 3 years ago by maurits
We started work on this at the Ploneconf 2012 sprint. From Encolpe Degoute's recent email to the conference list:
The main goal was to move all cpy/cpt/vpy out of Products.CMFPlone. The destinations were plone.app.content, plone.app.users and the browser folder of Products.CMFPlone for generic views.
We hit some difficulties with a mixin of PloneTestCase and plone.app.testing. At last we choose to stay with PloneTestCase for plone.app.content. And also with missing functional tests. We need to do some more works on this point before the next step. David Glick ask us to make extra check on forms attacks that we didn't made.
The next step is to make a review of the code of the plip before to start it again in a new sprint.
We identify some other packages where we can start to work on this migration :
plone.app.controlpanel plone.app.kss (<= until its removal)
You can find formcontroller files in all products using this command in a shell :
find src -path "*deprecated" -prune -o -name "*\.?p[y|t]" -print
To work on this topic choose the 4.3 branch on buildout.coredev and use the plips/plip13260-cpy-removal.cfg file as buildout configuration file.
bin/buildout -c plips/plip13260-cpy-removal.cfg
comment:4 Changed 3 years ago by eleddy
- Status changed from new to confirmed
- Cc encolpe added
- Description modified (diff)
comment:5 Changed 3 years ago by eleddy
- Cc phgross, marciomazza, gforcada, buchi, tschanzt added
- Description modified (diff)
comment:6 Changed 3 years ago by eleddy
For reference from gil for later: For the ones working on moving the login infrastructure out of cpy files I just pushed this: https://github.com/gforcada/login_forms which contains all forms moved to z3c.form.
comment:7 Changed 3 years ago by timo
There is a related PLIP for p.a.controlpanel already: https://dev.plone.org/ticket/10359
comment:14 Changed 3 years ago by eleddy
- Milestone changed from Future to 4.4
- Description modified (diff)
- Summary changed from Move core CMFFormController scripts to views to Remove CMFFormController dependency from Products.CMFPlone
comment:15 Changed 3 years ago by eleddy
- Priority changed from minor to major
- Component changed from Unknown to Backend (Python)
- Description modified (diff)
comment:18 Changed 3 years ago by sureshvv
will take gforcada's login_forms and fold them into Products.CMFPlone under a separate folder called login_forms on the 4.3 branch.
comment:19 Changed 3 years ago by sureshvv
@gforcada:
Where is example.membership located?
comment:20 Changed 3 years ago by gforcada
@sureshvv:
unfortunately nowhere to be found :S
That's part of our freitag.de deployment, but we have it so tied to our own needs that it didn't make sense to just push the whole thing together. We do a double-pass registration (first ask the mail and then the registration), we also log the user IP (as there is community based discussions and so on) and we even added a third registration step (after registration giving them a chance of getting a free issue of the newspaper).
So as you see, all this does not make sense to be within Products.CMFPlone. All references to example.membership though are just interfaces which are z3c.form schema based ones, so you just need to (re)define Plone's default fields and that should be it.
Still, let me know if there's something not clear enough in the code (gforcada AT gnome DOT org).
comment:21 Changed 3 years ago by toutpt
- Description modified (diff)
I have updated the ticket about plone_3rdParty skin directory, it will be deleted by #12453.
comment:22 follow-up: ↓ 23 Changed 3 years ago by thet
regarding the plone_deprecated skins folder: why not moving this one to a dedicated core-package, say: plone.app.deprecated ?
this would make a removal of deprecated stuff in the future even easier and people who really need it can later depend on it.
EDIT: plone.app.deprecated could take over any deprecated skin layer now and in the future and provide backwards compatibility by just depending on this package.
we can also skin layers to de/activate specific BBB skin folders.
comment:23 in reply to: ↑ 22 Changed 3 years ago by eleddy
Replying to thet:
regarding the plone_deprecated skins folder: why not moving this one to a dedicated core-package, say: plone.app.deprecated ?
this would make a removal of deprecated stuff in the future even easier and people who really need it can later depend on it.
EDIT: plone.app.deprecated could take over any deprecated skin layer now and in the future and provide backwards compatibility by just depending on this package.
we can also skin layers to de/activate specific BBB skin folders.
I'm game for this. Great idea.
comment:24 Changed 3 years ago by davisagli
- Milestone changed from 4.4 to 5.0
The framework team has approved this for Plone 5.0.
comment:25 follow-up: ↓ 27 Changed 3 years ago by toutpt
I would be glade to join this plip.
To be sure, you want to remove skin directories not only cmf forms, right ? In this case, could we update the plip title please ? It's confusing.
I have checkout the plip and I don't understand why we find in the default profile skins.xml:
<skin-path name="Basic"> <layer name="custom"/> <layer name="zpt_content"/> <layer name="zpt_generic"/> <layer name="zpt_control"/> <layer name="Images"/> </skin-path>
comment:26 Changed 3 years ago by eleddy
- Summary changed from Remove CMFFormController dependency from Products.CMFPlone to Remove portal_skins from Products.CMFPlone
comment:27 in reply to: ↑ 25 Changed 3 years ago by eleddy
Replying to toutpt:
I would be glade to join this plip.
To be sure, you want to remove skin directories not only cmf forms, right ? In this case, could we update the plip title please ? It's confusing.
Updated. Originally it was just cpy but hey, why not.
I have checkout the plip and I don't understand why we find in the default profile skins.xml:
<skin-path name="Basic"> <layer name="custom"/> <layer name="zpt_content"/> <layer name="zpt_generic"/> <layer name="zpt_control"/> <layer name="Images"/> </skin-path>
Voodoo. Its a Plone 5 release so I would say remove it and see what happens? Definitely ZMI stuff.
comment:31 Changed 3 years ago by esteele
- Description modified (diff)
select_default_view has been migrated to plone.app.content
comment:32 Changed 3 years ago by thet
this is related: #13544 https://dev.plone.org/ticket/13544
"prefs_install_products_form shows INonInstallable profiles"
comment:36 Changed 3 years ago by eleddy
Just had a long discussion aout how this package is integrated and how to better prepare people for it. Many of these changes could be used now without issue, meaning they could be tested/interated/customized ahead of the actual release. With this notion I will work on making a future package where it makes sense and migrating changes if it makes sense. Will update as it happens here.
In addition, if people are using future packages they will depending on the imports so before release create a script that looks at the code and warns of references to packages that haven't been merged yet and give warnings to update. Integrators can't be expected to understand how to use grep so this will make a nicer transition.
comment:37 Changed 3 years ago by eleddy
We should make this script available ahead of time to all add on integrators and run it on their code for them. Easier for cpy and cpt, harder for css/js changes. We'll need a great how to before and how to after.
comment:38 Changed 3 years ago by eleddy
In addition to adding tests, documentation will be set up soon so that each method has old vs new way to call it, and the script is updated to indicate issues in code and rovide suggestions.
comment:39 Changed 3 years ago by esteele
The Framework Team has decided to move on to Plone 5. Updating milestones accordingly.
comment:40 Changed 3 years ago by frapell
Does anybody knows the status of this and if there is someone working on it ?
comment:41 Changed 3 years ago by eleddy
I haven't worked in the last 2 months but will pick up on it when relieved from board duties. Do you want to help? Probably next sprint will bring more to fruit
comment:42 Changed 3 years ago by frapell
unfortunately I can't at the moment... I was assigned as champion of this ticket by the FWT, so I was checking status...
I think we should definitely try to push this to be worked in any of the upcoming sprints, since this is a blocker for Plone 5.
comment:43 Changed 2 years ago by eleddy
Hi everyone - sorry for the silence on this for... forever. It is basically impossible to keep track of this plip as is so I moved just aboute every remaining task to github issues as small succinct pieces. Each issue should be associated with a pull. I think that some of these issues have been done but I've lost track of the work so hopefully this will get things better organized. For all the pieces detailed, please see:
https://github.com/plone/Products.CMFPlone/issues?labels=PLIP+13260%3A+Remove+CMF&state=open
I still need to make issues for a few things but thats more than plenty to move forward. It's all on master as pull requests now. That should make things easier. Thanks to bloodbare for spearheading that!
comment:44 Changed 2 years ago by encolpe
bloodbare brief me on that today during the Plone 5 sprint. We will push this plip some step further.
comment:45 Changed 2 years ago by saily
- Cc saily added
- Description modified (diff)
hi, during wine and beer sprint we've pushed this forward. We migrated some code of the cpy-removal branch into step-by-step plip. 13260-cpy-removal is now obsolete. Currently we have really big problems with plone.app.linkintegrity package because it relys on IReferenceablebehavior which is not activated by default.
Done are:
- cut/copy/paste actions
- delete/delete_confirmation actions
- site_feedback
- select default page/view
please see the plip/plip13260-step-by-step.cfg buildout for details, there were some additional packages involved. Hope to get a final pull this weekend. for any questions, please ask.
saily was team leader of the cleanup team in munich.
comment:46 Changed 2 years ago by saily
The major issue about that thing above is that plone.app.referenceablebehavior relys on reference_catalog and uid_catalog to be there, but they were decided to get removed.
So how should we continue working on that?
We have to be sure all addons run tests again dexterity types. currently each package builds it's own testlayer, some using collective.testcaselayer, PloneTestCase, others plone.app.testing. Most of them install Archetypes types and run tests on them, but we need tests to run on both contenttypes - since we want to support both in Plone 5.
comment:47 Changed 21 months ago by seanupton
Anyone know what the status is for replacing portal_skins/plone_scripts/check_id is? Without this, the name chooser will check against OFS and CMF _checkId() methods, which will not prevent users from putting spaces in shortname/ids/. I would like to solve this for rename, but whatever validation is done for rename vs. creation should be common, IMHO.
Ref: https://github.com/plone/Products.CMFPlone/issues/97#issuecomment-48767214
Would it be reasonable to implement this on plone.app.folder.base.BaseBTreeFolder._checkId (which name chooser would use on creation) and then have the folder-contents (1..* items) and action-based (single item) rename components call as well?