Ticket #14261 (closed PLIP: fixed)

Opened 21 months ago

Last modified 17 months ago

CSS and JS integration/ResourceRegistries rethink (require.js/mockup/less...)

Reported by: ramon Owned by:
Priority: minor Milestone: 5.0
Component: Unknown Version:
Keywords: Cc:

Description (last modified by vangheem) (diff)

Proposer: Ramon Navarro Bosch Seconder: Nathan Van Gheem

Abstract

With the actual js history on plone 4.3 - raw js and css on js/css registry - plus the mockup project that created a separated from plone solution for patterns/widgets we have a missing step on how the integration is going to be. This PLIP proposes a way to integrate modern js/css solutions with the actual Plone infrastructure. The main idea is to implement a requirejs/less* integration that makes easy to register patterns. It's also the intention to modernize the js/css registry.

Motivation

In order to make easy integration of mockup with plone and allowing front-end projects to be easily developed using plone it's necessary to find an easy solution that enables js/css development process to be close to plone. Currently, it's rather difficult for anyone to integrate their own mockup related integration.

Assumptions

It's assumed that by default plone will support LESS and requirejs frameworks because mockup is using them. The proposal doesn't prevent to implement a solution for SASS with the same idea or any other css/js framework as it's going to be adaptable.

As we don't want to assume that it's needed node for running plone on development we want to find a solution that does not depend on any node technology.

Proposal & Implementation

We propose that we modernize the concepts provided in Resource Registries. With consideration of the fact that most js components, or as we can call them, patterns, consist of one or more js AND css(or less) files coupled together, we propose combining the portal_css and portal_javascripts into one registry, a pattern registry.

Most of the functionality of portal_css and portal_javascript will still be used; however, patterns can depends on other patterns and all the dependencies are resolved and rendered in the correct order. For each bundle we can generate a config.js that will be used to configure where all the dependencies are going to be find on development AMD loading. In case we want to generate a production js using the r.js compiler we can create different bundles of js optimized that are going to be stored for cache proposals.

Enhancing the actual registry's css features with the option to define less dependencies and enabling the option to compile the less file on the browser(if lessc is installed). That will allow to define a mixins.less with the urls of the different resources to load a decide to compile them as less on browser, compiled css or production css.

Also, patterns will be able to couple json configuration with them. There will be a simple way to render this json config for a pattern so we are able to use it to configure widgets and other parts of the site.

Existing js/css will be migrated to the new registry. A stub portal_css/javascripts will be provided for backward compatibility. Also, the old portal_css.xml and portal_javascripts.xml will still work for now, importing those assets into the new patterns registry.

Last, a control panel will be implemented to make it more easy for admins to manage and configure registered patterns.

Deliverables

  • A branch of CMFPlone and ResourceRegistry

Risks

Participants

Ramon Navarro Bosch <bloodbare> Victor Fernandez <sneridagh> Nathan Van Gheem <vangheem@…> Rob Gietma <rob.gietema@…>

Progress

  • JS registry integration is done for development with requirejs

Change History

comment:1 Changed 21 months ago by ramon

  • Type changed from Bug to PLIP
  • Milestone changed from 4.x to 5.0

comment:2 Changed 21 months ago by thet

I'm not sure, if I understood it in every detail. This PLIP seems to make it easier to manage existing JS/CSS resources. But doesn't this PLIP remove much of the goals mockup was addressing?

  • node.js shouldn't be seen as a hurdle for doing frontend developing. It's just a dependency of the state-of-the-art frontend developing toolchain Grunt and Bower. I don't see a point in not using it.
  • The seperation of frontend development from Plone backend is another great feature, mockup gives us. With binding the frontend again tighter to Plone, like in pre Plone 5, we are loosing the ability of involving people, who do not have any Plone core development experience.

I like mockup's bundling concept. Actually, I think we need mockup's development environment to catch up again with recent frontend development standards.

comment:3 Changed 21 months ago by ramon

It's not the goal of the PLIP to remove much of the goals mockup was addressing, its finding a good integration of mockup in Plone, extending mockup PLIP. In my opinion mockup plip was so generic and big so I addressed this issue on a separate PLIP so it can be discussed.

Mockup is great for frontend development, but there are two issues that shouldn't be addessed on mockup:

  • bundling all the plone css and js to a unique file to integrate on Plone, that leads to a lot of problems when you are developing add-ons. It's really hard to debug, integrate or expand, if you want to create your own modal, if you want to integrate with some other js or you want to extend its css right now is a nightmare.
  • on the css world having a unique css for all the css on Plone it's hard to maintain and means that everybody needs to rely on mockup css (so using bootstrap and less).

All the modifications I did for doing the PLIP didn't touch any file at Mockup, and what should be changed there is more a matter of urls and defining a mixins.less with the url of all the dependencies that can be generated on Plone in case you want to load the resources.

There are some other issues that I would like to see happening, cleaning the mockup repo ( right now its so big )

Don't get me wrong, I love grunt and bower and it's the correct aproach for front-end development, patterns is a great idea and I love it, but I think that mockup bundle-story make so magic ( pushing the resource files to plone, bundle with a lot of things,... )

comment:4 Changed 21 months ago by davisagli

I like that this is working on the question of how add-ons and themes can supply javascript that depends on Plone's core mockup. That's something that we haven't done a good job at planning.

But there are some specific things I'm not sure about the implementation:

  • I don't like that the requirejs configuration gets specified in XML. It means that even if someone already knows about configuring require.js, they have to learn something new. Can't we configure it with the same JSON as in config.js?
  • I don't understand why you moved the resources into CMFPlone instead of keeping them in (a branch of) mockup.
  • It's not clear to me what the process is for creating bundles, testing them, and serving them on a production server. Can you describe the steps that a developer would take to do this?

comment:5 Changed 21 months ago by ramon

Hey david,

Yea, we can configure a config.js, my initial goal was to redo the resourceregistry with plone.registry so its easy to import a config.js as it is and makes easier for people to understand it. Throwing resourceregistry to plone.registry is a huge task, and means creating some new infraestructure, so I was afraid it's too ambicious.

About moving the components:

  • adding the bower.json on CMFPlone sets a bunch of default js that is needed for Plone to run, so its clear for integrators which js you have and which versions
  • in mockup we have a lot of js that is used for testing mockup and grunt
  • I really want to have the bower components downloaded on the plone repository (or mockup), it doesn't make sense to download everytime you need it ( and without phantomjs and other testing js it's lighter)

I didn't wanted to touch anything on mockup repository for not breaking anything there that works.

So creating a bundle :

as you have js/css on the register I created a view optimize_js that generates a bundle for all the js you have on your registry (it automatic sets the non-require-js to be loaded after requirejs finish). It uses r.js on the browser, but I would prefer to use node, meaning that we should have node on your development machine. On the css side, we can register less (that gets compiled one by one to css so we should use (reference)) or css. On the css side we can use the bundling cache option of resourceregistry.

One possible option for serving the bundle is to store on customize static folder so it can be rewritten, I have no strong opinion on this side.

At the end a developer would register it's js/css, with its dependencies, its exports. A control panel on plone would allow you to see the js/css linked by the component name, so you can disable a module with its css, and some action there would allow you to create a unique css/js file you can use on diazo and is stored for usage on production mode (another switcher on the control panel). This is not implemented because I don't see if it's a good enought solution.

comment:6 Changed 21 months ago by thet

Still I'm not convinced. That would shift a lot of development focus from mockup to a completly new concept.

I'm OK with the mockup development process, as far as I have worked with it. I see it as an advantage that the management of JS and CSS resources has been moved out of Plone. This way, we can use the whole modern JS and CSS toolchain. With the seperation of mockup-core, add-on developers should be able to create completly seperated bundles while still depending on mockup-core or even mockup. If they cannot use mockup, they have still the option to use the ResourceRegistry as it is. For easier development, we can the original resources, which are not uglified or minified. How to automatically use these when Zope starts in development mode, is another story we might have to think about.

If we use plone.registry instead of P.ResourceRegistry, we need a UI that sucks less (I admit, that would have some advantages like different resources per context, etc.). The current resource management screens are OK, but don't have the best usability. The current plone.registry management screen is OK for simple settings, but not for resources which group a lot of settings per resource.

comment:7 Changed 21 months ago by ramon

There are some issues with the actual concept I'l love to point :

  • If you have a bundle (imagine plone bundle) and you are doing your theme (imagine barceloneta theme with its js), then you need to create a unique bundle with all the js from the plone bundle and your js ? but if you want to install an add-on that have it's own js you also need to integrate that js on your bundle. Imagine you have a site with a lot of products, each one with some js, pattern, ... then how is supposed to work for an integrator ?
  • Imagine that you need to use a modern bootstrap version, pat-modal has the .modal namespace and it will breake everythink, so you want to remove the pat-modal or adapt just it as an integrator, how is supposed to manage this situation?

I've been facing a lot of situations (now if you want to use the add-on wildcard.foldercontents with plone.app.widgets on 4.3 you get jquery 2 or 3 times, plus different css bootstrap versions on the same page), on plone 5 if you check for some bootstrap css you have it 3 or 4 times loaded on the site.

In development mode you can't have inline scripts, old-fashion scripts because they are not require and js is not loaded, you need to support less compilation on browser....

In my opinion all this situations an integrator or somebody that loves js may end crazy.

The situation of a css/js out of plone solution is lovely if you control everything, as an idea I love it, and I'm not suggesting that mockup idea is bad, development of js/css on plone should and must be done on a js/css enviroment so you can test, develop, and use the modern js/css tools, what I'm suggesting is that once you have your js/css code that works you need to integrate it with a lot of other addons/plugins/frameworks and there is where I think that registration should be done on plone so bundles are generated on plone, and development mode is less painfull.

comment:8 Changed 21 months ago by thet

hmhm... I'm getting you point.

I think this problems can be solved like so:

1) Addons depend on mockup, jQuery, Bootstrap, whatever they need in their bower.json. They depend on them via require js in their JavaScript code like normal. But they don't include it in their grunt task. This grunt task creates a bundle targeted for Plone, but without the common assets already included by mockup or barceloneta. Addons register their bundle via GenericSetup for the portal_registry/portal_css and include it after Plone's main dependencies (mockup and barceloneta). I haven't tried this out, but as long as require.js keep it's registry in global namespace (which it does, AFAIK), this could work. Developers who don't want to use require.js can still develop their JavaScript the old way and register it via the portal_javascript registry (jQuery and the like are still available without require.js, or not?).

2) If we want to replace pat-modal or customize it, we may need to make sure, that the mockup registry supports overloading or extension points.

I have much less experience with the mockup stack than you, Ramon. I'm currently getting my way into it. Therefore this answers are naive and maybe even don't work out.

I agree, we really have to avoid to deliver libraries twice, because the payload is already quite high. And we have to make sure to keep Frontend developing easy for all experience levels.

I'm looking forward to hear more opinions on this. And I'm looking forward to work on this at the Zidanca sprint!

comment:9 Changed 20 months ago by vangheem

  • Description modified (diff)
  • Summary changed from CSS and JS integration (require.js/mockup/less...) to CSS and JS integration/ResourceRegistries rethink (require.js/mockup/less...)

comment:10 Changed 20 months ago by vangheem

  • Description modified (diff)

comment:11 Changed 17 months ago by ramon

  • Status changed from new to closed
  • Resolution set to fixed
Note: See TracTickets for help on using tickets.