Ticket #11310 (confirmed Bug)

Opened 5 years ago

Last modified 4 years ago

Exception entering non-ASCII chars as content item's short name/ID

Reported by: eallik Owned by:
Priority: major Milestone: 4.x
Component: General Version: 4.1
Keywords: unicode, tuneup Cc: eallik, neaj, frisi

Description

When renaming a content item to contain non-ASCII characters (to get Unicode URLs for better SEO), instead of a validation error message, I get:

Traceback (innermost last):
  Module ZPublisher.Publish, line 127, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 47, in call_object
  Module Products.CMFFormController.FSControllerPageTemplate, line 91, in __call__
  Module Products.CMFFormController.BaseControllerPageTemplate, line 26, in _call
  Module Products.CMFFormController.FormController, line 384, in validate
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 47, in call_object
  Module Products.CMFFormController.FSControllerValidator, line 58, in __call__
  Module Products.CMFFormController.Script, line 145, in __call__
  Module Products.CMFCore.FSPythonScript, line 130, in __call__
  Module Shared.DC.Scripts.Bindings, line 324, in __call__
  Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec
  Module Products.PythonScripts.PythonScript, line 344, in _exec
  Module script, line 30, in validate_folder_rename
   - <FSControllerValidator at /news/validate_folder_rename used for /news/uudised>
   - Line 30
  Module Products.CMFCore.FSPythonScript, line 130, in __call__
  Module Shared.DC.Scripts.Bindings, line 324, in __call__
  Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec
  Module Products.PythonScripts.PythonScript, line 344, in _exec
  Module script, line 67, in check_id
   - <FSPythonScript at /news/check_id used for /news/uudised>
   - Line 67
  Module script, line 31, in xlate
   - <FSPythonScript at /news/check_id used for /news/uudised>
   - Line 31
  Module Products.CMFPlone.TranslationServiceTool, line 49, in translate
  Module zope.i18n, line 126, in translate
  Module zope.i18n.translationdomain, line 78, in translate
  Module zope.i18n.translationdomain, line 140, in _recursive_translate
  Module zope.i18n, line 164, in interpolate
  Module zope.i18n, line 159, in replace
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

Change History

comment:1 Changed 5 years ago by eallik

  • Cc eallik added

comment:2 Changed 5 years ago by eallik

Also, what is the simplest way to add Unicode URL support to Plone?

comment:3 Changed 5 years ago by hannosch

You should indeed get a validation error here and not a traceback.

But there's no easy way of using non-ascii ids/urls in Plone. The entire Plone / Zope 2 stack doesn't support it. So it's going to take a major effort to update the entire stack and quite some time before we can roll this into any official release.

comment:4 Changed 5 years ago by neaj

  • Cc neaj added

comment:5 Changed 4 years ago by kleist

  • Component changed from Unknown to Infrastructure
  • Milestone set to Ongoing

comment:6 Changed 4 years ago by frisi

  • Priority changed from minor to major
  • Cc frisi added
  • Milestone changed from Ongoing to 4.2

object_rename and folder_rename_form should really not resut in a site-error in case users provide non ascii characters for short names. i'll set the milestone to plone4.2 so the ticket gets more attention.

comment:7 Changed 4 years ago by kleist

  • Version set to 4.1

Seems to be fixed in the 4.2 coredev buildout. I get a proper error message when trying to rename the id of a page to "åäö":

åäö is not a legal name. The following characters are invalid: åäö

comment:8 Changed 4 years ago by kleist

  • Status changed from new to confirmed
  • Component changed from Infrastructure to General

Should the fix (wherever it is) be backported to 4.1.x?

comment:9 Changed 4 years ago by frisi

yes, i think it should be backportet.

the check is done in the script check_id  https://github.com/plone/Products.CMFPlone/blob/4.2/Products/CMFPlone/skins/plone_scripts/check_id.py#L80

and can be easily backportet in the next tuneup

plone 4.1:

# check for bad characters
plone_utils = getToolByName(container, 'plone_utils', None)
if plone_utils is not None:
    bad_chars = plone_utils.bad_chars(id)
    if len(bad_chars) > 0:
        return xlate(
            _(u'${name} is not a legal name. The following characters are invalid: ${characters}',
                mapping={u'name': id, u'characters': ''.join(bad_chars)}))

plone 4.2

# check for bad characters
plone_utils = getToolByName(container, 'plone_utils', None)
if plone_utils is not None:
    bad_chars = plone_utils.bad_chars(id)
    if len(bad_chars) > 0:
        pprop = getToolByName(context, 'portal_properties')
        charset = getattr(pprop.site_properties, 'default_charset', 'utf-8')
        bad_chars = ''.join(bad_chars).decode(charset)
        decoded_id = id.decode(charset)
        return xlate(
            _(u'${name} is not a legal name. The following characters are invalid: ${characters}',
                mapping={u'name': decoded_id, u'characters': bad_chars}))

comment:10 Changed 4 years ago by kleist

  • Keywords unicode, tuneup added; unicode removed
Note: See TracTickets for help on using tickets.