Ticket #12871 (confirmed Bug)

Opened 4 years ago

Last modified 3 years ago

plone.app.discussion + plone.app.async = ZODB.POSException.StorageTransactionError

Reported by: gyst Owned by: tisto
Priority: minor Milestone: 4.x
Component: Commenting/Discussion Version: 4.1
Keywords: plone.app.async Cc: vangheem

Description

Using plone.app.discussion in combination with plone.app.async breaks on transaction errors. Full traceback below.

Intent

plonesocial.microblogging uses plone.app.discussion adaptation on the Plone SiteRoot, to store microblogging status updates in a Conversation container that is set as annotationstorage on the siteroot.

In order to improve scalability and developer flexibility, the p.a.discussion adapter factory is replaced with a custom plonesocial.microblogging.conversation adapter (for the siteroot).

This override intercepts the .addComment(comment) call made from the browser form handler. On the user facing instance, with plone.app.async installed, it will schedule this for execution on the async worker instance. The worker instance receives the call and then propagates it to the plone.app.discussion .addComment method.

Adding a status update in this setup triggers the error described below.

The whole call flow has been extensively tested and validated, both in async mode with pdb inspection and in sync mode with immediate execution on the user facing instance.

Doing an async delegation of a similar but simple action, by annotating the SiteRoot with a persistent dummy object, works just fine. Only when the plone.app.discussion addComment handler is integrated into the callflow do we get transaction errors.

Bisecting plone.app.discussion.conversation.addComment shows multiple sources that trigger this error. Any BTree manipulation will trigger it. But even with all BTree manipulations disabled, the error still occurs.

What makes this hard to debug, is that the exception is raised only after addComment has returned, when plone.app.ascync.service does a transaction commit. It's completely opaque to me where the conflicting tpc_begin is coming from.

Traceback:

2012-05-04 11:24:43 INFO zc.async.trace starting in thread 140536691984144: <zc.async.job.Job (oid 55934, db 'main') ``plone.app.async.service._executeAsUser(('', 'Plone'), ('', 'Plone'), ('', 'acl_users'), 'admin', plonesocial.microblog.conversation.addStatus, plone.app.discussion.comment.Comment (oid 55952, db 'main'))``>
2012-05-04 11:24:44 ERROR zc.async.events <zc.async.job.Job (oid 55934, db 'main') ``plone.app.async.service._executeAsUser(('', 'Plone'), ('', 'Plone'), ('', 'acl_users'), 'admin', plonesocial.microblog.conversation.addStatus, plone.app.discussion.comment.Comment (oid 55952, db 'main'))``> failed with traceback:
*--- Failure #6 (pickled) ---
/home/gyst/cachebuildout/eggs/zc.async-1.5.4-py2.6.egg/zc/async/job.py:597: __call__(...)
/home/gyst/cachebuildout/eggs/plone.app.async-1.2-py2.6.egg/plone/app/async/service.py:63: _executeAsUser(...)
/home/gyst/cachebuildout/eggs/Zope2-2.13.13-py2.6.egg/Zope2/App/startup.py:301: commit(...)
/home/gyst/cachebuildout/eggs/transaction-1.1.1-py2.6.egg/transaction/_manager.py:89: commit(...)
/home/gyst/cachebuildout/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py:329: commit(...)
/home/gyst/cachebuildout/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py:441: _commitResources(...)
/home/gyst/cachebuildout/eggs/ZODB3-3.10.5-py2.6-linux-x86_64.egg/ZODB/Connection.py:551: tpc_begin(...)
/home/gyst/cachebuildout/eggs/ZODB3-3.10.5-py2.6-linux-x86_64.egg/ZEO/ClientStorage.py:1116: tpc_begin(...)
 [Capture of Locals and Globals disabled (use captureVars=True)]
ZODB.POSException.StorageTransactionError: Duplicate tpc_begin calls for same transaction
*--- End of Failure #6 ---

2012-05-04 11:24:44 INFO zc.async.trace <zc.async.job.Job (oid 55943, db 'main') ``plone.app.async.service.job_failure_callback()``> succeeded with result: None
2012-05-04 11:24:44 INFO zc.async.trace completed in thread 140536691984144: <zc.async.job.Job (oid 55934, db 'main') ``plone.app.async.service._executeAsUser(('', 'Plone'), ('', 'Plone'), ('', 'acl_users'), 'admin', plonesocial.microblog.conversation.addStatus, plone.app.discussion.comment.Comment (oid 55952, db 'main'))``>

Steps to reproduce

git clone https://github.com/cosent/plonesocial.buildout.git
cd plonesocial.buildout
make async
bin/zeo start; bin/instance start; bin/worker fg

point your browser at  http://localhost:9050/manage

username: admin, password: admin

add Plone site, select add-on 'plonesocial.suite'

Submit a comment on the homepage.

Worker console shows the traceback cited above.

Change History

comment:1 Changed 4 years ago by gyst

Update: I've just tagged the whole bundle as "0.2" and released the stuff as 0.2 on pypi. That way you can easily do the reproduction above, even if I should break stuff in newer commits.

comment:2 Changed 4 years ago by eleddy

  • Cc vangheem added
  • Keywords plone.app.async added; discussion, async removed
  • Component changed from Unknown to Commenting/Discussion

Man, this is a pretty specific bug. Do you think that plone.app.discussion is the source of this error?

comment:3 Changed 4 years ago by kleist

  • Status changed from new to confirmed

comment:4 Changed 4 years ago by vangheem

I just tried this out and can confirm the issue happening. I don't have the slightest idea what is causing the problem though yet.

The part that causes the issue in plone.app.discussion is:

self._comments[id] = comment

in the addComment method of the Conversation in conversation.py

comment:5 Changed 3 years ago by eastxing

It's an issue of plone.app.async. I found in your "async job function", calling 'transaction.commit()' before any ZODB changes will get rid of the issue.

Note: See TracTickets for help on using tickets.