Plone

RedTurtle: 2010 Content Technology Vendor Map

Posted 14 days ago

Webmeisterei: Searching the ideal commenting product for Plone

Posted 14 days ago

We plan to add commenting infrastructure to a portal currently running on plone 3.3.x so I did some research and came up with a (probably uncomplete) list of candidates that will be described below in more detail.

iqpp.plone.commenting (aka ... [More] EasyCommenting)
collective.disqus
quintagroup.plonecomments (aka qPloneComments)
collective.discussionplus
plone.app.discussion

Hope this will kick off a discussion with the product authors / people using them and will help others to decide which package (not) to use.

Summary
I did not look really deep into all of these packages (installed and tested plonecomments and plone.app.discussion) so please feel free to comment and put me into the right direction.

collective.disqus looks promising and has great advantages in the area of social web.
on the other side comments are not as well integrated into plone (eg search) and it’s hard or even impossible? to implement custom notifications or other additional features within your application.

quintagroup.plonecomments is a very mature product with lots of configuration options and notification features.
Editing of comments is possible (maybe there should be a way to turn this evil? feature off).
Comments are compatible with standard plone comments (right?) so they might be migrateable to the new commenting system for plone 4.x (plone.app.discussion).

collective.discussionplus peps up plone’s standard discussion items with workflow while keeping them migrateable to plone.app.discussion later. it’s more leight-weight than plonecomments with the downside of no notifications and less configuration options.

plone.app.discussion is currently in alpha state but offers basic configuration options and a moderation view for batch managing comments. Its much more leightweight than standard plone comment types (plonecomments and collective.dicsussionplus are using). I don’t know how many people are contrubuting currently and i hope timo is still motivated after his plip did not make it into plone 4.0.

I’m not sure if it’s a good idea to use plone.app.discussion in alpha1 state on production servers (comment of the authors welcome). But it seems by using collective.discussionplus or quintagroup.plonecomments (not sure about that) one can migrate all comments with the migration script of plone.app.discussion as soon as there is a more stable release or it even is shipped with plone 4.x

This is just a quick overview, more details for each package follow below. There is a project on coactivate about commenting in plone, too. Founded in 2007 it currently seems to be not very active or completely up to date (eg products like disqus missing) but there are reviews of some other commenting products and a collection of commenting related usecases.

iqpp.plone.commenting
Seems unmaintained (has not been updated on pypi since october 2007), so i did not look into that further.

Martin mentioned in his blogpost on commenting that its comparable to PloneComments and he used it for a big deployment.

collective.disqus
homepage: http://plone.org/products/collective.disqus
demo: http://lichota.pl/blog/

Uses the webservice disqus.com/ so your comments are stored “somewhere in the cloud”.
There is an export for comments though, so you’re able to migrate back to plone comments with some coding.

Seems that you can only set one language in the account settings, so it might won’t work for multilingual sites.

Users can comment using their twitter, facebook, openid,… accounts and choose whether or not to get email notifications on new comments / replies. Comments can be posted on Social Networks with might generate more interest and therefore visits to your site.

Comments are centrally managed using disqus.com.

Since comments are not stored in your plonesite they are not searchable by default.

quintagroup.plonecomments
actively developed (latest release 4.1.1 nov, 2009) and already there for many years.

has it’s own api but pretends to internally use plone’s standard commenting types so migrating to plone.app.discussion might work.

captcha support via quintagroup.plonecaptchas (did not test that)

Configlet for configuring settings (eg activate moderation, or enable anonymous comments) and managing pending comments .

Users with role DiscussionManager can publish comments via publish button below the comments, and can access the configlet and review list (/prefs_recent_comments_form) too.

Notifications for new pending comments are only sent to one single address configured in the control panel.
After Managers have approved the comment users get a notification that their content has been commented and commenter that his comment got published. Commenters get notifications for replies to their comment too (these notifications can be turned on/off in configlet)

Logged in or respectively anonymous users can report spam comments. A configuration option enables an “abuse” button that opens a form users can submit with an optional comment why they consider this comment spam. An email is sent to the notification address.

collective.discussionplus
Martin’s interim solution to enable moderation of standard plone comments.

Uses DCWorkflow, adds indexes to catalog to search for comments of a certain author or the number of comments for a content item. Events are fired for comment creation and removal, so on can hook up email notifications on workflow transitions and/or these events.

In his blogpost on commenting in plone Martin outlined the reasons for not using EasyComment or PloneComments and writing discussionplus.

According to the readme it’s likely not to work with plone4.
However comments should be migrateable to plone.app.discussion since they’re standard plone comments.

plone.app.discussion
Google summer of code project outlined and mentored by martin and mainly developed by timo.

Aimed to be the new default commenting solution for plone 4.0 but the plip9288 has been rejected in october since the framework team considered it not finished enought.

project page: http://plone.org/products/plone.app.discussion/
tracker: http://www.pivotaltracker.com/projects/15135
online demo: http://gsoc.timostollenwerk.net/

For my research I used version 1.0a2 (rev31840) in the development buildout

Configlet to configure if anon comments are allowed, which captcha to use, whether or not to show the commenter’s image and a toggle to switch commenting off globally.

Moderation is configured in the types control panel by setting the workflow of the Comment type.

Captcha support via plone.formwidget.(re)captcha

Migration script for importing existing plone comments is in place (did not test it)

Moderating comments is done using a site action (@@moderate-comments)

Email notifications are not yet available, but planned for 2010 (according to the tracker) [Less]

Izak Burger: Varnish, Zope and Backend Checking

Posted 14 days ago

I sent this explanation about some trouble we had with varnish's backend probing to a client a while ago. The information is useful enough that it should be in a blog post.

David Glick: Recombining ZODB storages

Posted 14 days ago

I recently faced the task of joining back together a Plone site composed of 4 ZODB filestorages that had been (mostly through cavalier naïveté on my part) split asunder some time ago.

Normally I would probably just do a ZEXP export of each ... [More] of the folders that lived in its own mountpoint, then remove the mountpoints and reimport the ZEXP files into the main database. However, that wasn’t going to work in this case because the database included some cross-database references.

Some background: Normally in Zope, mountpoints are the only place where one filestorage references another one, but the ZODB has some support for *any* object to link to any other object in any other database, and this can happen within Zope if you copy an object from one filestorage to another. This is generally bad, since the ZODB’s support for cross-database references is partial — when you pack one filestorage, the garbage collection routine doesn’t know about the cross-database references (unless you use zc.zodbdgc), so an object might get removed even if some other filestorage still refers to it, and you’ll get POSKeyErrors. Also, in ZODB 3.7.x, the code that handles packing doesn’t know about cross-database references, so you’ll get KeyError: ‘m’ or KeyError: ‘n’ while packing.

Well, this is what had happened to my multi-database, and I wanted to keep those cross-database references intact while I merged the site back into one monolithic filestorage. So I ended up adapting the ZEXP export code to:

traverse cross-database references (the standard ZEXP export ignores them and will not include objects in different filestorages from the starting object),
traverse ZODB mountpoints (removing them in the process),
and rewrite all the oids to avoid collisions in the new merged database.

Here is the script I ended up with. If you need to use it, you should:

Edit the final line to pass the object you want to start traversing from, and the filename you want to write the ZEXP dump to.
Run the script using bin/instance run multiexport.py

"""Support for export of multidatabases."""

##############################################################################
#
# Based on the ZODB import/export code.
# Copyright (c) 2009 David Glick.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################

import logging
import cPickle, cStringIO
from ZODB.utils import p64, u64
from ZODB.ExportImport import export_end_marker
from ZODB.DemoStorage import DemoStorage

logger = logging.getLogger('multiexport')

def export_zexp(self, fname):
context = self
f = open(fname, 'wb')
f.write('ZEXP')
for oid, p in flatten_multidatabase(context):
f.writelines((oid, p64(len(p)), p))
f.write(export_end_marker)
f.close()

def flatten_multidatabase(context):
"""Walk a multidatabase and yield rewritten pickles with oids for a single database"""
base_oid = context._p_oid
base_conn = context._p_jar
dbs = base_conn.connections

dummy_storage = DemoStorage()

oids = [(base_conn._db.database_name, base_oid)]
done_oids = {}
# table to keep track of mapping old oids to new oids
ooid_to_oid = {oids[0]: dummy_storage.new_oid()}
while oids:
# loop while references remain to objects we haven't exported yet
(dbname, ooid) = oids.pop(0)
if (dbname, ooid) in done_oids:
continue
done_oids[(dbname, ooid)] = True

db = dbs[dbname]
try:
# get pickle
p, serial = db._storage.load(ooid, db._version)
except:
logger.debug("broken reference for db %s, oid %s", (dbname, repr(ooid)),
exc_info=True)
else:
def persistent_load(ref):
""" Remap a persistent id to a new ID and create a ghost for it.

This is called by the unpickler for each reference found.
"""

# resolve the reference to a database name and oid
if isinstance(ref, tuple):
rdbname, roid = (dbname, ref[0])
elif isinstance(ref, str):
rdbname, roid = (dbname, ref)
else:
try:
ref_type, args = ref
except ValueError:
# weakref
return
else:
if ref_type in ('m', 'n'):
rdbname, roid = (args[0], args[1])
else:
return

# traverse Products.ZODBMountpoint mountpoints to the mounted location
rdb = dbs[rdbname]
p, serial = rdb._storage.load(roid, rdb._version)
klass = p.split()[0]
if 'MountedObject' in klass:
mountpoint = rdb.get(roid)
# get the object with the root as a parent, then unwrap,
# since there's no API to get the unwrapped object
mounted = mountpoint._getOrOpenObject(app).aq_base
rdbname = mounted._p_jar._db.database_name
roid = mounted._p_oid

if roid:
print '%s:%s -> %s:%s' % (dbname, u64(ooid), rdbname, u64(roid))
oids.append((rdbname, roid))

try:
oid = ooid_to_oid[(rdbname, roid)]
except KeyError:
# generate a new oid and associate it with this old db/oid
ooid_to_oid[(rdbname, roid)] = oid = dummy_storage.new_oid()
return Ghost(oid)

# do the repickling dance to rewrite references

pfile = cStringIO.StringIO(p)
unpickler = cPickle.Unpickler(pfile)
unpickler.persistent_load = persistent_load

newp = cStringIO.StringIO()
pickler = cPickle.Pickler(newp, 1)
pickler.persistent_id = persistent_id

pickler.dump(unpickler.load())
pickler.dump(unpickler.load())
p = newp.getvalue()

yield ooid_to_oid[(dbname, ooid)], p

class Ghost(object):
__slots__ = ("oid",)
def __init__(self, oid):
self.oid = oid

def persistent_id(obj):
if isinstance(obj, Ghost):
return obj.oid

export_zexp(app.mysite, '/tmp/mysite.zexp')

Download multiexport.py

I’ve used this script with apparent success, but it has not been extensively tested and your mileage may of course vary. [Less]

Blue Dynamics: Tasty Tasty Soup: cornerstone.soup

Posted 14 days ago

Yesterday I meet Robert to finish work on cornerstone.soup after our cologne alliance partner Sven found an odd bug. We simplified the soup recipe, but now its really yummy. Version 1.0 is released.

cornerstone.soup is a genric storage for ... [More] mass-data in an isolated container. Light-weight records are stored in an IOBTree. A Zope-Tool-Kit catalog in used to index values of interest. cornerstone.soup is no out-of-the-box package. Its addressed to developers needing to solve the problem of storing tiny entities of mass-data, where heavy weight archetypes or dexterity are too much effort and are to slow. I.e if you need a container for non-CMSish content, like votes, data from a poll, orders in a webshop, measuring data, or alike.

Isolated means, for each soup we store all in one persistent local utility. Stored are generic „Records“ which are persistent items with attributes passed in at construction time as keyword arguments. Also the catalog with its indexes is part of the local utility. Items are accessed by query.

We tested the soup with Zope 2.10 and Zope 2.12. It works in a plain Zope. We are using it at the moment in a Plone 3.3

For more information read the documentation at pypi: cornerstone.soup. [Less]

Reinout van Rees: Quotes

Posted 14 days ago

If I see a nice quote, I store it in a textfile somewhere. There've been a
few good ones lately, so here are some. From websites, blogs and mailinglist
messages. No guarantees about correct attribution, though I've tried to write
down ... [More] where I found it.

Culinary mayhem from my brother Maurits:

When dinner is calling, you need a rifle.
Talking about dinner:

It will be like loosing a dragon in the chicken coop.
Project management by Robert Bloch:

The man who smiles when things go wrong has thought of someone to blame it
on.
kcunning has a dire warning:

She will come out like a cat you've dunked in cold bathwater. It will only
be amusing for the person watching it on YouTube later.
Sane programming advice by Brian W. Kernighan:

Debugging is twice as hard as writing the code in the first place. Therefore,
if you write the code as cleverly as possible, you are, by definition, not
smart enough to debug it.
Elizabeth Warren about the financial bump in the road:

The reason banks lost confidence in each other is because they looked at their
own books.
Keep at it, keep at it. Paul Graham:

I've been surprised again and again by just how much more important
persistence is than raw intelligence.
Make noise, embarrass yourself, get famous, be happy. Martijn Faassen:

I'm a natural noise-maker, and I discovered that while as a result of this I
embarrass myself in public on a regular basis, it also means a lot of people
know who I am. That's a good thing.
In Dutch you have a saying "killing a mosquito with a canon". This is a nice
alternative:

It really feels like killing an ant with a sledgehammer.
ESR about the faking of climate data, specifically some of the comments in the
also-released source code:

This isn’t just a smoking gun, it’s a siege cannon with the barrel still
hot.
Chris McDonough about the need for documenting your software:

Document the shit out of it, like it was for your grandma. [Less]

Chris McDonough: My ZCA Thoughts Summary

Posted 14 days ago

Lately there's been some conversation on zope-dev about the future of the "zope.component" package, which is the package that provides "the Zope Component Architecture", more or less. I make some proposals here and take some positions.

Plone.org: Plone domain DNS changes

Posted 14 days ago

Switching DNS servers for Plone domain names.

Netsight Developers: Day Two of IMS 2009

Posted 14 days ago

This year we have been representing Plone at the IMS / Online 2009 expo at Earl Court Olympia in London.

After a great start with day one we were back to start day two of the expo. I went over to the venue a bit earlier today for a meeting ... [More] with Janus Boye of JBoye to talk a bit about the state of Plone and to bounce a quite interesting idea around, which hopefully we will reveal soon.

The main show floor then opened at 10:00 and people started coming to the Plone stand to chat about Plone. I was presenting a talk in one of the show floor lecture theatres on 'The Flexibility of Open Source: A Case Study of a Large Corporate Intranet' at 11:15, so Astra and Chris set about wandering the show floor to hand out more flyers we'd created for the show. One side of the flyers was the Flexible Plone advert we created, and the other side some promo about my talk.

My talk was the second talk of the day, and was preceded by a talk entitled 'Moving a large institution to an Open Source WCM system' by Richard Morgan from the Victoria & Albert Museum here in London. His talk focussed on the evaluation and procurement process they went through to eventually end up at a decision to use Squiz's MySource Matrix as a CMS. One of the key points he made was the importance of the relationship between the client and the vendor and how he really enjoyed the 'colocation days' they did (customer sprints in the Plone world). As it happened this turned out to be a great introduction to my talk, which was then taking the Open Source story a bit further and presenting a specific case study of a Plone based intranet for a large client of ours. My talk was packed to capacity with standing room left only, about 60 people in total.

The talk went really very well, and I even managed to keep to time pretty accurately. The talk described the relationship with one of our clients over the past three years and how their initial intranet has grown and evolved over that time to encompass a number of specific business processes.

Following the talk we had a rush on the Plone stand and were doing demos non stop for the next couple of hours until lunch time. As usual I'll be putting a Slidecast of my talk on Slideshare.net as soon as I've edited the audio.

After lunch thing quietened down a bit at the show, but by the end of the day we had handed out about 150 brochures over the first two days and spoken to even more people than we did yesterday.

We did have one quite amusing incident towards the end of the day, where some random guy decided to impart his aged wisdom to us and loudly proclaimed that we needed a 'gobble gobble' machine and needed to buy up all the competing CMS systems in the market (as there are too many to choose from) kill the competitors off and then sell stock options. I think in all his wisdom he had completely missed the point of Open Source. He told us we were wrong and that we needed a USP. I told him our USP was that by being Open Source, companies who spend time and money investing in using Plone as their CMS can rest safe in the knowledge that their CMS won't be bought out by a 'gobble gobble' machine and killed off by someone following his advice. He didn't get it. Oh well. I'm sure he'll catch up with the modern world one day.

Once the show closed at 5pm we headed over to the Hilton next door for the vendors drinks reception and caught up with the Squiz guys for a chat and some beers.

As we were wrapping up for the day, the Gilbane conference in Boston seemed to be revving up to full swing and pics were coming in of the Plone stand there. It was great to see the pic of the stand there and to see the new plone brochures on display. With Thanksgiving just last weekend, Nate, Ken et al must have worked incredibly hard to get them printed in time. As I headed back to my hotel room, checking twitter, a panel discussion between Plone, Drupal, Joomla! and Alfresco was just kicking off... go Plone!

Tomorrow is the last day of both IMS and Gilbane, so if you want to come along and find out more about Plone and you are in either London or Boston then please do come along for a chat. [Less]

Josh Johnson: JOB: Looking for Two Programmers

Posted 14 days ago

NOTE: All of the information in this post is to be considered unofficial and informal in nature. Be sure to read the actual postings carefully and apply through the proper channels.

I’m building a Python-centric, Plone-adjacent, web ... [More] application development team. To that end, I’ve been given two mid-tier Applications Analyst positions to fill. These have been opened, posted, and we are actively taking applications.

This is a “ground up” development team; we have some pre-existing Plone infrastructure and applications we want to integrate, but there’s opportunity for development with different web frameworks if they will meet our requirements more completely than Plone can (but we will still be running Plone for general web sites and will want to try to integrate as much as we can).

The infrastructure is being established right now (by Yours Truly), and is being designed to be highly available and low maintenance. It will be under the full control of the team.

My focus is on finding people who really enjoy application development, have strong communications and organizational skills, like working in a highly productive team environment, and don’t mind getting their hands dirty when it comes to hardware.

Another primary function of the team will be to produce publications. It will require us to be especially diligent in documenting everything we do. It also allows us to choose solutions with innovation in mind. This is a great opportunity to get very public recognition for your work, and forge new territory.

These positions are for the University of North Carolina at Chapel Hill, and will be administered through the Lineberger Comprehensive Cancer Center.

You can view the postings and apply for the positions on the UNC HR website:

https://s4.its.unc.edu/RAMS4/details.do?reqId=0911083&type=S
https://s4.its.unc.edu/RAMS4/details.do?reqId=0911084&type=S

If you’d like to apply for either of these positions, please use the “Apply for this position” link at the bottom of the page.

I’d be happy to answer any questions (in an unofficial manner) via e-mail at josh underscore johnson at unc dot edu.

Read on for more info about the jobs, working at UNC, and living in this part of North Carolina :)

About the jobs
These are permanent, full-time, mid-level positions. The primary focus will be on application development and publication, with secondary emphasis on user support/training, maintaining and scaling the hardware infrastructure, and keeping on top of innovations in software development, hardware management concepts, and data management techniques.

We’ll be building applications to assist with a few different facets of research, that, to over simplify, represent some form of pathology. This means dealing with the marshaling, processing, and analyzing of data collected from specimens of varying origin.

It’s essentially bioinformatics work, but there will be some specialization in areas like in silico experimentation (developing experiments that occur entirely within a computer system), image analysis, inventory management, financial tracking (purchasing), business process automation, animal colony management, and systems integration (much of the equipment used has some sort of API that can be tapped into).

For project management, we will be imploring a variant of the SCRUM development methodology. I want to see the team actively cultivate a highly productive methodology that works best for us (as opposed to just adopting something whole-hog).

The goal is short, productive development cycles that focus on functionality.

We will be forging new ground with much of our work, so research and preliminary development practices (prototyping, testing, evaluation) will be essential.

As I mentioned earlier, publication is a primary focus. Everything that we do is novel given the current state of tools used for this type of research. We want to be able to collaborate with other departments, other universities, and generate publications that showcase what we’re doing to the entire world.

Documentation is always important, but in the case of this team, we must be especially careful to explain and document what we do in not only a highly technical way (e.g. inline documentation, API references, user manuals), but also in a less technical, more narrative way. Something that can be adapted easily to various scientific and industry journal formats, and conveyed to investigators and department personnel that may not have a firm understanding of the technology we’re using.

The hardware infrastructure is virtual machine based, with a large AoE SAN. We will not have a dedicated systems administrator, so we must participate in the ongoing maintenance and development of the hardware infrastructure as a group. The work plan for the positions require 20% of each analyst’s time to be available for hardware tasks. My hope is that the actual amount of needed time will be much less, and I am designing the infrastructure so that it will require as little maintenance as possible. There’s a focus on high availability, so we don’t have to address problems right away.

On a work-culture level, the work environment will be very nurturing to a highly productive team. We will have our own work space free from most distractions (but not far from the labs we support or our hosting facilities), flexible work hours and location. Retreats, field trips, and other “lets get out of the office and think” activities are all part of our job.

We’re located right in the middle of the UNC medical campus, not far from many food options, cafes, and beautiful landscapes with free wi-fi :)

We’ll have a faithful simulation of our production environment for testing and development, will employ a VCS system (Mercurial at present) and some sort of issue tracking.

We’ll participate in community efforts (conferences, sprints, local user group activities) as much as possible.

Working at UNC
UNC is a state university. UNC itself provides many great benefits to its employees, and the state provides even more. We get generous leave accruals, retirement fund (pension as well as 401k), very good medical benefits, longevity pay, early retirement options, flexible spending account, tuition reimbursement, employee discounts and more.

More information (and real details) can be found on the UNC HR website http://hr.unc.edu

I’ve worked for the university for 6 years and it’s been great.

Living in the Triangle
If you’re not from around here, UNC-CH is located in what’s known as the “Triangle” of North Carolina, which consists of three major metropolitan areas, Durham, Raleigh, and Chapel Hill. It’s a great place to live and work.

There’s a wide range of places to live depending on your taste; everything from highly urban downtown areas to country farmland to suburban neighborhoods. Chapel Hill itself is sort of a combination of the three. Nearly any town in the Triangle area is about a 30 minute drive from UNC-CH.

Beyond UNC, the Triangle is host to a lot of technology companies (IBM, SAS, Red Hat), as well as a few other major universities (Duke, NCSU), and biotech companies (GlaxoSmithKline, RTI). (see http://www.gonetocarolina.net/employers for more)

Chapel Hill itself, where I live now, is great. It’s a somewhat small town, but because of the university it’s been built up quite substantially (yet it’s just minutes from the country). There are many, many trees :)

We have a free public transit system, and a wide variety of things to do, both cultural and culinary in nature. And if Chapel Hill doesn’t have it, chances are Durham or Raleigh will.

The Triangle is also the home of one of the best user groups ever, the Triangle Zope/Python User Group, better known as TriZPUG. It provides a multitude of opportunities to network with other Pythonistas doing very similar types of work.

We’ve also got the TriLUG, the Triangle Linux User Group, which has also been a great networking resource.

This part of North Carolina is sort of moderate when it comes to weather. Prior to moving here I lived in Florida and New England, and I feel like the Triangle is sort of right in the middle between the two. We’ve got oceans and mountains about three hours in either direction, it gets warm in the summer and cool in the winter but never too hot or too cold for too long. We’re far enough from the coast to avoid most hurricanes or nor-easters (although we are susceptible to gulf coast weather), far enough away from the mountains to avoid most flash flooding; we get some snow some times, leaves change in the fall, and there’s a lot of outdoorsy activities all around.

[Less]