Tuesday, September 25, 2012

Easier log rotation for buildout users


If you are using buildout and the plone.recipe.zope2instance (>= 4.2.5) to create your zope installation, two parameters are available to enable log rotation. For example:
  • event-log-max-size = 10mb
  • event-log-old-files = 3
This will rotate the event log when it reaches 10mb in size. It will retain a maximum of 3 files. Similar direcives are also available for the access log.
  • access-log-max-size = 100mb
  • access-log-old-files = 10
Enjoy!

Saturday, March 26, 2011

Deleting a member from a large site

The other day my client complained about not being able to delete a member from his site. When I tried it, each time it would spin around for a while and then the zeo client would die with ClientDisconnected. When I poked into the code, i discovered that every object in the site was being visited and reindexed in order to clear out any local roles that the member may have on the object. Of course the transaction becomes huge and the zeo client runs out of memory and ends up disconnecting from the server.

So I added a transaction.commit() in the body of the deleteLocalRoles function in MembershipTool.py. This helped somewhat. The zeo client did not run out of memory and was able to stay connected. Although I ended up with a proxy timeout at the browser, Plone was able to plod along and delete the member and when I went back into User and Groups management, the member was gone.

But nobody likes this half-baked "fix" as you can see at

http://www.gossamer-threads.com/lists/zope/cmf/230407

Friday, February 25, 2011

How to import a nested folder structure containing files



Problem

I have a deeply nested folder structure containing file objects (.doc, .xls, .ppt etc.) and I want to import them into Plone


Approach

By declaring file objects to be IDAVAware, we could use GenericSetup to import them, an idea I got from here:

http://reinout.vanrees.org/weblog/2006/09/13/creating-content-with-genericsetup.html


Steps

1. Mark ATFile as IDAVAware by adding the following in your configure.zcml:


<five:implements
        class="Products.ATContentTypes.content.file.ATFile"
        interface="Products.GenericSetup.interfaces.IDAVAware"
   />

2. Create a structure folder under the default profile of your product and create the desired folder structure there.

3. Recursively create the .objects, .preserve and .properties files under the structure folder. To make this easy, you can use the prepare_content recipe from collective.migrator. Your buildout step should look something like this:



[prepare_content]
recipe = collective.migrator:prepare_content
top_folder = /plone/src/my.product/my/product/profiles/default/structure/My Folder Structure/


4. Reinstall your product using the portal_quickinstaller



Wednesday, February 23, 2011

How to export a nested folder structure containing Files



Problem

I have a deeply nested folder structure containing file objects (.doc, .xls, .ppt etc.) and I want to export them to the file system


Approach

By declaring file objects to be IDAVAware, we could use GenericSetup to export them, an idea I got from here:

http://reinout.vanrees.org/weblog/2006/09/13/creating-content-with-genericsetup.html


Steps

1. Mark ATFile as IDAVAware by adding the following in your configure.zcml:

  
   <five:implements
      class="Products.ATContentTypes.content.file.ATFile"
      interface="Products.GenericSetup.interfaces.IDAVAware"
   />

2. Go to portal_setup at the root of the site

3. Go to the Export tab, select the step called Content

4. Click on Export Selected Steps at the bottom

Caveat

A small change in GenericSetup which will land in version 1.6.3 made this possible,

https://bugs.launchpad.net/zope-cmf/+bug/722726


Thursday, November 4, 2010

Removing a file from nginx cache

$ cd <cache-dir>

$ rm -f `grep -rl '^KEY: [^/]*<full-path>$' .`

Tuesday, September 14, 2010

MailHost fails with gaierror: [Errno 2] Temporary failure in name resolution on MacOS

Start your instance as follows:

$ /usr/libexec/StartupItemContext bin/instance start

Sunday, September 12, 2010

Staying logged in over multiple browser sessions


Portal Properties -> Site Properties -> auth_cookie_length

Specifies the validity period of the login cookie.

But just make sure that credentials_cookie_auth is activated in

acl_users -> plugins -> Update credentials plugins.

Monday, July 5, 2010

AttributeError: <ATFolder at ...> has no assigned layout, perhaps it needs an FTI

Recently as part of migrating a large site, I moved a lot of content from a rather ancient version of Plone (2.5) to Plone 4. When navigating to some of the folders after migration, I came across the following error.

It appears that the portal_type for these folders was set to "ATFolder" rather than just "Folder". This also has the effect of the portal_icon showing up with the default one for DTML documents (Has <> in the icon) in the ZMI.

I then fixed with the the [fix_portal_type] buildout step using the magnificent collective.migrator.

Wednesday, June 16, 2010

collective.migrator


collective.migrator is a buildout based tool to help migrate content between Plone/Zope instances. It can be installed as follows:

$ easy_install collective.migrator


Once installed you can run the tool to set up the migration environment

$ migrator


This creates a folder called migrator and installs a buildout environment there. All further actions are run from this folder.

The first thing that you want to do at this point is customize the instance.cfg file.

By default it looks like this:


[remote]
host = xxx.webfactional.com
user = ssh_user
port = 8080
extensions = /usr/Plone-2.5.5/zeocluster/client1/Extensions
zmi_user = admin
zmi_pwd = admin
root = Plone
export = /usr/Plone-2.5.5/zeocluster/client1/var

[local]
host = localhost
port = 8080
extensions = /home/suresh/plone4/parts/instance/Extensions
zmi_user = admin
zmi_pwd = admin
root = Plone
import = /home/suresh/plone4/var/instance/import


This defines the settings for all the Plone instances involved in the migration.

The buildout.cfg defines the steps that will be executed as part of the migration.

Here is the default content:


[buildout]
extends = instance.cfg
migrate_frontpage.cfg
migrate_users.cfg
migrate_props.cfg
parts =
tbd =
${migrate_frontpage:parts}
${migrate_users:parts}
${migrate_props:parts}


As you can see, parts has been intentionally left blank. Also instance.cfg described previously is being used here. The other migrate_*.cfg files contain some sample steps to move various objects between the instances.

As a simple test, you can change parts in buildout.cfg to look like this:


parts = export_frontpage


This step is defined in migrate_frontpage.cfg.

Now after you run buildout as follows:

bin/buildout


you should notice that the front-page object has been exported in the remote Plone instance. Once you gain more confidence in the tool, you can even try to run the other steps found in the migrate_*.cfg files.

PS: This may not be the "coolest" way to manipulate your Plone and some of these actions may be better done with GenericSetup profiles, but this worked for me!

Monday, June 7, 2010

Login problems using zope.testbrowser.browser


Recently, while writing some automated scripts for the ZMI with zope.testbrowser, I ran into some problems.

Here was my first try straight out of the pypi page of zope.testbrowser:


from zope.testbrowser.browser import Browser
br = Browser()
br.addHeader('Authorization', 'Basic %s:%s' % (zmi_user, zmi_passwd))
url = 'http://%s:%s/manage_main' % (zmi_host, zmi_port)
br.open(url)

Contrary to expectations, this resulted in

mechanize._response.HTTPError: HTTP Error 500: Internal Server Error

Looking at the server error log was more puzzling.

Traceback (innermost last):
Module ZPublisher.Publish, line 106, in publish
Module ZPublisher.BaseRequest, line 452, in traverse
Module Products.PluggableAuthService.PluggableAuthService, line 234, in validate
Module Products.PluggableAuthService.PluggableAuthService, line 627, in _extractUserIds
Module Products.PluggableAuthService.PluggableAuthService, line 123, in extractCredentials
Module ZPublisher.HTTPRequest, line 1343, in _authUserPW
ValueError: need more than 1 value to unpack

Suspecting that this may be due to a Zope/zope.testbrowser version mismatch as this server was running Zope 2.9.8, I then tried the same with a brand new Plone 4 instance. It still failed but the traceback gave me a clue

Traceback (innermost last):
Module ZPublisher.Publish, line 116, in publish
Module ZPublisher.BaseRequest, line 593, in traverse
Module Products.PluggableAuthService.PluggableAuthService, line 232, in validate
Module Products.PluggableAuthService.PluggableAuthService, line 558, in _extractUserIds
Module Products.PluggableAuthService.plugins.HTTPBasicAuthHelper, line 76, in extractCredentials
Module ZPublisher.HTTPRequest, line 1519, in _authUserPW
Module base64, line 321, in decodestring
Error: Incorrect padding

I tried the obvious

import base64
br.addHeader('Authorization', base64.encodestring('Basic %s:%s' % (zmi_user, zmi_passwd)))

but was now left with

mechanize._response.HTTPError: HTTP Error 401: Unauthorized

After poking some more at ZPublisher.HTTPRequest.py I came up with the winning

br.addHeader('Authorization', 'Basic %s' % base64.encodestring('%s:%s' % (zmi_user, zmi_passwd)))

Monday, May 31, 2010

migrate-btrees to the rescue

I had some old folderish content from Plone 2.5 and when i tried to migrate this to Plone 4 it was not working correctly. I was getting

AttributeError: has_key

exceptions from Products.BTreeFolder. I also found this article describing this problem:

http://plone.org/documentation/error/nonetype-keys

but no clear solution.

It turns out that ATContentTypes 2.0 changes the ATFolder implementation to use BTreeFolder as the base class. Thankfully a migration view is provided called "migrate-btrees" that will migrate all the folderish items under the given path.

At the end I got a nice message in my log:

2010-05-31 22:23:35 INFO plone.app.folder.migration processed 3490 object(s) in 10.864s (10.640s cpu time)

Tuesday, May 25, 2010

Zopeskel/Paster created plone3_theme shows up without the "Plone default" css

This is by design.

"Plone default" does not mean the standard Plone look and feel. If you want that, you should use the "Plone Classic Theme" or the "Sunburst Theme" as the "Skin Base" which you will be prompted for if you use the "expert" mode of the wizard.

Saturday, May 22, 2010

collective.MockMailHost

Here is a product that will help you with integration testing email notifications generated from your custom Plone add-on.

DO NOT install this on your running Plone site. Simply use this from your tests. If you are running Plone 4, just add this egg to the [test] runner section of your buildout rather than to the [instance].

The product egg is available from pypi. The source is in the collective svn repository.

Enjoy and do remember to give me some feedback!

ImportError: <module 'setuptools.dist' from ...> has no 'check_packages' attribute

This error when you run buildout can be solved by upgrading setuptools as follows:

(plone3) $ easy_install -U setuptools

Thursday, May 20, 2010

Products.PloneboardSubscription

Here is a product that will enable email notification for comments added to Ploneboard fora or conversations.

Once you install the product using the QuickInstaller, a Subscribe/Unsubscribe link is available when viewing a Forum or a Conversation. Once subscribed, an email notification is sent when a comment is added to the particular Forum or conversation.

The product egg is available from pypi. The source is in the collective svn repository.

Enjoy and do remember to give me some feedback!