The purpose of this document is to describe the software design of the Vasudeva Service Application, Generation 3. This document uses the terminology established in the System Requirement Specifications.

ZEO/Zope/Storage Configuration

The configuration described here applies specifically to the instance of the Vasudeva Service Application on the Vasudeva Servers.

Production ZEO server

The production ZEO server runs in the directory /opt/zope/vsa_3/zeo/. It manages several filestorages (ZODBs) under /opt/zope/vsa_3/var. Some of these are used by Core Developers, Site Administrators, Editors, and Designers only, while others are used by all users. This is described below.

Staging Zope Instance

A staging Zope instance for Core Developers and Designers currently runs in the directory
/opt/zope/vsa_3/stage_client/. This Zope instance is used for testing new content-type components and diagnostic components. Through a process involving the staging Zope instance, components are tested in a staging area and later pushed to a production area, where they are visible to all users. This work flow is described later.

The staging Zope instance mounts a filestorage in its root, and mounts zeoclients in staging and production Folders, and in a Folder for Zope External Methods. One of the production zeoclients is a development fishbowl. The root filestorage contains User Folders for the Core Developers.

In the future, it is expected that the staging Zope instance will move to another machine, to free up machine resources for Members and Anonymous Users. At that point, only activity under the production Folder would impact the physical server on which the production ZEO server runs.

Public Zope Instance(s)

A public Zope instance for Site Administrators, Editors, Members, and Anonymous Users runs in the directory /opt/zope/vsa_3/pub_client/. In the future, this instance will be replicated on at least one additional physical server.

The public Zope instance mounts a filestorage in its root, a zeoclient for persistent session storage (so sessions are not lost on restarting the Zope instance), and several zeoclients in a production Folder in Zope.

Note that Members may generate frequent writes against the production ZEO server. The public Zope instance manages all Zope user accounts and Plone Member Folders. The Zope acl_users Folder for some of these accounts will be managed outside of Plone Sites so that several Plone Sites may share memberships, while maintaining distinct Member home pages.

Administrative Zope Instance

An administrative Zope instance runs in /opt/zope/vsa_3/adm_client/. The Zope Administrator uses this instance to create new sites and to diagnose problems with components of the Vasudeva Service Application. The administrative Zope instance has a flexible storage configuration. It is typically shut down when not in use.

Storage

Public

/opt/zope/vsa_3/pub_client/var/Data.fs

This is mounted as a filestorage at the root of the public Zope instance(s).

/opt/zope/vsa_3/var/pub/session/Data.fs

This is mounted as a zeoclient by the public Zope instance(s). This provides persistent storage for all Member and Anonymous User sessions.

Admin

/opt/zope/vsa_3/adm_client/var/Data.fs

This is mounted as a filestorage in some configurations of the administrative Zope instance.

Staging

/opt/zope/vsa_3/stage_client/var/Data.fs

This is mounted as a filestorage at the root of the staging Zope instance. It holds developer User Folders.

/opt/zope/vsa_3/var/stage/srichinmoyraces.org/Data.fs

/opt/zope/vsa_3/var/stage/srichinmoycentre.org/Data.fs

/opt/zope/vsa_3/var/stage/srichinmoy.org/Data.fs

/opt/zope/vsa_3/var/stage/srichinmoylibrary.com/Data.fs

/opt/zope/vsa_3/var/stage/vs_sites/Data.fs

These are staging areas for the respective domains. Each domain may comprise several locales. In the development Zope instance, work is pushed here from off-server developer sandboxes and tested. The staging Zope mounts these as zeoclients under /home.

The production ZEO server will continue to serve the staging Folder after the staging Zope instance moves off-server, so that the staging area can mimic the production area as closely as possible.

The filestorage under vs_sites contains a group of sites, such as radiosrichinmoy.org and vasudevaservice.com, the development fishbowl.

Production

/opt/zope/vsa_3/var/pub/srichinmoyraces.org/Data.fs

/opt/zope/vsa_3/var/pub/srichinmoycentre.org/Data.fs

/opt/zope/vsa_3/var/pub/srichinmoy.org/Data.fs

/opt/zope/vsa_3/var/pub/srichinmoylibrary.com/Data.fs

/opt/zope/vsa_3/var/pub/vs_sites/Data.fs

These are the production ("live") areas for the respective domains. Each domain may comprise several locales. In the staging Zope instance, work that has been tested is pushed into these areas.

In the staging Zope instance, these are mounted as zeoclients under /pub. In the public Zope instance(s), these are mounted as zeoclients under /home.

The filestorage under vs_sites contains a group of sites, such as radiosrichinmoy.org and vasudevaservice.com, the development fishbowl.

Example

The following summarizes the intended use of the storages described above, using srichinmoyraces.org as an example.

The public Zope instance(s) ZMI has the following:

a. /home/srichinmoyraces.org (production area)

Members will have write access to this Zope, and will interact with the Plone Site object srichinmoyraces.org through the Plone administrative interface.

The development Zope instance ZMI has the following:

a. /pub/srichinmoyraces.org (production area)

b. /home/srichinmoyraces.org (testing area)

c. /dev/medur/srichinmoyraces.org (sandbox)

d. /dev/tejaswi/srichinmoyraces.org (sandbox)

The staging Zope instance and public Zope instance(s) should have the same access rules in their respective Zope roots.

The two mounts indicated by "a." refer to the same physical filestorage on the file system, even though their mount points are named differently.

Procedures

Sandboxes

Each Core Developer, Designer, and Site Administrator uses off-server sandbox areas as they see fit.

Staging

The following procedure pertains to work done on a single site through the staging Zope instance. The Zope Administrator is responsible for ensuring that the process is followed.

When work on a site, such as a portal slot, is a candidate for prime time, a staging area for the site is prepared. In order to duplicate the production site, the filestorage containing the staging site is replaced by the filestorage containing the production site. This requires a restart of the staging Zope instance.

The site is represented as a mounted Plone Site object in the /home Folder. If this Plone Site object (=mount point) is deleted or replaced, the mount is broken and the actual Plone Site object inside the filestorage managed by the associated zeoclient is untouched. If not remounted but replaced, the new object is stored in the root filestorage of the staging Zope instance, not in the zeoclient for that staging site. Further, replacing the large Plone Site object by copy-and-paste will be very expensive; it will take a long time and will bloat the root filestorage with undoable transactions. In order to keep the testing activity in the zeoclient (to most closely duplicate production) do not delete or replace the mounted Plone Site object. If it is accidentally deleted, just remount it.

Testing is carried out by a user who did not create the new objects. The Zope Administrator is responsible for ensuring that adequate testing is done.

Special procedures must be followed for objects that can not simply be copied and pasted through the ZMI. These include the following.

• Site-specific External Methods

Methodology: External methods follow the Zope instance. Python code is managed in cvs on the file system. Tests are performed against the head revision.

• Products, including custom Archetypes

Methodology: Products follow the Zope instance. Python and configuration code for custom archetypes is managed in cvs on the file system. Tests are performed against the head revision.

If changes to a staging site cause the staging Zope instance to spin or crash, that site will be unmounted and taken off-line until it is corrected.

Production Deployment

The following procedure pertains to work done on a single site through the staging Zope instance. The Zope Administrator is responsible for ensuring that the process is followed.

After successful testing, all new or changed objects are identified and copied to the production server. The Zope Administrator is responsible for the process of copying objects to the production Folder.

The site is represented as a mounted Plone Site object in the /pub Folder in the staging Zope instance. If this Plone Site object (=mount point) is deleted or replaced, the mount is broken and the actual Plone Site object inside the filestorage managed by the associated zeoclient is untouched. This means that the public Zope instance does not see any change. If not remounted but replaced, the new object is stored in the root filestorage of the staging Zope instance, not in the zeoclient for that staging site. Again, this means that the public Zope instance will not see any changes, particularly the changes. When pushing changes to production, do not delete or replace the mounted Plone Site object in the /pub Folder. If it is accidentally deleted, just remount it.

Special procedures must be followed for objects that can not simply be copied and pasted through the ZMI. These include the following.

• Site-specific External Methods

Methodology: External methods follow the Zope instance. Python code is managed in cvs on the file system. Tests are performed against the head revision.

• Products, including custom Archetypes

Methodology: Products follow the Zope instance. Python and configuration code for custom archetypes is managed in cvs on the file system. Tests are performed against the head revision.

If changes to a production site cause the public Zope instance to spin or crash, that site will be unmounted and taken off-line until it is corrected.

Maintenance and Diagnostic Scripts

The Vasudeva Service Application includes scripts to handle the following tasks.

• ZEO daemon restart on system startup

• Zope instance daemons restart on system startup

• Zope instance scheduled restart

• Demonstration that session is preserved across public Zope instance restart.

• filestorage scheduled packing

• Log rotation

Application Installation

Following are the components of the Vasudeva Service Application, including both third party products and in-house contributions.

python 2.3.3

Install this in /usr/local/lib/python2.3.3. Install as root.

Zope 2.7.0

Installed in /usr/local/zope-2.7.0. Install as zope.

Zope Products

Core products sit in the Zope installation area. Other products travel with the Zope instances, and go in the Products directory in $INSTANCE_HOME.

Archetypes 1.2.4 on python 2.3 requires that you install the generator and validation python packages. They go in the site-packages directory of the python installation.

Plone 2.0 travels with the Zope instance.

External Methods

External Methods travel with the Zope instance and are held in cvs on the file system.

Miscellaneous Topics to Incorporate Later

Using / Not Using zdctl

http://zope.org/Wikis/ZODB/HowtoUseZdctl/zdctl.html

This may be fine for testing, but a bad idea for running a ZEO server in a production environment. In production, you want the ZEO server to be run as a daemon process, you want the log output to go to a file, you want the ZEO server to be started when the system is rebooted, and (usually) you want the ZEO server to be automatically restarted when it crashes. You should also have a log rotation policy in place so that your disk doesn't fill up with log messages.

The zdctl/zdrun combo can take care of running a server as a daemon process and restarting it when it crashes. It can also be used to start it when the system is rebooted. Sending log output to a file is done by adjusting the ZEO server configuration. There are many fine existing tools to rotate log files, so we don't provide this functionality; zdctl has a command to send the server process a SIGUSR2 signal to tell it to reopen its log file after log rotation has taken place (the ZEO server has a signal handler that catches SIGUSR2 for this purpose).

Daemonizing ZEO

http://zope.org/Wikis/ZODB/HowtoRunAZEOServer/howto.html

Running the ZEO server as a daemon