photo

John Hurst

Version 1.5.5

20110806:163853

Table of Contents

1. The Main Program

photo.py is a Python program that organizes digital photographic images into a set of browsable web pages. The images are assumed to be organized into directories. (A separate program, filePics.py, is available to perform this function.) These directories form the basis for browsing amongst the generated web pages.

Within each directory, an index.xml file is generated. This XML file is renderable into HTML by a separate XSL program, albums.xsl, also available separately. The format and structure of these directories is user-controlled, and photos can be arranged by subject, date, location, etc., etc.. Within a directory, photos are arranged by the order specified in an associated descriptions file, containing the image name and an image description. The initial sequence of image names is generated automatically for any image found in the directory, but not present in the descriptions file. The descriptions file is created if one is not found. See the section The index.xml file format for details.

Within each directory, an albums.xml file describes the content of the directory, and is used to generate the associated index.xml file. The albums.xml file contains the path name of the directory relative to the root directory of all photos managed by the system, along with the directory description and name, and a path to a thumbnail image that will represent the directory. This thumbnail image can be one of the photos contained within the directory, or elsewhere, and thus can be stylised to suit the directory contents.

1.1 Setup

Initialize the debug flag. This is currently set manually, but eventually we expect to make it a command line option.

Initialize the variable 'lists'. This is a python directory, where the keys are (os) directories paths rooted at the album subdirectory (the cli parameters), and the entries are lists of image file names within that directory. This is used to build a list of images within each (os) directory traversed. The list/directory is initially empty.

Initialize the variable 'descriptions'. This is a python directory, where the keys are image file names, and the entries are descriptions/captions for the image concerned. It is built from the eponymously named file 'descriptions' in the directory currently being scanned.

Define the XSL script used to render all XML files built by this program.

maxmissing, maxdir are variables that track the directory with the maximum number of missing descriptions. This is purely a convenient device to point the user to where most documenting effort is required!

1.2 Define miscellaneous subroutines

1.3 Collect the Command Line Options

There are four command line options:

d

turn on debugging

f

Force the generate of the image XML files. This is useful if other relevant components have changed, such as the XSL files, which are not known about here.

l

Do not generate the large subimages.

r

Recursively traverse any subdirectories for further images.

t

Generate only the thumbnail subimages.

1.4 Perform the Top Level Visit

We visit every directory given as arguments to the program invocation. If no arguments are given, then visit the current directory.

All the real work of visiting a directory and computing the index XML files is done by the visit procedure, which returns a tuple of potentially changed values that are of relevance to higher level directory index XML files. These are, respectively: the title of the visited directory; the number of photos or images contained in the directory and its sub-directories; the number of albums ditto; the thumbnail to characterize the directory; the description of the directory; and the number of missing descriptions in the directory and its sub-directories.

These values are then propogated to higher level photo albums (directories) via the chunk <explore next higher level to update counts >, which also recomputes updated values of the tuple described above.

The next higher directory is identified and the index.xml file parsed. This is used to identify all the directory elements, which are scanned to update the key parameters. The title, thumbnail and descriptions are reset, since these will not change in the higher level summaries.

The summary element is then recomputed, and the index.xml file rewritten.

Note that this rewriting may not be necessary if none of the key variables have changed. This is not currently tested.

1.5 write list file

Open the file list in each of the directories visited, and write a list of images found in that directory to the file. This may not be entirely accurate, and needs to be confirmed that it does work correctly. It has not been used recently, and may be superflous to needs. It has now been commented out (v1.2.3).

1.6 Plan for new data structure

In order to develop this program further, it is suggested that a full data structure for each image and album be developed. This data structure would be populated in some way, and would then provide the data for writing the index.xml file. How is it populated?

1. from the current index.xml file
2. from the album.xml file
3. from the image file
4. from the descriptions file
5. from the command line (?)

There would be a list of such data components for each directory visited (hence a local variable to routine visit). This list could be sorted on one or more of its attributes to provide a range of views.

Components of the new data structure:

2. The visit routine

2.1 initialize variables for visit routine

nphotos

the number of image files found in this subdirectory (album)

nalbums

the number of subalbums (subdirectories containing images) found in this subdirectory

missingdescrs

the number of images for which no description was found in the descriptions file

gotdescrfile

set if the descriptions file was succesfully read

listmissingimages

a list of those images for which a description was found, but no image file

albumdom

the XML-parsed album file

title

the title of this album (subdirectory)

thumb

the thumbnail image used to represent this album

description

the description of this album

maxmissing

maxdir

saved

mdescr

images

2.2 open index.xml and write header

2.3 get information from album XML file

2.4 write tree element to index.xml

2.5 sort fnames into directories and images

Sort the list of all files in this directory into directories and images. We detect the first from an os.path.isdir call, and the second by checking its extension, which must be either .JPG or .jpg (or capitalisations in between).

2.6 Visit All Directories

2.7 scan all images and process them

{Note 2.9.1}

dud descriptions file entry, ignore it

2.8 wind up index file

3. Descriptions

This is a revision of the literate structure to bring all descriptions related material together.

descrpat is a pattern to match image names. Note that the first pattern must be non-greedy, or the '.JPG' gets swallowed, and the fourth pattern likewise, or the ' # ' gets swallowed.

The third pattern has been added (v1.3.0) to flag protected images that must not be shown publically.

3.1 read descriptions file and create dictionary

Build the python dictionary of descriptions. We attempt to open a file in the current directory called descriptions, and use that to initialize the dictionary. The file has the format of one entry per line for each photo, with the name of the photo at the start of the line (with or without the .JPG extension), followed by at least one blank, followed by the descritive text, terminated by the end of line.

3.2 wind up descriptions file

We finalise the descriptions file. Two scenarios are relevant: either a) the descriptions file does not exist, in which case we build a new one, with just the image names of those images found in the directory scan, or b) the descriptions file exists, but there are images found not mentioned in it. In this latter case, we append image names to the file. No attempt is made to insert them in their 'correct' position.

If there are no changes required, the descriptions file is untouched. The existence of the descriptions file is determined by whether the flag gotdescrfile is set.

4. the addToIndex routine definition

I've changed my mind a couple of times about escaping the description string. Currently the string is not escaped, meaning that any special XML characters (ampersand, less than, etc.) must be entered in escaped form. But this model does allow URLs and the like to appear in the description captions.

5. the addToList routine definition

6. the makeImage routine

7. the makeSubImages routine

This routine is responsible for building all the smaller images from the master image.

7.1 New SubImage generalized size definitions

We define here data structures to handle generalized sub-image size definitions. Not yet active.

Initialize the image size parameters. imageSizeDefs is normative, and defines all required image sizes. The first entry is deemed to be the thumbnail size, and an entry of (0,0) refers to the original size. imageSizes is an equivalent list of strings in the form %dx%d, and derivedImageMatch is an equivalent list of patterns to match image names of those sizes. (Except the last, which fix!)

Check that the image file exists. If it doesn't, then there are two possible reasons. Firstly, the image file has been removed for space reasons, in which case there will be an entry in the descriptions database, and we return that without attempting to make any sub images.

Alternatively, no such descriptions entry means the image is genuinely missing, and therefore we should print a warning message, and return a missing image flag.

Get all the modification times. The default modification times are that mview is the oldest, mfile is the youngest, and all the others are in between.

(20110717:172812) The variables smallwd, medwd, largeht are introduced to ensure that the resulting images all have a consistent height, if not width. This was introduced because panoramas came out with the same width as convention 4x3s, making the height impossible small.

See comment under <the addToIndex routine definition 4.1> regarding escaping the description string.

8. the retrieve support routines definition

These two support routines for retrieve encapsulate data extraction from the DOM model. They perform the same basic operation, differeing only in the default parameters required.

retrieve is called when we do not wish to recursively visit a subdirectory. Instead, relevant details from the subdirectory are extracted ('retrieved') from an index.xml file, which has been constructed when the relevant subdirectory was visited.

There is a slight problem with identifying a thumbnail for this album, since the definite source is that in the album.xml file. However, this may be blank, in which has we need to promote the first thumbnail within the directory (or subdirectories where the directory has no images of its own).

9. The index.xml file format

The following description defines the collection of XML elements used to build an index.xml file.

Note that lowercase names are the names of the element tags; capitalized names are element content; names starting with @ are attributes (with string values: attributes that must be digit strings are so identified).

indexfile   = title description commentary tree 
              directory* thumbnail image*
              summary 
            .
title       = TextNode .
description = TextNode .
commentary  = TextNode .
tree        = TextNode .
directory   = @title @name @albums @images @missing @thumb @description Empty .
thumbnail   = TextNode .
image       = @name TextNode .
summary     = @albums @maxdir @maxmissing @missing @photos .
@albums     = Digits .
@images     = Digits .
@missing    = Digits .
@maxmissing = Digits .
@photos     = Digits .

Explanation of attributes:

@title

The title of the directory or album (for the human reader). This attribute is derived from the title element of the specified (sub-)directory

@name

The name of the directory (the Unix name)

@albums

digit string defining the number of sub-directories

@images

digit string defining the number of photos/images in this directory and any sub-directories (albums and sub-albums)

@missing

digit string defining the number of missing descriptions for all enclosed images

10. Indices

Document History