App/tar.py fix for Zope 2.6.1/Python 2.1.3

rev. date: 27-Nov-2004

rev. by: Fred Morris

Abstract

The version of Zope/Python 2.1 which ships with SuSE 8.2 contains a bug in App/tar.py which manifests when attempting to create a distribution of a product. There is a bug in the tar header creation code which raises an exception condition. Unfortunately this error is further masked by a different error handler which may fail when the error is caught in an outer context. While research indicated that this problem has been fixed in a later version, no description of the defect was found.

Disclaimer

The fixes provided below worked for me. No warranty is expressed or implied. The fixes, and the code which implements them, are provided as-is, in the hope that they will be useful. I have not compared it to the fix ostensibly made in a more recent version. Use at your own risk.

Problem Description/Fix - App/tar.py

The mtime header field is written one byte too long, which causes an internal consistency check to fail and raise the error Bad Header Length at line 65 in the original file. The following can be used to replace the shipped version. Changed/added lines are colored red.

##############################################################################
#
# Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
__doc__='''Simple module for writing tar files

$Id: tar.py,v 1.6 2002/08/14 21:31:41 mj Exp $'''
__version__='$Revision: 1.6 $'[11:-2]

import sys, time, zlib
try:
    from newstruct import pack
except:
    from struct import pack

def oct8(i):
    i=oct(i)
    if len(i) > 6: raise ValueError, 'oct8 conversion error'
    return '0'*(6-len(i))+i+' \0'

def oct12(i):
    i=oct(i)
    if len(i) > 12: raise ValueError, 'oct12 conversion error'
    if len(i) <= 11: i = '0'*(11-len(i))+i+' '
    return i

def pad(s,l):
    ls=len(s)
    if ls >= l: raise ValueError, 'value, %s, too wide for field (%d)' % (s,l)
    return s+'\0'*(l-ls)

class TarEntry:

    def __init__(self, path, data,
                 mode=0644, uid=0, gid=0, mtime=None, typeflag='0',
                 linkname='', uname='jim', gname='system', prefix=''):
        "Initialize a Tar archive entry"
        self.data=data
        if mtime is None: mtime=int(time.time())
        header=''.join([
            pad(path,      100),
            oct8(mode),
            oct8(uid),
            oct8(gid),
            oct12(len(data)),
            oct12(mtime),
            ' ' * 8,
            typeflag,
            pad(linkname,  100),
            'ustar\0',
            '00',
            pad(uname,      32),
            pad(gname,      32),
            '000000 \0',
            '000000 \0',
            pad(prefix,    155),
            '\0'*12,
            ])
        if len(header) != 512: raise 'Bad Header Length', len(header)
        header=(header[:148]+
                oct8(reduce(lambda a,b: a+b, map(ord,header)))+
                header[156:])
        self.header=header

    def __str__(self):
        data=self.data
        l=len(data)
        if l%512: data=data+'\0'*(512-l%512)
        return self.header+data

def tar(entries):
    r=[]
    ra=r.append
    for name, data in entries:
        ra(str(TarEntry(name,data)))
    ra('\0'*1024)
    return ''.join(r)

def tgz(entries):
    c=zlib.compressobj()
    compress=c.compress
    r=[]
    ra=r.append
    for name, data in entries:
        ra(compress(str(TarEntry(name,data))))
    ra(compress('\0'*1024))
    ra(c.flush())
    return ''.join(r)

class tgzarchive:

    def __init__(self, name, time=None):
        self._f=gzFile('%s.tar' % name, time)

    def add(self, name, data):
        self._f.write(str(TarEntry(name,data)))

    def finish(self):
        self._f.write('\0'*1024)

    def __str__(self):
        return self._f.getdata()

class gzFile:
    _l=0
    _crc=zlib.crc32("")

    def __init__(self, name, t=None):
        self._c=zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS,
                                 zlib.DEF_MEM_LEVEL, 0)
        if t is None: t=time.time()
        self._r=['\037\213\010\010',
                 pack("<i", int(t)),
                 '\2\377',
                 name,
                 '\0'
                 ]

    def write(self, s):
        self._crc=zlib.crc32(s, self._crc)
        self._r.append(self._c.compress(s))
        self._l=self._l+len(s)

    def getdata(self):
        r=self._r
        append=r.append
        append(self._c.flush())
        append(pack("<i", self._crc))
        append(pack("<i", self._l))
        return ''.join(r)

Additional Debugging Information - Zope/App/startup.py

When the exception is raised by the above module, it is caught by a logging module at an outer scope. On the system where the problem was first noted, the logging module in turn kicked an error, raising the exception AttributeError for the symbol __error_log__. This effectively masked the underlying problem. For the purposes of tracking down the problem, the offending code was disabled in Zope/App/startup.py. Only a diff is provided below, but it should be sufficient to illustrate the technique:

145c145,148
<             log = aq_acquire(published, '__error_log__', containment=1)
---
>             # This seems to be raising an exception, which I do not
>             # understand, so trick it. FWM, 27-Nov-2004
>             # log = aq_acquire(published, '__error_log__', containment=1)
>             error_log_url = ''
148,149c151,152
<         else:
<             error_log_url = log.raising((t, v, traceback))
---
>         # else:
>             # error_log_url = log.raising((t, v, traceback))