CC-tea/org-hyperion-cules
1
0
Fork 0
mirror of https://github.com/SDL-Hercules-390/hyperion.git synced 2026-05-28 21:17:53 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
ce193eddcea00e3d8947fd5fcc751c032ca8a13e
org-hyperion-cules/html/cckddasd.html
T
Jan Jaeger 396093b802 Merge 2.13 changes back in 
git-svn-id: file:///home/jj/hercules.svn/trunk@120 956126f8-22a0-4046-8f4a-272fa8102e63
2001-05-08 15:38:00 +00:00

1164 lines
59 KiB
HTML
Raw Blame History

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 3.0//EN" "html.dtd">
<HTML>
<HEAD><TITLE>
Hercules: Compressed CKD Dasd Emulation</TITLE>
<LINK REL=STYLESHEET TYPE="text/css" HREF="hercules.css">
</HEAD>
<BODY BGCOLOR="#ffffcc" TEXT="#000000" LINK="#0000A0"
VLINK="#008040" ALINK="#000000">
<h1>Compressed CKD Dasd Emulation</h1>
<hr noshade>
<h2>Contents</h2>
<ul>
<li><a href="#introduction"> Introduction </a>
<li><a href="#shadowfiles"> Shadow Files </a>
<li><a href="#filestructure"> File Structure </a>
<li><a href="#methodology"> Methodology </a>
<li><a href="#quickstart"> Quick Start </a>
<li><a href="#usingsfiles"> Using Shadow Files </a>
<li><a href="#options"> Options </a>
<li><a href="#utilities"> Utilities </a>
<li><a href="#faq"> FAQ </a>
<li><a href="#changes"> Changes </a>
<li><a href="#bugs"> Bugs </a>
<li><a href="#cckddump"> cckddump os/390 hlasm program</a>
</ul>
<hr noshade>
<h3><a NAME="introduction">Introduction</a></h3>
Using compressed CKD files, or <b>cckd</b> files, you can significantly
reduce the file space required for emulated CKD dasd and possibly gain
a performance benefit because less physical i/o occurs. Using the
<i>shadow</i> function, you can minimize the amount of data loss in the
event of file corruption.
<p>
A <b>cckd</b> file contains <i>track images</i>, which may be <i>compressed</i>
or <i>uncompressed</i>, and overhead blocks which are <i>headers</i>,
<i>lookup tables</i>, and <i>free space</i>. Compressed track images
may be compressed by
<a href="http://www.info-zip.org/pub/infozip/zlib/"><b>zlib</b></a> or
<a href="http://sourceware.cygnus.com/bzip2/"><b>bzip2</b></a>.
<p>
Track images are addressed by <i>track number</i> using a two table
lookup method: <i>track number</i> divided by 256 (<code>trk >> 8</code>)
indexes into the <i>primary lookup table</i>, which contains the file offset
to the <i>secondary lookup table</i>; the remainder of <i>track number</i>
divided by 256 (<code>trk & 0xff</code>) indexes into the corresponding
<i>secondary lookup table</i>, which contains the <i>offset</i> and
<i>length</i> of the track image.
<p>
There is a single <i>primary lookup table</i> and a variable number of
<i>secondary lookup tables</i>. The maximum number of <i>secondary lookup
tables</i> is the number of tracks for the device type divided by 256, rounded up.
For example, a 3390-3 contains 50085 tracks and would require at most 196
<i>secondary lookup tables</i>.
<p>
<a NAME="formula1">
A regular CKD file contains a 512 byte <i>header</i> followed by <i>track
images</i>, each taking the same amount of space: the maximum track size.
The offset of a track image can be readily calculated by the track number:
<center>
<code>offset = 512 + trk * maxtrksz</code>
</center>
</a>
<p>
A <b>cckd</b> file can take significantly less file space than a regular
CKD file because
<ul><li>A track image only occupies its <i>length</i> in the file and
not the <i>maximum track size</i>
<li>A track image may be compressed, reducing its <i>length</i>
<li>Unused or <i>null</i> tracks do not occupy any space at all
</ul>
Performance improvements may also occur because less data is read and
written to the hard drive.
<br>However, the <i>lookup tables</i> must be
accurately maintained, <i>track images</i> must be compressed and
uncompressed, and <i>free space</i> must be kept track of, and dealt
with by a <i>garbage collector</i>. This results in a more complicated
file structure, more CPU activity, and the possibility of file corruption
due to program failure or bug. The introduction of <i>shadow files</i>,
however, reduces the impact of possible file corruption.
<p>
<hr noshade>
<p><h3><a NAME="shadowfiles">Shadow Files</a></h3>
Malcom Beattie originally introduced the concept of <i>shadow</i>
files in a <a href="http://www.egroups.com/message/hercules-390/6695">
post</a> to the <a href="http://www.egroups.com/community/hercules-390">newsgroup</a>
8 December 2000. The function is actually implemented as a kind of
<i>snapshot</i>, where a new shadow file can be created on demand.
A CKD emulated dasd is represented by a <i>base</i> file and 0 or more
shadow files. The base file can be either a regular CKD file
(with some restrictions) or a cckd file. All files are opened <i>read-only</i>
except for the <i>current</i> file, which is opened <i>read-write</i>.
<p>
Shadow files are implemented using the same file structure as base <b>cckd</b> files.
By default, there can be up to 8 shadow files in use at any time for an
emulated CKD device. The base file is designated file <b>[0]</b> and
the shadow files are files <b>[1]</b> up to file <b>[8]</b>.
The <i>highest</i> numbered file in use at a given time is the <i>current</i>
file, where all writes will occur. Track reads start at the <i>current</i>
file and proceed down until a file is found that actually contains the track
image.
<p>
A shadow file, then, contains all the changes made to the emulated CKD dasd
since its creation, until the creation of the next shadow file. The moment
of the shadow file's creation can be thought of as taking a <i>snapshot</i>
of the current emulated CKD dasd at that time, because if the shadow file is
later removed, then the emulated CKD dasd will revert to the state it was at
when the <i>snapshot</i> was taken.
<p>
Using shadow files, you can keep the base CKD file on a read-only device
such as cdrom, or change the base CKD file attributes to read-only,
ensuring that this file can never be corrupted.
<p>
Hercules console commands are provided to add a new shadow file, remove
the current shadow file (with or without backward merge), and display
the shadow file status and statistics.
<p>
<hr noshade>
<p><h3><a NAME="filestructure">CCKD File Structure</a></h3>
Like a regular CKD emulation file, the first 512 bytes
of a compressed or shadow file contains a <code>CKDDASD_DEVHDR</code>
block. The eye-catcher at the beginning is different to distinguish
the file:
<ul><li><b>CKD_P370</b>&nbsp&nbsp&nbsp Regular CKD file
<li><b>CKD_C370</b>&nbsp&nbsp&nbsp Compressed CKD file
<li><b>CKD_S370</b>&nbsp&nbsp&nbsp Shadow CKD file
</ul>
The next 512 bytes contain a compressed device
header or <code>CCKDDASD_DEVHDR</code> block. This contains the
version-release-mod level of the file, options, space statistics,
and total number of cylinders for the device. Next is
the primary lookup table or the <code>L1TAB</code>. Each 4 byte
entry in the <code>L1TAB</code> contains the file offset to a secondary
lookup table (or <code>L2TAB</code>) or <code>0x00000000</code> (indicating that
the secondary lookup table is <i>null</i>), or <code>0xffffffff</code> (indicating
that the previous file should be searched instead).<br>
The size of the <code>L1TAB</code> is dependent on the number
of tracks on the emulated device.
<p>
<center>
<table border=1>
<tr><td align="left"><code>CKDDASD_DEVHDR</code><br><br><br><br></td>
<tr><td align="left"><code>CCKDDASD_DEVHDR</code><br><br><br><br></td>
<tr><td align="left"><code>L1TAB</code><br><br>
<center>.&nbsp&nbsp.&nbsp&nbsp.</center><br><br></td>
</table>
</center>
<p>
Following the <code>L1TAB</code>,
in no particular order, are <code>L2TAB</code>s, compressed track
images, and free spaces.
<p>
<code><b>L2TAB</b></code>s contain 256 8-byte
entries,and each are, consequently, 2048 bytes in length. Each entry
contains the <i>offset</i> and <i>length</i> of a track image. If
the <i>offset</i> is <code>0x00000000</code> then the track image is <i>null</i>;
if the <i>offset</i> is <code>0xffffffff</code> then the previous file should be
searched instead.
<p>
<center><code><b>L2TAB</b></code> entry
<table border=1>
<tr><td><code><b>offset</b></code><br>4 bytes</td>
<td><code><b>length</b></code><br>2 bytes</td>
<td><code>[unused]</code><br>2 bytes</td>
</table>
</center>
<p>
A <b>compressed track image</b> contains the following two fields:
<ul>
<li> A <i>track header</i>, also called a <i>home address</i> or a <i>track index</i>.
The track header is <b>never</b> compressed.
<li> The track image, beginning with the <i>R0 count</i> and ending with the
<i>end-of-track</i> marker, which is a count field containing all hex 0xff's.
The track image may or may not be compressed.
</ul>
<center><table border=1>
<tr><td><code><b>HA</b></code><br>5 bytes</td>
<td><b>track image</b> (compressed or uncompressed)</br><i>length</i>-5 bytes</td>
</table></center>
<p>
The <b>HA</b> contains <code>0CCHH</code>, that is, a byte of zeroes, 2 bytes indicating
the <i>cylinder</i> of the track, and 2 bytes indicating the <i>head</i> of the track
on the cylinder. Both <code>CC</code> and <code>HH</code> are stored in
<i>big-endian</i> byte order. The track is computed by
<br><center><code>
trk = (((CC[0] << 8) + CC[1]) * <i>trks_per_cyl</i>) + (HH[0] << 8) + HH[1]
</code></center><br>
Since the first byte of the <code>HA</code> is always 0x00 (at least in emulated
CKD files), this byte as stored in the file actually indicates the compression
algorithm used for the remainder of the track image
(<b>0</b> = no compression, <b>1</b> = zlib compression, <b>2</b> = bzip2 compression).
<p>
<b>Free space</b> contains a 4-byte <i>offset</i> to the next free space,
a 4-byte <i>length</i> of the free space, and zero or more bytes of residual data.
<p>
<center><b>Free Space</b> entry
<table border=1>
<tr><td><code><b>offset</b></code><br>4 bytes</td>
<td><code><b>length</b></code><br>4 bytes</td>
<td><code><b>residual</b></code><br>(<i>length</i> - 8) bytes</td>
</table>
</center>
<p>The minimum length of a free space is 8 bytes.
Since free space is ordered by file offset and no two free spaces are adjacent,
<i>offset</i> in the free space entry is always greater than the current free space
offset + the current free space <i>length</i>, unless the <i>offset</i> is zero,
which indicates the free space chain is terminated.<br>
The free space chain is read when the file is opened for read-write and
written when the file is closed; while the file is opened, the free space chain is
maintained in storage.
<p>
<hr noshade>
<p><h3><a NAME="methodology">Methodology</a></h3>
This section is tedious; you probably want to skip to the
<a href="#quickstart">next section</a> unless you are genuinely
curious as to how cckd actually works. This section is for my edification
as much as anything...
<p>
<b>Initialization</b>
<p>
When a CKD dasd emulation file is initialized, function <code>ckddasd_init_handler</code>
in <code>ckddasd.c</code> is called. After <code>ckddasd_init_handler</code>
has completed its initialization, if the file is a <b>cckd</b> file, or if
<i>shadowing</i> was specified, function <code>cckddasd_init_handler</code>
in <code>cckddasd.c</code> is called.
<p>
<code>cckddasd_init_handler</code> obtains a <i>cckd extension</i> and stores
its address in field <code>cckd_ext</code> in the <code>DEVBLK</code> (the
control block that represents the device).
<p>
The compressed device header (<code>CCKDDASD_DEVHDR</code>) and <code>L1TAB</code>
for the file is read; however, if the file is a regular file, a dummy
<code>CCKD_DEVHDR</code> is built (all zeroes) and a dummy <code>L1TAB</code>
is built (all 0xff's).
<p>
If shadow files exist, they are opened and their <code>CCKD_DEVHDR</code>s and
<code>L1TAB</code>s are read. If the last file opened could only be opened read-only,
then a new shadow file is created.
<p>
The basic point is that the <code>CCKD_DEVHDR</code> and the <code>L1TAB</code>
for the base file and each shadow file is read and stored in an array in the
<i>cckd extension</i>, and each file is opened read-only, except the
current file, which is opened read-write.
<p>
<b>File I/O</b>
<p>
In the course of executing a channel program, routines in <code>ckddasd.c</code>
normally call the <i>lseek</i>, <i>read</i>, and <i>write</i> c library routines.
These routines (as we all know;-) perform the following functions:
<p>
<table border=0>
<tr><td align="left" valign="top"><b>lseek</b></td>
<td>Sets the current file offset to the specified value</td>
<tr><td align="left" valign="top"><b>read </b></td>
<td>Reads the specified number of bytes from the current file offset
to a buffer and increments the current file offset accordingly</td>
<tr><td align="left" valign="top"><b>write</b></td>
<td>Writes the specified number of bytes from a buffer to the current
file offset and increments the current file offset accordingly</td>
</table>
<p>
If the <i>cckd extension</i> is present, however, routines <i>cckd_lseek</i>,
<i>cckd_read</i> and <i>cckd_write</i> in <code>cckddasd.c</code> are called instead.
<p>
Routines <i>cckd_read</i> and <i>cckd_write</i> are not very interesting,
they merely copy data to/from the caller's buffer from/to the <i>active</i>
uncompressed track image buffer. <i>cckd_write</i>, it should be noted,
sets a bit indicating that the active track image has been <i>updated</i>.
All the interesting work results from <i>cckd_lseek</i> being called...
<p>
<b>Track Switching</b>
<p>
From the offset passed to <i>cckd_lseek</i> by <code>ckddasd.c</code>, using
the formula described <a href="#formula1">above</a>, the requested track can
be calculated using:
<center>
<code>trk = (offset - 512) / maxtrksz</code>
</center>
If the calculated track number is the same as the <i>current</i> track number,
then the new offset is noted and the function returns. Otherwise, an event
known as a <i>track switch</i> occurs, and high-level function <i>cckd_read_trk</i>
is called to make the new track the <i>active</i> track and the new track number
the <i>current</i> track number.
<p>
<b>cckd_read_trk</b>
<p>
Function <i>cckd_read_trk</i> scans the <i>track cache</i> array to see if the new track
image is cached. (By default, a cylinder's worth of track images are cached).
If the new track is found to be cached, then the <i>timestamp</i> in the cache entry
is updated, and the track image pointed to by the cache entry is made <i>active</i>
(this is known as a <i>cache hit</i>).
Otherwise, the cache entry with the oldest <i>timestamp</i> (called the
<i>least recently used</i> or <i>lru</i> entry) is <i>stolen</i>.
<p>
If the <i>stolen</i> cache entry has had it's track image buffer updated
(by <i>cckd_write</i>), then high-level routine <i>cckd_write_trk</i> is
called to place the buffer on the <i>deferred write queue</i> and a new
buffer is obtained.
<p>
If writes have previously occurred for the file, then the <i>deferred write queue</i>
is scanned to see if the new track image has been queued to be written. If the
new track image is in the <i>deferred write queue</i> then the current <i>lru</i>
buffer is discarded and replaced with buffer scheduled to be written and
<i>cckd_read_trk</i> returns, counting the encounter as a <i>cache hit</i> since
no physical i/o to be performed. (Also, a cache bit, called the <i>writing</i>
bit is turned on, indicating that if this cache entry is later stolen, then
a new buffer must be obtained).
<p>
Otherwise, low-level routine <i>cckd_read_trkimg</i> is called to physically
read the track image; the image is uncompressed (if necessary), and the cache entry
<i>timestamp</i> is updated.
<p>
If <i>cckd_read_trk</i> was called by the i/o thread (ie by <i>cckd_lseek</i>),
and the new track number is one more than the <i>current</i> track number,
function <i>cckd_readahead</i> is called to asynchronously read 1 or more
following track images, if those track images are not already in the
<i>track cache</i> and <i>readahead</i> is enabled. <i>cckd_readahead</i>
signals 1 or more readahead threads, implemented in function <i>cckd_ra</i>.
<i>cckd_ra</i>, when signalled, calls <i>cckd_read_trk</i> to read the requested
track. <b>Note</b> readahead is currently <i>disabled</i> for windows32 due
to some, as yet unknown, problem in the pthreads implementation.
<p>
If the <i>stolen</i> cache entry had had its track image buffer updated (and
<i>cckd_write_trk</i> was called), then the <i>deferred-write-thread</i> is
signalled to actually begin the process of writing the old updated track image.
We see that the updated track image is not sheduled to be written until <i>after</i>
the new track image has been read and the readahead threads have been signalled.
Further, we see that an updated track image is not scheduled to be written until
its cache entry has been stolen; hence the moniker <i>very-lazy-write</i>.
<p>
After all this, <i>cckd_read_trk</i> returns, and the new track image buffer
is made <i>active</i> and the new track number is made <i>current</i>.
<p>
<b>cckd_write_trk</b>
<p>
Function <i>cckd_write_trk</i> is called by <i>cckd_read_trk</i> whenever
a <i>stolen</i> cache entry's track image buffer has been updated.
The function obtains a <i>deferred-write</i> entry and places the
entry at the head of the <i>deferred-write</i> queue. If this is the
first time that <i>cckd_write_trk</i> has been called for the device
(ie the first write), then a bit is set on in the <code>CCKDDASD_DEVHDR</code>
indicating that writes have occurred for the file, and the <i>deferred-write</i>
threads and the <i>garbage collection</i> thread are created.
It is the responsibility of the caller of <i>cckd_write_trk</i> to
actually signal the <i>deferred-write</i> thread (<i>cckd_dfw</i>) to initiate
the write process.
<p>
<b>cckd_dfw</b>
<p>
Function <i>cckd_dfw</i> is a high-level routine that actually causes a
track image to be written. It pops an entry off the <i>deferred-write</i>
queue, compresses the track image, calls the low-level routine
<i>cckd_write_trkimg</i> to perform the physical i/o, and releases the
track image buffer (unless the track is still in the <i>track cache</i>,
then the <i>updated</i> bit is turned off).
<p>
<i>cckd_dfw</i> will also <i>throttle</i> the <i>deferred-write</i> queue if
it becomes too large; this causes the callers of <i>cckd_write_trk</i>
(eg <i>cckd_read_trk</i>) to be suspended until the queue drops below its threshold.
<p>
More than one <i>deferred-write</i> thread can be created, by specifying a parameter
on the device statement. The benefits, if any, of multiple threads has not
yet been shown.
<p>
<b>High level vs Low level routines</b>
<p>
It has been casually mentioned above that <i>cckd_read_trk</i>, <i>cckd_write_trk</i>
and <i>cckd_dfw</i> are <i>high-level</i> routines and <i>cckd_read_trkimg</i>
and <i>cckd_write_trkimg</i> are <i>low-level</i> routines. In this context,
<i>high-level</i> routines have no dependcy on the underlying file structure
while the <i>low-level</i> routines do. The <i>high-level</i> routines are
thread-aware while the <i>low-level</i> routines are not.
<p>
<b>The Low level routines</b>
<p>
There are low level routines to read and write each of the components
of the <b>cckd</b> file structure (headers, l1tabs, l2tabs, track images,
and free spaces); each are cognizant of the base file and shadow files, if
any. These routines consist of the following functions
<table>
<tr><td>&nbsp <td valign="top"><b>cckd_read_chdr <td>read the compressed header
<tr><td>&nbsp <td valign="top"><b>cckd_write_chdr <td>write the compressed header
<tr><td>&nbsp <td valign="top"><b>cckd_read_l1 <td>read the primary lookup table
<tr><td>&nbsp <td valign="top"><b>cckd_write_l1 <td>write the primary lookup table
<tr><td>&nbsp <td valign="top"><b>cckd_write_l1ent <td>write a primary lookup table entry
<tr><td>&nbsp <td valign="top"><b>cckd_read_fsp <td>read the free space chain
<tr><td>&nbsp <td valign="top"><b>cckd_write_fsp <td>write the free space chain
<tr><td>&nbsp <td valign="top"><b>cckd_read_l2 <td>read a secondary lookup table
<tr><td>&nbsp <td valign="top"><b>cckd_write_l2 <td>write a secondary lookup table
<tr><td>&nbsp <td valign="top"><b>cckd_read_l2ent <td>read a secondary lookup table entry
<tr><td>&nbsp <td valign="top"><b>cckd_write_l2ent <td>write a secondary lookup table entry
<tr><td>&nbsp <td valign="top"><b>cckd_read_trkimg <td>read a track image
<tr><td>&nbsp <td valign="top"><b>cckd_write_trkimg <td>write a track image
</table>
All writes occur to the <i>current</i> file. The compressed header and the primary
lookup table are kept in storage, so they are read once. Free space is read when the
first write occurs for the file and is written when the file is closed.
Secondary lookup tables are <i>cached</i>, so <i>cckd_read_l2</i> doesn't necessarily
perform physical file i/o. The <i>base</i> file can be a regular CKD file.
<p>
<b>Shadow file routines</b>
<p>
The routines that manipulate the shadow files are
<table>
<tr><td>&nbsp <td valign="top"><b>cckd_sf_name</b> <td>generates a file name for a given shadow
or base file
<tr><td>&nbsp <td valign="top"><b>cckd_sf_init</b> <td>performs shadow file initialization
<tr><td>&nbsp <td valign="top"><b>cckd_sf_new</b> <td>creates a new shadow file
<tr><td>&nbsp <td valign="top"><b>cckd_sf_add</b> <td>adds a shadow file (<b>sf+</b> panel command)
<tr><td>&nbsp <td valign="top"><b>cckd_sf_remove</b> <td>removes a shadow file, with or without
backwards merge (<b>sf-</b> panel command)
<tr><td>&nbsp <td valign="top"><b>cckd_sf_newname</b><td>sets a new shadow file name if shadowing
is not currently active
(<b>sf=</b> panel command)
<tr><td>&nbsp <td valign="top"><b>cckd_sf_stats</b> <td>display base and shadow file statistics
(<b>sfd</b> panel command)
</table>
<p>
<b>The garbage collector</b>
<p>
The garbage collection thread is created when the first write occurs to
the file. The garbage collection thread is only active for the <i>current</i>
file. When a new track image is written, the space it previously occupied in
the file is freed, and new space is acquired. The garbage collector moves
track images and secondary lookup tables to combine free spaces and tends to
move free space towards the end of the file so it can drop off. The garbage
collector also schedules track images to be written if they haven't been referenced
in some amount of time.
<p>
<b>Byte order</b>
<p>
As described above, a number of fields in the various blocks that comprise the
spaces in a compressed CKD Dasd emulation file contain offsets and lengths that
are more than 1 byte in length. Values in multiple bytes may be stored in
either <i>little-endian</i> or <i>big-endian</i> byte order. For example,
<b>Intel</b> architecture stores values in little-endian byte order and <b>S390</b>
architecture stores values in big-endian byte order. Consider the value <code>0x00010203</code>;
stored in little-endian byte order, we would see "<code>03020100</code>"; stored in big-endian
byte order, we would see "<code>00010203</code>". The values in the compressed CKD Dasd emulation
file are stored in byte order of the host machine; a bit in the <code>CCKDDASD_DEVHDR</code>
indicates which order its values are stored. If a file is opened with the wrong
byte order, then the initialization routine will automatically reverse all the values
before continuing.
<p>
If a base file or shadow is read-only and contains the wrong byte order, then the fields are
automatically converted when the blocks are read.
<hr noshade>
<p><h3><a NAME="quickstart">Quick Start</a></h3>
The <i>ckd2cckd</i> utility can be used to create a new compressed CKD file from a
regular CKD file. Your disk images can be a combination of regular CKD files and
compressed CKD files. Simply specify the names of your new compressed ckd files in
<code>hercules.cnf</code> in place of the regular CKD file names.
You can also use the <i>cckddump</i> program on an os/390 system to build a
compressed CKD file from a real disk that can be transferred to your Hercules machine
and used right away.
<hr noshade>
<p><h3><a NAME="usingsfiles">Using Shadow Files</a></h3>
Shadow files enable you to make updates to <b>cckd</b> emulation files and
not worry about possibly corrupting your entire disk image. I strongly urge
those of you who use <b>cckd</b> to start using shadow files immediately and
change your base file to read-only. This, in turn, reduces the amount of
data you have to back up, increasing the amount of file savings cckd has to
offer. You can even change shadow files to read-only, as long as a new shadow
file can be created. You can also use shadow files for regular (non-cckd)
files.
<p>
Shadow files are automatically enabled for <b>cckd</b> files; you <em>must</em>
explicitly enable them for regular CKD files. To enable shadowing for a CKD
device, specify
<p><center>sf=<i>shadow_file_name</i></center><br>
on the device statement in the <code>hercules.cnf</i> file. <i>shadow_file_name</i>
should include a spot in the file name, similar to multiple CKD dasd files,
that can be used as a sequence number, for example, <b>sf=../mvs/shadows/mvsres_1.500</b>.
The naming convention substitutes the shadow file number (1 thru 8) on the
character preceding the period after the last slash, or the last character
if no period follows the last slash. Example <p>
<b>0500 3390 ../mvs/disks/mvsres.500 sf=../mvs/shadows/mvsres_1.500</b>
<p>
If you did not specify <i>sf=</i> for a cckd file, or you wish to change
the shadow file name for a cckd or regular file, but no shadow files are
in use, then you can issue the following command on the Hercules console:
<br><center>sf=<i>xxxx</i> &nbsp <i>shadow_file_name</i></center><br>
where <i>xxxx</i> is the device unit address. For example,
<b>sf=0500 ../mvs/shadows/mvsres_1.500</b>.
<p>
Specifying a <i>shadow_file_name</i> does not explicitly create a shadow file
if the base file or current shadow file
is able to be opened read-write. Otherwise, if the base file and all existing
shadow files (if any) can only be opened read-only, then a new shadow file is created.
<p>
To explicitly create a new shadow file, issue the following command on the Hercules console:
<p><center>sf+<i>xxxx</i></center><br>
where <i>xxxx</i> is the device unit address. For example, <b>sf+0500</b>.
All updated track images that haven't been written are written and the <i>current</i>
file is hardened. Note that if a lot of write activity is ocurring at the time the
<b>sf+</b> command is entered, then the exact state of the hardened file can not
be predicted. A new shadow file is created and all new writes are directed to it.
<p>
To remove the <i>current</i> shadow file, issue either of the following commands
on the Hercules console:
<p><center>sf-<i>xxxx</i></center>
<center>sf-<i>xxxx</i> nomerge</center><br>
where <i>xxxx</i> is the device unit address. For example, <b>sf-0500</b>.
If <b>nomerge</b> was not specified, then the <i>current</i> shadow file contents
are merged into the preceding shadow file or base file. The <i>current</i> shadow
file is deleted and the preceding shadow file or base file is made the <i>current</i>
file. If the preceding file is read-only, then an error message is issued.
If possible, you can make the preceding file read-write and re-issue the command.
Note that if <b>merge</b> is specified or implied, then the command may take some
amount of time depending on the size of the old <i>shadow</i> file.
<br>[<i>hmmm... note to myself -- if sf-xxxx nomerge was specified and preceding file is
read-only, then delete the current file and recreate it ??</i>]
<p>
To display the status and statistics for a shadow-enabled file,
issue the following command on the Hercules console:
<p><center>sfd<i>xxxx</i></center><br>
This command displays status and statistics for the base file and all shadow files
representing the emulated dasd. The following data is displayed:
<table>
<tr><td>&nbsp <td valign="top"><b>size</b> <td>The total size of the file
<tr><td>&nbsp <td valign="top"><b>free</b> <td>The amount of free space in the file as a percentage
of the file size
<tr><td>&nbsp <td valign="top"><b>nbr</b> <td>The number of free spaces in a file
<tr><td>&nbsp <td valign="top"><b>st</b> <td>File open status - <b>ro</b>=read-only;
<b>rd</b>=read-only, but can be opened read-write;
<b>rw</b>=read-write
<tr><td>&nbsp <td valign="top"><b>reads</b> <td>Number of times <i>cckd_read_trkimg</i> performed
physical read i/o
<tr><td>&nbsp <td valign="top"><b>writes</b> <td>Number of times <i>cckd_write_trkimg</i> performed
physical write i/o
<tr><td>&nbsp <td valign="top"><b>l2reads</b> <td>Number of times a secondary lookup table was read
<tr><td>&nbsp <td valign="top"><b>hits</b> <td>Number of times <i>cckd_read_trk</i> found a track
image in the <i>track cache</i> when called by the
i/o thread
<tr><td>&nbsp <td valign="top"><b>switches</b><td>Number of times <i>cckd_read_trk</i> was called by
the i/o thread (<i>cckd_lseek</i>)
<tr><td>&nbsp <td valign="top"><b>readaheads</b><td>Number of track images read by the <i>readahead</i>
threads
<tr><td>&nbsp <td valign="top"><b>misses</b> <td>Number of track images read by the <i>readahead</i>
threads that were never referenced when the
<i>track cache</i> entry was <i>stolen</i>
</table>
<p>
<b>Special note when using shadowing for regular CKD files</b>
<p>
You can use shadow files with regular CKD files providing that the regular
CKD file is contained in a single file (since only 1 base file is supported)
and if the <b>sf=</b> option was specified on the device initialization statement.
If shadowing is active for a regular CKD file, then all i/o for the file is
performed by the cckd code. Interestingly, if shadowing is specified for a
regular CKD file, but the CKD file is opened read-write, and no <b>sf+</b>
command is issued to create a shadow file, then the regular CKD file is processed
as a cckd file, with asynchronous readaheads, deferred writes and garbage collection
(all the garbage collector does in this case is schedule updated track images for
write after a specified amount of time). The final caveat is that cckd files, and
by extension shadow files, are always the size of the device type, while regular
CKD files can be less. For example, you can specify a 100 cylinder 3390 regular
CKD file, but with shadowing, the file size will appear to be 1113 cylinders
(the size of a 3390-1 device). The device may have to be varied offline and back
online to the operating system (or the equivalent) for the new space to be recognized.
However, if you do write data to the newly provided space, then a backwards merge
cannot be performed (<b>sf-</b>).
<hr noshade>
<p><h3><a NAME="options">CKD Options</a></h3>
In this section I will attempt to document all the options that can be
specified for a CKD file (regular or compressed) in the <code>hercules.cnf</code>
file (or on the <i>attach</i> panel command).
<table border=1>
<th><td>Regular<td>cckd<td>Function
<tr><td valign="top" align="left">lazywrite<br>nolazywrite<td align="center">X<td> &nbsp
<td>Data written to a cached track image will not be immediately written,
but will be written when a track switch occurs. Thus, only one write
will occur for a track image while it is the <i>active</i> image.
<i>nolazywrite</i>, the default, specifies that all writes are performed
when requested.
<tr><td valign="top" align="left"> fulltrackio<br> fulltrkio<br>ftio<br>
nofulltrackio<br>nofulltrkio<br>noftio<td align="center">X<td> &nbsp
<td>Specifies whether or not a full track will be read when a track switch
occurs. Subsequent reads to this track image will not cause any physical I/Os.
Turning on <i>fulltrackio</i> can considerably enhance CKD device response time.
However, if you are sharing CKD disk images with more than 1 instance of Hercules
at the same time when writes could occur, you should specify <i>nofulltrackio</i>.
The default is <i>fulltrackio</i>.
<tr><td valign="top" align="left">readonly<br>rdonly<br>ro<br><td align="center">X<td align="center"> X
<td>Causes the CKD file image to be opened <i>read-only</i>. Attempts to write to
the emulated device will cause an I/O error unless option <i>fakewrite</i> is
also specified. If <i>readonly</i> is specified for shadowed
file images, then the base file will be opened readonly and a shadow file will be
created if one doesn't exist.
<tr><td valign="top" align="left">fakewrite<br>fakewrt<br>fw<br>
<td align="center">X<td align="center"> X
<td>Writes to a <i>readonly</i> file will be considered successful even though no write
actually occurred. This option is only meaningful if <i>readonly</i> is also specified.
<i>Fakewrite</i> is ignored for shadowed file images.
<tr><td valign="top" align="left">cache=<i>n</i><td align="center">X<td align="center">X
<td>Specifies the number of track images that will be cached. The default is the number
of tracks per cylinder for the device. [For <b>cckd</b> files, the default is the
number of tracks per cylinder <i>plus</i> the number of readahead threads]. If
<i>nofulltrackio</i> is specified for a regular CKD file, then no caching occurs.
Caching <em>always</em> occurs for <b>cckd</b> files, although you can set the cache
value to 1.
<tr><td valign="top" align="left">sf=<i>file_name</i><td align="center">X<td align="center">X
<td>Specifies the name of the shadow file(s) for the emulated device. The name should have
a spot where the shadow file number can be inserted into the name (see
<a href="#usingsfiles">above</a>).
<tr><td valign="top" align="left">l2cache=<i>n</i><td align="center">* <td align="center">X
<td>Specifies the number of Secondary Lookup Tables (l2tabs) that will be cached for the
<b>cckd</b> or <i>shadowed</i> device. (Each l2tab is 2048 bytes). The default is 32.
<tr><td valign="top" align="left">dfwq=<i>n</i><td align="center">* <td align="center">X
<td>Specifies a threshold for the size of <i>deferred-write-queue</i> where processing will be
<i>throttled</i> if the size exceeds this number. Each entry in the <i>deferred-
write-queue</i> contains a pointer to a buffer whose size is <i>max-track-size</i>.
The default is 64.
<tr><td valign="top" align="left">wt=<i>n</i><td align="center">*<td align="center">X
<td>Specifies the time in seconds that an updated track image will be written after its
last reference. The <i>garbage collector</i> is responsible for scheduling these
old track images to be updated. The default is 60 seconds.
<tr><td valign="top" align="left">ra=<i>n</i><td align="center">*<td align="center">X
<td>Specifies the number of <i>readahead</i> threads (and number of tracks to be read ahead)
when sequential access to the emulated device is detected. That is, each track that
is read ahead of time is read by a different thread. A value between 0 and 9 can be
specified. Currently, readahead should be <em>disabled</em> for Windows32 due to
an unknown error involving the pthreads implementation. Default for WIN32 is 0
otherwise the default is 2.
<tr><td valign="top" align="left">dfw=<i>n</i><td align="center">*<td align="center">X
<td>Specifies the number of <i>deferred write</i> threads. A number between 1 and 9
may be specified; the default is 1. It has not been shown that specifying a greater
number results in any performance improvements.
</table>
<font size=-1><b> * </b>Option is only applicable if shadowing is active for the regular CKD file.
</font>
<p>
Generally, the defaults for all options (except <b>sf=</b>) should not be changed unless there
is an explicit reason for doing so. If you use <b>cckd</b> files, then I strongly recommend
that you start using <i>shadow</i> files. If you use regular CKD files, then you can use
<i>shadow</i> files if you want to gain the <i>snapshot</i> benefit .
<hr noshade>
<p><h3><a NAME="utilities">Utilities</a></h3>
<a NAME="ckd2cckd">
<li><b>ckd2cckd</b> <i>[options] source-file target-file</i>
<ul><li><small><b>Description</b></small> Copies a regular CKD Dasd emulation
file to a compressed CKD Dasd emulation file. The target
file cannot previously exist. If the emulated Dasd device
is in more than 1 file then specify the <em>first</em> file.
After the copy completes, the target file contains no
free space, imbedded or otherwise.
<li><small><b>Options</b></small>
<ul><li><b>-c</b>ompress <i>n</i><br>Compression Algorithm
<ul><li><b>0</b> don't compress
<li><b>1</b> compress using zlib
<li><b>2</b> compress using bzip2
</ul>
<li><b>-d</b>ontcompress <i>n</i><br>Same as <i>-compress 0</i>
<li><b>-m</b>axerrs <i>errs</i><br>Maximum number of errors
that can occur before the copy is terminated;
if 0 then errors are ignored. Default is 5.
<li><b>-n</b>ofudge<br>[deprecated]
<li><b>-q</b>uiet<br>Quiet mode; don't display status
<li><b>-z</b> <i>parm</i><br>Parameter passed to compression
<br>
<br>zlib compression level:
<br>0 = no compression
<br>1=fastest ... 9=best
<br>
<br>bzip2 blockSize100k value:
<br>1=fastest ... 9=best
</ul>
</ul>
</ul>
<a NAME="cckd2ckd">
<li><b>cckd2ckd</b> <i>[options] source-file target-file</i>
<ul><li><small><b>Description</b></small> Copies a compressed CKD Dasd emulation
file to a regular CKD Dasd emulation file. The target
file cannot previously exist. More than 1 target file may
be created.
<li><small><b>Options</b></small>
<ul><li><b>-c</b>yls <i>n</i><br>Number of cylinders to copy
if the entire file isn't to be copied. If <b>0</b>
then only the number of cylinders in use are copied.
<li><b>-m</b>axerrs <i>errs</i><br>Maximum number of errors
that can occur before the copy is terminated;
if 0 then errors are ignored. Default is 5.
<li><b>-q</b>uiet<br>Quiet mode; don't display status
<li><b>-v</b>alidate<br>Validate track images [default]
<li><b>-n</b>ovalidate<br>Don't Validate track images
</ul>
</ul>
</ul>
<a NAME="cckdcdsk">
<li><b>cckdcdsk</b> <i>[-level] file-name</i>
<ul><li><small><b>Description</b></small> Performs compressed or shadowed CKD Dasd emulation
file integrity verification and recovery and repair.
<li><small><b>Options</b></small>
<ul><li>-<i>level</i><br>A digit 0, 1 or 3 that specifies
the level of checking. The higher the level, the
longer the integrity check takes.
<ul><li><b>0</b> Minimal checking. Device headers are verified,
free space is verified, primary lookup table and secondary
lookup tables are verified.
<li><b>1</b> Same checks as level 0 plus all 5-byte track headers
are verified.
<li><b>3</b> Same checks as level 1 plus all track images are
read, uncompressed and verified.
</ul>
</ul>
</ul>
</ul>
<a NAME="cckdcomp">
<li><b>cckdcomp</b> <i>[-level] file-name</i>
<ul><li><small><b>Description</b></small> Removes all free space from a compressed
or shadow CKD Dasd emulation file. (Compresses or compacts a cckd
file ... your choice!).
If <i>level</i> is specified, then <b>cckdcdsk</b> is called first
with the specified level; this is a short-hand method to call both
functions in one utility call.
<li><small><b>Options</b></small>
<ul><li>-<i>level</i><br>A digit 0, 1 or 3 that specifies
the level of checking. The higher the level, the
longer the integrity check takes.
<ul><li><b>0</b> Minimal checking. Device headers are verified,
free space is verified, primary lookup table and secondary
lookup tables are verified.
<li><b>1</b> Same checks as level 0 plus all 5-byte track headers
are verified.
<li><b>3</b> Same checks as level 1 plus all track images are
read, uncompressed and verified.
</ul>
</ul>
</ul>
</ul>
<a NAME="cckdfix">
<li><b>cckdfix</b> <i>file-name</i>
<ul><li><small><b>Description</b></small> This is a skeleton program that is
not compiled during make. It can be edited to change/repair
the device headers.
<li><small><b>Compiling</b></small> Enter `<i>cc -o cckdfix -DARCH=390 cckdfix.c</i>'
to compile and link the edited program.
</ul>
<li><b>cckddump</b>
<ul><li><small><b>Description</b></small> This is an os/390 hlasm (High Level
Assembler) program that will create a compressed CKD emulation file
from an actual CKD device. See <a href="#cckddump">below</a> for
a description on how to build and run this program.
</ul>
</ul>
<hr noshade>
<p><h3><a NAME="faq">FAQ</a></h3>
<table>
<tr><td valign="top"><b>Q.</b><td>
What devices are supported ?
<tr><td valign="top"><b>A.</b><td>
2311, 2314, 3330, 3340, 3350, 3375, 3380 and 3390.
However, I have only tested using 3390 devices.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
Is a 3390 model 9 supported ?
<tr><td valign="top"><b>A.</b><td>
The short answer is "no". Long answer, "sort of".
A 3390-9 should compress to a file size less than
the 2G limit. However, the compressed dasd program
"hooks" into <code>ckddasd.c</code> by replacing
the lseek, read and write library calls with a call
to an intermediate function. The file offset parameter
passed to lseek is a 32-bit signed number. For a
compressed file, the cckd code treats this number as
unsigned (for SEEK_SET) and uses this number to
calculate the dasd track and offset. That is, for a
compressed file, the file offset maintained by
<code>ckddasd.c</code> is just a number that indicates
a track and the offset into the track. That means
that the largest offset is 4G-1, which is not a problem
for a 3390-3 but only references about half of a 3390-9.
It would be possible to modify <code>ckddasd.c</code>
to use <code>long long</code> when dealing with file
offsets, but I wanted to minimize changes to
<code>ckddasd.c</code> and this change seemed a
little too intrusive.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
When I start hercules, I get these messages showing
all this free space in my compressed files. How do
I get rid of that free space ?
<tr><td valign="top"><b>A.</b><td>
Once the total amount of free space falls below 6% of
the total file size, the garbage collector is not very
aggressive about eliminating free space. To remove
all free space from the file, copy the compressed
file to a regular file using the
<a href=#cckd2ckd><code>cckd2ckd</code></a> utility
and then rebuild the compressed file by using the
<a href=#ckd2cckd><code>ckd2cckd</code></a> utility.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
How can I display the space statistics for a compressed
file ?
<tr><td valign="top"><b>A.</b><td>
The statistics are displayed when the compressed file
is opened. Currently, there is no supplied method to
display these statistics at any other time. However,
it shouldn't be too hard to write a shell script
(similar to <code>dasdlist</code>) to display these
statistics. The statistics are contained in the
<code>CCKDDASD_DEVHDR</code> which is at offset 512
in the compressed file; the header is mapped in
<code>hercules.h</code>.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
What is a "null track" anyway ?
<tr><td valign="top"><b>A.</b><td>
The term "null track" is just something I made up. It is
what is returned when a zero offset is found in either the
primary or secondary lookup table for the track. It contains
the folllowing fields:
<table>
<tr><td><code>0CCHH</code></td><td>Home address</td>
<tr><td><code>CCHH0008 00000000</code></td><td>standard R0</td>
<tr><td><code>CCHH1000</code></td><td>end-of-file marker</td>
<tr><td><code>ffffffff</code></td><td>end-of-track marker</td>
</table>
When a null track is written, space previously occupied by
the track is freed and the offset in the secondary lookup table
is set to zero. If all offsets in the secondary lookup table
are zero, then the secondary lookup table is freed and the
primary lookup table entry is zeroed.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
I want to try bzip2 but I'm getting compiler errors.
What am I doing wrong ?
<tr><td valign="top"><b>A.</b><td>
Probably bzip2 is not installed or is not installed
properly. You can obtain bzip2 from
<a href="http://sourceware.cygnus.com/bzip2/">here</a>.
If bzip2 is installed, then you need to find the directory
where <code>bzlib.h</code> is installed and the
directory where <code>libbz2.a</code> is installed.
You can then add "-I <i>bzlib.h-directory</i>" to the
CFLAGS in the make file and add "-L <i>libbz2.a-directory</i>"
to the LFLAGS.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
Which is better, zlib or bzip2 ?
<tr><td valign="top"><b>A.</b><td>
This is a religious question. I have no actual preference,
I just wanted to make a choice available.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
Can other compression programs be used ?
<tr><td valign="top"><b>A.</b><td>
Yes. The program is architecturally structured so that other
compression algorithms can be added rather painlessly. This
will require, of course, an update to the source.
<br><br>
<tr><td valign="top"><b>Q.</b><td>
Can this compression scheme be used for FBA devices too ?
<tr><td valign="top"><b>A.</b><td>
I have not worked with FBA devices for over 20 years.
However, it seems to me that a similar program for FBA
devices should be simpler than this program for CKD devices
(none of those count/key/data fields mucking everything
up). Since an FBA block is 512 bytes, it might not
be efficient to have each block compressed individually;
it might be better to compress blocks in 32K or 64K chunks.
If someone asks very nicely, I may consider looking into it;-)
<br><br>
</table>
<hr noshade>
<p><h3><a NAME="changes">Changes</a></h3>
<ul>
<li>0.2.0
<ul>
<li>This release greatly enhances the stability of cckd files.
<li>Free spaces are read at file open time, maintained in storage,
and written at file close time. If a file is not successfully
closed, then free spaces are easily recovered by the chkdsk
function when the file is opened next.
<li>Imbedded free space is deprecated. Because the free space chain
is now in storage, the overhead required to keep an updated track
image at its same location in the file is no longer necessary;
the penalty for traversing the free space chain to find a new location
is greatly reduced. Also, because a free space header is no longer
written at the beginning of a freed space when it is freed, that space
is eligible for recovery in the event of a failure.
<li>Garbage collection is more efficient because it can combine the
most free spaces per iteration now that the penalty for scanning
free space is gone.
<li>Secondary lookup tables (or l2tabs) are now cached. Most overhead
I/O for cckd files is now eliminated.
<li>Support for read-only files is added.
<li>Utility <i>cckdcomp</i> is provided to remove all free space from
a cckd file.
</ul>
<li>0.2.1
<ul>
<li>The concept of shadow files is implemented, which logically performs
a snapshot function.
<li>Windows 32 support is enabled.
<li>Seems like there should be more but I can't mush my brain much farther.
Some new options have been added so #defines in hercules.cnf don't have
to be changed; overflow tracks are supported (I hope) -- Thanks Valery!!
</ul>
</ul>
<hr noshade>
<p><h3><a NAME="bugs">BUGS</a></h3>
This code is absolutely bug free; if you encounter any problems then
it must be a personal problem and you've done something wrong. Also,
there are no enhancements that can be made because I've already thought
of them all and implemented them. By the way, I have some prime soon to be
ocean front property in Tennessee to sell to the highest bidder;-)
<hr noshade>
<p><h3><a NAME="cckddump">cckddump os/390 hlasm program</a></h3>
The cckddump program (supplied in file <code>cckddump.hla</code>) is an
os/390 assembler language program that creates a compressed CKD Dasd emulation
file from a real DASD volume. This program must be APF-authorized since
it modifies the DEB to be able to read all tracks from the real device.
The program executes 16 or so instructions while in supervisor state/key 0;
otherwise the program runs entirely in problem state/key 8.
It is not the prettiest assembler language program I've ever written, and
there are plenty of enhancements that I originally intended to put into the
program that I haven't yet; once I got the program working good enough, I
spent the rest of my time writing the fun stuff, the Hercules part.
<p>The real CKD Dasd volume that is dumped must be an ECKD device (ie support
'Locate Record' and 'Read Track' CCWs); this shouldn't be a problem because
I don't think any os/390 release supports a non-ECKD device. The output file
must be a DASD file; its characteristics are LRECL=4096, BLKSIZE=4096, RECFM=F.
The program only dumps allocated tracks (plus track 0) and only dumps tracks up
to DS1LSTAR for DSORG=PS and DSORG=PO files. The program will call zlib
to compress the track images if the zlib routines have been linked with the
program; however, I don't think the program will be advantageous if it can't
call zlib.
<p>
<h4>Preparing zlib</h4>
<ul>
<li>zlib can be obtained from
<a href="http://www.info-zip.org/pub/infozip/zlib/"><b>here</b></a>
<li>Copy or ftp the *.c files to a LRECL=255,RECFM=VB partitioned dataset;
here we will call the dataset <i>prefix</i>.ZLIB.C
<li>Similarly, copy or ftp the *.h files to a LRECL=255,RECFM=VB partitioned
dataset; we'll call it <i>prefix</i>.ZLIB.H
<li>Edit member <i>prefix</i>.ZLIB.H(ZCONF). Near the bottom, before the 2nd
to last <code>#endif</code>, add the following lines:<br><pre>
# pragma map(compress,"COMPRESS")
# pragma map(compress2,"COMPRES2")
# pragma map(uncompress,"UNCOMPRE")
</pre>
<li>Allocate an object partitioned dataset <i>prefix</i>.ZLIB.OBJ;
LRECL=80,BLKSIZE=3200,RECFM=FB.<br>Submit the following job to compile zlib:
<pre>
// JOB
//CC JCLLIB ORDER=(CBC.SCBCPRC)
//*
//ADLER32 EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(ADLER32)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(ADLER32),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//COMPRESS EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(COMPRESS)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(COMPRESS),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//CRC32 EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(CRC32)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(CRC32),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//DEFLATE EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(DEFLATE)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(DEFLATE),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//EXAMPLE EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(EXAMPLE)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(EXAMPLE),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//GZIO EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(GZIO)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(GZIO),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFBLOCK EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFBLOCK)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFBLOCK),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFCODES EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFCODES)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFCODES),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFFAST EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFFAST)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFFAST),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFLATE EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFLATE)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFLATE),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFTREES EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFTREES)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFTREES),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//INFUTIL EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(INFUTIL)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(INFUTIL),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//TREES EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(TREES)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(TREES),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//UNCOMPR EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(UNCOMPR)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(UNCOMPR),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
//*
//ZUTIL EXEC EDCC,INFILE='<i>prefix</i>.ZLIB.C(ZUTIL)',
// CPARM='RENT,LIST,SOURCE,LONGNAME,AGG,OPT(2)',
// OUTFILE='<i>prefix</i>.ZLIB.OBJ(ZUTIL),DISP=SHR'
//USERLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.H
</pre>
<li>Prelink zlib using the following job:
<pre>
// JOB
//PLKED EXEC PGM=EDCPRLK
//SYSMSGS DD DISP=SHR,DSN=CEE.SCEEMSGP(EDCPMSGE)
//SYSLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ
// DD DISP=SHR,DSN=CEE.SCEEOBJ
//SYSOUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSIN DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(ADLER32)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(COMPRESS)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(CRC32)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(DEFLATE)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(GZIO)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFBLOCK)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFCODES)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFFAST)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFLATE)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFTREES)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(INFUTIL)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(TREES)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(UNCOMPR)
// DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(ZUTIL)
//SYSMOD DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ(ZLIB)
</pre>
</ul>
<h4>Assemble and linkedit cckddump</h4>
<ul>
<li>Allocate partitioned dataset <i>prefix.cckddump.source</i>;
LRECL=80,RECFM=FB and copy or ftp file cckddump.hla
<li>Submit the following job:
<pre>
// JOB
//C EXEC PGM=ASMA90
//SYSLIB DD DISP=SHR,DSN=SYS1.MACLIB
// DD DISP=SHR,DSN=SYS1.MODGEN
//SYSPRINT DD SYSOUT=*
//SYSIN DD DISP=SHR,DSN=<i>prefix.cckddump.source</i>(CCKDDUMP)
//SYSUT1 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSLIN DD DISP=(,PASS),DSN=&&OBJ,UNIT=SYSDA,SPACE=(CYL,(1,1))
// LRECL=80,BLKSIZE=3200,RECFM=FB
//L EXEC PGM=HEWL
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSLIB DD DISP=SHR,DSN=CEE.SCEESPC
// DD DISP=SHR,DSN=CEE.SCEELKED
//ZLIB DD DISP=SHR,DSN=<i>prefix</i>.ZLIB.OBJ
//SYSLMOD DD DISP=SHR,DSN=<i>apfauth.load</i>
//SYSLIN DD DISP=(OLD,DELETE),DSN=&&OBJ
// DD *
INCLUDE ZLIB(ZLIB)
INCLUDE SYSLIB(EDCXHOTL)
INCLUDE SYSLIB(EDCXHOTU)
INCLUDE SYSLIB(EDCXHOTT)
ORDER MAIN(P)
ENTRY MAIN
SETCODE AC(1)
NAME CCKDDUMP(R)
</pre>
<li>The assemble step (C) should complete with condition code 4.
This is a `feature' due to the way IBM macro IECSDSL1 is coded.
The linkedit step (L) should complete with condition code 0.
</ul>
<h4>Executing cckddump</h4>
<ul>
<li>The volume to be dumped is identified by the SYSUT1 DD statement;
the output compressed CKD Dasd emulation file is identified by the
SYSUT2 DD statement.
<li>Submit a job similar to the following:
<pre>
// JOB
//S1 EXEC PGM=CCKDDUMP
//STEPLIB DD DISP=SHR,DSN=<i>apfauth.load</i>
//SYSPRINT DD SYSOUT=*,RECFM=VB,LRECL=255,BLKSIZE=4096
//SYSUT1 DD DISP=OLD,UNIT=SYSDA,VOL=SER=<i>volser</i>
//SYSUT2 DD DISP=(,CATLG),DSN=<i>prefix.volser.cckd</i>,
// UNIT=SYSDA,SPACE=(TRK,(7500,1500),RLSE),
// LRECL=4096,BLKSIZE=4096,RECFM=F
</pre>
</ul>
<h4>Make the file available to Hercules</h4>
<ul>
<li>Copy or ftp <i>prefix.volser.cckd</i> in <b>binary</b> mode
to your platform running Hercules.
</ul>
<hr noshade>
<p><h3><a NAME="feedback">Feedback</a></h3>
Questions ?? Problems ?? Comments ?? Suggestions ?? Corrections ?? Bugs ??<br>
Let me know at <a href="mailto:gsmith@nc.rr.com"><em>gsmith</em>&#064;<em>nc.rr.com</em></a>
<p>
greg smith
<p><small>Last updated 11 January 2001</small>
</BODY>
</HTML>
Reference in New Issue View Git Blame Copy Permalink