Date: Sat, 13 Jan 2007 07:42:01 -0500 (EST)
From: Herbert J. Bernstein <yaya@bernstein-plus-sons dot com>
To: Miroslav Kobas <miroslav.kobas@gmail dot com>
Cc: eric@eikenberry dot com Wolfgang.Kabsch@mpimf-heidelberg.mpg dot de,
     wright@esrf dot fr, hammersl@esrf dot fr, sweet@bnl dot gov
Subject: Thank you and status

Dear Miro,

  I am nearing the end of my stay and, before I lose email
contact for a while, I want to be certain the say thank you
to you, Eric, and all the rest at PSI and SLS and, especially
to Wolfgang, for making this both a productive and pleasurable
visit.  Your facility and your hospitality are both excellent.
Walking through the SLS ring was great fun and dinner
in Baden was a treat.

  I am pleased that we have come to an agreement on
how to proceed.  Allow me to use this email message
to review my understanding of what we are doing.
Please feel free to correct or add to what I say.
I am a bit tired and may make mistakes or forget
something.

  1.  I will add additional mime headers reported
at the start of each binary section.  To ensure
compatability with existing CBF files, the new
headers will be optional, so that CBFlib will
be able to read existing files without declaring
an error, but it will default to writing them.
The full list will now be:

     Content-Type:
     Content-Transfer-Encoding:
     Content-MD5:
     X-Binary-Size:
     X-Binary-ID:
     X-Binary-Element-Type:
     X-Binary-Element-Byte-Order:
     X-Binary-Size-Fastest-Dimension:
     X-Binary-Size-Second-Dimension:
     X-Binary-Size-Third-Dimesion:
     X-Binary-Size-Padding:

I will provide more detail of how these are filled in
as I revise the documentation in doing the code, but
one detail that I should note for Eric is that
currently CBFlib will only write small_endian files
(even from within a big-endian machine), but it
is a good idea to add this information into the
mini-header now to avoid errors.

I have made good progress on these headers and hope
to be able to finish them during my time at ESRF.
I will discuss them with Any and Jon to see if
they need any additions to the list, and am
copying them and Bob on this message.  I have saved
a copy of what I have done so far in a tarball upload
to the states, so all should be well even if
my computer gets clobbered.

  2.  I will be adding code for two compressions
to CBFlib:  Wolgang's version of Andy's byte_offsets
algorithm (this will fill a long-standing gap
in CBFlib and should be useful to many people), and
a full 32-bit 4-way averaging version of the algorithm
in JPA's pack_c.c.  The second one will share
a lot of code with the current packed compression
in CBFlib, but it is sufficently different that
I will add it as a new compression option: pack_jpa.
I have gotten approval from Martyn Winn at ccp4 and
am waiting to hear back from Abrahams, but I am
confident that we will be able to incorporate
it within CBFlib as part of the GPL/LGPL'd code.

  3.  As I get to the point of having a properly
working version of each of these new compressions,
I will send sample files to Wolfgang and Eric
along with my suggestions on how to do a quick
read in Fortran-90 code, just scanning for
and using the mini-header.  I will try to clean out the
flat-field 32 bit bug in the current jpa code
that Eric found before sending Wolfgang the samples.

  4.  I will be working with the sample files and
code that Eric provided to try and make realistic
examples of how I think your data would look
in imgCIF.  I will have to do that part at home
on a bigger machine (or on a faster network
connection).

  5.  I will look at Eric's need for memory-based
I/O in place of disk-based I/O, since I have to
do the same thing for RasMol to handle maps,
but the priority of the above tasks means that
I probably won't be able to do more than give
Eric friendly advice and suggestions on that
topic before your February run.

  6.  When time permits, I will float the idea
of deprecating tags in the imgCIF dictionary that
are no longer used or which should no longer
be used, and then moving them to a separate
imgCIF_obsolete dictionary.

  7.  I will try to take what I learning from
my very helful discussions with Eric and try
to make the documentation of both imgCIF
and CBFlib clearer.

As I understand from Eric, the target timing
for me to get code back to him is 3 weeks
from now.  I will try to get samples and
Fortran-90 code to Wolfgang sooner than that.

I hope I covered everything.

Thank you again and please let me know what I missed
or got wrong.

  Regards,
    Herbert

P.S to Eric -- thank you for pointing out the
gcc 4 warnings.  Cleaning those out was a
good idea.  I am glad it is done.

P.P.S. And thank somebody for the nice weather.
=====================================================
 Herbert J. Bernstein, Professor of Computer Science
   Dowling College, Kramer Science Center, KSC 121
        Idle Hour Blvd, Oakdale, NY, 11769

                 +1-631-244-3035
                 yaya@dowling dot edu
=====================================================