| IUCr Home Page
| CIF Home Page
| BioSync | MEDSBIO list | MEDSBIO list archive | Meetings |
| Make a Submission to the MEDSBIO web site |
Third imgCIF workshop (new series) at BSR 2007 in Manchester and at Diamond:
Sponsored by DOE under grant ER64212-1027708-0011962, NSF under grant DBI-0610407 and NIH under grant 1R13RR023192-01A1
14 Aug 07
14 Aug 07
17 Aug 07
17 Aug 07
17 Aug 07
The pace of data collection and the volume of data collected at synchrotron beam lines is increasing. The ACA Data, Standards, and Computing Committee spearheaded an effort to improve the efficiency of the handling and storage of these data by encouraging the adoption of common data formats and standard software interfaces. The goal of this was firstly to have the data be self defining, therefore equally accessible to data-reduction and -visualization codes. The second goal, for the purposes of secure archiving, was to provide robust internal documentation of the source of the data.
The current effort began in 2005, building on work started in the mid 1990's on a Crystallographic Binary Format (CBF) proposed by Andy Hammersley. This effort was the basis for the image-supporting Crystallographic Information Format/Crystallographic Binary Format (imgCIF/CBF). The first imgCIF/CBF workshop took place at Brookhaven National Laboratory in 1997 and proposed a format combining support for an efficient binary representation of images with a fully CIF-compliant ASCII equivalent. An imgCIF/CBF dictionary and software to support the format were created, are available on the web, and are described in Volume G of the IUCr International Tables for Crystallography. Now the community should adopt a consensus standard for management of data at synchrotron beam lines and to make it easier for users to process data taken from various beam lines. Also, as our science evolves, new concepts will be considered: possibilities include NeXus and XML.
The first workshop in the new series on "Management of Synchrotron Image Data: imgCIF File System and Beyond", was held on 22 July 2006 as part of the 2006 ACA meeting in Honolulu, Hawaii. That workshop concluded that that was "the right time for more widespread use of imgCIF ... [and that] SR sources should start writing imgCIF image files as soon as possible, employing the imgCIF dictionary already adopted by the IUCr Committee on the Maintenance of the CIF Standard (COMCIFS) and published on the web and in International Tables Volume G.] " [from the report of the workshop, see http://www.medsbio.org/meetings/ACA_2006_WK02_Report.html].
Subsequent to the Hawaii workshop intensive work was started in response to these recommendations. Both the imgCIF dictionary and the supporting software library were reviewed and, after meetings at SLS and ESRF, extended. See SLS_report.html and ESRF_report.html for more information on those meetings. The work continued in collaboration with members of the community (see the imgCIF mailing list http://www.iucr.org/iucr-top/cif/cbf/imgcif-l/.
In light of this activity a workshop on data formats for synchrotron image data was held after the NSLS/CFN meeting on 24 May 2007 at BNL in the Biology Department. Topics discussed included proposed extensions to imgCIF, the use of NeXus, progress on software and the status of imgCIF at Diamond and at SLS. That workshop concluded that work was needed on support for the handling of uncorrected images and true bitmaps, creation of a utility to "tidy" CIFS, creation of an agreed interface with XML, NeXus and HDF, clarification of the specification of the relationship between detector specification and the physical locations of pixels in the laboratory, tags for robotics and remotes access, and the creation of more cookbooks. See http://www.medsbio.org/mettings/BNL_May07_imgCIF_Workshop_Report.html.
In the short time between the second and third workshops, work on many of the items on this task list began. In addition, building on discussions with J. Steinbrener at the second workshop, discussions with Matt Dougherty on the image needs in the microscopy community, and, through Matt Dougherty, with Mike Folk of the HDF Group on techniques for integration between imgCIF and NeXus, HDF and XML were started. Work on CBFlib continued and version 0.7.8 was released.
The third imgCIF workshop was held in two sessions at BSR 2007 in Manchester and at Diamond. Herbert Bernstein and Alun Ashton organized this workshop. The purpose of this workshop was to provide a review of the status of imgCIF and CBFlib for the European user community and to discuss the integration of imgCIF with NeXus, HDF and XML. The Manchester session was used for an introduction and review of the status of imgCIF and for some discussion. The Diamond Light Source session was used to deal with more detailed technical issues and further discussion.
The first session was in Manchester on Tuesday, 14 August 2007. It provided an introduction to imgCIF and NeXus and a brief review of current progress. This session took place in the Barbirolli Room at Bridgewater Hall from 12.45 to 13.45. The session began with a review of the status of imgCIF by H. Bernstein, followed by a review of the status of use of imgCIF with the Pilatus 6M detector at the Swiss Light Source by C. Broennimann and a presentation of a developer's perspective on imgCIF by G. Winter. The major discussion at this session was about what information, in what format whould be in the header of an image presented as a CBF.
The second session was at the Diamond Light Source (DLS) on Friday, 17 August 2007, to discuss recent changes in the imgCIF dictionary and software and the interaction with NeXus. This session took place in room 1.17 of Diamond House. It began with short presentations and some discussion from 13:15 to 14:20. Then there was a break, and more substantive discussions continued from 14:30 until about 17:15. The presentations began with H. Bernstein giving a recap of the status of imgCIF and proposals on versioning and integration between imgCIF and NeXus. A. Ashton and C. Nielsen then presented various aspects of the use of imgCIF at DLS. M. Dougherty presented the status of data formats in cryo-electron microscopy. M. Folk reviewed the capabilities and status of HDF.
The Manchester session was attended by 20 people, 14 of whom were formally registered, and the DLS session was attended by 14 people, 11 of whom were able to remain for the full discussions. A total of 28 people participated in one or both sessions.
The discussions at both sessions were constructive and productive and, at times, passionate. While there were sharp disagreements, they were over how best to use imgCIF, not over whether to use imgCIF, and there seemed to be the reasonable propsect that eventual compromise and consensus will be achived after further discussions and email exchanges. The major points of discussion and their status were:
After some discussion, it was agreed that the important issue was not to specify what SLS might choose to use or produce but to specify what the major processing programs might require and to clearly identify each CBF file as to the completeness of the headers that they produce for purposes of processing by various programs. Alternatives considered included a special comment for incomplete headers, versus the special looped tag within each image data block specifying compliance with the needs of particular processing programs (e.g. _software_compliance.complies_with mosflm _software_compliance.complies_with_version "7.0.1" ) Harry Powell has provided a sample of the header items he proposes should be required (see this sample of header proposed by H. Powell). Discussions with SLS and Dectris representatives between the two workshop sessions and by email afterwards suggest that agreement on a full header to be offered to beamlines and users who want them is likely to be achievable.
(The pre-workshop agenda and charge is at http://www.medsbio.org/meetings/BSR_2007_imgCIF_Workshop.html).
The agenda for the workshop as held was
The workshop is completed. The following information is to assist participants in completing their travel paperwork.
The 14 May 2007 workshop session was held at the BSR 2007 meeting venue in Manchester. For travel information for that session, see http://www.srs.ac.uk/bsr2007/pages/travel.htm.
The 17 May 2007 workshop session was held as the Diamond Light Source (DLS). Registered participants at BSR 2007 were able to avail themselves of the transportation to DLS by making arrangements with the meeting organizers (anita at ya-ya dot co dot uk). Alternatively DLS travel information was available at http://www.diamond.ac.uk/AboutDiamond/Location
For those for whom travel reimbursement had been arranged, forms were provided at the workshops. A copy is available as a PDF BSR_Aug07_expense.pdf, which is to be filled out, signed and returned with receipts to Herbert J. Bernstein, Kramer Science Center KSC-121, Dowling College, Oakdale, NY 11769-1999, USA.
||Attend||Freddie Akeroyd|| F.A.Akeroyd at rl.ac dot uk
||Present||Alun W. Ashton|| Alun.Ashton at diamond.ac dot UK
||Attend||Mark Basham|| Mark.Basham at diamond.ac dot UK
||Present ||Present||Herbert J. Bernstein||
yaya at bernstein-plus-sons dot com
||Attend||Frances C. Bernstein||
fcb at bernstein-plus-sons dot com
christian.broennimann at psi dot ch
||Attend||Ian Clifton|| ian.clifton at chem.ox.ac dot uk
||Attend|| Attend ||Georgi Darakev|| darakevg at gmail dot com
||Attend|| Attend ||Nikolay Darakev|| darakevn at gmail dot com
||Present||Matt Dougherty|| matthewd at bcm.tmc dot edu
||Attend|| Attend ||Elizabeth M. Duke||
elizabeth.duke at diamond.ac dot uk
||Attend||Attend ||Judy Flippen-Anderson ||
flippen at rcsb.rutgers dot edu
Folk|| mfolk at hdfgroup dot org
|| Attend||John Jemilawon|| damie225 at gmail dot com
||Attend||Louise Jones|| lj at iucr dot org
|| Attend||Laurent Lerusse|| L.Lerusse at rl.ac dot uk
||Attend||Brian McMahon|| bm at iucr dot org
||Attend||Present ||Chris Nielsen|| cn at adsc-xray dot com
||Attend||Harry Powell|| harry at mrc-lmb.cam.ac dot uk
||Attend||Francois Remacle|| f.remacle at dl.ac dot uk
Stuart.Robinson at diamond.ac dot UK
clemens.schulze at psi.ch
||Present||Graeme Winter|| g.winter at dl.ac dot uk
|| Attend||Jonathan Wright|| wright at esrf dot fr