Announcing the release of version 4.8 of the Silo Library

This is the first, official BSD Licensed release of the Silo Library!

Organization of information in this announcement

1. Bugs fixed in this release.
2. Performance-oriented features in this release.
3. Other new features in this release.
4. Important information regarding BSD vs. Legacy Licensed release tarballs.

1. Bugs fixed in this release

• A bug where the Force Single behavior of the HDF5 driver was resulting in all objects' arrays being converted to float but then neglecting to set the associated 'datatype' member to DB_FLOAT was corrected.
• A few cases where Silo was erroneously leaving HDF5 objects open in a file, thereby causing the file itself to be left open in HDF5 even after DBClose was called have been corrected.
• Some problems building Silo on windows with newer compilers and with various configurations have been corrected. Thanks to Kathleen Bonnell for these fixes as well as porting a number of code updates to Windows and for laborious testing on Windows.
  
• The silofile utility was re-written to NOT use magic file numbers as that proved to be unreliable. Now the tool is much more robust at identifying silo files as well as their underlying format (PDB or HDF5).
• A bug swapping the slash character in the filename part of multi-block object block names when moving files between windows and Linux was fixed. Thanks to Al Nichols for reporting the and the fix.
  
• Leaks of optional species names and colors were corrected.

2. Performance-oriented features in this release

• To take advantage of some of the performance improvements described here, HDF5 Version 1.8.4 or newer is required.
• A number of options for using various Virtual File Drivers (VFDs) that come pre-packaged with the HDF5 library were added to Silo. The interface to controlling these VFDs was enhanced. See documentation on DBRegisterFileOptionsSet() in the Silo 4.8 user's manual for more information. See Overview of Silo block-based VFD and Scaling Studies of Silo block-based VFD for some details.
  
• There is a new, Silo-specific VFD designed specifically for BlueGene (e.g. BG/P and BG/Q) systems called the Silo block-based VFD. Use of this VFD is essential for adequate performance on BG/P systems. Thanks to Mike Collette and Richard Hedges for performing scaling studies to help identify performance issues on BG/P. Read more about this VFD and its options as well as a number of other VFDs and their options in the Silo 4.8 User's manual.
• Silo can now optionally link with and use PDB Proper in PACT using the --with-pdb-proper=path-to-pact-libs and 'DB_PDBP' for the 'type' argument in DBCreate/DBOopen calls. Thanks to Marty Marinak for testing and Stewart Brown for guidance in implementing this feature. This feature may be particularly important to codes using PDB driver on BlueGene systems (e.g. BG/P) because the PDB Proper in PACT has undergone substantial performance tuning for those systems.
• A new I/O performance benchmarking tool, ioperf, was added to Silo tests. This tool can be useful in comparing performance of the Poor Man's Parallel I/O strategy with any I/O library including sec2, stdio, PDB, HDF5, Silo as well as any others a user might wish to write plugins to test.
  
• A new function, DBSortObjectsByOffset() was added that given a list of object names an application wishes to read from a Silo file, will return the order in which the named objects appear in the file. This is useful in optimizing read operations from the file by potentially minimizing the frequency/number of seeks as well as improving efficiency of read-ahead.
• New options for using nameschemes instead of explicit lists of names of blocks for multi-block objects were added. These options can profoundly reduce the total size of multi-block objects as well as the root files into which they are written as well as the I/O time to read and write them.
  
• A performance issue where HDF5 driver was constantly writing and re-writing cycle, time and dtime for every DBPutXXX operation was corrected.
• A new friendly names option was added such that HDF5 datasets are in fact written as their friendly named variant in the current working directory instead of writing them in the magic /.silo directory with the friendly names being links to them. This may be important for scalability as the number of objects in the /.silo directory can otherwise grow quite large causing possible performance issues for HDF5 to handle the related metadata.

3. Other new features in this release

• Note that Silo Version 4.8 is NOT link-time, backward compatible with any previously released version. You will need to re-compile all code that includes the Silo header file, silo.h.
• Silo is now officially released with a BSD Open source license. Thanks to Eric Brugger for reviewing the approximately 150,000 lines of code as well as the LLNL IM Team for completing the review and release process.
  
• There is a new, public, redmine bug tracker for Silo; https://visitbugs.ornl.gov. Thanks to Sean Ahern and Oak Ridge National Labs for hosting it and thanks to Cyrus Harrison for setting it up. Before users can submit requests there, you will be required to register and create an account. But, that is a one-time operation. If you are already registered for the VisIt tracker, then you can get the Silo tracker automatically.
• A significant reduction in compiler symbol table global namespace clutter has been eliminated from Silo reducing possibility of symbol collision when linking large applications.
• A new error reporting level was added, DB_ALL_AND_DRVR. This level of error reporting is the same as DB_ALL except that it also includes any additional error reporting from the underlying driver.
• Support for arrays of strings of variable length was added to a number of functions in the Fortran interface that accept arrays of strings. Thanks to Olivier Cessenat for this patch.
• DBPUTQV which accepts a multi-component quadvar was added to the Fortran interface. Thanks to Olivier Cessenat for this implementation.
• A restriction on the number of components of a multi-component ucdvar or quadvar was relaxed so that now, for example, DBPutUcdvar can write a variable with 100 sub-variables.
• DBnamescheme objects now support references to external arrays of string value.
• DBErrFunc was deprecated. Use DBErrFuncname now. Also, a new method, DBErrfunc was added to return the function pointer for the error function set in the last most-recent DBShowErrors() call.
• A new function, DBErrlvl, was added to return the error reporting level set in the last, most-recent DBShowErrors() call.
• The test suite was enhanced substantially to run test on a variety of HDF5 virtual file drivers and library settings.
• Functions for catenating an array of strings into a single, semi-colon separated string and the reverse operation were added to the public API. See DBStringArrayToStringList and DBStringListToStringArray. These functions used to be solely internal to Silo but have proven to be useful for Silo applications needing to read and write arbitrary collections of strings.
• A new function, DBSetUnknownDriverPriorities was added to control the behavior of the unknown (e.g. DB_UNKNOWN) driver when opening files of unspecified low-level format type. Previously, the Silo library had a hard-coded sequence of drivers it would try to open a file using DB_UNKNOWN driver type. Now, that sequence can be specified by the Silo application.

4. Important information regarding BSD vs. Legacy Licenses

Copyright (C) 1994-2016 Lawrence Livermore National Security, LLC.
All rights reserved. This work was produced at Lawrence Livermore
National Laboratory under Contract No. DE-AC52-07NA27344 with the
DOE.

DISCLAIMER

This work was prepared as an account of work sponsored by an
agency of the United States Government. Neither the United States
Government nor the University of California nor any of their
employees, makes any warranty, express or implied, or assumes any
liability or responsibility for the accuracy, completeness, or
usefulness of any information, apparatus, product, or process
disclosed, or represents that its use would not infringe privately-
owned rights. Reference herein to any specific commercial products,
process, or service by trade name, trademark, manufacturer or
otherwise does not necessarily constitute or imply its endorsement,
recommendation, or favoring by the United States Government or the
University of California. The views and opinions of authors
expressed herein do not necessarily state or reflect those of the
United States Government or the University of California, and shall
not be used for advertising or product endorsement purposes.

NOTIFICATION OF COMMERCIAL USE

Commercialization of this product is prohibited without notifying
the Department of Energy (DOE) or Lawrence Livermore National
Laboratory (LLNL).
----------------------------------------------------------------

The BSD Open Source release distributions do not contain the Hzip and
FPzip compression libraries while the legacy release distributions do.
This means any objects within a Silo file containing data that is
compressed with Hzip or FPzip from a legacy distribution will not be
readable in a BSD distribution. Since compression features must be
explicitly turned on by a Silo application, this impact can be easily
mitigating by ensuring that Hzip and FPzip compression are used only
for objects within Silo files not intended to be read by BSD releases.

For more information on availability of Hzip and FPzip compression,
contact the developer, Peter Lindstrom, lindstrom2@llnl.gov.