snapraid/HISTORY

SnapRAID HISTORY
================

4.0 2013/7
==========
 * New 'scrub' command to periodically check the oldest blocks for silent
   errors without the need to scan the whole array.
 * New 'status' command to check the fragmentation, the last check time
   distribution, and the silent error status of the array.
 * Added the new Spooky hash. It's faster in 64 bit architectures.
   To convert you can use the new 'rehash' command.
 * Added a hash seed to make harder intentional collision attacks.
 * When insering files for the first time, sort them by their physical
   address to improve read performance. 
 * Better selection of the RAID6 implementation for different CPUs.   
 * Added RAID5/RAID6 mmx and sse2 implementations with unrolling by 4.
   They are a little faster than the previous unroll by 2.
 * Added a lock file to avoid multiple running instances on the same array.
   The file is named as parity file adding the .lock extension.
   There is also the undocumented --test-skip-lock to avoid to check it.
 * Automatically ignores, with warning, mount points inside the array
   directory tree.
 * Changes the 'dup' output format to include the size of each duplicate file.

3.1 2013/5
==========
 * Direct use of Windows API for disk access to improve error reporting.
 * If the 'fix' process is aborted, it removes all the new files partially
   recovered, to allow to reuse again the '-m, --filter-missing' flag.
 * In Windows don't exclude anymore system files. Only system directories are
   excluded.
 * In Windows applies filters in case insensitive way.
 * The Windows binaries are now built with gcc 4.7.2.
 * Reduced memory occupation for hardlinks and directories.
 * In 'dup' don't list file with 0 size.

3.0 2013/3
==========
 * Added pooling support with the new 'pool' command. It creates a virtual
   view of the array using symbolic links pointing to the original files.
 * Added a new -m, --filter-missing option that allow to undelete files,
   without checking/fixing the others.
 * Added a new -i, --import option to automatically import deleted files
   when fixing.
 * Added a new -l, --log option to save to disk the detailed log.
 * Added support also for hardlinks and empty directories.
 * Added support to save symlinks to files in Windows. Note that only the
   symlink is saved and not the linked file.
   Symlinks to dirs and junctions are still not supported in Windows.
 * Files without read permission generate an error instead of a warning.
   You now must explicitely exclude them in the configuration file with
   exclusion rules.
 * In 'check' and 'fix', if verbose is enabled, prints the result for each
   processed file.
 * Added an UUID check to detect when a disk is replaced, and to prevent
   unwanted disk swaps.

2.1 2013/1
==========
 * Checks for wrong empty fields in the configuration file.
 * Filter rules for files are not anymore applied to directories.

2.0 2012/12
===========
 * Added a new -a option to make the 'check' command to only check file hashes
   without checking the parity data.
 * Added a new -d option to filter by disk name.
 * The file modification time is now saved using nanosecond precision.
   This allows to restore the exact modification time in 'fix'.
   The new 'content' files written with this version are not backward
   compatible, but it's still possible to read the old 'content' format.
 * Fixed hard-links automatic exclusion. All the hardlinks after the first one
   are now correctly ignored.
 * If it isn't possible to grow a parity file, prints the list of files
   outside the maximum size allocated.
 * Autosave isn't triggered if we are near the end of the 'sync' process.
 * Before starting a 'sync', we wait for two seconds, to workaround the FAT
   limitation of having two seconds modification time precision.
   This a safe measure to be 100% sure to always detect file changes.
 * Always fill the memory after allocating it to avoid the OOM (Out Of Memory)
   killer in Linux.
 * Fixed compilation in Solaris/OpenIndiana for lacking both futimes()
   and futimens().
 * Now 'sync' ensures that the parity files are not too small to contain the
   just loaded data.
 * Removed the '-H,--filter-nohidden' option. It doesn't make sense to
   have it as command line option.
   You must use the 'nohidden' option in the configuration file.
 * When opening files in read-only mode, also specify the noatime flag,
   to avoid to update the file access time.
 * Exclude rules for files are now also applied to directories.
   This allows to excludes some file/directory without the need to call
   the stat() function on them.
 * The -N, --find-by-name option also ignores the nanosecond part of
   timestamps to work with copy programs not supporting nanoseconds.
 * Fixed deduplicated files handling in Windows Server 2012.
 * Removed MD5 support.

1.13 2012/11
============
 * Fixed a Segmentation Fault when checking/fixing if there are three
   or more errors in a specific block.

1.12 2012/9
===========
 * Fixed file renaming in Windows during a 'fix' command.
   This is only a Windows only issue, no reason to upgrade for other platforms.

1.11 2012/7
===========
 * Fixed again directories inclusion. Exclusion rules for directories were
   ignored.

1.10 2012/6
===========
 * Fixed directory inclusion, in case the last rule is an "include" one.
 * Fixed very long paths in Windows. We now always use the special '\\?' prefix
   to remove the 260 chars limitation.
 * If a file is excluded, it prints explicitely which attribute caused the
   exclusion.   
 * Automatically excludes also the temporary copy of content file,
   the one with the ".tmp" extension.
 * Avoid the Windows system to go in automatic sleep mode when running.

1.9 2012/3
==========
 * Implemented a more sophisticated recovering in case a harddisk failure
   happens during a 'sync' command.
   When using RAID6 it improves the chances of recovering data after an
   aborted 'sync'.
 * Fixed the count of new files.
 * Added a new 'autosave' configuration option to save the intermediate 'sync' state.
 * Supported filesystems with read requests returning less data than requested.
 * In Windows ensures that the disk serial number is not zero.

1.8 2012/1
==========
 * Added a new "dup" command to find all the duplicate files.
 * Added a new option "--filter-nohidden" to exclude hidden files.
 * Faster and parallel writing of content files.
 * The example configuration files now put the content files in the data
   disks instead than in the parity disks.
 * Added a checksum at the content file to ensure its integrity.
 * Using fallocate() instead posix_fallocate() to avoid the very slow
   posix_fallocate() fallback of writing the whole file.

1.7 2011/11
===========
* If a file is modified or removed during a sync, the sync process doesn't
  stop anymore, but it will simply skip the file, resulting in an incomplete
  sync. Note that the sync will terminate with an error.
* If the content file is placed in a data disk, it's automatically excluded
  from the sync process.
* Increased by one the minimum number of content files. Before it was only
  a suggestion, but now it's a requirement because you are allowed to put
  content files in data disks.
* Added checks to ensure that data and parity disks are different, and to
  correctly count the number of copies of "content" files in different disks.
* Removed the dependency of the "disk" order specification in the configuration
  file. The used order is now saved in the content file to avoid to damage the
  dual parity in case the order is changed by the user.
  It easily allows to remove or add disks from the array when using a
  dual parity.
* Improved the "fix" performance when a lot of files or the parity have to be
  recreated from scratch.
* When getting unrecoverable errors, the printed log line now starts with
  "unrecoverable:" instead of "error:" to allow an easier identification.
* Added a new option "--find-by-name" to allow to sync using only the
  file path and not the inode. This is useful to avoid long sync
  when you replace one disk with another copying manually the files.
* If "fix" cannot recover a file, it's renamed adding the ".unrecoverable"
  extension.
* Checking and fixing also empty files with size 0.

1.6 2011/9
==========
* The content file is now saved also at the start of the "sync" command.
  This avoids parity errors if the sync process is aborted without
  saving the content file and you made changes at the disk array before
  another "sync" command was done.
  More specifically, deletions or truncations of not yet synched files
  after the aborted sync, and before the next sync command, may have damaged
  the parity data. New file additions were instead safe.
  If these conditions may have happened to you a "check" command (also
  with older version of the program) is recommended to ensure the correctness
  of your parity data.
* The "diff" command now recognizes the reuse of inodes.
* Windows hidden files are now saved like any other files.
* Symbolic links are now saved in *nix. Not supported in Windows.
* The "fix" command restores also the original modification time.
* The message asking to use the --force-empty option now lists all the
  empty disks.

1.5 2011/7
==========
* Ignores extra spaces in the configuration file.
* Changed the output of check/fix to allow a more easy postprocessing
  with other tools like awk and sort.
* Added the hidden option -G/--gui to enable the output of progress
  information for a potential GUI for SnapRAID.
* Added a new "diff" command to print the list of changes detected
  at file level.
* Faster loading of content file. Approx three times faster.

1.4 2011/6
==========
* Ignoring in sync System and Hidden files in Windows.
* Files without read permission are ignored in sync.
* If a file is ignored a warning message is printed. You have to exclude it
  to remove the warning.
* In fixing, if a file cannot be written for missing permission, an error is
  reported only if a write is effectively required.
* Ignores any symbolic links. They are not saved.

1.3 2011/5
==========
* Fixed the restore of directory with unicode chars in Windows.
* Fixed support of file names starting or ending with a space.
* Removes files before inserting new ones to minimize the parity size.

1.2 2011/5
==========
* Fixed use of file names out of the codepage in Windows. All the names
  are now stored in UTF8 in the content file.

1.1 2011/5
==========
* Fixed a bug in the check command when detecting garbage data over the
  expected end of the file.
  The parity data was anyway computed correctly, and no special action is
  required to update.
* Changed the default checksum to Murmur3 hash. It's a lot faster than MD5.
  You can check its speed with the "snapraid -T" command.
  MD5 is still supported for backward compatibility.
  To convert to the new Murmur3 hash, simply remove the 'content' file,
  and start a new complete 'sync'.
* Added RAID6 support. It's used the very good RAID6 library made by H. Peter
  Anvin also used in the Linux Kernel. It contains optimized implementations
  for SSE2 and MMX.
* Added support for multiple 'content' files. You can save extra copies to be
  able to verify the checksums also if you lose all the 'content' files in the
  parity disks.
* Added a filtering include logic, where anything not explicitely included
  is excluded. For example, it allow to include only the files in a predefined
  set of directories.
* The check command returns with an error code if any kind of error is present.
  Previously it was returning an error only if unrecoverable errors were
  present.
* Opening the files in sequential mode in Windows. This should give a speedup
  in Windows.
* In Windows you can use the backslash \ in the filter definitions instead
  of the forward slash /.

1.0 2011/4
==========
* No relevant change.

0.4 2011/4
==========
* Added hidden 'dry' command mainly for speed measurement.
* As default, uses the OpenSSL crypto MD5 implementation.

0.3 2011/4
==========
* Added --filter option to select a subset of file in check and fix.
* Better ETA estimation in all the commands.
* Added support for OpenSSL crypto library to use its optimized MD5 implementation.
* Added test vectors and a speed test for MD5.

0.2 2011/3
==========
* Second public test release of SnapRAID.
* Functionally complete in check and fix.
* Files are identified by inode and not anymore by name.
* Exclusion list of files and directories.
* Precise error management.
* More regression tests.

0.1 2011/3
==========
* First public test release of SnapRAID.