Package one.microstream.storage.types

Interface StorageDataFileEvaluator

All Superinterfaces:
StorageDataFileDissolvingEvaluator
All Known Implementing Classes:
StorageDataFileEvaluator.Default
public interface StorageDataFileEvaluator
extends StorageDataFileDissolvingEvaluator
Function type that evaluates if a storage file needs to be dissolved and its remaining data content be transferred to a new file or if the current head storage file needs to retire and be replaced by a new one.

Note that any implementation of this type must be safe enough to never throw an exception as this would doom the storage thread that executes it. Catching any exception would not prevent the problem for the channel thread as the function has to work in order for the channel to work properly. It is therefore strongly suggested that implementations only use "exception free" logic (like simple arithmetic) or handle any possible exception internally.

  • Method Details

    • needsDissolving

      boolean needsDissolving​(StorageLiveDataFile storageFile)
      Specified by:
      needsDissolving in interface StorageDataFileDissolvingEvaluator

    • needsRetirement

      boolean needsRetirement​(long fileTotalLength)

    • fileMinimumSize

      int fileMinimumSize()

    • fileMaximumSize

      int fileMaximumSize()

    • New

      static StorageDataFileEvaluator New()
      Pseudo-constructor method to create a new StorageDataFileEvaluator instance using default values specified by StorageDataFileEvaluator.Defaults.

      For explanations and customizing values, see New(int,int,double).

      Returns:
      a new StorageDataFileEvaluator instance.
      See Also:
      New(int,int), New(double), New(int,int,double), StorageDataFileEvaluator.Defaults

    • New

      static StorageDataFileEvaluator New​(double minimumUseRatio)
      Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed value and default values specified by StorageDataFileEvaluator.Defaults.

      For explanations and customizing values, see New(int,int,double).

      Parameters:
      minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to preventthe file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data,inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its lengthas a negative value length header).
      The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the morefile dissolving (data transfers to new files) is required and vice versa.
      Returns:
      a new StorageDataFileEvaluator instance.
      See Also:
      New(), New(int,int), New(int,int,double), StorageDataFileEvaluator.Defaults

    • New

      static StorageDataFileEvaluator New​(int fileMinimumSize, int fileMaximumSize)
      Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.

      For explanations and customizing values, see New(int,int,double).

      Parameters:
      fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller fileswill be dissolved.
      fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger fileswill be dissolved.
      Note that a file can exceed this limit if it contains a single entity that already exceeds the limit.E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file sizelimit of, for example, 30 MB.
      Returns:
      a new StorageDataFileEvaluator instance.
      See Also:
      New(), New(double), New(int,int,double), StorageDataFileEvaluator.Defaults

    • New

      static StorageDataFileEvaluator New​(int fileMinimumSize, int fileMaximumSize, double minimumUseRatio)
      Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values.

      A StorageDataFileEvaluator is used to evaluate storage data files regarding if they should be desolved, meaning copy all their still needed data to the current head file and then delete the file.
      This technique is used to optimize (or "clean up") the occupied storage space: No longer needed data, e.g. of entities that became unreachable or prior versions of a still reachable entity, is considered a logical "gap", that occupies storage space unnecessarily. These "gaps" have to be removed in order to only occupy as much storage space as required.

      The parameters defined here give an opportunity to configure how "aggressive" this clean up shall be performaned. The trade-off is between acceptale gap sizes and the required performance and disk-writes to clean them up.
      Without specifying them, default values will be used, defined in StorageDataFileEvaluator.Defaults.

      Parameters:
      fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller files will be dissolved.
      fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger files will be dissolved.
      Note that a file can exceed this limit if it contains a single entity that already exceeds the limit. E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file size limit of, for example, 30 MB.
      minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to prevent the file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data, inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its length as a negative value length header).
      The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the more file dissolving (data transfers to new files) is required and vice versa.
      Returns:
      a new StorageDataFileEvaluator instance.
      See Also:
      New(), New(double), New(int,int), StorageDataFileEvaluator.Defaults

    • New

      static StorageDataFileEvaluator New​(int fileMinimumSize, int fileMaximumSize, double minimumUseRatio, boolean cleanUpHeadFile)
      Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.

      For explanations and customizing values, see New(int,int,double).

      Parameters:
      fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller fileswill be dissolved.
      fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger fileswill be dissolved.
      Note that a file can exceed this limit if it contains a single entity that already exceeds the limit.E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file sizelimit of, for example, 30 MB.
      minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to preventthe file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data,inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its lengthas a negative value length header).
      The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the morefile dissolving (data transfers to new files) is required and vice versa.
      cleanUpHeadFile - a flag defining wether the current head file (the only file actively written to) shall be subjected to file cleanups as well.
      Returns:
      a new StorageDataFileEvaluator instance.
      See Also:
      New(), New(double), New(int,int), New(int,int,double), StorageDataFileEvaluator.Defaults