This project is sponsored by the Innovative Software AG!



Definitions and Objectives

Introduction


Definition of Terms
  • internal-fragmentation
  • external-fragmentation
  • standard formulation
  • fragmentationX
  • fragmented path


  • Link to my old page


    Copyright and Contact



    Tool: Fibmap

    Description
  • download



  • Tool: Read

    Description
  • download



  • Tool: Agesystem

    Description
  • download


  • Gauge Measurements

  • Test System

  • 500 MB Partition

  • file size distribution
  • results for Ext2FS
  • results for ReiserFS
  • results for JFS
  • results for XFS


  • 1GB Partition
  • file size distribution
  • all results


  • 4 GB Partition
  • file size distribution
  • all results


  • Gauge Verifications

  • Test System

  • 500 MB Partition

  • 1 GB Partition

  • 4 GB Partition


  • TODO



    Gauge Performance

    Description
  • download


  • Measurements on 2.4.5

  • Test System

  • Output Explanation

  • variable file size
  • create fixed size
  • append variable size


  • Measurements on 2.4.8

  • Test System

  • Output Explanation

  • variable file size



  • Append Tests

    Description
  • download


  • Measurements on 2.4.8

  • Test System

  • Output Explanation

  • fixed append size
  • variable append size
  • low statistics


  • With Preallocation

  • Output Explanation

  • fixed append size


  • Measurements on 2.4.10
  • Test System

  • Output Explanation

  • fixed append size


  • With Preallocation

  • Output Explanation

  • fixed append size


  • Tool: Agesystem3

    Description
  • download



  • Aging Tests

    Description
  • download


  • Measurements on 2.4.8

  • Test System

  • Explanation of Terms

  • generic file size dist


  • Measurements on 2.4.10
  • Test System

  • Explanation of Terms

  • generic file size dist



  • External Links

    Filesystem Homepages
  • ReiserFS
  • Ext2FS
  • Ext3FS
  • XFS
  • JFS


  • Benchmarks & Results
  • ReiserFS benchs
  • Denis'+Kresimir's site
  • Randy's tests
  • Yves' YAFB tests


  • Literature
  • Smith's site
  • My master thesis




  • Additional Notes

    Copyright 2001 by
    Constantin Loizides,
    loizides AT informatik DOT uni-frankfurt DOT de

    Last changes:
    Do Feb  5 09:40:02 CET 2004
    

    Journaling-Filesystem Fragmentation Project

    Aging Tests




    Description

    The following setup ages a file system through the creation and deletion of files using the
    agesystem3 tool. The idea is to simulate a file system usage pattern around a partition fullness of 75 % in order to measure how the performance drops down after having deleted and created a bunch of files.
    In Detail:
    • Choose a set of file systems to test (eg. reiser, reiser -o notail and ext2)
    • Choose the directory structure for agesystem (eg. 1x1) and choose the file size distribution for agesystem (eg. the standard for a 1 GB partition) or measure the distribution of files per directory and file sizes according to an interesting directory (eg. /usr/src/linux or your home directory).
    • Choose the minimum and maximum percentage between the file system should be aged (eg. between 60-80 %)
    • Choose the number of aging runs (eg. 10)
    • Choose the number of files to create through each aging run (eg. 1000)
    • Loop over the file systems to test
      • Format partition one
      • Gauge it up to the chosen minimum percentage
      • Loop over the aging run number
        • Age partition one using the agesystem3 tool with the set of chosen options
        • Format partition two and copy partition one onto it (cp -R . /partition2)
        • Run fibmap to get the written block layout on both partitions
        • Run read to measure the read performance on both partitions

    Download

    You may download the necessary files and try it yourself. After downloading the package create a directory where to untargz it. You probably need to adjust some paths, then type ''make agesystem; make fibmap; make read'' and look for the script ''agetest.sh'' that shows how to setup the options described above. If you like to get some graphs out of the resulting output, send the output files to me (targzed, please!).


    Measurements on 2.4.8

    Test System

    The hardware and settings are the same as in the gauge section.

    • AMD Duron 650 Mhz, 128 MB, 40 GB EIDE-harddisk for the system with the tests taken on
      • Adaptec AHA-2940U2/W host adapter
      • 9 GB IBM DNES-309170W SCSI-harddisk
    Due to lack of simulation time, problems with the Linux 2.4 VM and no time to patch each day another kernel the file system tested are ReiserFS with and without tails and Ext2 on Alan's 2.4.8-ac1 kernel.
    • Software: SuSE 7.2 (gcc 2.95-3) kernel 2.4.8-ac1 restricted to 32 MB memory without swap.


    Explanation of Terms



    Generic File Size Distribution

    The following graphs show that all file systems suffer of the aging. After 50000 creations the systems seem to reach their minimum performance. Best seen is the performance drop of up to 30 % when looking at the read or fread performance as it is not much influenced by lots of disc seeks as the rread performance. But also the random read performance significantly drops down to 50 to 70 %. A detailed analysis of the filesystem outputs shows that of the 5383 files which were created at the beginning in run zero after the first run 651, after the second 69, after the third 8 and after the fourth 3 files were left. Clearly the most interesting is what happens in the first 50000 creations. Therefore a second set of tests is being done at the moment.

    The following graph shows the normalized graphs where every value of the different measured curves is divided by their initial value at runtime zero (right after being gauged): compared normalized graphs


    Measurements on 2.4.10

    Test System

    The hardware and settings are the same as in the gauge section.

    • AMD Duron 650 Mhz, 128 MB, 40 GB EIDE-harddisk for the system with the tests taken on
      • Adaptec AHA-2940U2/W host adapter
      • 9 GB IBM DNES-309170W SCSI-harddisk
    The file system tested are
    • ReiserFS with and without tails with and without the bitmap hinting patch
    • XFS (linux-2.4.10-xfs-2001-10-03.patch)
    • JFS (jfs-2.4.7-1.0.7-patch
    • Ext2 on a
      2.4.10 kernel (patched with the mentioned patches).
    • Software: SuSE 7.2 (gcc 2.95-3) kernel 2.4.10 restricted to 32 MB memory without swap.


    Explanation of Terms


    Generic File Size Distribution

    The following graphs show that Reiser with the Bitmap Hinting Patch (BMP) is slightly slower in absolute numbers than the stock Reiser (RFS), but scales significantly better when the file system ages (see the normed graph).