We recently responded to a forum post on our YouTube channel regarding SSDs and Defragmentation - you can view the video here: http://www.youtube.com/watch?v=hznCSqb4Mzg

Below are some "before and after" graphs that provide proof that fragmentation affects SSDs:

 

In today’s day and age, many SMBs and enterprise-level businesses are “taking to the skies” with cloud computing. These companies realize that working in the cloud comes with many benefits – including reduced cost of in-house hardware, ease of implementation and seamless scalability. However, as you will read on and discover - performance-impacting file fragmentation and the need for defragmentation still exists and is actually amplified in these environments. Based on these factors, it must now be addressed with a two-fold proactive and preventative solution.

Let’s face it – we generate a tremendous amount of data and it’s only the beginning. In fact, findings included in a recent study by IDC titled “Extracting Value from Chaos” predict that in the next ten years we will create 50 times more information and 75 times more files. Now regardless of destination, most of this data is generated on Windows-based computers, which are known to fragment files. Therefore, when files are manipulated they become fragmented before even reaching the cloud. This occurs because as they are worked with, they get broken up into various pieces and scattered to numerous locations across the hard disk. The result is increased time necessary to access these files and affects system performance.

So how does the above scenario affect the big picture? To understand this, let’s take a closer look at your cloud environment. Your data, and in many cases, much of your infrastructure, has “gone virtual”. Users are able to access applications and work with their data basically anywhere in the world. In such an atmosphere, where the amount of RAM and CPU power available is dramatically increased and files are no longer stored locally, how can the need for defragmentation still be an issue?

Well, what do you think happens when all this fragmented data comes together? The answer is an alarming amount of fragmented Big Data that’s now sitting on the hard drives of your cloud solution. This causes bottlenecks that can severely impact your mission-critical applications due to the large-scale unnecessary I/O cycles needed to process the broken up information.

At the end of the day, traditional approaches to defragmentation just aren’t going to cut it anymore and it’s going take the latest software technology implemented on both sides of the cloud to get these issues resolved. It starts with software, such as Diskeeper 12, installed on every local workstation and server, to prevent fragmentation at its core. Added to this is deploying V-locity software across your virtualized network. This one-two punch defragmentation software solution addresses I/O performance concerns, optimizes productivity and will push cloud computing further than you ever thought possible. In these exciting times of emerging new technologies, Cloud computing can send your business soaring or keep it grounded - the choice is up to you.

In today’s well connected world of electronics and instant communications I received a text from a friend asking if I had seen the recent PC World magazine (February, 2012).  He said it had some tidbit of information concerning one of my favorite subjects; system performance, defragmentation, and SSDs.  I located a copy here at the office and found the article. As I read the first line I realized the debate on the virtues of defragmentation especially on SSDs will be one that goes on indefinitely as no one really talks about the issue with supporting hard facts and numbers.  Most articles are rehashing ideas and opinions long since debunked.  They continue to surface because very few truly understand the intricacies of the Windows NTFS file system and that of the storage media, whether it is rotating magnetic hard disks or electronic solid state disks.

So let’s set the record straight… Fragmentation is exponentially more of a problem with today’s data explosion. Defragmenting once a week will still cause the user to experience slowdowns from the degradation effects and doesn’t address the issue when files are initially being written.  And yes, never do a traditional defrag on SSDs.

NTFS file and free space fragmentation happens far more frequently than you might guess.  It has the potential to happen as soon as you install the operating system.  It can happen when you install applications or system updates, access the internet, download and save photos, create e-mail, office documents, etc…  It is a normal occurrence and behavior of the computer system, but does have a negative effect on over all application and system performance.  As fragmentation happens the computer system and underlying storage is performing more work than necessary.  Each I/O request takes a measurable amount of time.  Even in SSD environments there is no such thing as an “instant” I/O request.  Any time an application requests to read or write data and that request is split into additional I/O requests it causes more work to be done.   This extra work causes a delay right at that very moment in time.  Whoever thought that defragmenting once a month or weekly was good enough, simply didn’t understand fragmentation.

Disk drives have gotten faster over the years, but so have CPUs.  In fact, the gap between the difference in speed between hard disks and CPU has actually widened.  This means that applications can get plenty of CPU cycles, but they are still starving to get the data from the storage.  What’s more, the amount of data that is being stored has increased dramatically.  Just think of all those digital photos taken and shared over the holidays.  Each photo use to be approximately 1MB in size, now they are exceeding 15MB per photo and some go way beyond that.  Video editing and rendering and storage of digital movies have also become quite popular and as a result applications are manipulating hundreds of Gigabytes of data.  With typical disk cluster sizes of 4k, a 15MB size file could potentially be fragmented into nearly 4,000 extents.  This means an extra 4,000 disk I/O requests are required to read or write the file.  No matter what type of storage, it will simply take longer to complete the operation.

Suppose I chose to do some editing of my family videos on Tuesday evening.  Even the built-in defragmentation tool in Windows 7 doesn’t do me much good because it isn’t schedule to run until Wednesday morning at 1:00am.  This also means that quite a bit of fragmentation has built up since the previous week when it last ran.  Maybe I’ll manually run it, but that can take quite a while and I’ve wasted time that I would have rather spent on my project.  Unfortunately, the Windows built-in defragmentation utility doesn’t prevent fragmentation so even after running it manually; I still will wind up with fragmentation and slow access speed of my newly created files. 

I’ve often thought about why Wednesday at 1:00am was chosen as the time to schedule defragmentation.  Why isn’t it scheduled all the time?   It is because there could be system resource conflicts that either interfere with getting the task done or the defragmentation process has difficulty throttling back under a variety of conditions.  Regardless, this wait a week to clean up fragmentation doesn’t really help me when I need it most.

As pointed out in the article, the built-in defragmenter does not have the technology advancement to properly deal with fragmentation and SSDs. The physical placement of data on an SSD doesn’t really matter like it does on regular magnetic HDDs.  With an SSD there is no rotational latency or seek time to contend with.  Many experts assume that fragmentation is no longer a problem, but the application data access speed isn’t just defined in those terms.  Each and every I/O request performed takes a measurable amount of time.  SSD’s are fast, but they are not instantaneous.  Windows NTFS file system does not behave any differently because the underlying storage is an SSD vs. HDD and therefore fragmentation still occurs.  Reducing the unnecessary I/O’s by preventing and eradicating the fragmentation reduces the number of I/O requests and as a result speeds up application data response time and improve the overall lifespan of the SSD.  In essence, this makes for more sequential I/O operations which is generally faster and outperforms random writes.

In addition, SSD’s require that old data be erased before new data is written over it, rather than just writing over the old information as with HDDs.  This doubles the wear and tear and can cause major issues with the speed performance and lifespan of the SSD.  Most SSD manufactures have very sophisticated wear-leveling technologies to help with this. The principle issue is write speed degradation due to free space fragmentation.  Small free spaces scattered across the SSD causes the NTFS file system to write a file in fragmented pieces to those small available free spaces.  This has the effect of causing more random I/O traffic that is slower than sequential operations.

I think I have clearly made my point….  The built-in defragmenter in Windows 7 is not a solution for neither the consumer/home user, nor the enterprise business user.  Data access speeds are far more critical in the business world where time is money.  In the enterprise environment there are generally many more files that are used by higher number of users that are accessing data across shared type of storage such as SAN.  Even virtual platforms benefit from the same points covered.  This opens the door and is the reason why robust solutions such as Diskeeper exist.  More data about Diskeeper and the superior technology it offers can be found at http://www.diskeeper.com.

Diskeeper Corporation announced today the launch of a PC Speed Contest where contestants are asked to Show – via a picture or short video and Tell – the remarkable gains that they have received in accelerating PC performance using Diskeeper® 2011 performance software.

Diskeeper increases the speed and reliability of all PCs by improving system performance, faster boot-up times and increased speeds on everything from internet browsing to antivirus scans. Diskeeper 2011 gives PCs faster-than-new speed without any effort from the user.

The grand prize winner will receive a new Laptop computer and a copy of Diskeeper 2011 Pro-Premier software.

To enter the contest: http://tinyurl.com/3b37ayq

Dawn Richcreek, Vice President of Marketing has been recognized by Everything Channel’s CRN Magazine as one of the top Women of the Channel in 2011. This annual list recognizes female executives for their accomplishments over the past year, based on their achievements as executives and the amount of influence they wield over the technology channel.  This year’s Women of the Channel were chosen by the editors of CRN Magazine from a field of vendor channel organizations, distributors and solution providers. 

Over the past year, Richcreek successfully developed the Diskeeper Channel Awareness Campaign with Tech Data.   Additionally, she has launched two new products, Diskeeper^® 2011 data performance software and V-locity^®  2.0  virtual platform disk optimizer, that make the day-to-day lives of reseller customers easier and more productive.  Richcreek’s previous accomplishments include implementation of the Diskeeper Corporation Channel Marketing program where she provided profit generating training and sales tools for thousands of resellers on the Diskeeper Corporation suite of products enabling them to better service and provide solutions to their customers. She is a marketing veteran with over 25 years of experience in the industry. The complete article is located here:  http://tinyurl.com/3dakwmy
 

Overview 

The intensified demand for IT network efficiency and lower operating costs has been driving the phenomenal growth of virtualization in the past decade, with no signs of slowing. At present, many corporations run more virtualized servers than physical servers.

 

While virtualization provides opportunity for consolidation and better hardware utilization, it’s critically important to recognize and never exceed hardware capacities.  

The importance of ensuring sufficient CPU and memory are well understood, with many processes and management tools available to help plan and properly provision VMs for these critical resources. I/O traffic, network and disk, are more complicated to account for in virtual environments as they tend to be more unpredictable.

In order to better accommodate disk I/O, most virtualization platforms will implement a Storage Area Network (SAN) which can offer greater data throughput, and a dynamic environment to address fluctuations in I/O demands.

While a storage infrastructure can be built out to meet expected demands, there are uncontrollable behaviors that will still impede performance. 

File Fragmentation

As files are written to a general purpose local disk file systems, such as Windows NTFS, a natural byproduct is file fragmentation. File fragmentation is a state in which the data stream of a file is stored in non-contiguous clusters in the file system. Fragmentation occurs on logical volume, and by device drivers is translated to logical blocks, and eventually to physical sectors residing on a storage device. It can be demonstrated as pieces of a file located in a non-contiguous manner. The effect of this file fragmentation is increased I/O overhead, leading to slower system performance for the operating system.

In the case of virtual platforms, a guest operating systems is stored as a file (i.e. set of files) on the virtual platforms file system as a “virtual disk”. A virtual disk is essentially a container file, housing all the files that constitute the OS and user data of a VM.  A virtual disk files can fragment just as any other file can resulting in what amounts to a “logically” fragmented virtual hard disk, which still has typical file fragmentation contained within it. The picture represented to the right would appear as “VirtualServer1.vmdk, 30GB in size, in 4 pieces”.  

 

This situation equates to hierarchical fragmentation or more simply fragmentation-within-fragmentation. Given the relatively static nature and large size of virtual disks, and large allocation unit size of VMFS (typically 1MB), fragmentation of these files is unlikely to cause performance issues in most cases. The focus and solution to fragmentation should be directed at the guest operating system.

Fragmentation within a Windows VM will cause Windows to generate additional unnecessary I/O. This added I/O traffic can be discovered using Windows Performance Monitor, where it is one of the principal causes for Split I/O.  

 

Fragmentation prevention and defragmentation technologies exist to eliminate unnecessary I/O overhead, and improve system performance. Fragmentation prevention solves fragmentation at the source, by actively causing files to be written contiguously via advanced files system drivers. Defragmentation is the action in which file fragments are re-aligned within the file system, into a single extent, so that only the minimal amount of disk I/Os are required to access the file, thereby increasing access speed.  

Partition Alignment 

Depending on your storage protocol and virtual disk type, misaligned partitions can cause additional unnecessary I/O[1]. In the example below in which the ESX and SAN volumes are not properly aligned, a Word file spanning four NTFS clusters causes additional unnecessary I/O in both VMFS and the SAN LUN.  

 

Similarities between Partition Alignment and Fragmentation 

Much like misaligned partitions can cause additional I/O at multiple layers, so does fragmentation. While partitions can be properly aligned once and never require further corrective action, fragmentation will continue to occur, and needs to be regularly addressed.

In the example below, which assume proper partition alignment, a file in eight fragments in the guest OS, causes additional I/Os to be generated at the virtualization platform layer[2] and at the LUN.   

 

Defragmentation in the guest operating system (of this file), eliminates excess I/O when accessing the file as Windows only generates one I/O. This reduction in I/O traffic translates to the host file system and SAN LUN, ensuring efficiencies at each layer.   

 

Best Practices 

Defragmentation of Windows file systems is a VMware recommended performance solution. The VMware Knowledge Base article 1004004[3] states “Defragmenting a disk is required to address problems encountered with an operating system as a result of file system fragmentation. Fragmentation problems result in slow operating system performance.” In order to validate the Vmware statement, tests were performed.

 

Test Environment

  

Configuration

Test Environment Configuration Host OS: ESX Server 4.1 with VMFS (1MB blocks)

Guest OS: Windows Server 2008r2 x64 (3GB RAM, 1 vCPU)

Benchmarking Software: Iometer (http://www.iometer.org/)

Fragmentation Program: FragmentFile.exe (used to fragment a specified file)

Defragmentation Software: V-locity® 3.0 (http://www.diskeeper.com/business/v-locity/)

 

Storage: 10GB test volume in a 40GB virtual disk. VMFS Datastore of 410GB. HP Smart Array P400 controller. RAID 5 (4x 136GB SCSI at 10K RPM) Stripe size of 64KB with a 64KB offset (properly aligned).

Load Generation 

The industry standard benchmarking tool Iometer was used to generate I/O load for these experiments.  

Iometer configuration options used as variables in these experiments:

• Transfer request sizes: 1KB, 4KB, 8KB, 16KB, 32KB, 64KB, 72KB, and 128KB

• Percent random or sequential distribution: for each transfer request size, 0 percent and 100 percent random accesses were selected

• Percent read or write distribution: for each transfer request size, 0 percent and 100 percent read accesses were selected 

Iometer parameters that were held constant for all tests:

• Size of volume: 10GB

• Size of Iometer test file (iobw.tst): 8,131,204 KB (~7.75GB)

• Number of outstanding I/O operations: 16

• Runtime: 4 minutes

• Ramp-up time: 60 seconds

• Number of workers to spawn automatically: 1 

The following is excerpted from a VMware white paper[4], and helps to explain why the Iometer parameters were used. 

Servers typically run a mix of workloads consisting of different access patterns and I/O data sizes.

Within a workload there may be several data transfer sizes and more than one access pattern.There are a few applications in which access is either purely sequential or purely random. For example, database logs are written sequentially. Reading this data back during database recovery is done by means of a sequential read operation. Typically, online transaction processing (OLTP) database access is predominantly random in nature. 

The size of the data transfer depends on the application and is often a range rather than a single value. For Microsoft Exchange, the I/O size is generally small (from 4KB to 16KB), Microsoft SQL Server database random read and write accesses are 8KB, Oracle accesses are typically 8KB, and Lotus Domino uses 4KB. On the Windows platform, the I/O transfer size of an application can be determined using Perfmon.

In summary, I/O characteristics of a workload are defined in terms of the ratio of read operations to write operations, the ratio of sequential accesses to random accesses, and the data transfer size. Often, a range of data transfer sizes may be specified instead of a single value.  

Create Fragmentation 

The FragmentFile.exe tool was used to fragment the Iometer test file (iobw.tst) into 568,572 fragments, a mid-range amount of fragmentation for a production server. The statistics collected from an analysis of the volume (shown below) were performed with V-locity.

Test Procedure 

The primary objective was to characterize the performance of fragmented versus defragmented virtual machines for a range of data sizes across a variety of access patterns. The data sizes selected were 1KB, 4KB, 8KB, 16KB, 32KB, 64KB, 72KB, and 128KB. The access patterns were restricted to a combination of 100 percent read or write and 100 percent random or sequential. Each of these four workloads was tested for eight data sizes, for a total of 32 data points per workload.

In order to isolate the impact of fragmentation only the test VM was powered on and active for the duration of the tests.

For the initial run, Iometer created a non-fragmented file, and performance data was collected. Then FragmentFile.exe tool was used to fragment the Iometer test file, the VM rebooted, and the test procedure re-run. This resulted in data sets for both non-fragmented and fragmented scenarios. The results are graphed below.  

Performance Results  

As the graphs show, all workloads show an increase in throughput when the volume [file] is defragmented (i.e. not fragmented).  It also becomes clear that as the I/O read/write size increases, the fragmentation-induced I/O latency increases dramatically.  The greatest improvements of a contiguous file are found with file reads; both random and sequential. 

 

Random Reads  

 

Random Writes 

Sequential Reads

Sequential Writes

Conclusion

 

Fragmentation demonstratively impedes performance of Windows guest operating systems.  While the tests depicted were executed on a singular VM, the issue becomes exponentially worse in a multi-VM environment wherein each VM suffers from file fragmentation.  As server virtualization establishes a symbiotic relationship, it is important to remember that generating disk I/O in one virtual machine affects I/O requests from other virtual systems.  Therefore latencies in one VM will artificially inflate latency in co-located virtual machines (VMs that share a common platform).  

Fragmentation artificially inflates the amount of disk I/O requests which, on a virtual machine platform, compounds the disk bottleneck even more so than on conventional systems.

Eliminating fragmentation in VMs, and the corresponding unnecessary disk I/O traffic, is vital to platform-wide performance and enhances the ability to host more VMs on a shared infrastructure.

              

“We are now running Diskeeper 2011 EnterpriseServer on our a DFS server which houses all our corporate documents. This server has about 500GB of files and was heavily fragmented prior to installing Diskeeper. 

“Users were experiencing latency issues, login delays (roaming profiles) and sometimes apps would freeze upon saving to that server. After running Diskeeper for a few weeks, there was a major performance improvement. We went from getting several calls a week prior to the Diskeeper install, down to a couple of calls a month. 

“The other server is a backup repository that houses our SQL backups, BackupExec files and other archived info. This server has over 10TB of storage. Since running Diskeeper, the overall performance of the server and backup times improved greatly.” 

Sandy Hyde, Senior Systems Analyst, Business Support Services, Family Insurance Solutions

Diskeeper 2011 was covered on Wugnet.  Howard Sobel stated, “They introduced technology that slowed down and prevented fragmentation in Diskeeper 2010 so I thought it was impossible to improve on the concept of defrag much more. Not so! By increasing the efficiency of their algorithms, they have decreased the wear and tear on your hard disk and your computing performance while it works in the background. Decreasing the overall disk activity also decreases your electrical consumption. This may not be a huge savings on "your" electric bill but consider how much savings this amounts to in datacenters where thousands of hard disks are used. So being GREEN doesn't mean paying a penalty. In fact, the opposite is true for Diskeeper. I would go as far as declaring this the software utility "Product of the Year" if we didn't have 8 more months to go in 2011.” The full article is located here: http://www.wugnet.com/tips/this_week.asp  

 

Michael Vizard, Industry leader and IT Editor wrote an article on Diskeeper 2011 and stated,

"But as more applications begin to share the same IT infrastructure thanks to the advent of virtualization and cloud computing, the more fragmentation becomes an I/O performance optimization issue."

 The full article is located here: http://www.ctoedge.com/content/intelligent-disk-defragmentation

Daiwa Odakyu Construction Co., Ltd.  

Problem 

Daiwa Odakyu Construction Co., Ltd did back up of 15 Servers every day using Symantec Backup Exec. But the problem of latency of backup appeared. Initially the system was designed to finish backup within 3 hours at most. But the time grew to 8 hours. They thought additional hardware was needed but had no additional budget. At that time they discovered the cause of latency was fragmentation. First they tried to solve fragments with the built-in defrag. But it solved only 20%. Next a free defrag software was tried. It proceeded to 35% and shortened backup time by more than 1 hour. But 2 days after, the backup time got back to 8 hours. 

Solution 

Mr. Kawata, manager of the Information System Group of Daiwa Odakyu, found Diskeeper® via the internet and downloaded trialware from SOHEI. Diskeeper solved all fragmentation and shortened the backup times down to 1.3 hours. What’s more, IntelliWrite® prevented fragmentation up to 99%! In accordance with their trial, the system without Diskeeper added 1 hour a day to their backup times. 

	Back up Time(hour)	Number of Fragments	Number of fragments prevented
Initial design	3.0	-	-
Fragmented	8.5	10,000,000以上	-
After defrag by Diskeeper	1.3	250	3,000,000～7,200,000
Improvement	85%	99.9975%	（断片化防止率99％）

 “I couldn’t estimate such a big influence of fragmentation in the 50% used volume.  The incredible power of Diskeeper is surprising me. The massive fragmentation I gave up trying to solve is now easily defragmented and prevented by Diskeeper, and then this leads to prevent latency of back up completely.  

“The trial without Diskeeper, which resulted in adding 1 hour a day to backup time, scared me so much. With increasing numbers of files, the backup caused massive fragmentation and gets into a dangerous situation. Less than $1,000 investment in Diskeeper is equal to more than $10,000 worth of Servers. I think Diskeeper is vital for file servers and backup servers."  

“I’m thankful to the developers of this surprising software.” 

System: 

HP DL380G4 Xeon E5430 4GB Memory 410GB SAS RAID10(146GB SA*6）

HP StorageWorks MSA60 3.41TB SATA RAID10(750GB SATA*10）  

 

Diskeeper (data performance for physical systems) and V-locity (optimization for virtual systems) are designed to deliver performance, reliability, longer life and energy savings. Increased performance and saved energy from our software are relatively easy to empirically test and validate. Longer life is a matter of minimizing wear and tear on hard drives (MTTF) and providing an all around better experience for users so they can continue to be productive with aging equipment (rather than frequent hardware refreshes).

Reliability is far more difficult to pinpoint as the variables involved are difficult, if not impossible, to isolate in test cases. We have overwhelming anecdotal evidence from customers in surveys, studies, and success stories that application hangs, freezes, crashes, and the sort are all remedied or reduced with Diskeeper and/or V-locity.

However, there is a reliability "hard ceiling" in the NTFS file system; a point in which fragmentation/file attributes become so numerous reliability is jeopardized. In NTFS, files that hit the proverbial "fan", and spray out into hundreds of thousands and millions of fragments, result in a mess that is well... stinky.

In short, fragmentation can become so severe that it ultimately ends up in data loss/corruption. A Microsoft Knowledge Base article describes this phenomenon. I've posted it below for reference:

A heavily fragmented file in an NTFS file system volume may not grow beyond a certain size caused by an implementation limit in structures that are used to describe the allocations.

In this scenario, you may experience one of the following issues:

When you try to copy a file to a new location, you receive the following error message:
In Windows Vista or in later versions of Windows
The requested operation could not be completed due to a file system limitation
In versions of Windows that are earlier than Windows Vista
insufficient system resources exist to complete the requested service
When you try to write to a sparse file from the Application log, Microsoft SQL Server may log an event that resembles the following:
In Windows Vista or in later versions of Windows
Event Type: Information
Event Source: MSSQLSERVER

Description: ...
665(The requested operation could not be completed due to a file system limitation.) to SQL Server during write at 0x000024c8190000, in filename...
In versions of Windows that are earlier than Windows Vista
Event Type: Information
Event Source: MSSQLSERVER

Description: ...
1450(Insufficient system resources exist to complete the requested service.) to SQL Server during write at 0x000024c8190000, in file with handle 0000000000000FE8 ...
When a file is very fragmented, NTFS uses more space to save the description of the allocations that is associated with the fragments. The allocation information is stored in one or more file records. When the allocation information is stored in multiple file records, another structure, known as the ATTRIBUTE_LIST, stores information about those file records. The number of ATTRIBUTE_LIST_ENTRY structures that the file can have is limited.

We cannot give an exact file size limit for a compressed or a highly fragmented file. An estimate would depend on using certain average sizes to describe the structures. These, in turn, determine how many structures fit in other structures. If the level of fragmentation is high, the limit is reached earlier. When this limit is reached, you receive the following error message:

Windows Vista or later versions of Windows:
STATUS_FILE_SYSTEM_LIMITATION The requested operation could not be completed due to a file system limitation

Versions of Windows that are earlier than Windows Vista:
STATUS_INSUFFICIENT_RESOURCES insufficient system resources exist to complete the requested service

Compressed files are more likely to reach the limit because of the way the files are stored on disk. Compressed files require more extents to describe their layout. Also, decompressing and compressing a file increases fragmentation significantly. The limit can be reached when write operations occur to an already compressed chunk location. The limit can also be reached by a sparse file. This size limit is usually between 40 gigabytes (GB) and 90 GB for a very fragmented file.  

WORKAROUND
For files that are not compressed or sparse, the problem can be lessened by running Disk Defragmenter. Running Disk Defragmenter will not resolve this problem for compressed or sparse files.

Okay, so you've bought, installed, connected, configured, and then tuned/optimized your new storage virtualization solution, but somehow there are still latencies with apps (e.g. SQL).

You've run the Storage Area Network (SAN) vendor utilities that:

• did not see any contention on the disks in the RAID group(s). 
• noted that the average I/O to physical disks did not exceed a reasonable number of I/O's per second on each volume in the meta device.
• checked the utilization of the port that the Host Bus Adaptor (HBA) is zoned to and did not see any performance issues.
• noted the switch port that the HBA is connected to is not saturated or reporting any errors.

And basically surmised "at this time we do not see any issue on the array or with the SAN in reference to this server."

However....

When running PerfMon within Windows, it continues to uncover latencies in the 100ms+ range. What the hayel!

This is when it's important to consider what those SAN optimization and reporting tools are providing. SANs can optimize storage from HBA-to-spindle. Above the HBA other factors cause latencies outside the scope or control of the SAN, and ultimately it is the App/User Experience that needs to be addressed.

So, it's time to look further up the storage stack.

Here is a great chart (borrowed from VMware here):

The chart helps simplify that SAN and even VM based latency monitoring and storage optimization do not account for latencies that may exist in the Guest Operating System (GOS). They are only aware of, and able to optimize I/O from the point they receive the traffic to the physical storage.

Monitoring performance in Windows does not go away simply because you've left direct attached storage (DAS) and physical servers to go virtual. There are numerous causes for poor performance on the GOS side, from poorly written apps, to incorrect configurations, to bad partitioning strategies, file system fragmentation and more. Pretty much all the issues that could cause poor Windows I/O performance on physical servers with DAS, still exist.

It's important to continue to use GOS based solutions to determine application latency such as PerfMon, which can support counters for popular apps (like SQL).

To evaluate if file fragmentation is a potential cause, track these metrics with Perfmon. Fragmentation will show up in the logical disk statistics referred to in the document. You can also use a freeware solution from Diskeeper Corporation; called Disk Performance Analyzer for Networks (DPAN) to collect file fragmentation statistics from any Windows system (physical or virtual) on your LAN/WAN.  You can download DPAN here.

Sample DPAN Report:

Overview:

As high performing storage solutions based on block protocols (e.g. iSCSI, FC), SANs excel at optimizing block access. SANs work at a storage layer underneath the operating systems file system; usually NTFS when discussing Microsoft Windows®. That dictates that a SAN is unaware of “file” fragmentation and unable to solve this issue.

Fig 1.0: Diagram of Disk I/O as it travels from Operating System to SAN LUN.

With file fragmentation causing the host operating system to generate additional unnecessary disk I/Os (more overhead on CPU and RAM) performance suffers. In most cases the randomness of I/O requests, due to fragmentation and concurrent data requests, the blocks that make up the file will be physically scattered in uneven stripes across a SAN LUN/aggregate. This causes even greater degradation in performance.

Fig 1.1: Sample Windows Performance Monitor Report from fragmented SAN-attached NTFS volume.

Fortunately there are simple solutions to NTFS file system fragmentation; fragmentation prevention and defragmentation. Both approaches solve file fragmentation at the source, the local disk file system.

IntelliWrite® “The only way to prevent fragmentation before it happens™”

IntelliWrite is an advanced file system driver that leverages and improves upon modern Windows’ file system “Best Fit” file write design in order to write a file in a non-fragmented state on the initial write. Intelligently writing contiguous files to the disk provides four principal benefits above and beyond defragmentation, including:

• Prevents most fragmentation before it happens
• Better file write performance
• An energy friendly approach to improving performance, as defragmentation is not required for files handled by IntelliWrite
• 100% compatibility with copy-on-write technologies used in advanced storage management solutions (e.g. snapshots)

While eliminating fragmentation improves performance. it is important to properly configure and account for advanced SAN features.

With the increasing popularity of SANs, we've included instructions in the Diskeeper installation to ensure users properly configure Diskeeper:

We suggest reading this full document before executing any of the recommended configurations. These instructions apply to V-locity (used on VMs as well).

Best Practices:

Highlights:

Implementing Diskeeper on a SAN is simple and straightforward. There are two principal concepts to ensuring proper configuration and optimal results:

• Ensure IntelliWrite is enabled for all volumes.
• Find a time to schedule Automatic Defragmentation (more details below)

Details:

If you are implementing any of the following SAN based technologies such as Thin Provisioning, Replication, Snapshots, Continuous Data Protection (CDP) or Deduplication, it is recommended to follow these guidelines.

Defragmentation can cause unwanted side effects when any of the above referenced technologies are employed. These side effects include:

This is why it is important to enable the fragmentation prevention (IntelliWrite) and change the Automatic Defragmentation to occur during non-production periods to address the pre-existing fragmentation:

During Installation, disable Automatic Defragmentation;

Uncheck the “Enable Automatic Defragmentation” option during installation.

Upon installation ensure IntelliWrite is enabled on all volumes (default). IntelliWrite was specifically designed to be 100% compatible with all advanced SAN features, and should be enabled on all SAN LUNs. IntelliWrite configuration is enabled or disabled per volume, and can be used in conjunction with Automatic Defragmentation, or exclusively.

To ensure IntelliWrite is enabled, right click a volume(s) and select the feature.


Then confirm “Prevent Fragmentation on this volume” is selected, and click “OK” to complete.

Once installed, enable Automatic Defragmentation for any volumes that are not mapped to a SAN LUN. This may include the System Partition (e.g. C:\).

To enable Automatic Defragmentation, right click a volume(s) and select the feature.


Then check “Enable Automatic Defragmentation on the selected volumes” and click “OK” to complete.

If you are not using any advanced SAN features, it is recommended to enable Automatic Defragmentation for all days/times. However, note that pre-existing fragmentation will require significant effort from Diskeeper to clean up. This effort will generate disk I/O activity within the SAN.

Therefore, if existing fragmentation is significant, initially schedule Diskeeper to run during off-peak hours. As Diskeeper has robust scheduling capability, this is easily configured.

To enable Automatic Defragmentation during non-production periods, right click a volume(s) and select the feature.


Then check “Enable Automatic Defragmentation on the selected volumes”. Diskeeper is then scheduled by using your mouse to highlight over the 30 minute blocks in the interactive weekly calendar.

The above example disables defragmentation Monday through Friday. It also disables defragmentation Saturdays and Sundays except between 7pm until 3:30am the following morning. This would afford 17 hours of defragmentation availability per week. Immediately following these scheduled defragmentation periods is when SAN maintenance for advanced features should be addressed (e.g. thin reclamation, deduplication).

Should accommodating SAN maintenance be difficult (e.g. limited maintenance windows)using a weekly optimization process, very granular scheduling is also available with Diskeeper. Note, maintenance windows are not required in order to implement and benefit from IntelliWrite.

To schedule for specific non-reoccurring dates and times in the future, select the “Turn Automatic Defragmentation on or off based on specific dates” option. Click any multitude of dates and times using Shift-Select or Ctrl-Select. Once done, click OK to complete.

If you are implementing the above mentioned advanced technologies and your SAN provides hot block optimization / data tiering, it is also recommended to disable I-FAAST® (Intelligent File Access Acceleration Sequencing technology). I-FAAST sequences hot “files” (not blocks) in a Windows volume, after determining hardware performance characteristics. The sequencing process creates additional movement of data for those advanced SAN features, and is therefore generally recommended to disable when similar SAN solutions are in place.

To disable I-FAAST, right click a volume(s) and select the feature.

Note, I-FAAST requires Automatic Defragmentation be enabled. Also note that I-FAAST is disabled by default in Diskeeper 2011 in certain cases. Also note that I-FAAST generates additional disk I/Os and will therefore cause an increase in the aforementioned Automatic Defragmentation side effects.

Once pre-existing fragmentation has been removed, increase the periods in which Diskeeper actively optimizes the Windows file systems. With real-time defragmentation and InvisiTasking® technology, Diskeeper immediately cleans up fragmentation (that is not prevented by IntelliWrite). This minimal ongoing optimization generates only invisible, negligible I/O activity.

New features in Diskeeper 2011 to improve SAN performance:

Diskeeper 2011 introduces SAN specific solutions. These default solutions automate many of the configurations required for SAN-attached servers.

Diskeeper 2011’s new Instant Defrag™ technology dramatically minimizes I/O activity, and exponentially speeds up defragmentation. The Instant Defrag engine is provided fragmentation information, in real-time, by the IntelliWrite file system filter driver (those fragments that it does not prevent). Without the traditional need to run a time and resource intensive whole-volume fragmentation analysis, Instant Defrag can address the recently fragmented files as they occur. This dynamic approach prevents a buildup of fragmentation, which could incur additional I/O overhead to solve at a later date/time.

Diskeeper 2011’s new Efficiency Mode (default) maximizes performance, while minimizing disk I/O activity. By focusing on efficiency and performance and not on presenting a “pretty disk” visual display, Diskeeper 2011 minimizes negative side effects (e.g. reduce snapshot storage requirements or thin LUN growth, etc..) while maximizing performance benefits. It is a SAN-optimized defrag mode and our recommended solution for SAN-attached Windows volumes.

By default, Efficiency Mode also disables proprietary file placement features such as I-FAAST.

Also, by default, Diskeeper 2010/2011 moves data to lower NTFS clusters, and hence generally “forward” on SAN LUNs.

Best Practices Summary:

• Ensure IntelliWrite is enabled for all volumes.
• Automatic Defragmentation should be enabled at all times for all direct attached storage volumes.
• Use Efficiency Mode of Diskeeper 2011.
• Schedule Automatic Defragmentation on SAN LUNs, based on use of advanced SAN features.
• Run SAN processes such as space reclamation and/or deduplication on recently defragmented LUNs using advanced SAN features.

Want this in PDF form. Get it here: Best Practices for using Diskeeper on Storage Area Networks.pdf (3.00 mb)

One of the most significant features in Windows 2008R2 (for Hyper-V) is Cluster Shared Volumes (CSV) for virtual disks (vhd). This allows NTFS to behave similar to a clustered file system, addressing many limitations found in Hyper-V storage with the original release (Windows 2008).  

IntelliWrite is “DKRtWrt” (its code names in development stages was WriteRight and then later RightWrite -hence "RtWrt"). To see or load/unload filter drivers, use the Filter Manager Control Program (fltmc):

When Windows NT 4.0 was released, Diskeeper 2.0 hit the market. NT 4.0 had limitations about the type of data that could be safely moved online. So, a market-first innovation that Diskeeper brought to market with Diskeeper 3.0 was what we called Boot Time Defragmentation. Boot Time Defragmentation addressed these special data types during the computer boot process, when it was safe to do so. The files that Diskeeper optimized included metadata (data which "lives above" your data), directories (folders), and the paging file. 

Metadata are special files that the NTFS file system driver uses to manage an NTFS volume. The most famous piece of metadata is the MFT (Master File Table), which is a special file typically consisting of 1024-byte records. Each file or directory on the volume is described by at least one of these MFT records. It may take several MFT records to fully describe a file... especially if it is badly fragmented or compressed; A 271MB compressed file can require over 450 MFT records!

Defragmenting the MFT, data files, and folders are all vital for optimal performance. The example below of what occurs when NTFS goes to read in the 1-cluster file \Flintstone\Barney.txt, makes that case.
 
1. The volume's boot record is read to get the cluster address of the first cluster of the MFT.
2. The first cluster of the MFT is read, which is used to locate all of the pieces of the MFT.
3. MFT record 5 is read as it is predefined to be the MFT record of the root directory.
4. Data cluster 0 of the root directory is read in and searched for "Flintstone".
5. If "Flintstone" is not found, then at least one other data cluster of the root directory needs to be
read to find it.
6. The MFT record for the "Flintstone" directory is read in.
7. Data cluster 0 of the "Flintstone" directory is read in and searched for "Barney.txt".
8. If "barney.txt" is not found, then at least one other data cluster of the "Flintstone" directory needs.
to be read to find it.
9. The MFT record for the "Barney.txt" file is read in
10. Data cluster 0 of the "Barney.txt" file is read in.
 
This is a worst-case scenario. It presumes the volume is not yet mounted, so the NTFS cache is empty at step 1, and the MFT itself needs to be located.  But it shows how many I/Os are required to get at a file that is only one level removed from the root directory: 10. Each one of those 10 I/Os requires a head movement. Any fragmentation along that path only increases the amount of disk I/Os required to access the data - slowing the whole process down.

And, if you follow the step-by-step I/O sequence outlined above, you'll see that every time a new directory is encountered in the path is an additional two or three I/Os. For obvious performance reasons it is beneficial to keep the depth of your directory structure at a minimum.  It also makes the importance of defragmenting these special file types quite obvious. 

As Windows progressed with newer iterations, many of the files that required offline defragmentation, were supported in online defragmentation (including the MFT and directories), so while Boot Time defrag still exists today, the need to run it has diminished. As a great deal of metadata is typically cached from boot to shutdown, perhaps the last remaining system file that is vital to defragment "offline" is the paging file. We've heard arguments over the years that due to the random nature of the data in the paging file that defrag was not valuable, but anyone who has cleaned up a badly shredded paging file will tell you otherwise.

Diskeeper Corporation, Innovators in Performance and Reliability Technologies, announced that its Diskeeper 2010 performance software has received the Certificate of Networthiness (CoN) from the U.S. Army Network Enterprise Technology Command. The Certificate of Networthiness signifies that Diskeeper performance software is configured to the current Army Golden Master baseline and complies with all U.S. Army and Department of Defense (DoD) standards for security, compatibility and sustainability. A CoN is required for all enterprise software products deployed in the Army Enterprise Infrastructure Network and used by the U.S. Army, all National Guard, Army Reserve and DoD organizations that use the Army Enterprise Infrastructure.

On April 10, 2002, Diskeeper enjoyed 15 seconds of TV game-show fame on the TV show Jeopardy with the answer: "Diskeeper is software to do this, reorganize your computer’s files."  The contestant won $2,000 by correctly providing the question, "What is defragment?"

 

“The initial feedback we're getting from our end users is that the performance improved after we implemented Diskeeper. End users had been suffering from slow access to the databases and the huge systems images. This is much better lately due to using the Diskeeper EnterpriseServer. Our main goal for using the Diskeeper has been achieved and people are feeling the difference."

“We are mainly a Health Sector solutions provider for the whole Middle East and soon North Africa, for variety vendors with high focus on Philips solutions. We are vendor free when it comes to IT products, so we use different brands like IBM, HP with Microsoft platforms only, and on top of all that the medical and health solutions like Cardiac Pictures and Archiving solutions, CPACK, and patients monitoring.

 

 

“Please feel free posting my comments. This is the least thing we can afford in paying back your reputable company for availing this fine product.” 

Ayman A. Nimer

Technology Services Segment Manager

Al Faisaliah Medical Systems

The purpose of this blog post is to provide some data about fragmentation on the Mac, that I've not seen researched/published elsewhere.

Mac OSX has a defragmenter in the file system itself. Given Mac is open-source, we looked at the code.

During a file open the files get defragmented if the following conditions are met:

1. The file is less than 20MB in size

2. There are more than 7 fragments

3. System has been up for more than 3 minutes

4. A regular file

5. File system is journaled

6. And the file system is not read-only.

So what's Apple's take on the subject? An Apple technical article states this:

Do I need to optimize?

You probably won't need to optimize at all if you use Mac OS X. Here's why:

• Hard disk capacity is generally much greater now than a few years ago. With more free space available, the file system doesn't need to fill up every "nook and cranny." Mac OS Extended formatting (HFS Plus) avoids reusing space from deleted files as much as possible, to avoid prematurely filling small areas of recently-freed space.
• Mac OS X 10.2 and later includes delayed allocation for Mac OS X Extended-formatted volumes. This allows a number of small allocations to be combined into a single large allocation in one area of the disk.
• Fragmentation was often caused by continually appending data to existing files, especially with resource forks. With faster hard drives and better caching, as well as the new application packaging format, many applications simply rewrite the entire file each time. Mac OS X 10.3 Panther can also automatically defragment such slow-growing files. This process is sometimes known as "Hot-File-Adaptive-Clustering."
• Aggressive read-ahead and write-behind caching means that minor fragmentation has less effect on perceived system performance.

For these reasons, there is little benefit to defragmenting.

Note: Mac OS X systems use hundreds of thousands of small files, many of which are rarely accessed. Optimizing them can be a major effort for very little practical gain. There is also a chance that one of the files placed in the "hot band" for rapid reads during system startup might be moved during defragmentation, which would decrease performance.

If your disks are almost full, and you often modify or create large files (such as editing video, but see the Tip below if you use iMovie and Mac OS X 10.3), there's a chance the disks could be fragmented. In this case, you might benefit from defragmentation, which can be performed with some third-party disk utilities. 
 

Here is my take on that information:

While I have no problem with the lead-in which states probably, the reasons are theoretical. Expressing theory and then an opinion on that theory is fine, so long as you properly indicate it is an opinion. The problem I do have with this is the last sentence before the notation, "For these reasons, there is little benefit to defragmenting.", or more clearly; passing off theory as fact.

Theory, and therefore "reasons" need to be substantiated by actual scientific processes that apply the theory and then either validate or invalidate it. Common examples we hear of theory-as-fact are statements like "SSDs don't have moving parts and don't need to be defragmented". Given our primary business is large enterprise corporations, we hear a lot of theory about the need (or lack thereof) of defragmenting complex and expensive storage systems. In all those cases, testing proves fragmentation (files, free space or both) slows computers down. The reasons sound logical, which dupes readers/listeners into believing the statements are true.

On that note, while the first three are logical, the last "reason" is most likely wrong. Block-based read-ahead caching is predicated on files being sequentially located/interleaved on the same disk "tracks". File-based read-ahead would still have to issue additional I/Os due to fragmentation. Fragmentation of data essentially breaks read-ahead efforts. Could the Mac be predicting file access and pre-loading files into memory well in advance of use, sure. If that's the case I could agree with the last point (i.e. "perceived system performance), but I find this unlikely (anyone reading this is welcome to comment).

They do also qualify the reason by stating "minor fragmentation", to which I would add that that minor fragmentation on Windows may not have "perceived" impact either.

I do agree with the final statement that states "you might benefit from defragmentation" when using large files, although I think might is too indecisive.

Where my opinion comes from:

A few years ago (spring/summer of 2009) we did a research project to understand how much fragmentation existed on Apple Macs. We wrote and sent out a fragmentation/performance analysis tool to select customers who also had Macs at their homes/businesses. We collected data from 198 volumes on 82 Macs (OSX 10.4.x & 10.5.x). 30 of those systems were in use between 1 – 2 years. 

                               

While system specifics are confidential (testers provided us the data under non-disclosure agreements) we found that free space fragmentation was particularly bad in many cases (worse than Windows). We also found an average of a little over 26,000 fragments per Mac, with an average expected performance gain from defrag of about 8%.Our research also found that the more severe cases of fragmentation, where we saw 70k/100k+ fragments, were low on available free space (substantiating that last paragraph in the Apple tech article).

This article also provide some fragmentation studies as well as performance tests. Their data also validates Apple's last paragraph and makes the "might benefit" statement a bit understated.

Your Mileage May Vary (YMMV): 

So, in summary I would recommend defragmenting your Mac. As with Windows, the benefit from defragmenting is proportionate to the amount of fragmentation. Defrag will help. The question is "does defrag help enough to spend the time and money?". The good thing is most Mac defragmenters, just like Diskeeper for Windows, have free demo versions you can trial to see if its worth spending money.

 

Here are some options: 

+ iDefrag (used by a former Diskeeper Corp employee who did graphic design on a Mac)

+ Drive Genius suite (a company we have spoken with in the past)

+ Stellar Drive Defrag (relatively new)

Perhaps this article begs the question/rumor "will there be a Diskeeper for Mac?", to which I would answer "unlikely, but not impossible". The reason is that we already have a very full development schedule with opportunities in other areas that we plan to pursue.

We are keeping an "i" on it though ;-).

Dear CIO,

One of the most positive things to come out of the recession is a deeper appreciation of how much an efficient IT impacts corporate competiveness and profitability. And no wonder: envisioning and acting upon business opportunities requires having a powerful and flexible data center to work with.

In the real world, most networks are patchworks of legacy and cutting edge tech. Despite the best efforts of IT managers, IT investments take longer to recover and cost of operations creeps steadily upward.

Imagine a product that automatically ensured Windows systems ran at the peak speeds they were designed to deliver, reliably, using less energy and for a longer lifespan. Imagine the effect on operating costs and value return.

Hundreds of your peers in Global 1000 companies and Federal and State agencies don’t have to imagine this because this product exists and they refuse to run their networks without it: Diskeeper performance technology for Windows systems.

I am the CEO of the company that makes Diskeeper and I run the team behind the product. We’ve been around for 30 years; we’re in for the long term and we’d like an opportunity to help you achieve your IT goals faster.  

A simple evaluation on your IT network will show you the Diskeeper business value far better than I can say. Here is a link to quickly download a trial evaluation of Diskeeper. I suggest you have your IT manager test it out and report back to you. https://www.diskeeper.com/landing/diskeeper-30-day-trialware.aspx

Best,

Lisa Terrenzi

CEO

Diskeeper Corporation