SAP Sybase IQ SA CR 728597 / Linux Kernel direct i/o bug & huge pages

Last year, April -> October, I asked the question about IQ supporting Huge Pages on Linux. It was mentioned that under SA CR 728597 and Red Hat Bug 891857 that there was a bug in the Linux kernel handling of direct I/O while using transparent huge memory pages (a variant of Linux Huge memory pages).

CR 728597:
This problem is related to a possible bug in the transparent huge pages (THP) feature introduced in these operating system versions. Red Hat bug 891857 has been created to track this issue.

The problem can be triggered by calling an external environment, xp_cmdshell, or other procedure that causes a fork while other I/O is occurring. A known limitation with the Linux kernel limits the use of fork while doing O_DIRECT I/O operations. Essentially what can happen is that the data can come from or go to the wrong process’ memory after the fork. SQL Anywhere performs O_DIRECT I/O operations according to the documented safe usage. However, THP appears to cause further problems and the O_DIRECT I/O data comprising database page reads/writes appears to get lost.

http://scn.sap.com/thread/3338917 and http://froebe.net/blog/2013/06/17/does-anyone-have-any-details-on-redhat-linux-bug-891857/

Does anyone know the status of this ongoing FIVE year old issue?

http://scn.sap.com/thread/3505418

Share Button

Comparing Linux huge memory pages & Kernel Samepage Merging for KVM virtualization

A quick test (so take it with a grain of salt):

  1. Huge pages is slightly faster than not using huge pages (~10% with 4 winxp virtual machines copying 512MB from one memory location to another).
  2. KSM is slightly slower then not using KSM (~5% with 4 winxp virtual machines copying 512MB from one memory location to another).

So, at first glance it would appear that we can use a loose rule of thumb:

  1. to consolidate the maximum number of machines, use KSM as it will allow you to over commit the amount of memory on your box.
    1. Risk: if the memory pages are significantly different, you may start swapping in a very bad way. This is where monitoring comes in
  2. to give the best performance to a number of machines, use huge memory pages.. does not allow you to over commit
    1. Risk: if you don’t leave enough memory for the host os, you can crash your machine. This is also where monitoring comes in
Share Button