Skip to main content
Skip table of contents

Memory Configuration

Memory Settings

Running the CData Virtuality Server requires a considerable amount of memory. Providing and configuring the memory is crucial for a fast and stable server installation, especially when the CData Virtuality Server has to perform many operations or handle vast amounts of data. 

Memory for the CData Virtuality Server is configured via JVM-based memory settings and BufferManager settings.

The CData Virtuality Server has an auto-adjust mechanism to configure memory usage according to your local resources.

Auto-Adjusting

The CData Virtuality Server startup script configures memory settings as indicated in section "Memory Settings Recommendations" automatically on bootstrap. 

You can configure the script using the analytic_storage_on_different_host_dv parameter in %pathToDVserver%/bin/standalone.conf (Linux)/ %pathToDVserver%\bin\standalone.conf.bat (Windows) files:

ParameterValue
analytic_storage_on_different_host_dv
  • 0 (default) if the analytical storage and the CData Virtuality Server are on the same host
  • 1 if the analytical storage and the CData Virtuality Server use different hosts

Configuration Files

You can find memory settings in the following files:

Operating SystemFileComment
Linux%pathToDVserver%/bin/standalone.conf

Contains JVM-based memory settings

Windows%pathToDVserver%\bin\standalone.conf.batContains JVM-based memory settings
Linux/Windows%pathToDVserver%/standalone/configuration/dvserver-standalone.xmlContains memory settings of the application server

On Windows platforms, it is possible to configure JVM memory settings in different files.

We do not recommend doing this! If memory settings are found in different locations, the server will use the ones read last.

If you find duplicate startup options in the boot.log file, remove memory settings from all locations other then standalone.conf.bat. They may reside in the following places:

  • %pathToDVserver%\bin\standalone.bat
  • %pathToDVserver%\bin\service.bat

JVM Settings

You can configure the memory settings of the CData Virtuality Server process via the JVM settings. It is important to properly configure the amount of heap memory and allocated total memory.

JVM optionExampleComment
-Xms-Xms8912MMinimum amount of heap memory -Xms and -Xmx may be set to equal values
-Xmx-Xmx8912MMaximum amount of heap memory -Xms and -Xmx may be set to equal values
-XX:MaxDirectMemorySize-XX:MaxDirectMemorySize=6G

Maximum total size for new Java I/O package direct buffer allocations


-XX:MaxDirectMemorySize (JVM setting) may not be less than the sum of memory-buffer-space and max-storage-object-size
-XX:ReservedCodeCacheSize-XX:ReservedCodeCacheSize=128mSize of cache used to store code which Java's compiler will access

BufferManager Settings

BufferManager is responsible for tracking memory and disk usage by the CData Virtuality Server. Configuring the BufferManager properly is one of the most important parts of ensuring high performance.

Settings for the BufferManager are stored in the %pathToDVserver%\standalone\configuration\dvserver-standalone.xml file in the CData Virtuality Server folder, section buffer-service. Here is an example of how they may look:

XML
<buffer-service memory-buffer-off-heap="true" max-storage-object-size="134217728"/>
Application Server SettingDefault Value Description Remarks
max-reserve-kb-1

Total size of batches that the BufferManager can hold in memory.

The default value  -1 will auto-calculate a typical max based upon the max heap available to the VM

Specified in KB without a measurement unit
max-processing-kb-1

Total size of batches that can be used by active plans regardless of the memory held based on max-reserve-kb.

The default value of -1 will auto-calculate a typical max based upon the max heap available to the VM and max active plan

Specified in KB without a measurement unit
max-file-size2048 (in MB)
Specified in MB without a measurement unit
max-buffer-space51200 (in MB, i.e., 50GB)
Specified in MB without a measurement unit
max-storage-object-size134217728 (in bytes)

Sizing for max-storage-object-size is in terms of serialized size, which will be much closer to the raw data size than the Java memory footprint estimation used for max-reserve-kb.
max-storage-object-size should not be set too large relatively to memory-buffer-space since it will reduce the performance of the memory buffer. The memory buffer supports only one concurrent writer for each max-storage-object-size of memory-buffer-space

Specified in bytes without a measurement unit
memory-buffer-off-heapTRUE

Setting memory-buffer-off-heap to TRUE will allocate the CData Virtuality Server memory buffer off-heap.

Depending on whether your system runs only the CData Virtuality Server and the amount of system memory available, this may be preferable to on-heap allocation. The primary benefit is additional memory usage without additional garbage collection tuning. This becomes especially important in situations where more than 32GB of memory is desired for the VM. Note that when using off-heap allocation, the memory must still be available to the java process and that setting the value of memoryBufferSpace too high may cause the VM to swap rather than reside in memory.


memory-buffer-space-1

Determines how much memory exactly the server can use for buffers/memory-buffer-space setting controls the amount of on or off-heap memory allocated as byte buffers for use by the buffer manager.

The default value  -1 automatically determines a setting based upon whether it is on or off-heap, and the value for max-reserve-kb. You can use the buffer manager memory buffer to access system memory without allocating it to the heap.

buffer-service element of dvserver-standalone.xml

Specified in MB without a measurement unit

In systems where extensive intermediate results are normal (scrolling cursors or sorting over millions of rows), you can consider increasing max-processing-kb and decreasing max-reserve-kb so that each request has access to an effectively smaller buffer space. Each intermediate result buffer, temporary LOB, and the temporary table are stored in their own set of buffer files, where an individual file is limited to max-file-size megabytes. Consider increasing the storage space available to all such files if your installation uses internal materialization, makes heavy use of SQL/XML, or processes large row counts. 

Recommendations

General Recommendations


Analytical Storage on
Settingsamedifferent

host as DataVirtuality Server


-XX:MaxDirectMemorySize

6575% of available RAM

8085% of available RAM

-Xms, -Xmx

60% of MaxDirectMemorySize

-XX:MaxPermSize

10% of Xms

14% of Xms

-XX:UseLargePages

yes, if Xms >= 12GB RAM

memory-buffer-space

50% of MaxDirectMemorySize

-XX:ReservedCodeCacheSize256MB RAM

Example: Recommendations for a System with 32 GB of RAM


Analytical Storage on

Setting

same

different


host as CData Virtuality Server

-XX:MaxDirectMemorySize

20GB

27GB

-Xms, -Xmx

12GB

16 GB

-XX:MaxPermSize

512MB

768MB

-XX:UseLargePages

-UseLargePages

+UseLargePages

memory-buffer-space

10240

13312

-XX:ReservedCodeCacheSize256MB

See Also

Changing Directory of Temp Folder for a guide on how to change the location of your CData Virtuality Server temp folder

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.