Package org.apache.sysds.conf

Class DMLConfig

java.lang.Object

org.apache.sysds.conf.DMLConfig

public class DMLConfig
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ASYNC_PREFETCH	Asynchronous triggering of Spark OPs and operator placement
static final String	ASYNC_SPARK_BROADCAST
static final String	ASYNC_SPARK_CHECKPOINT
static final String	AVAILABLE_GPUS
static final String	BUFFERPOOL_LIMIT
static final String	CODEGEN
static final String	CODEGEN_API
static final String	CODEGEN_COMPILER
static final String	CODEGEN_LITERALS
static final String	CODEGEN_OPTIMIZER
static final String	CODEGEN_PLANCACHE
static final String	COMPILERASSISTED_RW
static final String	COMPRESSED_COCODE
static final String	COMPRESSED_COST_MODEL
static final String	COMPRESSED_LINALG
static final String	COMPRESSED_LINALG_INTERMEDIATE
static final String	COMPRESSED_LOSSY
static final String	COMPRESSED_OVERLAPPING
static final String	COMPRESSED_SAMPLING_RATIO
static final String	COMPRESSED_SOFT_REFERENCE_COUNT
static final String	COMPRESSED_TRANSFORMENCODE
static final String	COMPRESSED_TRANSPOSE
static final String	COMPRESSED_VALID_COMPRESSIONS
static final String	CP_PARALLEL_IO
static final String	CP_PARALLEL_OPS
static final String	DAG_LINEARIZATION
static final String	DEFAULT_BLOCK_SIZE
static final String	DEFAULT_FEDERATED_INITIALIZATION_TIMEOUT
static final int	DEFAULT_FEDERATED_PORT
static final int	DEFAULT_NUMBER_OF_FEDERATED_WORKER_THREADS
static final String	DEFAULT_SHARED_DIR_PERMISSION
static final String	DEFAULT_SYSTEMDS_CONFIG_FILEPATH
static final String	DELTA_READER_BATCH_SIZE
static final String	DELTA_WRITER_ADAPTIVE_FILE_SIZE
static final String	DELTA_WRITER_BATCH_SIZE
static final String	DELTA_WRITER_TARGET_FILE_SIZE
static final String	EAGER_CUDA_FREE
static final String	EVICTION_SHADOW_BUFFERSIZE
static final String	FEDERATED_COMPRESSION
static final String	FEDERATED_MONITOR_FREQUENCY	Trigger frequency of the collecting and parsing statistics process on registered workers for monitoring in seconds
static final String	FEDERATED_PAR_CONN
static final String	FEDERATED_PAR_INST
static final String	FEDERATED_PLANNER
static final String	FEDERATED_READCACHE
static final String	FEDERATED_TIMEOUT
static final String	FLOATING_POINT_PRECISION
static final String	FRAME_TO_MATRIX_WARN_CAST
static final String	GPU_EVICTION_POLICY
static final String	GPU_MEMORY_ALLOCATOR
static final String	GPU_MEMORY_UTILIZATION_FACTOR
static final String	GPU_RULE_BASED_PLACEMENT
static final String	IO_COMPRESSION_CODEC
static final String	LINEAGECACHESPILL
static final String	LOCAL_SPARK_NUM_THREADS
static final String	LOCAL_TMP_DIR
static final String	MEMORY_MANAGER
static final String	NATIVE_BLAS
static final String	NATIVE_BLAS_DIR
static final String	OPTIMIZATION_LEVEL
static final String	PARALLEL_ENCODE
static final String	PARALLEL_ENCODE_APPLY_BLOCKS
static final String	PARALLEL_ENCODE_BUILD_BLOCKS
static final String	PARALLEL_ENCODE_NUM_THREADS
static final String	PARALLEL_ENCODE_STAGED
static final String	PARALLEL_TOKENIZE
static final String	PARALLEL_TOKENIZE_NUM_BLOCKS
static final String	PRINT_GPU_MEMORY_INFO
static final String	PRIVACY_CONSTRAINT_MOCK
static final String	SCRATCH_SPACE
static final String	STATS_MAX_WRAP_LEN
static final String	SYNCHRONIZE_GPU
static final String	USE_LOCAL_SPARK_CONFIG
static final String	USE_SSL_FEDERATED_COMMUNICATION

Constructor Summary

Constructors

Constructor	Description
DMLConfig()
DMLConfig(String fileName)
DMLConfig(String fileName, boolean silent)
DMLConfig(DMLConfig dmlconf)
DMLConfig(Element root)

Method Summary

Modifier and Type	Method	Description
DMLConfig	clone()
boolean	getBooleanValue(String tagName)
String	getConfigInfo()
static String	getDefaultTextValue(String key)
double	getDoubleValue(String tagName)
int	getIntValue(String tagName)
String	getTextValue(String tagName)	Method to get string value of a configuration parameter Handles processing of configuration parameters
static DMLConfig	parseDMLConfig(String content)
static DMLConfig	readConfigurationFile(String configPath)	Start with the internal default settings, then merge in the settings from any specified configuration file, if available.
String	serializeDMLConfig()
void	set(DMLConfig dmlconf)
void	setTextValue(String paramName, String paramValue)	Method to update the key value

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_SYSTEMDS_CONFIG_FILEPATH

public static final String DEFAULT_SYSTEMDS_CONFIG_FILEPATH

See Also:

• Constant Field Values

LOCAL_TMP_DIR

public static final String LOCAL_TMP_DIR

See Also:

• Constant Field Values

SCRATCH_SPACE

public static final String SCRATCH_SPACE

See Also:

• Constant Field Values

OPTIMIZATION_LEVEL

public static final String OPTIMIZATION_LEVEL

See Also:

• Constant Field Values

DEFAULT_BLOCK_SIZE

public static final String DEFAULT_BLOCK_SIZE

See Also:

• Constant Field Values

CP_PARALLEL_OPS

public static final String CP_PARALLEL_OPS

See Also:

• Constant Field Values

CP_PARALLEL_IO

public static final String CP_PARALLEL_IO

See Also:

• Constant Field Values

IO_COMPRESSION_CODEC

public static final String IO_COMPRESSION_CODEC

See Also:

• Constant Field Values

DELTA_READER_BATCH_SIZE

public static final String DELTA_READER_BATCH_SIZE

See Also:

• Constant Field Values

DELTA_WRITER_BATCH_SIZE

public static final String DELTA_WRITER_BATCH_SIZE

See Also:

• Constant Field Values

DELTA_WRITER_TARGET_FILE_SIZE

public static final String DELTA_WRITER_TARGET_FILE_SIZE

See Also:

• Constant Field Values

DELTA_WRITER_ADAPTIVE_FILE_SIZE

public static final String DELTA_WRITER_ADAPTIVE_FILE_SIZE

See Also:

• Constant Field Values

PARALLEL_ENCODE

public static final String PARALLEL_ENCODE

See Also:

• Constant Field Values

PARALLEL_ENCODE_STAGED

public static final String PARALLEL_ENCODE_STAGED

See Also:

• Constant Field Values

PARALLEL_ENCODE_APPLY_BLOCKS

public static final String PARALLEL_ENCODE_APPLY_BLOCKS

See Also:

• Constant Field Values

PARALLEL_ENCODE_BUILD_BLOCKS

public static final String PARALLEL_ENCODE_BUILD_BLOCKS

See Also:

• Constant Field Values

PARALLEL_ENCODE_NUM_THREADS

public static final String PARALLEL_ENCODE_NUM_THREADS

See Also:

• Constant Field Values

PARALLEL_TOKENIZE

public static final String PARALLEL_TOKENIZE

See Also:

• Constant Field Values

PARALLEL_TOKENIZE_NUM_BLOCKS

public static final String PARALLEL_TOKENIZE_NUM_BLOCKS

See Also:

• Constant Field Values

FRAME_TO_MATRIX_WARN_CAST

public static final String FRAME_TO_MATRIX_WARN_CAST

See Also:

• Constant Field Values

COMPRESSED_LINALG

public static final String COMPRESSED_LINALG

See Also:

• Constant Field Values

COMPRESSED_LINALG_INTERMEDIATE

public static final String COMPRESSED_LINALG_INTERMEDIATE

See Also:

• Constant Field Values

COMPRESSED_LOSSY

public static final String COMPRESSED_LOSSY

See Also:

• Constant Field Values

COMPRESSED_VALID_COMPRESSIONS

public static final String COMPRESSED_VALID_COMPRESSIONS

See Also:

• Constant Field Values

COMPRESSED_OVERLAPPING

public static final String COMPRESSED_OVERLAPPING

See Also:

• Constant Field Values

COMPRESSED_SAMPLING_RATIO

public static final String COMPRESSED_SAMPLING_RATIO

See Also:

• Constant Field Values

COMPRESSED_SOFT_REFERENCE_COUNT

public static final String COMPRESSED_SOFT_REFERENCE_COUNT

See Also:

• Constant Field Values

COMPRESSED_COCODE

public static final String COMPRESSED_COCODE

See Also:

• Constant Field Values

COMPRESSED_COST_MODEL

public static final String COMPRESSED_COST_MODEL

See Also:

• Constant Field Values

COMPRESSED_TRANSPOSE

public static final String COMPRESSED_TRANSPOSE

See Also:

• Constant Field Values

COMPRESSED_TRANSFORMENCODE

public static final String COMPRESSED_TRANSFORMENCODE

See Also:

• Constant Field Values

NATIVE_BLAS

public static final String NATIVE_BLAS

See Also:

• Constant Field Values

NATIVE_BLAS_DIR

public static final String NATIVE_BLAS_DIR

See Also:

• Constant Field Values

DAG_LINEARIZATION

public static final String DAG_LINEARIZATION

See Also:

• Constant Field Values

CODEGEN

public static final String CODEGEN

See Also:

• Constant Field Values

CODEGEN_API

public static final String CODEGEN_API

See Also:

• Constant Field Values

CODEGEN_COMPILER

public static final String CODEGEN_COMPILER

See Also:

• Constant Field Values

CODEGEN_OPTIMIZER

public static final String CODEGEN_OPTIMIZER

See Also:

• Constant Field Values

CODEGEN_PLANCACHE

public static final String CODEGEN_PLANCACHE

See Also:

• Constant Field Values

CODEGEN_LITERALS

public static final String CODEGEN_LITERALS

See Also:

• Constant Field Values

STATS_MAX_WRAP_LEN

public static final String STATS_MAX_WRAP_LEN

See Also:

• Constant Field Values

AVAILABLE_GPUS

public static final String AVAILABLE_GPUS

See Also:

• Constant Field Values

SYNCHRONIZE_GPU

public static final String SYNCHRONIZE_GPU

See Also:

• Constant Field Values

EAGER_CUDA_FREE

public static final String EAGER_CUDA_FREE

See Also:

• Constant Field Values

GPU_EVICTION_POLICY

public static final String GPU_EVICTION_POLICY

See Also:

• Constant Field Values

GPU_RULE_BASED_PLACEMENT

public static final String GPU_RULE_BASED_PLACEMENT

See Also:

• Constant Field Values

USE_LOCAL_SPARK_CONFIG

public static final String USE_LOCAL_SPARK_CONFIG

See Also:

• Constant Field Values

LOCAL_SPARK_NUM_THREADS

public static final String LOCAL_SPARK_NUM_THREADS

See Also:

• Constant Field Values

LINEAGECACHESPILL

public static final String LINEAGECACHESPILL

See Also:

• Constant Field Values

COMPILERASSISTED_RW

public static final String COMPILERASSISTED_RW

See Also:

• Constant Field Values

BUFFERPOOL_LIMIT

public static final String BUFFERPOOL_LIMIT

See Also:

• Constant Field Values

MEMORY_MANAGER

public static final String MEMORY_MANAGER

See Also:

• Constant Field Values

GPU_MEMORY_UTILIZATION_FACTOR

public static final String GPU_MEMORY_UTILIZATION_FACTOR

See Also:

• Constant Field Values

GPU_MEMORY_ALLOCATOR

public static final String GPU_MEMORY_ALLOCATOR

See Also:

• Constant Field Values

FLOATING_POINT_PRECISION

public static final String FLOATING_POINT_PRECISION

See Also:

• Constant Field Values

PRINT_GPU_MEMORY_INFO

public static final String PRINT_GPU_MEMORY_INFO

See Also:

• Constant Field Values

EVICTION_SHADOW_BUFFERSIZE

public static final String EVICTION_SHADOW_BUFFERSIZE

See Also:

• Constant Field Values

USE_SSL_FEDERATED_COMMUNICATION

public static final String USE_SSL_FEDERATED_COMMUNICATION

See Also:

• Constant Field Values

DEFAULT_FEDERATED_INITIALIZATION_TIMEOUT

public static final String DEFAULT_FEDERATED_INITIALIZATION_TIMEOUT

See Also:

• Constant Field Values

FEDERATED_TIMEOUT

public static final String FEDERATED_TIMEOUT

See Also:

• Constant Field Values

FEDERATED_PLANNER

public static final String FEDERATED_PLANNER

See Also:

• Constant Field Values

FEDERATED_PAR_INST

public static final String FEDERATED_PAR_INST

See Also:

• Constant Field Values

FEDERATED_PAR_CONN

public static final String FEDERATED_PAR_CONN

See Also:

• Constant Field Values

FEDERATED_READCACHE

public static final String FEDERATED_READCACHE

See Also:

• Constant Field Values

FEDERATED_COMPRESSION

public static final String FEDERATED_COMPRESSION

See Also:

• Constant Field Values

PRIVACY_CONSTRAINT_MOCK

public static final String PRIVACY_CONSTRAINT_MOCK

See Also:

• Constant Field Values

FEDERATED_MONITOR_FREQUENCY

public static final String FEDERATED_MONITOR_FREQUENCY

Trigger frequency of the collecting and parsing statistics process on registered workers for monitoring in seconds

See Also:

• Constant Field Values

DEFAULT_FEDERATED_PORT

public static final int DEFAULT_FEDERATED_PORT

See Also:

• Constant Field Values

DEFAULT_NUMBER_OF_FEDERATED_WORKER_THREADS

public static final int DEFAULT_NUMBER_OF_FEDERATED_WORKER_THREADS

See Also:

• Constant Field Values

ASYNC_PREFETCH

public static final String ASYNC_PREFETCH

Asynchronous triggering of Spark OPs and operator placement

See Also:

• Constant Field Values

ASYNC_SPARK_BROADCAST

public static final String ASYNC_SPARK_BROADCAST

See Also:

• Constant Field Values

ASYNC_SPARK_CHECKPOINT

public static final String ASYNC_SPARK_CHECKPOINT

See Also:

• Constant Field Values

DEFAULT_SHARED_DIR_PERMISSION

public static final String DEFAULT_SHARED_DIR_PERMISSION

See Also:

• Constant Field Values

Constructor Details

DMLConfig

public DMLConfig()

DMLConfig

public DMLConfig(String fileName)
          throws FileNotFoundException

Throws:

FileNotFoundException

DMLConfig

public DMLConfig(String fileName,
 boolean silent)
          throws FileNotFoundException

Throws:

FileNotFoundException

DMLConfig

public DMLConfig(Element root)

DMLConfig

public DMLConfig(DMLConfig dmlconf)

Method Details

set

public void set(DMLConfig dmlconf)

getTextValue

public String getTextValue(String tagName)

Method to get string value of a configuration parameter Handles processing of configuration parameters

Parameters:

tagName - the name of the DMLConfig parameter being retrieved

Returns:

a string representation of the DMLConfig parameter value.

getIntValue

public int getIntValue(String tagName)

getBooleanValue

public boolean getBooleanValue(String tagName)

getDoubleValue

public double getDoubleValue(String tagName)

setTextValue

public void setTextValue(String paramName,
 String paramValue)

Method to update the key value

Parameters:

paramName - parameter name

paramValue - parameter value

serializeDMLConfig

public String serializeDMLConfig()

parseDMLConfig

public static DMLConfig parseDMLConfig(String content)

readConfigurationFile

public static DMLConfig readConfigurationFile(String configPath)
                                       throws FileNotFoundException

Start with the internal default settings, then merge in the settings from any specified configuration file, if available. If it is not explicitly given, then merge in settings from the default configuration file location, if available.

Parameters:

configPath - User-defined path of the configuration file.

Returns:

dml configuration

Throws:

FileNotFoundException - if FileNotFoundException occurs

getConfigInfo

public String getConfigInfo()

getDefaultTextValue

public static String getDefaultTextValue(String key)

clone

public DMLConfig clone()