Package org.apache.sysds.runtime.controlprogram.parfor

Class DataPartitioner

java.lang.Object

org.apache.sysds.runtime.controlprogram.parfor.DataPartitioner

Direct Known Subclasses:

DataPartitionerLocal, DataPartitionerRemoteSpark

public abstract class DataPartitioner
extends Object

This is the base class for all data partitioner.

Method Summary

Modifier and Type	Method	Description
MatrixObject	createPartitionedMatrixObject(MatrixObject in, String fnameNew)
MatrixObject	createPartitionedMatrixObject(MatrixObject in, String fnameNew, boolean force)
MatrixObject	createPartitionedMatrixObject(MatrixObject in, MatrixObject out, boolean force)	Creates a partitioned matrix object based on the given input matrix object, according to the specified split format.
static MatrixBlock	createReuseMatrixBlock(ParForProgramBlock.PDataPartitionFormat dpf, int rows, int cols)
void	disableBinaryCell()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
 String fnameNew)

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
 String fnameNew,
 boolean force)

createPartitionedMatrixObject

public MatrixObject createPartitionedMatrixObject(MatrixObject in,
 MatrixObject out,
 boolean force)

Creates a partitioned matrix object based on the given input matrix object, according to the specified split format. The input matrix can be in-memory or still on HDFS and the partitioned output matrix is written to HDFS. The created matrix object can be used transparently for obtaining the full matrix or reading 1 or multiple partitions based on given index ranges.

Parameters:

in - input matrix object

out - output matrix object

force - if false, try to optimize

Returns:

partitioned matrix object

disableBinaryCell

public void disableBinaryCell()

createReuseMatrixBlock

public static MatrixBlock createReuseMatrixBlock(ParForProgramBlock.PDataPartitionFormat dpf,
 int rows,
 int cols)