Package org.apache.sysds.runtime.io

Class WriterTextCSV

java.lang.Object

org.apache.sysds.runtime.io.MatrixWriter

org.apache.sysds.runtime.io.WriterTextCSV

Direct Known Subclasses:

WriterTextCSVParallel

public class WriterTextCSV
extends MatrixWriter

Field Summary

Fields

Modifier and Type	Field	Description
static final int	BLOCKSIZE_J

Constructor Summary

Constructors

Constructor	Description
WriterTextCSV(FileFormatPropertiesCSV props)

Method Summary

Modifier and Type	Method	Description
final void	addHeaderToCSV(String srcFileName, String destFileName, long rlen, long clen)
final void	writeEmptyMatrixToHDFS(String fname, long rlen, long clen, int blen)	Writes a minimal entry to represent an empty matrix on hdfs.
long	writeMatrixFromStream(String fname, OOCStream<IndexedMatrixValue> stream, long rlen, long clen, int blen)	Consumes an out-of-core stream of matrix blocks and writes them to the target output path.
final void	writeMatrixToHDFS(MatrixBlock src, String fname, long rlen, long clen, int blen, long nnz, boolean diag)

Methods inherited from class org.apache.sysds.runtime.io.MatrixWriter

createMatrixBlocksForReuse, getMatrixBlockForReuse, setForcedParallel, writeMatrixToHDFS

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

BLOCKSIZE_J

public static final int BLOCKSIZE_J

See Also:

• Constant Field Values

Constructor Details

WriterTextCSV

public WriterTextCSV(FileFormatPropertiesCSV props)

Method Details

writeMatrixToHDFS

public final void writeMatrixToHDFS(MatrixBlock src,
 String fname,
 long rlen,
 long clen,
 int blen,
 long nnz,
 boolean diag)
                             throws IOException,
DMLRuntimeException

Specified by:

writeMatrixToHDFS in class MatrixWriter

Throws:

IOException

DMLRuntimeException

writeEmptyMatrixToHDFS

public final void writeEmptyMatrixToHDFS(String fname,
 long rlen,
 long clen,
 int blen)
                                  throws IOException,
DMLRuntimeException

Description copied from class: MatrixWriter

Writes a minimal entry to represent an empty matrix on hdfs.

Specified by:

writeEmptyMatrixToHDFS in class MatrixWriter

Parameters:

fname - file name

rlen - number of rows

clen - number of columns

blen - number of rows/cols in block

Throws:

IOException - if IOException occurs

DMLRuntimeException

addHeaderToCSV

public final void addHeaderToCSV(String srcFileName,
 String destFileName,
 long rlen,
 long clen)
                          throws IOException

Throws:

IOException

writeMatrixFromStream

public long writeMatrixFromStream(String fname,
 OOCStream<IndexedMatrixValue> stream,
 long rlen,
 long clen,
 int blen)

Description copied from class: MatrixWriter

Consumes an out-of-core stream of matrix blocks and writes them to the target output path. Implementations may choose single-file or multipart output depending on format and parallelism.

Specified by:

writeMatrixFromStream in class MatrixWriter

Parameters:

fname - The target output filename

stream - The OOC stream of matrix blocks to consume

rlen - The total number of rows in the matrix

clen - The total number of columns in the matrix

blen - The block size