Package org.apache.sysds.runtime.io

Class FrameReaderDelta

java.lang.Object

org.apache.sysds.runtime.io.FrameReader

org.apache.sysds.runtime.io.FrameReaderDelta

Direct Known Subclasses:

FrameReaderDeltaParallel

public class FrameReaderDelta
extends FrameReader

Single-threaded native Delta Lake reader for frames, built on the Spark-free Delta Kernel library. It opens the latest snapshot of a Delta table through the kernel (log replay, schema, data-file listing) and decodes the parquet data files directly into pre-allocated columns via parquet-mr's column API; tables with deletion vectors, partition columns or missing row statistics are read through the kernel's default engine instead (which applies deletion vectors and splices partition values). Schema and column names are derived from the Delta table schema.

Data is extracted column-at-a-time into primitive arrays (no per-cell boxing or FrameBlock.set dispatch) and the frame is constructed directly from typed column Arrays. Supported column types map to SystemDS value types: double, float, long, int, short, byte, boolean, and string. Neither the schema nor the dimensions need to be supplied; they are discovered from the table.

Constructor Summary

Constructors

Constructor	Description
FrameReaderDelta()

Method Summary

Modifier and Type	Method	Description
FrameBlock	readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)
FrameBlock	readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)

Methods inherited from class org.apache.sysds.runtime.io.FrameReader

getDefColNames, getDefSchema, readFrameFromHDFS, readFrameFromHDFS, readFrameFromInputStream, readFrameFromInputStream

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FrameReaderDelta

public FrameReaderDelta()

Method Details

readFrameFromHDFS

public FrameBlock readFrameFromHDFS(String fname,
 Types.ValueType[] schema,
 String[] names,
 long rlen,
 long clen)
                             throws IOException,
DMLRuntimeException

Specified by:

readFrameFromHDFS in class FrameReader

Throws:

IOException

DMLRuntimeException

readFrameFromInputStream

public FrameBlock readFrameFromInputStream(InputStream is,
 Types.ValueType[] schema,
 String[] names,
 long rlen,
 long clen)
                                    throws IOException,
DMLRuntimeException

Specified by:

readFrameFromInputStream in class FrameReader

Throws:

IOException

DMLRuntimeException