Package org.apache.sysds.api.mlcontext

Class Frame

java.lang.Object
org.apache.sysds.api.mlcontext.Frame
public class Frame extends Object
Frame encapsulates a SystemDS frame.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
    Create a Frame, specifying the SystemDS binary-block frame and its metadata.
    Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
    Convert a Spark DataFrame to a SystemDS binary-block representation.
    Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, long numRows, long numCols)
    Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.
    Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)
    Convert a Spark DataFrame to a SystemDS binary-block representation.
    Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    FrameMetadata
    getFrameMetadata()
    Obtain the frame metadata
    boolean
    hasBinaryBlocks()
    Whether or not this frame contains data as binary blocks
    boolean
    hasFrameObject()
    Whether or not this frame contains data as a FrameObject
    String[][]
    to2DStringArray()
    Obtain the frame as a two-dimensional String array
    org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>
    toBinaryBlocks()
    Obtain the frame as a JavaPairRDD<Long, FrameBlock>
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
    toDF()
    Obtain the frame as a DataFrame
    FrameObject
    toFrameObject()
    Obtain the frame as a SystemDS FrameObject.
    org.apache.spark.api.java.JavaRDD<String>
    toJavaRDDStringCSV()
    Obtain the frame as a JavaRDD<String> in CSV format
    org.apache.spark.api.java.JavaRDD<String>
    toJavaRDDStringIJV()
    Obtain the frame as a JavaRDD<String> in IJV format
    org.apache.spark.rdd.RDD<String>
    toRDDStringCSV()
    Obtain the frame as a RDD<String> in CSV format
    org.apache.spark.rdd.RDD<String>
    toRDDStringIJV()
    Obtain the frame as a RDD<String> in IJV format
    String
    toString()
     

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • Frame

      public Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)

    • Frame

      public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)
      Convert a Spark DataFrame to a SystemDS binary-block representation.
      Parameters:
      dataFrame - the Spark DataFrame
      frameMetadata - frame metadata, such as number of rows and columns

    • Frame

      public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, long numRows, long numCols)
      Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.
      Parameters:
      dataFrame - the Spark DataFrame
      numRows - the number of rows
      numCols - the number of columns

    • Frame

      public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
      Convert a Spark DataFrame to a SystemDS binary-block representation.
      Parameters:
      dataFrame - the Spark DataFrame

    • Frame

      public Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
      Create a Frame, specifying the SystemDS binary-block frame and its metadata.
      Parameters:
      binaryBlocks - the JavaPairRDD<Long, FrameBlock> frame
      frameMetadata - frame metadata, such as number of rows and columnss

  • Method Details

    • toFrameObject

      public FrameObject toFrameObject()
      Obtain the frame as a SystemDS FrameObject.
      Returns:
      the frame as a SystemDS FrameObject

    • to2DStringArray

      public String[][] to2DStringArray()
      Obtain the frame as a two-dimensional String array
      Returns:
      the frame as a two-dimensional String array

    • toJavaRDDStringIJV

      public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringIJV()
      Obtain the frame as a JavaRDD<String> in IJV format
      Returns:
      the frame as a JavaRDD<String> in IJV format

    • toJavaRDDStringCSV

      public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringCSV()
      Obtain the frame as a JavaRDD<String> in CSV format
      Returns:
      the frame as a JavaRDD<String> in CSV format

    • toRDDStringCSV

      public org.apache.spark.rdd.RDD<String> toRDDStringCSV()
      Obtain the frame as a RDD<String> in CSV format
      Returns:
      the frame as a RDD<String> in CSV format

    • toRDDStringIJV

      public org.apache.spark.rdd.RDD<String> toRDDStringIJV()
      Obtain the frame as a RDD<String> in IJV format
      Returns:
      the frame as a RDD<String> in IJV format

    • toDF

      public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> toDF()
      Obtain the frame as a DataFrame
      Returns:
      the frame as a DataFrame

    • toBinaryBlocks

      public org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> toBinaryBlocks()
      Obtain the frame as a JavaPairRDD<Long, FrameBlock>
      Returns:
      the frame as a JavaPairRDD<Long, FrameBlock>

    • getFrameMetadata

      public FrameMetadata getFrameMetadata()
      Obtain the frame metadata
      Returns:
      the frame metadata

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • hasBinaryBlocks

      public boolean hasBinaryBlocks()
      Whether or not this frame contains data as binary blocks
      Returns:
      true if data as binary blocks are present, false otherwise.

    • hasFrameObject

      public boolean hasFrameObject()
      Whether or not this frame contains data as a FrameObject
      Returns:
      true if data as binary blocks are present, false otherwise.