public class Frame extends Object
| Constructor and Description |
|---|
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert a Spark DataFrame to a SystemDS binary-block representation.
|
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
FrameMetadata frameMetadata)
Convert a Spark DataFrame to a SystemDS binary-block representation.
|
Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
long numRows,
long numCols)
Convert a Spark DataFrame to a SystemDS binary-block representation,
specifying the number of rows and columns.
|
Frame(FrameObject frameObject,
SparkExecutionContext sparkExecutionContext) |
Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks,
FrameMetadata frameMetadata)
Create a Frame, specifying the SystemDS binary-block frame and its
metadata.
|
| Modifier and Type | Method and Description |
|---|---|
FrameMetadata |
getFrameMetadata()
Obtain the frame metadata
|
boolean |
hasBinaryBlocks()
Whether or not this frame contains data as binary blocks
|
boolean |
hasFrameObject()
Whether or not this frame contains data as a FrameObject
|
String[][] |
to2DStringArray()
Obtain the frame as a two-dimensional String array
|
org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> |
toBinaryBlocks()
Obtain the frame as a
JavaPairRDD<Long, FrameBlock> |
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
toDF()
Obtain the frame as a
DataFrame |
FrameObject |
toFrameObject()
Obtain the frame as a SystemDS FrameObject.
|
org.apache.spark.api.java.JavaRDD<String> |
toJavaRDDStringCSV()
Obtain the frame as a
JavaRDD<String> in CSV format |
org.apache.spark.api.java.JavaRDD<String> |
toJavaRDDStringIJV()
Obtain the frame as a
JavaRDD<String> in IJV format |
org.apache.spark.rdd.RDD<String> |
toRDDStringCSV()
Obtain the frame as a
RDD<String> in CSV format |
org.apache.spark.rdd.RDD<String> |
toRDDStringIJV()
Obtain the frame as a
RDD<String> in IJV format |
String |
toString() |
public Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)
public Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
FrameMetadata frameMetadata)
dataFrame - the Spark DataFrameframeMetadata - frame metadata, such as number of rows and columnspublic Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
long numRows,
long numCols)
dataFrame - the Spark DataFramenumRows - the number of rowsnumCols - the number of columnspublic Frame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
dataFrame - the Spark DataFramepublic Frame(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)
binaryBlocks - the JavaPairRDD<Long, FrameBlock> frameframeMetadata - frame metadata, such as number of rows and columnsspublic FrameObject toFrameObject()
public String[][] to2DStringArray()
public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringIJV()
JavaRDD<String> in IJV formatJavaRDD<String> in IJV formatpublic org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringCSV()
JavaRDD<String> in CSV formatJavaRDD<String> in CSV formatpublic org.apache.spark.rdd.RDD<String> toRDDStringCSV()
RDD<String> in CSV formatRDD<String> in CSV formatpublic org.apache.spark.rdd.RDD<String> toRDDStringIJV()
RDD<String> in IJV formatRDD<String> in IJV formatpublic org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> toDF()
DataFrameDataFramepublic org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> toBinaryBlocks()
JavaPairRDD<Long, FrameBlock>JavaPairRDD<Long, FrameBlock>public FrameMetadata getFrameMetadata()
public boolean hasBinaryBlocks()
true if data as binary blocks are present, false
otherwise.public boolean hasFrameObject()
true if data as binary blocks are present, false
otherwise.Copyright © 2021 The Apache Software Foundation. All rights reserved.