Package org.apache.sysds.lops
Class Data
- java.lang.Object
-
- org.apache.sysds.lops.Lop
-
- org.apache.sysds.lops.Data
-
public class Data extends Lop
Lop to represent data objects. Data objects represent matrices, vectors, variables, literals. Can be for both input and output.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.sysds.lops.Lop
Lop.SimpleInstType, Lop.Type, Lop.VisitStatus
-
-
Field Summary
Fields Modifier and Type Field Description static StringPREAD_PREFIX-
Fields inherited from class org.apache.sysds.lops.Lop
_beginColumn, _beginLine, _endColumn, _endLine, _filename, CP_CHILD_THREAD, CP_ROOT_THREAD_ID, DATATYPE_PREFIX, FILE_SEPARATOR, FRAME_VAR_NAME_PREFIX, INSTRUCTION_DELIMITOR, LITERAL_PREFIX, MATRIX_VAR_NAME_PREFIX, NAME_VALUE_SEPARATOR, OPERAND_DELIMITOR, PROCESS_PREFIX, SAMPLE_FRACTION, SCALAR_VAR_NAME_PREFIX, UPDATE_INPLACE_PREFIX, VALUETYPE_PREFIX, VARIABLE_NAME_PLACEHOLDER
-
-
Constructor Summary
Constructors Constructor Description Data(Types.OpOpData op, Lop input, HashMap<String,Lop> inputParametersLops, String name, String literal, Types.DataType dt, Types.ValueType vt, Types.FileFormat fmt)Constructor to setup read or write LOP In case of write:inputmust be provided.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static DatacreateLiteralLop(Types.ValueType vt, String literalValue)Method to create literal LOPs.booleangetBooleanValue()StringgetCreateVarInstructions(String outputFileName, String outputLabel)doublegetDoubleValue()Types.FileFormatgetFileFormatType()method to get format type for input, output files.HashMap<String,Lop>getInputParams()method to get inputParamsStringgetInstructions()Method to generate createvar instruction that updates symbol table with metadata, hdfsfile name, etc.StringgetInstructions(String outputFileName)Method should be overridden if neededStringgetInstructions(String input1, String input2)Method to get CP instructions for reading/writing scalars and matrices from/to HDFS.longgetLongValue()LopgetNamedInputLop(String name)LopgetNamedInputLop(String name, String defaultVal)Types.OpOpDatagetOperationType()method to get operation type, i.e.StringgetStringValue()booleanisLiteral()method to check if this data lop represents a literal.booleanisPersistentRead()booleanisPersistentWrite()voidsetExecType(Types.ExecType et)Data-Lop-specific method to set the execution type for persistent write.StringtoString()Method to have Lops print their state.-
Methods inherited from class org.apache.sysds.lops.Lop
activatePrefetch, addInput, addOutput, addToDag, createReachable, getAggType, getBeginColumn, getBeginLine, getBroadcastInput, getDataType, getEndColumn, getEndLine, getExecType, getFederatedOutput, getFilename, getID, getInputs, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getLevel, getOutputParameters, getOutputs, getPrivacyConstraint, getProducesIntermediateOutput, getReachable, getSimpleInstructionType, getType, getValueType, getVisited, isAllOutputsCP, isDataExecLocation, isVariable, prefetchActivated, prepInputOperand, prepInputOperand, prepOperand, prepOperand, prepOutputOperand, prepOutputOperand, prepOutputOperand, prepScalarInputOperand, prepScalarInputOperand, prepScalarLabel, prepScalarOperand, printErrorLocation, removeConsumer, removeOutput, replaceInput, resetVisitStatus, setAllPositions, setBeginColumn, setBeginLine, setConsumerCount, setDataType, setEndColumn, setEndLine, setFederatedOutput, setFilename, setPrivacyConstraint, setValueType, setVisited
-
-
-
-
Field Detail
-
PREAD_PREFIX
public static final String PREAD_PREFIX
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
Data
public Data(Types.OpOpData op, Lop input, HashMap<String,Lop> inputParametersLops, String name, String literal, Types.DataType dt, Types.ValueType vt, Types.FileFormat fmt)
Constructor to setup read or write LOP In case of write:inputmust be provided. This will always be added as the first element ininputarray. For literals: this function is invoked through the static methodcreateLiteralLop.- Parameters:
op- operation typeinput- low-level operatorinputParametersLops- input lopsname- string nameliteral- string literaldt- data typevt- value typefmt- file format
-
-
Method Detail
-
createLiteralLop
public static Data createLiteralLop(Types.ValueType vt, String literalValue)
Method to create literal LOPs.- Parameters:
vt- value typeliteralValue- literal value- Returns:
- literal low-level operator
-
setExecType
public void setExecType(Types.ExecType et)
Data-Lop-specific method to set the execution type for persistent write. TODO: split lops into MR/CP lop.- Overrides:
setExecTypein classLop- Parameters:
et- execution type
-
getFileFormatType
public Types.FileFormat getFileFormatType()
method to get format type for input, output files.- Returns:
- file format
-
toString
public String toString()
Description copied from class:LopMethod to have Lops print their state. This is for debugging purposes.
-
getOperationType
public Types.OpOpData getOperationType()
method to get operation type, i.e. read/write.- Returns:
- operation type
-
getInputParams
public HashMap<String,Lop> getInputParams()
method to get inputParams- Returns:
- input parameters
-
isLiteral
public boolean isLiteral()
method to check if this data lop represents a literal.- Returns:
- true if data lop is a literal
-
getBooleanValue
public boolean getBooleanValue()
-
getDoubleValue
public double getDoubleValue()
-
getLongValue
public long getLongValue()
-
getStringValue
public String getStringValue()
-
isPersistentWrite
public boolean isPersistentWrite()
-
isPersistentRead
public boolean isPersistentRead()
-
getInstructions
public String getInstructions(String input1, String input2)
Method to get CP instructions for reading/writing scalars and matrices from/to HDFS. This method generates CP read/write instructions.- Overrides:
getInstructionsin classLop- Parameters:
input1- input 1input2- output- Returns:
- instructions as string
-
getInstructions
public String getInstructions()
Method to generate createvar instruction that updates symbol table with metadata, hdfsfile name, etc.- Overrides:
getInstructionsin classLop- Returns:
- instructions as string
-
getInstructions
public String getInstructions(String outputFileName)
Description copied from class:LopMethod should be overridden if needed- Overrides:
getInstructionsin classLop- Parameters:
outputFileName- output- Returns:
- instructions as string
-
-