Package org.apache.sysds.runtime.util
Class LocalFileUtils
- java.lang.Object
-
- org.apache.sysds.runtime.util.LocalFileUtils
-
public class LocalFileUtils extends Object
-
-
Field Summary
Fields Modifier and Type Field Description static intBUFFER_SIZEstatic StringCATEGORY_CACHEstatic StringCATEGORY_CODEGENstatic StringCATEGORY_LINEAGEstatic StringCATEGORY_PARTITIONINGstatic StringCATEGORY_RESULTMERGEstatic StringCATEGORY_WORK
-
Constructor Summary
Constructors Constructor Description LocalFileUtils()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static StringcheckAndCreateStagingDir(String dir)static intcleanupRcWorkingDirectory(String dir)static voidcleanupWorkingDirectory()static voidcleanupWorkingDirectory(String dir)static booleancreateLocalFileIfNotExist(String dir)static booleancreateLocalFileIfNotExist(String dir, String permission)static StringcreateWorkingDirectory()static StringcreateWorkingDirectoryWithUUID(String uuid)static voiddeleteFileIfExists(String dir)static voiddeleteFileIfExists(String dir, boolean fileOnly)static StringgetUniqueWorkingDir(String category)static StringgetWorkingDir(String category)static booleanisExisting(String dir)static intrcDelete(File dir)Recursively deletes an entire local file system directory and returns the number of files deleted.static voidrDelete(File dir)Recursively deletes an entire local file system directory.static intreadBlockSequenceFromLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] outValues, HashMap<MatrixIndexes,Integer> outMap)static CacheBlockreadCacheBlockFromLocal(String fname, boolean matrix)Reads a matrix/frame block from local file system.static FrameBlockreadFrameBlockFromLocal(String fname)Reads a frame block from local file system.static FrameBlockreadFrameBlockFromStream(InputStream is)Reads a frame block from an input stream, using a fused buffered reader with special support for matrix blocks.static MatrixBlockreadMatrixBlockFromLocal(String fname)Reads a matrix block from local file system.static MatrixBlockreadMatrixBlockFromLocal(String fname, MatrixBlock reuse)Reads a matrix block from local file system.static MatrixBlockreadMatrixBlockFromStream(InputStream is)Reads a matrix block from an input stream, using a fused buffered reader with special support for matrix blocks.static org.apache.hadoop.io.WritablereadWritableFromLocal(String fname, org.apache.hadoop.io.Writable ret)Reads an arbitrary writable from local file system, using a fused buffered reader with special support for matrix blocks.static org.apache.hadoop.io.WritablereadWritableFromStream(InputStream is, org.apache.hadoop.io.Writable ret)Reads an arbitrary writable from an input stream, using a fused buffered reader with special support for matrix blocks.static voidsetLocalFilePermissions(File file, String permissions)static voidwriteBlockSequenceToLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] inValues, int len)static voidwriteByteArrayToLocal(String fname, byte[] data)static voidwriteCacheBlockToLocal(String fname, CacheBlock cb)Writes a matrix/frame block to local file system.static voidwriteFrameBlockToLocal(String fname, FrameBlock fb)Writes a frame block to local file system.static voidwriteMatrixBlockToLocal(String fname, MatrixBlock mb)Writes a matrix block to local file system.static voidwriteTextFile(File file, String text)Writes a simple text file to local file system.static voidwriteWritableToLocal(String fname, org.apache.hadoop.io.Writable mb)Writes an arbitrary writable to local file system, using a fused buffered writer with special support for matrix blocks.
-
-
-
Field Detail
-
BUFFER_SIZE
public static final int BUFFER_SIZE
- See Also:
- Constant Field Values
-
CATEGORY_CACHE
public static final String CATEGORY_CACHE
- See Also:
- Constant Field Values
-
CATEGORY_PARTITIONING
public static final String CATEGORY_PARTITIONING
- See Also:
- Constant Field Values
-
CATEGORY_RESULTMERGE
public static final String CATEGORY_RESULTMERGE
- See Also:
- Constant Field Values
-
CATEGORY_WORK
public static final String CATEGORY_WORK
- See Also:
- Constant Field Values
-
CATEGORY_CODEGEN
public static final String CATEGORY_CODEGEN
- See Also:
- Constant Field Values
-
CATEGORY_LINEAGE
public static final String CATEGORY_LINEAGE
- See Also:
- Constant Field Values
-
-
Method Detail
-
readMatrixBlockFromLocal
public static MatrixBlock readMatrixBlockFromLocal(String fname) throws IOException
Reads a matrix block from local file system.- Parameters:
fname- file name to read- Returns:
- matrix block
- Throws:
IOException- if IOException occurs
-
readMatrixBlockFromLocal
public static MatrixBlock readMatrixBlockFromLocal(String fname, MatrixBlock reuse) throws IOException
Reads a matrix block from local file system.- Parameters:
fname- file name to readreuse- matrix block to reuse- Returns:
- matrix block
- Throws:
IOException- if IOException occurs
-
readFrameBlockFromLocal
public static FrameBlock readFrameBlockFromLocal(String fname) throws IOException
Reads a frame block from local file system.- Parameters:
fname- file name to read- Returns:
- frame block
- Throws:
IOException- if IOException occurs
-
readCacheBlockFromLocal
public static CacheBlock readCacheBlockFromLocal(String fname, boolean matrix) throws IOException
Reads a matrix/frame block from local file system.- Parameters:
fname- file name to readmatrix- if true, read matrix. if false, read frame.- Returns:
- cache block (common interface to MatrixBlock and FrameBlock)
- Throws:
IOException- if IOException occurs
-
readWritableFromLocal
public static org.apache.hadoop.io.Writable readWritableFromLocal(String fname, org.apache.hadoop.io.Writable ret) throws IOException
Reads an arbitrary writable from local file system, using a fused buffered reader with special support for matrix blocks.- Parameters:
fname- file name to readret- hadoop writable- Returns:
- hadoop writable
- Throws:
IOException- if IOException occurs
-
readMatrixBlockFromStream
public static MatrixBlock readMatrixBlockFromStream(InputStream is) throws IOException
Reads a matrix block from an input stream, using a fused buffered reader with special support for matrix blocks.- Parameters:
is- input stream to read- Returns:
- matrix block
- Throws:
IOException- if IOException occurs
-
readFrameBlockFromStream
public static FrameBlock readFrameBlockFromStream(InputStream is) throws IOException
Reads a frame block from an input stream, using a fused buffered reader with special support for matrix blocks.- Parameters:
is- input stream to read- Returns:
- frame block
- Throws:
IOException- if IOException occurs
-
readWritableFromStream
public static org.apache.hadoop.io.Writable readWritableFromStream(InputStream is, org.apache.hadoop.io.Writable ret) throws IOException
Reads an arbitrary writable from an input stream, using a fused buffered reader with special support for matrix blocks.- Parameters:
is- input stream to readret- hadoop writable- Returns:
- hadoop writable
- Throws:
IOException- if IOException occurs
-
writeMatrixBlockToLocal
public static void writeMatrixBlockToLocal(String fname, MatrixBlock mb) throws IOException
Writes a matrix block to local file system.- Parameters:
fname- file name to writemb- matrix block- Throws:
IOException- if IOException occurs
-
writeFrameBlockToLocal
public static void writeFrameBlockToLocal(String fname, FrameBlock fb) throws IOException
Writes a frame block to local file system.- Parameters:
fname- file name to writefb- fame block- Throws:
IOException- if IOException occurs
-
writeCacheBlockToLocal
public static void writeCacheBlockToLocal(String fname, CacheBlock cb) throws IOException
Writes a matrix/frame block to local file system.- Parameters:
fname- file name to writecb- cache block (common interface to matrix block and frame block)- Throws:
IOException- if IOException occurs
-
writeWritableToLocal
public static void writeWritableToLocal(String fname, org.apache.hadoop.io.Writable mb) throws IOException
Writes an arbitrary writable to local file system, using a fused buffered writer with special support for matrix blocks.- Parameters:
fname- file name to writemb- Hadoop writable- Throws:
IOException- if IOException occurs
-
writeByteArrayToLocal
public static void writeByteArrayToLocal(String fname, byte[] data) throws IOException
- Throws:
IOException
-
readBlockSequenceFromLocal
public static int readBlockSequenceFromLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] outValues, HashMap<MatrixIndexes,Integer> outMap) throws IOException
- Throws:
IOException
-
writeBlockSequenceToLocal
public static void writeBlockSequenceToLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] inValues, int len) throws IOException
- Throws:
IOException
-
createLocalFileIfNotExist
public static boolean createLocalFileIfNotExist(String dir)
-
deleteFileIfExists
public static void deleteFileIfExists(String dir)
-
deleteFileIfExists
public static void deleteFileIfExists(String dir, boolean fileOnly)
-
isExisting
public static boolean isExisting(String dir)
-
createLocalFileIfNotExist
public static boolean createLocalFileIfNotExist(String dir, String permission)
-
createWorkingDirectory
public static String createWorkingDirectory()
-
cleanupWorkingDirectory
public static void cleanupWorkingDirectory()
-
cleanupWorkingDirectory
public static void cleanupWorkingDirectory(String dir)
-
cleanupRcWorkingDirectory
public static int cleanupRcWorkingDirectory(String dir)
-
rDelete
public static void rDelete(File dir)
Recursively deletes an entire local file system directory.- Parameters:
dir- directory to delete recursively
-
rcDelete
public static int rcDelete(File dir)
Recursively deletes an entire local file system directory and returns the number of files deleted.- Parameters:
dir- directory to delete recursively- Returns:
- number of files deleted
-
writeTextFile
public static void writeTextFile(File file, String text) throws IOException
Writes a simple text file to local file system.- Parameters:
file- output filetext- content of text file- Throws:
IOException- errors in file open/write/close
-
-