public class Utils
extends java.lang.Object
Constructor and Description |
---|
Utils() |
Modifier and Type | Method and Description |
---|---|
static FuncSpec |
buildSimpleFuncSpec(java.lang.String className,
byte... types) |
static boolean |
checkNullAndClass(java.lang.Object obj1,
java.lang.Object obj2)
This method is a helper for classes to implement
Object.equals(java.lang.Object)
The method checks whether the two arguments are both null or both not null and
whether they are of the same class |
static boolean |
checkNullEquals(java.lang.Object obj1,
java.lang.Object obj2,
boolean checkEquality)
This method is a helper for classes to implement
Object.equals(java.lang.Object)
checks if two objects are equals - two levels of checks are
made - first if both are null or not null. |
static java.io.InputStream |
getCompositeStream(java.io.InputStream in,
java.util.Properties properties) |
static ResourceSchema |
getSchema(LoadFunc wrappedLoadFunc,
java.lang.String location,
boolean checkExistence,
org.apache.hadoop.mapreduce.Job job) |
static Schema |
getSchemaFromBagSchemaString(java.lang.String schemaString)
getSchemaFromBagSchemaString
NOTE: use this call when you need to generate a Schema object
from the representation generated by
Schema#toString . |
static Schema |
getSchemaFromString(java.lang.String schemaString) |
static ResourceSchema |
getSchemaWithInputSourceTag(ResourceSchema schema,
java.lang.String fieldName)
This method adds FieldSchema of 'input source tag/path' as the first
field.
|
static Schema |
getScriptSchema(java.lang.String loadFuncSignature,
org.apache.hadoop.conf.Configuration conf)
A helper function for retrieving the script schema set by the LOLoad
function.
|
static java.lang.String |
getScriptSchemaKey(java.lang.String loadFuncSignature) |
static java.lang.String |
getStackStraceStr(java.lang.Throwable e) |
static java.lang.String |
getStringFromArray(java.lang.String[] arr) |
static java.lang.String |
getTmpFileCompressorName(PigContext pigContext) |
static java.lang.Class<? extends FileInputLoadFunc> |
getTmpFileStorageClass(java.util.Properties properties) |
static FileInputLoadFunc |
getTmpFileStorageObject(org.apache.hadoop.conf.Configuration conf) |
static boolean |
isLocal(PigContext pigContext,
org.apache.hadoop.conf.Configuration conf) |
static <O> java.util.Collection<O> |
mergeCollection(java.util.Collection<O> a,
java.util.Collection<O> b) |
static LogicalSchema |
parseSchema(java.lang.String schemaString) |
static void |
recomputeProperties(org.apache.hadoop.mapred.JobConf jobConf,
java.util.Properties properties)
Method to apply pig properties to JobConf (replaces properties with
resulting jobConf values).
|
static void |
setMapredCompressionCodecProps(org.apache.hadoop.conf.Configuration conf) |
static void |
setTmpFileCompressionOnConf(PigContext pigContext,
org.apache.hadoop.conf.Configuration conf) |
static java.lang.String |
slashisize(java.lang.String str)
Replace sequences of two slashes ("\\") with one slash ("\")
(not escaping a slash in grunt is disallowed, but a double slash doesn't get converted
into a regular slash, so we have to do it instead)
|
static java.lang.String |
substituteVars(java.lang.String expr) |
public static boolean checkNullEquals(java.lang.Object obj1, java.lang.Object obj2, boolean checkEquality)
Object.equals(java.lang.Object)
checks if two objects are equals - two levels of checks are
made - first if both are null or not null. If either is null,
check is made whether both are null.
If both are non null, equality also is checked if so indicatedobj1
- first object to be comparedobj2
- second object to be comparedcheckEquality
- flag to indicate whether object equality should
be checked if obj1 and obj2 are non-nullpublic static boolean checkNullAndClass(java.lang.Object obj1, java.lang.Object obj2)
Object.equals(java.lang.Object)
The method checks whether the two arguments are both null or both not null and
whether they are of the same classobj1
- first object to compareobj2
- second object to comparepublic static Schema getScriptSchema(java.lang.String loadFuncSignature, org.apache.hadoop.conf.Configuration conf) throws java.io.IOException
loadFuncSignature
- conf
- java.io.IOException
public static java.lang.String getScriptSchemaKey(java.lang.String loadFuncSignature)
public static ResourceSchema getSchema(LoadFunc wrappedLoadFunc, java.lang.String location, boolean checkExistence, org.apache.hadoop.mapreduce.Job job) throws java.io.IOException
java.io.IOException
public static Schema getSchemaFromString(java.lang.String schemaString) throws ParserException
schemaString
- a String representation of the Schema without
any enclosing curly-braces.Not for use with
Schema#toString
ParserException
public static Schema getSchemaFromBagSchemaString(java.lang.String schemaString) throws ParserException
Schema#toString
.
This call strips the enclosing outer curly braces from the toString
representation, which are placed there because the actual representation of
the schema data is as a Bag-type relation.schemaString
- a String representation of the Schema to instantiate,
in the form generated by Schema.toString()
ParserException
public static LogicalSchema parseSchema(java.lang.String schemaString) throws ParserException
ParserException
public static ResourceSchema getSchemaWithInputSourceTag(ResourceSchema schema, java.lang.String fieldName)
schema
- fieldName
- public static java.lang.String getTmpFileCompressorName(PigContext pigContext)
public static FileInputLoadFunc getTmpFileStorageObject(org.apache.hadoop.conf.Configuration conf) throws java.io.IOException
java.io.IOException
public static java.lang.Class<? extends FileInputLoadFunc> getTmpFileStorageClass(java.util.Properties properties)
public static void setMapredCompressionCodecProps(org.apache.hadoop.conf.Configuration conf)
public static void setTmpFileCompressionOnConf(PigContext pigContext, org.apache.hadoop.conf.Configuration conf) throws java.io.IOException
java.io.IOException
public static java.lang.String getStringFromArray(java.lang.String[] arr)
public static FuncSpec buildSimpleFuncSpec(java.lang.String className, byte... types)
public static java.lang.String slashisize(java.lang.String str)
str
- public static <O> java.util.Collection<O> mergeCollection(java.util.Collection<O> a, java.util.Collection<O> b)
public static java.io.InputStream getCompositeStream(java.io.InputStream in, java.util.Properties properties)
public static void recomputeProperties(org.apache.hadoop.mapred.JobConf jobConf, java.util.Properties properties)
conf
- JobConf with appropriate hadoop resource filesproperties
- Pig properties that will override hadoop properties;
properties might be modifiedpublic static java.lang.String getStackStraceStr(java.lang.Throwable e)
public static boolean isLocal(PigContext pigContext, org.apache.hadoop.conf.Configuration conf)
public static java.lang.String substituteVars(java.lang.String expr)
Copyright © 2007-2012 The Apache Software Foundation