Package | Description |
---|---|
parquet.avro |
Provides classes to store Avro data in Parquet files.
|
parquet.hadoop |
Provides classes to store use Parquet files in Hadoop
In a map reduce job:
|
parquet.hadoop.codec | |
parquet.hadoop.mapred | |
parquet.hadoop.metadata |
Constructor and Description |
---|
AvroParquetWriter(org.apache.hadoop.fs.Path file,
org.apache.avro.Schema avroSchema,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize)
Create a new
AvroParquetWriter . |
AvroParquetWriter(org.apache.hadoop.fs.Path file,
org.apache.avro.Schema avroSchema,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
boolean enableDictionary)
Create a new
AvroParquetWriter . |
AvroParquetWriter(org.apache.hadoop.fs.Path file,
org.apache.avro.Schema avroSchema,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
boolean enableDictionary,
org.apache.hadoop.conf.Configuration conf)
Create a new
AvroParquetWriter . |
Modifier and Type | Field and Description |
---|---|
static CompressionCodecName |
ParquetWriter.DEFAULT_COMPRESSION_CODEC_NAME |
Modifier and Type | Method and Description |
---|---|
static CompressionCodecName |
ParquetOutputFormat.getCompression(org.apache.hadoop.conf.Configuration configuration) |
static CompressionCodecName |
ParquetOutputFormat.getCompression(org.apache.hadoop.mapreduce.JobContext jobContext) |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.mapreduce.RecordWriter<Void,T> |
ParquetOutputFormat.getRecordWriter(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path file,
CompressionCodecName codec) |
static void |
ParquetOutputFormat.setCompression(org.apache.hadoop.mapreduce.Job job,
CompressionCodecName compression) |
void |
ParquetFileWriter.startColumn(ColumnDescriptor descriptor,
long valueCount,
CompressionCodecName compressionCodecName)
start a column inside a block
|
Constructor and Description |
---|
ParquetWriter(org.apache.hadoop.fs.Path file,
WriteSupport<T> writeSupport,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize)
Create a new ParquetWriter.
|
ParquetWriter(org.apache.hadoop.fs.Path file,
WriteSupport<T> writeSupport,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
boolean enableDictionary,
boolean validating)
Create a new ParquetWriter.
|
ParquetWriter(org.apache.hadoop.fs.Path file,
WriteSupport<T> writeSupport,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
int dictionaryPageSize,
boolean enableDictionary,
boolean validating)
Create a new ParquetWriter.
|
ParquetWriter(org.apache.hadoop.fs.Path file,
WriteSupport<T> writeSupport,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
int dictionaryPageSize,
boolean enableDictionary,
boolean validating,
ParquetProperties.WriterVersion writerVersion)
Create a new ParquetWriter.
|
ParquetWriter(org.apache.hadoop.fs.Path file,
WriteSupport<T> writeSupport,
CompressionCodecName compressionCodecName,
int blockSize,
int pageSize,
int dictionaryPageSize,
boolean enableDictionary,
boolean validating,
ParquetProperties.WriterVersion writerVersion,
org.apache.hadoop.conf.Configuration conf)
Create a new ParquetWriter.
|
Modifier and Type | Method and Description |
---|---|
CompressionCodecName |
CodecConfig.getCodec() |
static CompressionCodecName |
CodecConfig.getParquetCompressionCodec(org.apache.hadoop.conf.Configuration configuration) |
Modifier and Type | Method and Description |
---|---|
static void |
DeprecatedParquetOutputFormat.setCompression(org.apache.hadoop.conf.Configuration configuration,
CompressionCodecName compression) |
Modifier and Type | Method and Description |
---|---|
static CompressionCodecName |
CompressionCodecName.fromCompressionCodec(Class<?> clazz) |
static CompressionCodecName |
CompressionCodecName.fromConf(String name) |
static CompressionCodecName |
CompressionCodecName.fromParquet(parquet.format.CompressionCodec codec) |
CompressionCodecName |
ColumnChunkProperties.getCodec() |
CompressionCodecName |
ColumnChunkMetaData.getCodec() |
static CompressionCodecName |
CompressionCodecName.valueOf(String name)
Returns the enum constant of this type with the specified name.
|
static CompressionCodecName[] |
CompressionCodecName.values()
Returns an array containing the constants of this enum type, in
the order they are declared.
|
Modifier and Type | Method and Description |
---|---|
static ColumnChunkProperties |
ColumnChunkProperties.get(ColumnPath path,
PrimitiveType.PrimitiveTypeName type,
CompressionCodecName codec,
Set<Encoding> encodings) |
static ColumnChunkMetaData |
ColumnChunkMetaData.get(ColumnPath path,
PrimitiveType.PrimitiveTypeName type,
CompressionCodecName codec,
Set<Encoding> encodings,
long firstDataPage,
long dictionaryPageOffset,
long valueCount,
long totalSize,
long totalUncompressedSize)
Deprecated.
|
static ColumnChunkMetaData |
ColumnChunkMetaData.get(ColumnPath path,
PrimitiveType.PrimitiveTypeName type,
CompressionCodecName codec,
Set<Encoding> encodings,
Statistics statistics,
long firstDataPage,
long dictionaryPageOffset,
long valueCount,
long totalSize,
long totalUncompressedSize) |
Copyright © 2015. All rights reserved.