org.apache.hadoop.hbase.mapreduce

Class ImportTsv

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.mapreduce.ImportTsv

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@InterfaceAudience.Public
public class ImportTsv
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Tool to import data from a TSV file. This tool is rather simplistic - it doesn't do any quoting or escaping, but is useful for many data loads.

See Also:

usage(String)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ImportTsv.TsvParser

Field Summary

Fields

Modifier and Type	Field and Description
static String	ATTRIBUTE_SEPERATOR_CONF_KEY
static String	BULK_OUTPUT_CONF_KEY
static String	COLUMNS_CONF_KEY
static String	CREATE_TABLE_CONF_KEY
static String	CREDENTIALS_LOCATION
(package private) static String	DEFAULT_ATTRIBUTES_SEPERATOR
(package private) static Class	DEFAULT_MAPPER
(package private) static String	DEFAULT_MULTIPLE_ATTRIBUTES_SEPERATOR
(package private) static String	DEFAULT_SEPARATOR
static String	DRY_RUN_CONF_KEY
private static boolean	DRY_RUN_TABLE_CREATED If table didn't exist and was created in dry-run mode, this flag is flipped to delete it when MR ends.
static String	JOB_NAME_CONF_KEY
protected static org.slf4j.Logger	LOG
static String	LOG_BAD_LINES_CONF_KEY
static String	MAPPER_CONF_KEY
(package private) static String	NAME
static String	NO_STRICT_COL_FAMILY
static String	SEPARATOR_CONF_KEY
static String	SKIP_EMPTY_COLUMNS
static String	SKIP_LINES_CONF_KEY
static String	TIMESTAMP_CONF_KEY

Constructor Summary

Constructors

Constructor and Description
ImportTsv()

Method Summary

Modifier and Type	Method and Description
protected static org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args) Sets up the actual job.
private static void	createTable(Admin admin, TableName tableName, String[] columns)
private static void	deleteTable(org.apache.hadoop.conf.Configuration conf, String[] args)
private static Set<String>	getColumnFamilies(String[] columns)
static void	main(String[] args)
int	run(String[] args)
private static void	usage(String errorMsg)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

LOG

protected static final org.slf4j.Logger LOG

NAME

static final String NAME

See Also:

Constant Field Values

MAPPER_CONF_KEY

public static final String MAPPER_CONF_KEY

See Also:

Constant Field Values

BULK_OUTPUT_CONF_KEY

public static final String BULK_OUTPUT_CONF_KEY

See Also:

Constant Field Values

TIMESTAMP_CONF_KEY

public static final String TIMESTAMP_CONF_KEY

See Also:

Constant Field Values

JOB_NAME_CONF_KEY

public static final String JOB_NAME_CONF_KEY

See Also:

Constant Field Values

DRY_RUN_CONF_KEY

public static final String DRY_RUN_CONF_KEY

See Also:

Constant Field Values

LOG_BAD_LINES_CONF_KEY

public static final String LOG_BAD_LINES_CONF_KEY

See Also:

Constant Field Values

SKIP_LINES_CONF_KEY

public static final String SKIP_LINES_CONF_KEY

See Also:

Constant Field Values

SKIP_EMPTY_COLUMNS

public static final String SKIP_EMPTY_COLUMNS

See Also:

Constant Field Values

COLUMNS_CONF_KEY

public static final String COLUMNS_CONF_KEY

See Also:

Constant Field Values

SEPARATOR_CONF_KEY

public static final String SEPARATOR_CONF_KEY

See Also:

Constant Field Values

ATTRIBUTE_SEPERATOR_CONF_KEY

public static final String ATTRIBUTE_SEPERATOR_CONF_KEY

See Also:

Constant Field Values

CREDENTIALS_LOCATION

public static final String CREDENTIALS_LOCATION

See Also:

Constant Field Values

DEFAULT_SEPARATOR

static final String DEFAULT_SEPARATOR

See Also:

Constant Field Values

DEFAULT_ATTRIBUTES_SEPERATOR

static final String DEFAULT_ATTRIBUTES_SEPERATOR

See Also:

Constant Field Values

DEFAULT_MULTIPLE_ATTRIBUTES_SEPERATOR

static final String DEFAULT_MULTIPLE_ATTRIBUTES_SEPERATOR

See Also:

Constant Field Values

DEFAULT_MAPPER

static final Class DEFAULT_MAPPER

CREATE_TABLE_CONF_KEY

public static final String CREATE_TABLE_CONF_KEY

See Also:

Constant Field Values

NO_STRICT_COL_FAMILY

public static final String NO_STRICT_COL_FAMILY

See Also:

Constant Field Values

DRY_RUN_TABLE_CREATED

private static boolean DRY_RUN_TABLE_CREATED

If table didn't exist and was created in dry-run mode, this flag is flipped to delete it when MR ends.

Constructor Detail

ImportTsv

public ImportTsv()

Method Detail

createSubmittableJob

protected static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
                                                                      String[] args)
                                                               throws IOException,
                                                                      ClassNotFoundException

Sets up the actual job.

Parameters:

conf - The current configuration.

args - The command line parameters.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

ClassNotFoundException

createTable

private static void createTable(Admin admin,
                                TableName tableName,
                                String[] columns)
                         throws IOException

Throws:

IOException

deleteTable

private static void deleteTable(org.apache.hadoop.conf.Configuration conf,
                                String[] args)

getColumnFamilies

private static Set<String> getColumnFamilies(String[] columns)

usage

private static void usage(String errorMsg)

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception