SecureBulkLoadEndpoint (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService
- - org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint

All Implemented Interfaces:

com.google.protobuf.Service, Coprocessor, CoprocessorService
```
@InterfaceAudience.Private
public class SecureBulkLoadEndpoint
extends org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService
implements CoprocessorService, Coprocessor
```
Coprocessor service for bulk loads in secure mode. This coprocessor has to be installed as part of enabling security in HBase. This service addresses two issues: 1. Moving files in a secure filesystem wherein the HBase Client and HBase Server are different filesystem users. 2. Does moving in a secure manner. Assuming that the filesystem is POSIX compliant. The algorithm is as follows: 1. Create an hbase owned staging directory which is world traversable (711): /hbase/staging 2. A user writes out data to his secure output directory: /user/foo/data 3. A call is made to hbase to create a secret staging directory which globally rwx (777): /user/staging/averylongandrandomdirectoryname 4. The user moves the data into the random staging directory, then calls bulkLoadHFiles() Like delegation tokens the strength of the security lies in the length and randomness of the secret directory.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`(package private) static interface`	`SecureBulkLoadEndpoint.Consumer<T>`
`private static class`	`SecureBulkLoadEndpoint.SecureBulkLoadListener`

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService
org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService.BlockingInterface, org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService.Interface, org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService.Stub

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor
Coprocessor.State

Field Summary

Fields
Modifier and Type	Field and Description
`private org.apache.hadoop.fs.Path`	`baseStagingDir`
`private org.apache.hadoop.conf.Configuration`	`conf`
`private RegionCoprocessorEnvironment`	`env`
`private org.apache.hadoop.fs.FileSystem`	`fs`
`static String`	`FS_WITHOUT_SUPPORT_PERMISSION_DEFAULT`
`static String`	`FS_WITHOUT_SUPPORT_PERMISSION_KEY`
`private static SecureBulkLoadEndpoint.Consumer<Region>`	`fsCreatedListener`
`private static org.apache.commons.logging.Log`	`LOG`
`private static org.apache.hadoop.fs.permission.FsPermission`	`PERM_ALL_ACCESS`
`private static org.apache.hadoop.fs.permission.FsPermission`	`PERM_HIDDEN`
`private SecureRandom`	`random`
`private static int`	`RANDOM_RADIX`
`private static int`	`RANDOM_WIDTH`
`private static HashMap<org.apache.hadoop.security.UserGroupInformation,org.apache.commons.lang.mutable.MutableInt>`	`ugiReferenceCounter`
`private UserProvider`	`userProvider`
`static long`	`VERSION`

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor
PRIORITY_HIGHEST, PRIORITY_LOWEST, PRIORITY_SYSTEM, PRIORITY_USER

Constructor Summary

Constructors
Constructor and Description

SecureBulkLoadEndpoint()

Constructors
Constructor and Description
`SecureBulkLoadEndpoint()`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`cleanupBulkLoad(com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.CleanupBulkLoadRequest request, com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.CleanupBulkLoadResponse> done)` `rpc CleanupBulkLoad(.CleanupBulkLoadRequest) returns (.CleanupBulkLoadResponse);`
`private org.apache.hadoop.fs.Path`	`createStagingDir(org.apache.hadoop.fs.Path baseDir, User user, String randomDir)`
`private org.apache.hadoop.fs.Path`	`createStagingDir(org.apache.hadoop.fs.Path baseDir, User user, TableName tableName)`
`private void`	`decrementUgiReference(org.apache.hadoop.security.UserGroupInformation ugi)`
`private User`	`getActiveUser()`
`private List<BulkLoadObserver>`	`getBulkLoadObservers()`
`(package private) Set<String>`	`getFileSystemSchemesWithoutPermissionSupport(org.apache.hadoop.conf.Configuration conf)`
`com.google.protobuf.Service`	`getService()`
`private void`	`incrementUgiReference(org.apache.hadoop.security.UserGroupInformation ugi)`
`private boolean`	`isUserReferenced(org.apache.hadoop.security.UserGroupInformation ugi)`
`void`	`prepareBulkLoad(com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.PrepareBulkLoadRequest request, com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.PrepareBulkLoadResponse> done)` `rpc PrepareBulkLoad(.PrepareBulkLoadRequest) returns (.PrepareBulkLoadResponse);`
`void`	`secureBulkLoadHFiles(com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadHFilesRequest request, com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadHFilesResponse> done)` `rpc SecureBulkLoadHFiles(.SecureBulkLoadHFilesRequest) returns (.SecureBulkLoadHFilesResponse);`
`(package private) static void`	`setFsCreatedListener(SecureBulkLoadEndpoint.Consumer<Region> listener)`
`void`	`start(CoprocessorEnvironment env)`
`void`	`stop(CoprocessorEnvironment env)`

Methods inherited from class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService
callMethod, getDescriptor, getDescriptorForType, getRequestPrototype, getResponsePrototype, newBlockingStub, newReflectiveBlockingService, newReflectiveService, newStub

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERSION
```
public static final long VERSION
```
See Also:
Constant Field Values

RANDOM_WIDTH
```
private static final int RANDOM_WIDTH
```
See Also:
Constant Field Values

RANDOM_RADIX
```
private static final int RANDOM_RADIX
```
See Also:
Constant Field Values

LOG

private static final org.apache.commons.logging.Log LOG

PERM_ALL_ACCESS

private static final org.apache.hadoop.fs.permission.FsPermission PERM_ALL_ACCESS

PERM_HIDDEN

private static final org.apache.hadoop.fs.permission.FsPermission PERM_HIDDEN

FS_WITHOUT_SUPPORT_PERMISSION_KEY

public static final String FS_WITHOUT_SUPPORT_PERMISSION_KEY

See Also:: Constant Field Values

FS_WITHOUT_SUPPORT_PERMISSION_DEFAULT

public static final String FS_WITHOUT_SUPPORT_PERMISSION_DEFAULT

See Also:: Constant Field Values

random
```
private SecureRandom random
```

fs

private org.apache.hadoop.fs.FileSystem fs

conf

private org.apache.hadoop.conf.Configuration conf

baseStagingDir

private org.apache.hadoop.fs.Path baseStagingDir

env

private RegionCoprocessorEnvironment env

userProvider
```
private UserProvider userProvider
```

ugiReferenceCounter

private static HashMap<org.apache.hadoop.security.UserGroupInformation,org.apache.commons.lang.mutable.MutableInt> ugiReferenceCounter

fsCreatedListener

private static SecureBulkLoadEndpoint.Consumer<Region> fsCreatedListener

Constructor Detail
- SecureBulkLoadEndpoint
```
public SecureBulkLoadEndpoint()
```

Method Detail

start

public void start(CoprocessorEnvironment env)

Specified by:: start in interface Coprocessor

getFileSystemSchemesWithoutPermissionSupport

Set<String> getFileSystemSchemesWithoutPermissionSupport(org.apache.hadoop.conf.Configuration conf)

stop

public void stop(CoprocessorEnvironment env)
          throws IOException

Specified by:: stop in interface Coprocessor
Throws:: IOException

prepareBulkLoad

public void prepareBulkLoad(com.google.protobuf.RpcController controller,
                   org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.PrepareBulkLoadRequest request,
                   com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.PrepareBulkLoadResponse> done)

Description copied from class: org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

rpc PrepareBulkLoad(.PrepareBulkLoadRequest) returns (.PrepareBulkLoadResponse);

Specified by:: prepareBulkLoad in class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

cleanupBulkLoad

public void cleanupBulkLoad(com.google.protobuf.RpcController controller,
                   org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.CleanupBulkLoadRequest request,
                   com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.CleanupBulkLoadResponse> done)

Description copied from class: org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

rpc CleanupBulkLoad(.CleanupBulkLoadRequest) returns (.CleanupBulkLoadResponse);

Specified by:: cleanupBulkLoad in class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

setFsCreatedListener

static void setFsCreatedListener(SecureBulkLoadEndpoint.Consumer<Region> listener)

incrementUgiReference

private void incrementUgiReference(org.apache.hadoop.security.UserGroupInformation ugi)

decrementUgiReference

private void decrementUgiReference(org.apache.hadoop.security.UserGroupInformation ugi)

isUserReferenced

private boolean isUserReferenced(org.apache.hadoop.security.UserGroupInformation ugi)

secureBulkLoadHFiles

public void secureBulkLoadHFiles(com.google.protobuf.RpcController controller,
                        org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadHFilesRequest request,
                        com.google.protobuf.RpcCallback<org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadHFilesResponse> done)

Description copied from class: org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

rpc SecureBulkLoadHFiles(.SecureBulkLoadHFilesRequest) returns (.SecureBulkLoadHFilesResponse);

Specified by:: secureBulkLoadHFiles in class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

getBulkLoadObservers

private List<BulkLoadObserver> getBulkLoadObservers()

createStagingDir

private org.apache.hadoop.fs.Path createStagingDir(org.apache.hadoop.fs.Path baseDir,
                                         User user,
                                         TableName tableName)
                                            throws IOException

Throws:: IOException

createStagingDir

private org.apache.hadoop.fs.Path createStagingDir(org.apache.hadoop.fs.Path baseDir,
                                         User user,
                                         String randomDir)
                                            throws IOException

Throws:: IOException

getActiveUser
```
private User getActiveUser()
```

getService
```
public com.google.protobuf.Service getService()
```
Specified by:

getService in interface CoprocessorService

Class SecureBulkLoadEndpoint

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor

Field Summary

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos.SecureBulkLoadService

Methods inherited from class java.lang.Object

Field Detail

VERSION

RANDOM_WIDTH

RANDOM_RADIX

LOG

PERM_ALL_ACCESS

PERM_HIDDEN

FS_WITHOUT_SUPPORT_PERMISSION_KEY

FS_WITHOUT_SUPPORT_PERMISSION_DEFAULT

random

fs

conf

baseStagingDir

env

userProvider

ugiReferenceCounter

fsCreatedListener

Constructor Detail

SecureBulkLoadEndpoint

Method Detail

start

getFileSystemSchemesWithoutPermissionSupport

stop

prepareBulkLoad

cleanupBulkLoad

setFsCreatedListener

incrementUgiReference

decrementUgiReference

isUserReferenced

secureBulkLoadHFiles

getBulkLoadObservers

createStagingDir

createStagingDir

getActiveUser

getService