MasterFileSystem (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.master.MasterFileSystem

```
@InterfaceAudience.Private
public class MasterFileSystem
extends Object
```
This class abstracts a bunch of operations the HMaster needs to interact with the underlying file system, including splitting log files, checking file system status, etc.

Field Summary

Fields
Modifier and Type	Field and Description
`private ClusterId`	`clusterId`
`(package private) org.apache.hadoop.conf.Configuration`	`conf`
`(package private) boolean`	`distributedLogReplay`
`private org.apache.hadoop.fs.FileSystem`	`fs`
`private boolean`	`fsOk`
`private static org.apache.commons.logging.Log`	`LOG`
`(package private) Server`	`master`
`(package private) static org.apache.hadoop.fs.PathFilter`	`META_FILTER`
`private MetricsMasterFileSystem`	`metricsMasterFilesystem`
`(package private) static org.apache.hadoop.fs.PathFilter`	`NON_META_FILTER`
`private org.apache.hadoop.fs.Path`	`oldLogDir`
`private org.apache.hadoop.fs.Path`	`rootdir`
`private MasterServices`	`services`
`(package private) Lock`	`splitLogLock`
`(package private) SplitLogManager`	`splitLogManager`
`private org.apache.hadoop.fs.Path`	`tempdir`

Constructor Summary

Constructors
Constructor and Description

MasterFileSystem(Server master, MasterServices services)

Constructors
Constructor and Description
`MasterFileSystem(Server master, MasterServices services)`

Method Summary

Methods
Modifier and Type	Method and Description
`HTableDescriptor`	`addColumn(TableName tableName, HColumnDescriptor hcd)` Add column to a table
`private static void`	`bootstrap(org.apache.hadoop.fs.Path rd, org.apache.hadoop.conf.Configuration c)`
`boolean`	`checkFileSystem()` Checks to see if the file system is still accessible.
`private org.apache.hadoop.fs.Path`	`checkRootDir(org.apache.hadoop.fs.Path rd, org.apache.hadoop.conf.Configuration c, org.apache.hadoop.fs.FileSystem fs)` Get the rootdir.
`private void`	`checkTempDir(org.apache.hadoop.fs.Path tmpdir, org.apache.hadoop.conf.Configuration c, org.apache.hadoop.fs.FileSystem fs)` Make sure the hbase temp directory exists and is empty.
`private org.apache.hadoop.fs.Path`	`createInitialFileSystemLayout()` Create initial layout in filesystem.
`HTableDescriptor`	`deleteColumn(TableName tableName, byte[] familyName)` Delete column of a table
`void`	`deleteFamilyFromFS(HRegionInfo region, byte[] familyName)`
`void`	`deleteRegion(HRegionInfo region)`
`void`	`deleteTable(TableName tableName)`
`ClusterId`	`getClusterId()`
`(package private) Set<ServerName>`	`getFailedServersFromLogFolders()` Inspect the log directory to find dead servers which need recovery work
`org.apache.hadoop.fs.FileSystem`	`getFileSystem()`
`private List<org.apache.hadoop.fs.Path>`	`getLogDirs(Set<ServerName> serverNames)`
`org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode`	`getLogRecoveryMode()`
`org.apache.hadoop.fs.Path`	`getOldLogDir()` Get the directory where old logs go
`org.apache.hadoop.fs.Path`	`getRootDir()`
`(package private) SplitLogManager`	`getSplitLogManager()`
`org.apache.hadoop.fs.Path`	`getTempDir()`
`void`	`logFileSystemState(org.apache.commons.logging.Log log)`
`HTableDescriptor`	`modifyColumn(TableName tableName, HColumnDescriptor hcd)` Modify Column of a table
`org.apache.hadoop.fs.Path`	`moveTableToTemp(TableName tableName)` Move the specified table to the hbase temp directory
`void`	`prepareLogReplay(ServerName serverName, Set<HRegionInfo> regions)` Mark regions in recovering state when distributedLogReplay are set true
`(package private) void`	`removeStaleRecoveringRegionsFromZK(Set<ServerName> failedServers)` Wrapper function on `SplitLogManager.removeStaleRecoveringRegions(Set)`
`static void`	`setInfoFamilyCachingForMeta(HTableDescriptor metaDescriptor, boolean b)` Enable in memory caching for hbase:meta
`void`	`setLogRecoveryMode()` The function is used in SSH to set recovery mode based on configuration after all outstanding log split tasks drained.
`void`	`splitLog(ServerName serverName)`
`void`	`splitLog(Set<ServerName> serverNames)`
`void`	`splitLog(Set<ServerName> serverNames, org.apache.hadoop.fs.PathFilter filter)` This method is the base split method that splits WAL files matching a filter.
`void`	`splitMetaLog(ServerName serverName)` Specialized method to handle the splitting for meta WAL
`void`	`splitMetaLog(Set<ServerName> serverNames)` Specialized method to handle the splitting for meta WAL
`void`	`stop()`
`void`	`updateRegionInfo(HRegionInfo region)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.apache.commons.logging.Log LOG

conf

org.apache.hadoop.conf.Configuration conf

master
```
Server master
```

metricsMasterFilesystem

private final MetricsMasterFileSystem metricsMasterFilesystem

clusterId
```
private ClusterId clusterId
```

fs

private final org.apache.hadoop.fs.FileSystem fs

fsOk
```
private volatile boolean fsOk
```

oldLogDir

private final org.apache.hadoop.fs.Path oldLogDir

rootdir

private final org.apache.hadoop.fs.Path rootdir

tempdir

private final org.apache.hadoop.fs.Path tempdir

splitLogLock
```
final Lock splitLogLock
```

distributedLogReplay
```
final boolean distributedLogReplay
```

splitLogManager
```
final SplitLogManager splitLogManager
```

services
```
private final MasterServices services
```

META_FILTER

static final org.apache.hadoop.fs.PathFilter META_FILTER

NON_META_FILTER

static final org.apache.hadoop.fs.PathFilter NON_META_FILTER

Constructor Detail

MasterFileSystem

public MasterFileSystem(Server master,
                MasterServices services)
                 throws IOException

Throws:: IOException

Method Detail

getSplitLogManager
```
SplitLogManager getSplitLogManager()
```

createInitialFileSystemLayout
```
private org.apache.hadoop.fs.Path createInitialFileSystemLayout()
                                                         throws IOException
```
Create initial layout in filesystem.
1. Check if the meta region exists and is readable, if not create it. Create hbase.version and the hbase:meta directory if not one.
2. Create a log archive directory for RS to put archived logs
Idempotent.
Throws:

IOException

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

getOldLogDir
```
public org.apache.hadoop.fs.Path getOldLogDir()
```
Get the directory where old logs go

Returns:
the dir

checkFileSystem
```
public boolean checkFileSystem()
```
Checks to see if the file system is still accessible. If not, sets closed

Returns:
false if file system is not available

getRootDir

public org.apache.hadoop.fs.Path getRootDir()

Returns:: HBase root dir.

getTempDir

public org.apache.hadoop.fs.Path getTempDir()

Returns:: HBase temp dir.

getClusterId
```
public ClusterId getClusterId()
```
Returns:
The unique identifier generated for this cluster

getFailedServersFromLogFolders
```
Set<ServerName> getFailedServersFromLogFolders()
```
Inspect the log directory to find dead servers which need recovery work

Returns:
A set of ServerNames which aren't running but still have WAL files left in file system

splitLog

public void splitLog(ServerName serverName)
              throws IOException

Throws:: IOException

splitMetaLog
```
public void splitMetaLog(ServerName serverName)
                  throws IOException
```
Specialized method to handle the splitting for meta WAL

Parameters:
serverName -

Throws:

IOException

splitMetaLog
```
public void splitMetaLog(Set<ServerName> serverNames)
                  throws IOException
```
Specialized method to handle the splitting for meta WAL

Parameters:
serverNames -

Throws:

IOException

getLogDirs

private List<org.apache.hadoop.fs.Path> getLogDirs(Set<ServerName> serverNames)
                                            throws IOException

Throws:: IOException

prepareLogReplay
```
public void prepareLogReplay(ServerName serverName,
                    Set<HRegionInfo> regions)
                      throws IOException
```
Mark regions in recovering state when distributedLogReplay are set true

Parameters:
serverName - Failed region server whose wals to be replayed
regions - Set of regions to be recovered

Throws:

IOException

splitLog

public void splitLog(Set<ServerName> serverNames)
              throws IOException

Throws:: IOException

removeStaleRecoveringRegionsFromZK

void removeStaleRecoveringRegionsFromZK(Set<ServerName> failedServers)
                                  throws IOException,
                                         InterruptedIOException

Wrapper function on SplitLogManager.removeStaleRecoveringRegions(Set)

Parameters:: failedServers -
Throws:: IOException; InterruptedIOException

splitLog
```
public void splitLog(Set<ServerName> serverNames,
            org.apache.hadoop.fs.PathFilter filter)
              throws IOException
```
This method is the base split method that splits WAL files matching a filter. Callers should pass the appropriate filter for meta and non-meta WALs.

Parameters:
serverNames - logs belonging to these servers will be split; this will rename the log directory out from under a soft-failed server
filter -

Throws:

IOException

checkRootDir

private org.apache.hadoop.fs.Path checkRootDir(org.apache.hadoop.fs.Path rd,
                                     org.apache.hadoop.conf.Configuration c,
                                     org.apache.hadoop.fs.FileSystem fs)
                                        throws IOException

Get the rootdir. Make sure its wholesome and exists before returning.

Parameters:: rd -; c -; fs -
Returns:: hbase.rootdir (after checks for existence and bootstrapping if needed populating the directory with necessary bootup files).
Throws:: IOException

checkTempDir

private void checkTempDir(org.apache.hadoop.fs.Path tmpdir,
                org.apache.hadoop.conf.Configuration c,
                org.apache.hadoop.fs.FileSystem fs)
                   throws IOException

Make sure the hbase temp directory exists and is empty. NOTE that this method is only executed once just after the master becomes the active one.

Throws:: IOException

bootstrap

private static void bootstrap(org.apache.hadoop.fs.Path rd,
             org.apache.hadoop.conf.Configuration c)
                       throws IOException

Throws:: IOException

setInfoFamilyCachingForMeta

public static void setInfoFamilyCachingForMeta(HTableDescriptor metaDescriptor,
                               boolean b)

Enable in memory caching for hbase:meta

deleteRegion

public void deleteRegion(HRegionInfo region)
                  throws IOException

Throws:: IOException

deleteTable

public void deleteTable(TableName tableName)
                 throws IOException

Throws:: IOException

moveTableToTemp
```
public org.apache.hadoop.fs.Path moveTableToTemp(TableName tableName)
                                          throws IOException
```
Move the specified table to the hbase temp directory

Parameters:
tableName - Table name to move

Returns:
The temp location of the table moved

Throws:

IOException - in case of file-system failure

updateRegionInfo

public void updateRegionInfo(HRegionInfo region)

deleteFamilyFromFS

public void deleteFamilyFromFS(HRegionInfo region,
                      byte[] familyName)
                        throws IOException

Throws:: IOException

stop
```
public void stop()
```

deleteColumn

public HTableDescriptor deleteColumn(TableName tableName,
                            byte[] familyName)
                              throws IOException

Delete column of a table

Parameters:: tableName -; familyName -
Returns:: Modified HTableDescriptor with requested column deleted.
Throws:: IOException

modifyColumn

public HTableDescriptor modifyColumn(TableName tableName,
                            HColumnDescriptor hcd)
                              throws IOException

Modify Column of a table

Parameters:: tableName -; hcd - HColumnDesciptor
Returns:: Modified HTableDescriptor with the column modified.
Throws:: IOException

addColumn

public HTableDescriptor addColumn(TableName tableName,
                         HColumnDescriptor hcd)
                           throws IOException

Add column to a table

Parameters:: tableName -; hcd -
Returns:: Modified HTableDescriptor with new column added.
Throws:: IOException

setLogRecoveryMode
```
public void setLogRecoveryMode()
                        throws IOException
```
The function is used in SSH to set recovery mode based on configuration after all outstanding log split tasks drained.

Throws:

IOException

getLogRecoveryMode

public org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode getLogRecoveryMode()

logFileSystemState

public void logFileSystemState(org.apache.commons.logging.Log log)
                        throws IOException

Throws:: IOException

Class MasterFileSystem

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

conf

master

metricsMasterFilesystem

clusterId

fs

fsOk

oldLogDir

rootdir

tempdir

splitLogLock

distributedLogReplay

splitLogManager

services

META_FILTER

NON_META_FILTER

Constructor Detail

MasterFileSystem

Method Detail

getSplitLogManager

createInitialFileSystemLayout

getFileSystem

getOldLogDir

checkFileSystem

getRootDir

getTempDir

getClusterId

getFailedServersFromLogFolders

splitLog

splitMetaLog

splitMetaLog

getLogDirs

prepareLogReplay

splitLog

removeStaleRecoveringRegionsFromZK

splitLog

checkRootDir

checkTempDir

bootstrap

setInfoFamilyCachingForMeta

deleteRegion

deleteTable

moveTableToTemp

updateRegionInfo

deleteFamilyFromFS

stop

deleteColumn

modifyColumn

addColumn

setLogRecoveryMode

getLogRecoveryMode

logFileSystemState