When repairing a corrupted HBase, it is best to repair the lowest risk inconsistencies first. These are generally region consistency repairs -- localized single region repairs, that only modify in-memory data, ephemeral zookeeper data, or patch holes in the META table. Region consistency requires that the HBase instance has the state of the region’s data in HDFS (.regioninfo files), the region’s row in the .META. table., and region’s deployment/assignments on region servers and the master in accordance. Options for repairing region consistency include:
-fixAssignments(equivalent to the 0.90
-fixoption) repairs unassigned, incorrectly assigned or multiply assigned regions.
-fixMetawhich removes meta rows when corresponding regions are not present in HDFS and adds new meta rows if they regions are present in HDFS while not in META.
To fix deployment and assignment problems you can run this command:
$ ./bin/hbase hbck -fixAssignmentsTo fix deployment and assignment problems as well as repairing incorrect meta rows you can run this command:.
$ ./bin/hbase hbck -fixAssignments -fixMetaThere are a few classes of table integrity problems that are low risk repairs. The first two are degenerate (startkey == endkey) regions and backwards regions (startkey > endkey). These are automatically handled by sidelining the data to a temporary directory (/hbck/xxxx). The third low-risk class is hdfs region holes. This can be repaired by using the:
-fixHdfsHolesoption for fabricating new empty regions on the file system. If holes are detected you can use -fixHdfsHoles and should include -fixMeta and -fixAssignments to make the new region consistent.
$ ./bin/hbase hbck -fixAssignments -fixMeta -fixHdfsHolesSince this is a common operation, we’ve added a the
-repairHolesflag that is equivalent to the previous command:
$ ./bin/hbase hbck -repairHolesIf inconsistencies still remain after these steps, you most likely have table integrity problems related to orphaned or overlapping regions.