18.4. Building Apache HBase

18.4.1. Basic Compile

HBase is compiled using Maven. You must use Maven 3.x. To check your Maven version, run the command mvn -version.

JDK Version Requirements

Starting with HBase 1.0 you must use Java 7 or later to build from source code. See Table 2.1, “Java” for more complete information about supported JDK versions.

18.4.1.1. Maven Build Commands

All commands are executed from the local HBase project directory.

18.4.1.1.1. Package

The simplest command to compile HBase from its java source code is to use the package target, which builds JARs with the compiled files.

mvn package -DskipTests

Or, to clean up before compiling:

mvn clean package -DskipTests

With Eclipse set up as explained above in Section 18.3.1, “Eclipse”, you can also use the Build command in Eclipse. To create the full installable HBase package takes a little bit more work, so read on.

18.4.1.1.2. Compile

The compile target does not create the JARs with the compiled files.

mvn compile
mvn clean compile
18.4.1.1.3. Install

To install the JARs in your ~/.m2/ directory, use the install target.

mvn install
mvn clean install
mvn clean install -DskipTests

18.4.1.2. Running all or individual Unit Tests

See the Section 18.9.3, “Running tests” section in Section 18.9.2, “Unit Tests”

18.4.1.3. Building against various hadoop versions.

As of 0.96, Apache HBase supports building against Apache Hadoop versions: 1.0.3, 2.0.0-alpha and 3.0.0-SNAPSHOT. By default, in 0.96 and earlier, we will build with Hadoop-1.0.x. As of 0.98, Hadoop 1.x is deprecated and Hadoop 2.x is the default. To change the version to build against, add a hadoop.profile property when you invoke mvn:

mvn -Dhadoop.profile=1.0 ...

The above will build against whatever explicit hadoop 1.x version we have in our pom.xml as our '1.0' version. Tests may not all pass so you may need to pass -DskipTests unless you are inclined to fix the failing tests.

'dependencyManagement.dependencies.dependency.artifactId' for org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does not match a valid id pattern

You will see ERRORs like the above title if you pass the default profile; e.g. if you pass hadoop.profile=1.1 when building 0.96 or hadoop.profile=2.0 when building hadoop 0.98; just drop the hadoop.profile stipulation in this case to get your build to run again. This seems to be a maven pecularity that is probably fixable but we've not spent the time trying to figure it.

Similarly, for 3.0, you would just replace the profile value. Note that Hadoop-3.0.0-SNAPSHOT does not currently have a deployed maven artificat - you will need to build and install your own in your local maven repository if you want to run against this profile.

In earilier versions of Apache HBase, you can build against older versions of Apache Hadoop, notably, Hadoop 0.22.x and 0.23.x. If you are running, for example HBase-0.94 and wanted to build against Hadoop 0.23.x, you would run with:

mvn -Dhadoop.profile=22 ...

18.4.1.4. Build Protobuf

You may need to change the protobuf definitions that reside in the hbase-protocol module or other modules.

The protobuf files are located in hbase-protocol/src/main/protobuf. For the change to be effective, you will need to regenerate the classes. You can use maven profile compile-protobuf to do this.

mvn compile -Pcompile-protobuf

You may also want to define protoc.path for the protoc binary, using the following command:

mvn compile -Pcompile-protobuf -Dprotoc.path=/opt/local/bin/protoc
             

Read the hbase-protocol/README.txt for more details.

18.4.1.5. Build Thrift

You may need to change the thrift definitions that reside in the hbase-thrift module or other modules.

The thrift files are located in hbase-thrift/src/main/resources. For the change to be effective, you will need to regenerate the classes. You can use maven profile compile-thrift to do this.

mvn compile -Pcompile-thrift

You may also want to define thrift.path for the thrift binary, using the following command:

                  mvn compile -Pcompile-thrift -Dthrift.path=/opt/local/bin/thrift
                

18.4.1.6. Build a Tarball

You can build a tarball without going through the release process described in Section 18.5, “Releasing Apache HBase”, by running the following command:

mvn -DskipTests clean install && mvn -DskipTests package assembly:single

The distribution tarball is built in hbase-assembly/target/hbase-<version>-bin.tar.gz.

18.4.1.7. Build Gotchas

If you see Unable to find resource 'VM_global_library.vm', ignore it. Its not an error. It is officially ugly though.

18.4.1.8. Building in snappy compression support

Pass -Psnappy to trigger the hadoop-snappy maven profile for building Google Snappy native libraries into HBase. See also Section E.3.1.5, “Install Snappy Support”

comments powered by Disqus