Skip to main content

Accumulo 数据库环境搭建

· 3 min read
Le Dai
Le Dai
Sr Soft Engineer

Accumulo 安全权限认证K V 型数据库

官网:http://accumulo.apache.org/

文档:https://accumulo.apache.org/1.9/accumulo_user_manual.html#_mapreduce

ps:文档写的很多坑......

官方examples:https://github.com/apache/accumulo-examples/blob/master/src/main/java/org/apache/accumulo/examples/client/ReadWriteExample.java

旧的版本bug很多 并且CDH目前只支持到 1.7.2 所以打算将Accumulo 单独分离CDH 存储依然使用CDH 的HDFS

accumulo-1.9.2

下载地址：https://accumulo.apache.org/

1.解压安装

tar -zxvf *.tar.gz -C /usr/local

2.添加依赖 bin目录运行./build_native_library.sh(若提示command make not... 记得安装make c++ 依赖库) 该语句运行成功后会在lib目录下生成一个native目录，如下图：

3.修改配置(注意 默认有templates examples 实际配置文件在 conf下 所以先将配置文件拷贝 再修改) cp /usr/local/accumulo-1.9.2/conf/templates/* /usr/local/accumulo-1.9.2/conf

4.配置master、slaves、monitor、gc Tracer 节点 这里就不写了 主要的配置信息accumulo-env.sh 和 accumulo-site.xml

accumulo-env.sh:

HADOOP_PREFIX hadoop home 目录 这里主要是添加hadoop 相关依赖(ps:我是将依赖拷贝到accumulo 目录下的方式所以随便配一个目录也不会有问题)

HADOOP_CONF_DIR hadoop 配置目录 CDH的默认为/etc/hadoop/conf

JAVA_HOME

ZOOKEEPER_HOME CDH 默认zookeeper路径 /opt/cloudera/parcels/CDH/lib/zookeeper

然后注意配置系统环境变量 ACCUMULO_HOME 为accumulo 安装目录

accumulo-site.xml 主要需要注意修改的

instance.volumes hdfs 存储路径

<property>
    <name>instance.volumes</name>
    <value>hdfs://nameservice1/accumulo2</value>
    <description>comma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo</description>
  </property>

instance.zookeeper.host zk路径

  <property>
    <name>instance.zookeeper.host</name>
    <value>sinan01:2181,sinan02:2181,sinan03:2181</value>
    <description>comma separated list of zookeeper servers</description>
  </property>

general.classpaths 依赖环境 主要需要 hadoop配置文件 accumulo环境 hadoop 依赖环境 这里将hadoop目录下share目录拷贝到 accumulo目录下

<property>
    <name>general.classpaths</name>

    <value>
      <!-- Accumulo requirements -->
      $ACCUMULO_HOME/lib/accumulo-server.jar,
      $ACCUMULO_HOME/lib/accumulo-core.jar,
      $ACCUMULO_HOME/lib/accumulo-start.jar,
      $ACCUMULO_HOME/lib/accumulo-fate.jar,
      $ACCUMULO_HOME/lib/accumulo-proxy.jar,
      $ACCUMULO_HOME/lib/[^.].*.jar,
      <!-- ZooKeeper requirements -->
      $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
      <!-- Common Hadoop requirements -->
      $HADOOP_CONF_DIR,
      <!-- Hadoop 2 requirements
	  -->
	  $ACCUMULO_HOME/share/hadoop/common/[^.].*.jar,
      $ACCUMULO_HOME/share/hadoop/common/lib/(?!slf4j)[^.].*.jar,
      $ACCUMULO_HOME/share/hadoop/hdfs/[^.].*.jar,
      $ACCUMULO_HOME/share/hadoop/mapreduce/[^.].*.jar,
      $ACCUMULO_HOME/share/hadoop/yarn/[^.].*.jar,
      $ACCUMULO_HOME/share/hadoop/yarn/lib/jersey.*.jar,
	  
    </value>
    <description>Classpaths that accumulo checks for updates and class files.</description>
  </property>

5.将配置文件scp到其余节点

6.初始化accumulo，命令：bin/accumulo init，会提示输入实例名称和秘密，

7.在node01主节点启动服务，命令：bin/start-all.sh

后续我会上传JAVA API client代码到githup