正文

初识python 之 离线搭建pyhive环境(含python3安装)

IP云
IP云V管理员/5 阅读
08/12
提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

系统版本： centos6.5

python版本：python3.6.8

相关包存放目录：software

注意：以下操作需要用到root权限

安装python3 root操作

cd /lzh/software
tar -zxvf Python-3.6.8.tgz
cd /lzh/software/Python-3.6.8
./configure --prefix=/usr/local/
make
make install

make 报错: Fatal Python error: Py_Initialize: Unable to get the locale encoding

解决：

在make之前加上：

export LANG=zh_CN.UTF-8
export LANGUAGE=zh_CN.UTF-8
make
make install

配置python环境

cd /usr/bin
mv python python.backup
ln -s /usr/local/bin/python3.6 /usr/bin/python

安装pyhive包

cd /lzh/software/py
tar -zxvf PyHive-0.6.4.tar.gz
cd /lzh/software/py/PyHive-0.6.4
su root
python setup.py install

测试pyhive

vi  test_hive.py

from pyhive import hive
print('hive测试')
conn = hive.Connection(host='xxxx',
port=10000,
auth="CUSTOM",
database='xxx',
username='xxx',
password='xxx')
cursor = conn.cursor()
cursor.execute('select * from tmp_811')
for result in cursor.fetchall():
print(result)
cursor.close()
conn.close()

运行test_hive.py

python test_hive.py

报错：  ModuleNotFoundError: No module named 'thrift'

安装 thrift 包

cd /lzh/software/py
tar -zxvf thrift-0.10.0.tar.gz
cd cd /lzh/software/py/thrift-0.10.0
su root
./configure
make
make install

添加环境变量
vi ~/.bash_profile
export PYTHONPATH=/usr/lib/python3.6/site-packages

继续执行test_hive.py

python test_hive.py

报错：ModuleNotFoundError: No module named 'six'

安装 six

cd /lzh/software/py
tar -zxvf six-1.16.0.tar.gz
cd /lzh/software/py/six-1.16.0
su root
python setup.py install

继续执行test_hive.py

python test_hive.py

报错：ModuleNotFoundError: No module named 'past'

安装 future(包含 past)

cd /lzh/software/py
tar -zxvf future-0.18.2.tar.gz
cd /lzh/software/py/future-0.18.2
su root
python setup.py install

继续执行test_hive.py

python test_hive.py

报错：ModuleNotFoundError: No module named 'sasl'

安装 sasl

第一次安装的是0.1.3版本，与GCC版本不符，安装失败。

cd /lzh/software/py
tar -zxvf sasl-0.2.1.tar.gz
cd /lzh/software/py/sasl-0.2.1
su root
python setup.py install

继续执行test_hive.py

python test_hive.py

报错：ImportError: libsasl2.so.3: cannot open shared object file: No such file or directory

解决：

su root
ln -s /usr/lib64/libsasl2.so.2.0.23 /usr/lib64/libsasl2.so.3

继续执行test_hive.py

python test_hive.py

报错：ModuleNotFoundError: No module named 'thrift_sasl'

安装 thrift_sasl

cd /lzh/software/py
tar -zxvf thrift_sasl-0.4.3.tar.gz
cd /lzh/software/py/thrift_sasl-0.4.3
su root
python setup.py install

继续执行test_hive.py

python test_hive.py

OK，执行结果如下：