python 操作hive pyhs2方式
作者:bigdataf
今天小编就为大家分享一篇python 操作hive pyhs2方式,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧
使用kerberos时
import pyhs2 class HiveClient: # 初始化 def __init__(self, db_host, user, password, database, port=10000, authMechanism="PLAIN", configuration=None): self.conn = pyhs2.connect(host=db_host, port=port, authMechanism=authMechanism, user=user, password=password, database=database, configuration=configuration, ) # 查询方法 def query(self, sql): with self.conn.cursor() as cursor: cursor.execute(sql) return cursor.fetch() def close(self): self.conn.close() def getHiveData(sql): config = {"mapreduce.job.queuename": "default", 'krb_host': 'hiveserve2ip', 'krb_service': 'hive'} hive_client = HiveClient(db_host='hiveserve2ip', port=10000, user='user@kdc.com', password='hive', database='dw.usee', authMechanism='KERBEROS', configuration=config) print sql result = hive_client.query(sql) return result
Could not start SASL: Error in sasl_client_start (-1) SASL(-1)
安装
yum install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi
pyhs2 安装 sasl问题
yum install cyrus-sasl-devel yum install cyrus-sasl-lib yum install libgsasl-devel yum install saslwrapper
对接superset hive kerberos
SQLAlchemy URI
hive://herverser2ip:10000/db
扩展 连接参数
{ "metadata_params": {}, "engine_params": { "connect_args": { "auth": "KERBEROS", "kerberos_service_name": "hive", "username" : "user@KDC.COM" } } }
以上这篇python 操作hive pyhs2方式就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持脚本之家。