Happybase 사용 방법 (HappybaseTest.py)

0. 참고 링크

Happybase 설치 안내
http://happybase.readthedocs.org/en/latest/installation.html#installing-the-happybase-package

Happybase를 사용하기 위해서는 python을 위한 가상환경(?)을 설정해야 한다.
왜 이런게 필요한지 모르겠지만..

# 최초 설치시 아래 명령어 실행. 하위 디렉토리로 envname 이 생성된다
$ virtualenv envname
# 설치 후에는 activate 스크립트를 실행만 시켜주면 된다. 사용시마다 먼저 실행
$ source envname/bin/activate

Happybase 사용 매뉴얼
https://media.readthedocs.org/pdf/happybase/latest/happybase.pdf
http://happybase.readthedocs.org/en/latest/user.html#establishing-a-connection

Happybase Basic Usage Code

import happybase

connection = happybase.Connection(’hostname’)

table = connection.table(’table-name’)
table.put(’row-key’, {’family:qual1’: ’value1’,’family:qual2’: ’value2’})

row = table.row(’row-key’)
print row[’family:qual1’] # prints ’value1’
for key, data in table.rows([’row-key-1’, ’row-key-2’]):
  print key, data # prints row key and data for each row

for key, data in table.scan(row_prefix=’row’):
  print key, data # prints ’value1’ and ’value2’

row = table.delete(’row-key’)

 

Thrift 설치 방법
http://www.programkr.com/blog/MgTOyADMwYT4.html

간단한 Thrift 사용 코드

from thrift.transport import TSocket
from thrift.protocol import TBinaryProtocol
from hbase import Hbase

transport=TSocket.TSocket('localhost',9090)

protocol=TBinaryProtocol.TBinaryProtocol(transport)

client=Hbase.Client(protocol)

transport.open()

client.getTableNames()

Thrift에서는 버전을 지정하여 값을 불러올 수 없다.
“Unfortunately the Thrift API doesn’t support scanners returning multiple versions for the same row. You can only get multiple values for a specified cell.”
http://stackoverflow.com/questions/13582235/how-to-use-python-thrift-client-to-scan-hbase-table-with-maxversions

테이블을 비활성화한 후, 테이블을 삭제
> disable ‘test’
> drop ‘test’

timestamp=30 을 포함한 그 이전 데이터 삭제
delete ‘t1’, ‘r1’, ‘c1’, 30

버전을 지정하여 값 가져오기
> get ‘t1’, ‘r1’, {COLUMN => ‘c1’, VERSIONS => 10}
COLUMN                     CELL
c1:                       timestamp=40, value=444
1 row(s) in 0.0040 seconds

1. HBase Shell 사용 예제

hbase(main):010:0> list
TABLE                                                                                                                                                                                                                                         
demo_table                                                                                                                                                                                                                                    
short_urls                                                                                                                                                                                                                                    
test                                                                                                                                                                                                                                          
3 row(s) in 0.0100 seconds

=> ["demo_table", "short_urls", "test"]

hbase(main):008:0> scan 'test'
ROW COLUMN+CELL 
 row1 column=cf:a, timestamp=1414721171438, value=value1 
 row2 column=cf:b, timestamp=1414721179483, value=value2 
 row3 column=cf:c, timestamp=1414721186425, value=value3 
3 row(s) in 0.0810 seconds

hbase(main):015:0> describe 'test'
DESCRIPTION ENABLED 
 'test', {NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIO true 
 NS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'} 
1 row(s) in 0.1160 seconds

hbase(main):018:0> t = get_table 'test'
0 row(s) in 0.0040 seconds

=> Hbase::Table - test
hbase(main):019:0> t.scan
ROW COLUMN+CELL 
 row1 column=cf:a, timestamp=1414721171438, value=value1 
 row2 column=cf:b, timestamp=1414721179483, value=value2 
 row3 column=cf:c, timestamp=1414721186425, value=value3 
3 row(s) in 0.0230 seconds

hbase(main):023:0> t.count
3 row(s) in 0.0560 seconds
=> 3

hbase(main):011:0> t.put 'row5', 'cf:b', 'value5'
0 row(s) in 0.0490 seconds
hbase(main):012:0> t.get 'row5'
COLUMN CELL 
 cf:b timestamp=1425969404982, value=value5 
1 row(s) in 0.0040 seconds
hbase(main):024:0> put 'test', 'row4', 'cf:a', 'value4'
0 row(s) in 0.1820 seconds

hbase(main):030:0> scan 'test',{COLUMNS => 'cf:a'}
ROW COLUMN+CELL 
 row1 column=cf:a, timestamp=1414721171438, value=value1 
 row4 column=cf:a, timestamp=1425964460613, value=value4 
2 row(s) in 0.0330 seconds

hbase(main):035:0> scan 'test'
ROW COLUMN+CELL 
 row1 column=cf:a, timestamp=1414721171438, value=value1 
 row2 column=cf:b, timestamp=1414721179483, value=value2 
 row3 column=cf:c, timestamp=1414721186425, value=value3 
 row4 column=cf:a, timestamp=1425964460613, value=value4 
 row5 column=cf:b, timestamp=1425969404982, value=value5 
 row6 column=cf:c, timestamp=1425977109517, value=value6-1 
 row7 column=cf:c, timestamp=1425972501370, value=value7 
 row8 column=cf:c, timestamp=1, value=value8 
8 row(s) in 0.0270 seconds

 

2. Happybase 사용 예제

#!/usr/bin/python
import happybase

conn = happybase.Connection('localhost',port=9090)

conn.tables()
## ==> ['demo_tables','test']

t = conn.table('test')


TEST_TABLE_NAME = 'test_1'

if TEST_TABLE_NAME in conn.tables():
  conn.delete_table( TEST_TABLE_NAME, disable=True)

cfs = {
  'cf1': {},
  'cf2': None,
  'cf3': {'max_versions': 1},
}
conn.create_table( TEST_TABLE_NAME, families=cfs)

# compact_table(): 압축하기(??)
conn.compact_table( TEST_TABLE_NAME )

# table properties
conn.table('test').name
conn.table('test').regions()
'''
[{'name': 'test,,1414721145599.24b83825261afa2ff6ea16c83bbaa666.', 'server_name': 'maxsvr14', 'port': 49802, 'end_key': '', 'version': 1, 'start_key': '', 'id': 1414721145599}]
'''

for name, fdesc in families.iteritems():
  print name, '=>', fdesc
'''
cf => {'max_versions': 1, 'bloom_filter_vector_size': 0, 'name': 'cf:', 'bloom_filter_type': 'ROW', 'bloom_filter_nb_hashes': 0, 'time_to_live': 2147483647, 'in_memory': False, 'block_cache_enabled': True, 'compression': 'NONE'}
'''

######################################################
import happybase
conn = happybase.Connection('localhost',port=9090)
conn.tables()
## ==> ['demo_table', 'short_urls', 'test']

t = conn.table('test')

conn.table('test').families()
## ==> {'cf': {'max_versions': 1, 'bloom_filter_vector_size': 0, 'name': 'cf:', 'bloom_filter_type': 'ROW', 'bloom_filter_nb_hashes': 0, 'time_to_live': 2147483647, 'in_memory': False, 'block_cache_enabled': True, 'compression': 'NONE'}}

families = t.families()
for name, fdesc in families.iteritems():
  print name, fdesc
## ==> cf {'max_versions': 1, 'bloom_filter_vector_size': 0, 'name': 'cf:', 'bloom_filter_type': 'ROW', 'bloom_filter_nb_hashes': 0, 'time_to_live': 2147483647, 'in_memory': False, 'block_cache_enabled': True, 'compression': 'NONE'}

row = t.row('row5')
print row['cf:b']
## ==> value5
print row
## ==> {'cf:b': 'value5'}

t.put('row6', {'cf:c': 'value6'})

values = t.cells('row6','cf:c')
for value in values:
  print value
## ==> value6


for key, data in t.scan():
  print key, data
''' ==> 
row1 {'cf:a': 'value1'}
row2 {'cf:b': 'value2'}
row3 {'cf:c': 'value3'}
row4 {'cf:a': 'value4'}
row5 {'cf:b': 'value5'}
row6 {'cf:c': 'value6-1'}
row7 {'cf:c': 'value7'}
'''

row = t.row('row6', columns=['cf:a','cf:b','cf:c'], include_timestamp=True)
value, timestamp = row['cf:c']
print value, timestamp
## ==> value6-1 1425977109517


t.put('row7', {'cf:c': 'value7'}, 1)
t.put('row8', {'cf:c': 'value8'}, 1)

 

 

 

  1. Happybase support search by timestamp?

    좋아하기

    응답

    1. Sure, you can.
      As Happybase Manual,
      Third argument of PUT function is “timestamp”.

      table.put(‘row-key’, {‘cf:col1’: ‘value1’}, timestamp=123456789)

      좋아하기

답글 남기기

아래 항목을 채우거나 오른쪽 아이콘 중 하나를 클릭하여 로그 인 하세요:

WordPress.com 로고

WordPress.com의 계정을 사용하여 댓글을 남깁니다. 로그아웃 / 변경 )

Twitter 사진

Twitter의 계정을 사용하여 댓글을 남깁니다. 로그아웃 / 변경 )

Facebook 사진

Facebook의 계정을 사용하여 댓글을 남깁니다. 로그아웃 / 변경 )

Google+ photo

Google+의 계정을 사용하여 댓글을 남깁니다. 로그아웃 / 변경 )

%s에 연결하는 중

%d 블로거가 이것을 좋아합니다: