Geek Answers Handbook

Effective Hbase Access

I use Java as a client for an Hbase request.

The My Hbase table is configured as follows:

ROWKEY     |     HOST     |     EVENT
-----------|--------------|----------
21_1465435 | host.hst.com |  clicked
22_1463456 | hlo.wrld.com |  dragged
    .             .             .
    .             .             .
    .             .             .

The first thing I need to do is get a list of all ROWKEYsthat are related to host.hst.com.

I can create a scanner in a column hostand for each row value with, column value = host.hst.comI will add the corresponding one ROWKEYto the list. Seems pretty effective. O(n)to get all the rows.

Now this is the hard part. For everyone ROWKEYon the list, I need to get the appropriate one EVENT.

If I use the usual command GETto get the cell in (ROWKEY, EVENT), I believe that the scanner is created in EVENT, which takes O(n)time to find the desired cell and return the value. This is a rather difficult time complexity for each person ROWKEY. Combining the two gives us O(n^2).

Is there a better way around this?

Thanks so much for any help in advance!

+4
source share
2 answers

What is nhere? With a RowKey in hand - I suppose you mean HBase rowkey - not some kind of manual one? - It's fast and easy for HBase. Consider that O (1).

ROWKEY - , .. . HBase row.

, - () hbase rowkey, , .

get (rowkey, EVENT) :

Perform a `get` with the given `rowkey`. 
In your result then filter out EVENT in <yourEventValues for that rowkey>

, ( ) . , -, 'n'?? .

, multiget. HBase / master/region.

OP: . "host |" . /.

HBase . , foobarRow1, foobarRow2,.. .., (foobarRow, foobarRowz), , foobarRow, - .

HBase (Easy): hbase

:

SingleColumnValueFilter filter = new SingleColumnValueFilter(
   Bytes.toBytes("columnfamily"),
   Bytes.toBytes("storenumber"),
   CompareFilter.CompareOp.NOT_EQUAL,
   Bytes.toBytes(15)
);
filter.setFilterIfMissing(true);
Scan scan = new Scan(
   Bytes.toBytes("20110103-1"),
   Bytes.toBytes("20110105-1")
);
scan.setFilter(filter);

, 20110103-1 20110105-1 .

+2

, rowkey , .

1) Get , ,

, , .

/**
     * Method getDetailRecords.
     * 
     * @param listOfRowKeys List<String>
     * @return Result[]
     * @throws IOException
     */
    private Result[] getDetailRecords(final List<String> listOfRowKeys) throws IOException {
        final HTableInterface table = HBaseConnection.getHTable(TBL_DETAIL);
        final List<Get> listOFGets = new ArrayList<Get>();
        Result[] results = null;
        try {
            for (final String rowkey : listOfRowKeys) {// prepare batch of get with row keys
   // System.err.println("get 'yourtablename', '" + saltIndexPrefix + rowkey + "'");
                final Get get = new Get(Bytes.toBytes(saltedRowKey(rowkey)));
                get.addColumn(COLUMN_FAMILY, Bytes.toBytes(yourcolumnname));
                listOFGets.add(get);
            }
            results = table.get(listOFGets);

        } finally {
            table.close();
        }
        return results;
    }

2)

Hbase Scan , . , .

FuzzyRowFilter (. hbase-the-definitive) , map-reduce, hbase.

, . , , byte [], . :

FuzzyRowFilter(List<Pair<byte[], byte[]>> fuzzyKeysData)

fuzzyKeysData - , :

0 , . 1 , .

: , , - . _, , 4, 2, 4 2 . , ( 99) . :

"99" _01 ", "? " , . = "\ x01\x01\x01\x01\x00\x00\x00\x00\x01\x01\x01\x01\x00\x00\x00" , , "???? 99???? _01", "?" .

, , , , . getNextCellHint(), , . , . 4-12 .

List<Pair<byte[], byte[]>> keys = new ArrayList<Pair<byte[], byte[]>>();
keys.add(new Pair<byte[], byte[]>(
  Bytes.toBytes("row-?5"), new byte[] { 0, 0, 0, 0, 1, 0 }));
Filter filter = new FuzzyRowFilter(keys);

Scan scan = new Scan()
  .addColumn(Bytes.toBytes("colfam1"), Bytes.toBytes("col-5"))
  .setFilter(filter);
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner) {
  System.out.println(result);
}
scanner.close();

, :

... :

keyvalues={row-05/colfam1:col-01/1/Put/vlen=9/seqid=0,
           row-05/colfam1:col-02/2/Put/vlen=9/seqid=0,
           ...
           row-05/colfam1:col-09/9/Put/vlen=9/seqid=0,
           row-05/colfam1:col-10/10/Put/vlen=9/seqid=0}
keyvalues={row-15/colfam1:col-01/1/Put/vlen=9/seqid=0,
           row-15/colfam1:col-02/2/Put/vlen=9/seqid=0,
           ...
           row-15/colfam1:col-09/9/Put/vlen=9/seqid=0,
           row-15/colfam1:col-10/10/Put/vlen=9/seqid=0}

20 row-01 to row-20. , -? 5, , , 5. .

+3

More articles:


All Articles