Geek Answers Handbook

Data structure / algorithm for storing variable-length records and searching on disk using only primary keys

I am looking for an algorithm / data structure that works well for large block devices (like a mechanical hard drive) that is optimized for insertion, retrieval, updating and deletion, when the search is always performed using the data identifier and where the data fields for any ID have a variable length.

The B-Tree appears to be a commonly quoted structure, but mainly for fixed-length records. I also expect significantly more updates and updates than inserts and deletes. Can I get rid of the O (log m) B-tree search?

I am very pleased that this is a combined system, for example, ISAM combines the B-tree and linear file storage, which looks like it can be made to work with variable-length records as an approach. Is there something better?

Some additional restrictions:

1) Identifiers are potentially sparse, but they can be forced to enter blocks with linear numbers, but in a large range (64 bits)

2) I do not want to use a DBMS, the performance for my specific problem was not very good. I don’t need any operations that a full DBMS uses, I don’t need a search. I need something that I can easily customize and optimize. Call it academic curiosity, if it runs MySQL, I will use it, but I need to try it faster.

3) , , , , , . , , - 1 .

4) . , , (, B- ).

+5
4

: - Berkeley DB. . " " , .

-: ( ), B-Tree node . append . , , B-Tree (, node, node) ). . , root node B-Tree, . :

  • , " " node .
  • " " , .
  • , .
  • B-Tree .
+9

, (, ) , . O (1) //, .

0

.

O (log m) B-, , { " " " " } - ( )... , , ( , ), bdonlan, .

, API, : ( O/S ) / ( O/S ) API- -.

0
source

If the database is heavy for you, consider a key value repository.

If you really want to implement it, use a disk-based hash table or a B-tree. To avoid problems with variable-length values, save the values ​​in a separate file and use the B-tree as an index for the data file. Spatial reclamation after deleting values ​​will be difficult, but it is possible (for example, a bit for free space in the data file).

0
source

More articles:


All Articles