Large-scale dataset processing

Question

From online discussion groups and blogs, I have seen that many of the interview questions are related to processing a large-scale array. I wonder if there is a systematic approach to analyzing this type of question? Or, more specifically, is there any data structure or algorithms that can be used to solve this problem? Any suggestions really appreciated.

Answer 1

Large-scale datasets belong to several categories that I have seen, each of which presents various problems for you.

• , . :
  • ,
  • , , ( , !)
  • .
• , , - . - , , .
• , . , , , :-) .
  • , . . / , CouchDB Hadoop, .
• , . ( ) ( , ). , , , . :
  • ( )
  • ( , )

, , , :

• , . , .
• , . , ?

Answer 2

"" - , "" ( , SMP - ).

Answer 3

. , , . , , , B + Trees.

Answer 4

, , . , .

- . , , , .

- - , .

- , .

The DBMS can greatly simplify data access, but it adds some system overhead.