Thread safety in a huge data model

Question

Background: I have a (more or less) huge data model in memory. The model contains about 3.150.000 to 12.600.000 objects that can be changed directly. In addition, there are about 75 million objects that can be changed only through these 3.150.000 to 12.600.000 objects.

On the other hand, there are about 10 modules that access the model. These modules can be grouped into:

• reading and changing some objects every 250 ms to 1000 ms
• reading and changing some objects upon request
• reading some objects if they were changed

Question: How to synchronize such a data model? In my mind there are the following thoughts:

(1) A lock in each class that can be directly changed. Advantage: only objects that have been modified should be locked. Disadvantage: high synchronization and a huge number of lock instances (from 3.150.000 to 12.600.000 additional objects / locks). There is a great danger of doing something wrong in synchronization (dead ends, etc.).

(2) Central interface for access to the entire data model. This interface locks the entire model with each modification with a single lock. Advantage: only one lock → less synchronization effort. Disadvantage: the entire model is locked regardless of the type of change.

(3) Dispatch Thread ( AWT/Swing). , (). /, (2). , . GUI-tollkits. , " " . , , .

: https://weblogs.java.net/blog/kgh/archive/2004/10/multithreaded_t.html

, ? , .

EDIT: . .

:)

EDIT 2: , :

enum Ware { WOOD, COAL, STONE }
class Stock { Map<Ware, Integer> internalStock; }
class Coordinate { int x; int y; }
interface ILand {}

class World {
  Map<Coordinate, ILand> land;
  Map<Coordinate, Ship> ships;
}

class Island implements ILand { Stock resources; }
class Ship { Stock stock; }
class Building {Stock stock; }

class Colony implements ILand {
  Island builtOn;
  Set<Building> building;
}

class Character {
  Set<Colony> colonies;
  Set<Ship> fleet;
}

:

Model
   World     <>--- ILand
             <>--- Ship
   Character <>--- Colony <>--- Building <>--- Stock
                          <>--- Island   <>---Stock
             <>--- Ship   <>--- Stock

Answer 1

, .

, Scala Clojure. , :

http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey

, : :

• . , .
• . , , .
• " " . , , .
• - : , .
• , . , , " " .

Answer 2

a) * , . * equals/hashCode, ConcurrentHashMap. . 40 .

b) -, , .. 100 ( , ). , . ~ 12 (.. ArrayList).

c) , AtomicBitSet java. . . , , , (~ 40 ConcurrentHashMap AtomicBitSet).

, / . :

 lock map: objectId -> {true | false}

 lock map: bucket of objectIds -> {true | false}

:

 lock map: objectId -> {ReadWriteLock lock, Thread owner, long writeLockGrantedAtMs}

, . , ReadWriteLock. writeLockedAtMs owner, .

ADDED

, , , , .. hashCode . , . :

void lockObjects (f, e, a) {

    reorder (f, e, a)

    if(!tryLock(a, timeout: 10ms)){
        throw "could not lock a";
    }

    if(!tryLock(e, timeout: 10ms)){
        throw "could not lock e";
    }

    if(!tryLock(f, timeout: 10ms)){
        throw "could not lock f";
    }

    // now these objects are locked, deadlocks avoided
}

) 3 , 1-2 . , , .

, Colony Character, . , / , .

Colony Character s, , .. Colony Character ( ) . , - , .

Answer 3

. , , synchronized. , , ConcurrentHashMaps .

Answer 4

2 3 parallelism ( ). 1 parallelism ( , ). ( ). 75 . ( ) ( ).

Answer 5

• , - . . , JMM , .

• , Flyweight, , . , , , , , , hashCode , .

• synchronization tryLock Lock , .

• ReadWriteLock. Read Lock .

• , . , :

---

public void fun(MyClass1 o1, MyClass2 o2)
{
    synchronized(o1)
      {
        synchronized(o2){
            .........
            .........
            .........
      }
}

, , :

---

public void fun(MyClass1 o1, MyClass2 o2)
{
    long l1 = System.identityHashCode(o1);
    long l2 = System.identityHashCode(o2);
    if(l1>l2){
      synchronized(o1)
      {
        synchronized(o2)
        {
            .........
            .........
            .........
       }
    }
   }
   else if (l1<l2)
   {
      synchronized(o2)
      {
        synchronized(o1)
        {
            .........
            .........
            .........
       }
    }
   }
 else//if equal than resolve by another mutex
 {
     synchronize(o)
     {
          synchronize(o1)
          {
              synchronize(o2)
              {
                  .........
                  .........
                  .........

              }
          }
      }
 }
}

identityHashCode, long1 > long2, , long2 > long1, , , mutex o . .

• , ​​ ConcurrentLinkedQueue, compareAndSwap, .

• ConcurrentHashMap, , .

• Lock , , . : , A , B CPU. , , . C CPU, , C, . , B , , C . .