Geek Answers Handbook

Regardless of the file locking platform?

I have very intensive computational scientific work that spits out the results from time to time. The job is basically to just simulate the same thing a whole bunch of times, so it was shared between several computers using different operating systems. I would like to direct the output from all these instances to the same file, since all computers can see the same file system through NFS / Samba. Here are the limitations:

  • Secure concurrent uploads must be enabled. Should be blocked if any other instance is currently being added on another computer.
  • Performance is not . I / O for each instance is just a few bytes per minute.
  • Easy to count. The whole point of this (besides pure curiosity) is that I can stop writing each instance to another file and merge these files together manually.
  • You should not depend on the details of the file system. You must work with an unknown file system on an NFS or Samba mount.

The language I use is D, in case that matters. I looked, there is nothing in the standard library that seems to do this. Both D-specific and general, linguistic agnostic responses are fully acceptable and appreciated.

+5
source share
5 answers

NFS . NFS . [datafile].lock NFS. , , - [ ].lock, , , , , , , [datafile].lock. [ ]. [PID]. - , NFS, . - , , , , , [datafile].lock

+7

, , . - , :

  • , -

, CVS, . , .

+2

, - , .

, , , , .

, :

  • , , , , , .
    , , .
    , .

  • , , ( , ).

, -.
, , , .

, , .
, , , 00 , , , .

NFS

, , Jiri Klouda , NFS , .

:

  • NFS noac sync. , , , .

  • O_DIRECT, O_SYNC O_DSYNC. .
    , .

  • flock() , , , NFS. .
    , , , .
    , , , SMB, .

  • NFS Samba: NFS, , .
    .

  • Jiri .

, , , NFS , .

-

NFS/SMB , .
.

+2

, ?

, - , , .

In my opinion, creating a server would be much easier than trying to use a network file system.

+2
source

I do not know D, but I use the mutex file to complete the task. Here is some pseudo code that might come in handy:

do {
  // Try to create a new file to use as mutex.
  // If it already created, it will throw some kind of error.
  mutex = create_file_for_writing('lock_file');
} while (mutex == null);

// Open your log file and write results
log_file = open_file_for_reading('the_log_file');
write(log_file, data);
close_file(log_file);

close_file(mutex);
// Free mutex and allow other processes to create the same file.
delete_file(mutex);

So, all processes will try to create a mutex file, but only the one who wins will be able to continue. When you write your output, close and delete the mutexes so that other processes can do the same.

+1
source

More articles:


All Articles