mtsudsm

All shared memory is maintained at the process with rank 0. It responds to requests by the other processes. Rank 0 may be a bottleneck.

The API is:

int dsm_init(void)
Start extra thread(s) as necessary, setup communication, etc.
int dsm_rank(void)
Return my rank in the set of processes. -1 indicates failure.
int dsm_nprocs(void)
Return the total number of processes in this run. -1 indicates failure.
void dsm_malloc(void)
Collective. Allocates a pagesize block of shared memory.
int dsm_barrier(void)
Barrier.
int dsm_sync(void)
Makes sure all pages are current at rank 0, i.e. all
pages chgd locally are sent back. Resets the protection bits
for all pages to PROT_NONE.
int dsm_finalize(void)
Barrier. Indicates shared memory will no longer be used.
Terminate threads.
Cleanup.
int dsm_mutex_lock(int)
Locks the mutex specified by the integer arg. There are 16
total mutexes all managed by rank 0. Hangs if already locked
by another process.
int dsm_mutex_unlock(int)
Unlocks the mutex specified by the integer arg.

All newly malloc'ed pages start out with no read or write access, but controlled by rank 0. If a process does a read access (load) from a page, it is first retrieved from rank 0 and then used locally. If a process does a write access (store) to a page, it is first retrieved from rank 0 and then used locally. No altered pages need be sent back to rank 0 until a dsm_sync at the rank changing the page. Note that there is a race condition if 2 or more ranks try to alter a page at the same time. These issues can be avoided by using dsm_mutex_lock.

When pages are first created, they are marked as no access. Then, on the first attempted access, the latest copy is retrieved from rank 0 and it can then be marked as read-and-write accessible. (Note that we do not really need a read-only access.) However, if you wish to experiment with read and or write access, when a page is first accessed, you may mark it as read-only. Then, when restarting the instruction, if the access is really a write, you can promote the page to write access on that second attempt.

Consider what happens with this small pgm with 2 processes:

#include 'dsm.h'
main()
{
int myrank;
char *p;
dsm_init();
myrank = dsm_rank();
p = (char*) dsm_malloc();
if (myrank == 0)
{
while (p[0] != 'X')
;
}
else
{
p[0] = 'X';
}
dsm_sync(); /* really only needs to be done by rank 1 */
dsm_finalize();
}Let's assume that rank 0 is ahead of rank 1 for some time. Then rank 0 will sit in the while loop for several iterations. On the first iteration, it will become a reader of the page. When rank 1 finally gets to the dsm_sync, the changed page will be flushed back to rank 0. So, rank 0 will eventually come out of the loop. The way that rank 0 can accomplish performing both tasks at once is via threads. Rank 1 becomes a writer to the page when it tries to perform the assignment to p0.

Turnin a dir and its contents including all nec C/C++ and header files and a makefile. When I type:

make
it should build libdsm.a. When I type: make test1
it should compile a program named test1.c (which contains an include for dsm.h) and link it with libdsm.a producing test1.

Tags

  mpi mpich mpich2 distributedsharedmemory

Share