states_elec_all_to_all_communications_oct_m Module Reference

This module provides routines for communicating all batches in a ring-pattern scheme. More...

Detailed Description

This module provides routines for communicating all batches in a ring-pattern scheme.

Communication is done in two steps: the ring_pattern module determines which task talk to each task for each step of the ring-pattern scheme.

For a given step of the ring-pattern scheme, this module determines how many steps of send/received are needed given the number of batches owned by the pairs of tasks involved (local, sender, receiver).

This module also provides the routines that handle the communication of batches

Data Types
type	states_elec_all_to_all_communications_t

Functions/Subroutines
subroutine	states_elec_all_to_all_communications_start (this, st, task_from, task_to, gpu_aware)
	Given a task to send to, and a task to receive from, initializes a states_elec_all_to_all_communications_t object. More...
integer function	set_nblock_to_receive (st, task_from)
	How many blocks we will receive from task_from. More...
integer pure function	states_elec_all_to_all_communications_get_ncom (this)
	Returns the number of communications. More...
integer pure function	states_elec_all_to_all_communications_get_nreceive (this)
	Returns the number of receiv calls. More...
integer pure function	states_elec_all_to_all_communications_get_nsend (this)
	Returns the number send calls. More...
logical pure function	states_elec_all_to_all_communications_process_receives (this, icom)
	Returns true if communication step icom receives a batch from task_from. More...
logical pure function	states_elec_all_to_all_communications_process_sends (this, icom)
	Returns true if communication step icom sends a batch to task_to. More...
subroutine	states_elec_all_to_all_communications_alloc_receive_batch (this, st, icom, np, psib)
	Given the icom step, allocate the receiv buffer (wfs_elec_t) More...
subroutine	states_elec_all_to_all_communications_get_send_indices (this, st, icom, ib, ik)
	Given the icom step, returns the block and k-point indices to be sent. More...
subroutine	states_elec_all_to_all_communications_get_receive_indices (this, st, icom, ib, ik)
	Given the icom step, returns the block and k-point indices to be received. More...
subroutine	states_elec_all_to_all_communications_wait_all_isend (this, st)
	Do an MPI waitall for the isend requests. More...
subroutine	dstates_elec_all_to_all_communications_post_all_mpi_isend (this, st, np, node_to)
	Post all isend commands for all batches of a given task. More...
subroutine	dstates_elec_all_to_all_communications_mpi_irecv_batch (this, st, np, node_fr, icom, psib_receiv, recv_req)
	Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch. More...
subroutine	dstates_elec_all_to_all_communications_isend_batch (this, st, np, ib_send, ik_send, node_to, icom, send_req)
	Post a single MPI isend for the batch (ib_send, ik_send) to node_to. More...
subroutine	dstates_elec_all_to_all_communications_isend_batch_to (this, st, np, psib, node_to, icom, send_req)
	Post a single MPI isend for an explicit batch to node_to. More...
subroutine	dstates_elec_all_to_all_communications_recv_batch_from (this, st, np, psib, node_fr, icom)
	Blocking MPI recv into an already-allocated batch from node_fr. More...
subroutine	zstates_elec_all_to_all_communications_post_all_mpi_isend (this, st, np, node_to)
	Post all isend commands for all batches of a given task. More...
subroutine	zstates_elec_all_to_all_communications_mpi_irecv_batch (this, st, np, node_fr, icom, psib_receiv, recv_req)
	Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch. More...
subroutine	zstates_elec_all_to_all_communications_isend_batch (this, st, np, ib_send, ik_send, node_to, icom, send_req)
	Post a single MPI isend for the batch (ib_send, ik_send) to node_to. More...
subroutine	zstates_elec_all_to_all_communications_isend_batch_to (this, st, np, psib, node_to, icom, send_req)
	Post a single MPI isend for an explicit batch to node_to. More...
subroutine	zstates_elec_all_to_all_communications_recv_batch_from (this, st, np, psib, node_fr, icom)
	Blocking MPI recv into an already-allocated batch from node_fr. More...
subroutine	sstates_elec_all_to_all_communications_post_all_mpi_isend (this, st, np, node_to)
	Post all isend commands for all batches of a given task. More...
subroutine	sstates_elec_all_to_all_communications_mpi_irecv_batch (this, st, np, node_fr, icom, psib_receiv, recv_req)
	Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch. More...
subroutine	sstates_elec_all_to_all_communications_isend_batch (this, st, np, ib_send, ik_send, node_to, icom, send_req)
	Post a single MPI isend for the batch (ib_send, ik_send) to node_to. More...
subroutine	sstates_elec_all_to_all_communications_isend_batch_to (this, st, np, psib, node_to, icom, send_req)
	Post a single MPI isend for an explicit batch to node_to. More...
subroutine	sstates_elec_all_to_all_communications_recv_batch_from (this, st, np, psib, node_fr, icom)
	Blocking MPI recv into an already-allocated batch from node_fr. More...
subroutine	cstates_elec_all_to_all_communications_post_all_mpi_isend (this, st, np, node_to)
	Post all isend commands for all batches of a given task. More...
subroutine	cstates_elec_all_to_all_communications_mpi_irecv_batch (this, st, np, node_fr, icom, psib_receiv, recv_req)
	Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch. More...
subroutine	cstates_elec_all_to_all_communications_isend_batch (this, st, np, ib_send, ik_send, node_to, icom, send_req)
	Post a single MPI isend for the batch (ib_send, ik_send) to node_to. More...
subroutine	cstates_elec_all_to_all_communications_isend_batch_to (this, st, np, psib, node_to, icom, send_req)
	Post a single MPI isend for an explicit batch to node_to. More...
subroutine	cstates_elec_all_to_all_communications_recv_batch_from (this, st, np, psib, node_fr, icom)
	Blocking MPI recv into an already-allocated batch from node_fr. More...

Function/Subroutine Documentation

states_elec_all_to_all_communications_start()

subroutine states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_start ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  task_from, integer, intent(in)  task_to, logical, intent(in), optional  gpu_aware  )

private

Given a task to send to, and a task to receive from, initializes a states_elec_all_to_all_communications_t object.

Setting gpu_aware to true will use GPU-aware MPI for the send/receive routines.

Definition at line 202 of file states_elec_all_to_all_communications.F90.

set_nblock_to_receive()

integer function states_elec_all_to_all_communications_oct_m::set_nblock_to_receive ( type(states_elec_t), intent(in)  st, integer, intent(in)  task_from  )

private

How many blocks we will receive from task_from.

Definition at line 247 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_get_ncom()

integer pure function states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_get_ncom ( class(states_elec_all_to_all_communications_t), intent(in)  this )

private

Returns the number of communications.

Definition at line 277 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_get_nreceive()

integer pure function states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_get_nreceive ( class(states_elec_all_to_all_communications_t), intent(in)  this )

private

Returns the number of receiv calls.

Definition at line 285 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_get_nsend()

integer pure function states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_get_nsend ( class(states_elec_all_to_all_communications_t), intent(in)  this )

private

Returns the number send calls.

Definition at line 293 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_process_receives()

logical pure function states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_process_receives ( class(states_elec_all_to_all_communications_t), intent(in)  this, integer, intent(in)  icom  )

private

Returns true if communication step icom receives a batch from task_from.

Definition at line 301 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_process_sends()

logical pure function states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_process_sends ( class(states_elec_all_to_all_communications_t), intent(in)  this, integer, intent(in)  icom  )

private

Returns true if communication step icom sends a batch to task_to.

Definition at line 310 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_alloc_receive_batch()

subroutine states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_alloc_receive_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  icom, integer, intent(in)  np, type(wfs_elec_t), intent(out)  psib  )

private

Given the icom step, allocate the receiv buffer (wfs_elec_t)

Parameters

[in]	icom	Communication step
[in]	np	Number of points in the batch (meshnp or meshnp_part)
[out]	psib	Allocated batch

Definition at line 319 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_get_send_indices()

subroutine states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_get_send_indices ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  icom, integer, intent(out)  ib, integer, intent(out)  ik  )

private

Given the icom step, returns the block and k-point indices to be sent.

Parameters

[in]	icom	Communication step
[out]	ib	Block index of the batch to send
[out]	ik	k-point index of batch to send

Definition at line 351 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_get_receive_indices()

subroutine states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_get_receive_indices ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  icom, integer, intent(out)  ib, integer, intent(out)  ik  )

private

Given the icom step, returns the block and k-point indices to be received.

Parameters

[in]	icom	Communication step
[out]	ib	Block index of the batch to receive
[out]	ik	k-point index of batch to receive

Definition at line 374 of file states_elec_all_to_all_communications.F90.

states_elec_all_to_all_communications_wait_all_isend()

subroutine states_elec_all_to_all_communications_oct_m::states_elec_all_to_all_communications_wait_all_isend ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(in)  st  )

private

Do an MPI waitall for the isend requests.

Definition at line 399 of file states_elec_all_to_all_communications.F90.

dstates_elec_all_to_all_communications_post_all_mpi_isend()

subroutine states_elec_all_to_all_communications_oct_m::dstates_elec_all_to_all_communications_post_all_mpi_isend ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  node_to  )

private

Post all isend commands for all batches of a given task.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

Definition at line 488 of file states_elec_all_to_all_communications.F90.

dstates_elec_all_to_all_communications_mpi_irecv_batch()

subroutine states_elec_all_to_all_communications_oct_m::dstates_elec_all_to_all_communications_mpi_irecv_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, integer, intent(in)  node_fr, integer, intent(in)  icom, type(wfs_elec_t), intent(inout)  psib_receiv, type(mpi_request), intent(out)  recv_req  )

private

Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 517 of file states_elec_all_to_all_communications.F90.

dstates_elec_all_to_all_communications_isend_batch()

subroutine states_elec_all_to_all_communications_oct_m::dstates_elec_all_to_all_communications_isend_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  ib_send, integer, intent(in)  ik_send, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for the batch (ib_send, ik_send) to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 560 of file states_elec_all_to_all_communications.F90.

dstates_elec_all_to_all_communications_isend_batch_to()

subroutine states_elec_all_to_all_communications_oct_m::dstates_elec_all_to_all_communications_isend_batch_to ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for an explicit batch to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to send

Definition at line 585 of file states_elec_all_to_all_communications.F90.

dstates_elec_all_to_all_communications_recv_batch_from()

subroutine states_elec_all_to_all_communications_oct_m::dstates_elec_all_to_all_communications_recv_batch_from ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_fr, integer, intent(in)  icom  )

private

Blocking MPI recv into an already-allocated batch from node_fr.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to receive into

Definition at line 632 of file states_elec_all_to_all_communications.F90.

zstates_elec_all_to_all_communications_post_all_mpi_isend()

subroutine states_elec_all_to_all_communications_oct_m::zstates_elec_all_to_all_communications_post_all_mpi_isend ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  node_to  )

private

Post all isend commands for all batches of a given task.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

Definition at line 748 of file states_elec_all_to_all_communications.F90.

zstates_elec_all_to_all_communications_mpi_irecv_batch()

subroutine states_elec_all_to_all_communications_oct_m::zstates_elec_all_to_all_communications_mpi_irecv_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, integer, intent(in)  node_fr, integer, intent(in)  icom, type(wfs_elec_t), intent(inout)  psib_receiv, type(mpi_request), intent(out)  recv_req  )

private

Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 777 of file states_elec_all_to_all_communications.F90.

zstates_elec_all_to_all_communications_isend_batch()

subroutine states_elec_all_to_all_communications_oct_m::zstates_elec_all_to_all_communications_isend_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  ib_send, integer, intent(in)  ik_send, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for the batch (ib_send, ik_send) to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 820 of file states_elec_all_to_all_communications.F90.

zstates_elec_all_to_all_communications_isend_batch_to()

subroutine states_elec_all_to_all_communications_oct_m::zstates_elec_all_to_all_communications_isend_batch_to ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for an explicit batch to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to send

Definition at line 845 of file states_elec_all_to_all_communications.F90.

zstates_elec_all_to_all_communications_recv_batch_from()

subroutine states_elec_all_to_all_communications_oct_m::zstates_elec_all_to_all_communications_recv_batch_from ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_fr, integer, intent(in)  icom  )

private

Blocking MPI recv into an already-allocated batch from node_fr.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to receive into

Definition at line 892 of file states_elec_all_to_all_communications.F90.

sstates_elec_all_to_all_communications_post_all_mpi_isend()

subroutine states_elec_all_to_all_communications_oct_m::sstates_elec_all_to_all_communications_post_all_mpi_isend ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  node_to  )

private

Post all isend commands for all batches of a given task.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

Definition at line 1012 of file states_elec_all_to_all_communications.F90.

sstates_elec_all_to_all_communications_mpi_irecv_batch()

subroutine states_elec_all_to_all_communications_oct_m::sstates_elec_all_to_all_communications_mpi_irecv_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, integer, intent(in)  node_fr, integer, intent(in)  icom, type(wfs_elec_t), intent(inout)  psib_receiv, type(mpi_request), intent(out)  recv_req  )

private

Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 1041 of file states_elec_all_to_all_communications.F90.

sstates_elec_all_to_all_communications_isend_batch()

subroutine states_elec_all_to_all_communications_oct_m::sstates_elec_all_to_all_communications_isend_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  ib_send, integer, intent(in)  ik_send, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for the batch (ib_send, ik_send) to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 1084 of file states_elec_all_to_all_communications.F90.

sstates_elec_all_to_all_communications_isend_batch_to()

subroutine states_elec_all_to_all_communications_oct_m::sstates_elec_all_to_all_communications_isend_batch_to ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for an explicit batch to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to send

Definition at line 1109 of file states_elec_all_to_all_communications.F90.

sstates_elec_all_to_all_communications_recv_batch_from()

subroutine states_elec_all_to_all_communications_oct_m::sstates_elec_all_to_all_communications_recv_batch_from ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_fr, integer, intent(in)  icom  )

private

Blocking MPI recv into an already-allocated batch from node_fr.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to receive into

Definition at line 1156 of file states_elec_all_to_all_communications.F90.

cstates_elec_all_to_all_communications_post_all_mpi_isend()

subroutine states_elec_all_to_all_communications_oct_m::cstates_elec_all_to_all_communications_post_all_mpi_isend ( class(states_elec_all_to_all_communications_t), intent(inout)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  node_to  )

private

Post all isend commands for all batches of a given task.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

Definition at line 1276 of file states_elec_all_to_all_communications.F90.

cstates_elec_all_to_all_communications_mpi_irecv_batch()

subroutine states_elec_all_to_all_communications_oct_m::cstates_elec_all_to_all_communications_mpi_irecv_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, integer, intent(in)  node_fr, integer, intent(in)  icom, type(wfs_elec_t), intent(inout)  psib_receiv, type(mpi_request), intent(out)  recv_req  )

private

Allocate a batch and post the MPI_Irecv for it. The caller must wait on recv_req before using the batch.

Note that we use here a unique request ID that is labelled by the sending task ID and the batch index, to help the MPI library.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 1305 of file states_elec_all_to_all_communications.F90.

cstates_elec_all_to_all_communications_isend_batch()

subroutine states_elec_all_to_all_communications_oct_m::cstates_elec_all_to_all_communications_isend_batch ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(inout)  st, integer, intent(in)  np, integer, intent(in)  ib_send, integer, intent(in)  ik_send, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for the batch (ib_send, ik_send) to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Definition at line 1348 of file states_elec_all_to_all_communications.F90.

cstates_elec_all_to_all_communications_isend_batch_to()

subroutine states_elec_all_to_all_communications_oct_m::cstates_elec_all_to_all_communications_isend_batch_to ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_to, integer, intent(in)  icom, type(mpi_request), intent(out)  send_req  )

private

Post a single MPI isend for an explicit batch to node_to.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to send

Definition at line 1373 of file states_elec_all_to_all_communications.F90.

cstates_elec_all_to_all_communications_recv_batch_from()

subroutine states_elec_all_to_all_communications_oct_m::cstates_elec_all_to_all_communications_recv_batch_from ( class(states_elec_all_to_all_communications_t), intent(in)  this, type(states_elec_t), intent(in)  st, integer, intent(in)  np, class(batch_t), intent(inout)  psib, integer, intent(in)  node_fr, integer, intent(in)  icom  )

private

Blocking MPI recv into an already-allocated batch from node_fr.

CPU or GPU-aware MPI is handled depending on thisgpu_aware.

Parameters

[in,out]

psib

Batch to receive into

Definition at line 1420 of file states_elec_all_to_all_communications.F90.