Interface between ns-3 and MPI for the Null Message distributed simulation implementation. More...

#include "null-message-mpi-interface.h"

Inheritance diagram for ns3::NullMessageMpiInterface:

Collaboration diagram for ns3::NullMessageMpiInterface:

Public Member Functions
	NullMessageMpiInterface ()

	~NullMessageMpiInterface () override

void	Destroy () override
	Deletes storage used by the parallel environment.

void	Disable () override
	Clean up the ns-3 parallel communications interface.

void	Enable (int pargc, char **pargv) override
	Setup the parallel communication interface.

void	Enable (MPI_Comm communicator) override
	Setup the parallel communication interface using the specified communicator.

MPI_Comm	GetCommunicator () override
	Return the communicator used to run ns-3.

uint32_t	GetSize () override
	Get the number of ranks used by ns-3.

uint32_t	GetSystemId () override
	Get the id number of this rank.

bool	IsEnabled () override
	Returns enabled state of parallel environment.

void	SendPacket (Ptr< Packet > p, const Time &rxTime, uint32_t node, uint32_t dev) override
	Send a packet to a remote node.

Public Member Functions inherited from ns3::ParallelCommunicationInterface
virtual	~ParallelCommunicationInterface ()
	Destructor.

virtual void	Destroy ()=0
	Deletes storage used by the parallel environment.

virtual void	Disable ()=0
	Clean up the ns-3 parallel communications interface.

virtual void	Enable (int pargc, char **pargv)=0
	Setup the parallel communication interface.

virtual void	Enable (MPI_Comm communicator)=0
	Setup the parallel communication interface using the specified communicator.

virtual MPI_Comm	GetCommunicator ()=0
	Return the communicator used to run ns-3.

virtual uint32_t	GetSize ()=0
	Get the number of ranks used by ns-3.

virtual uint32_t	GetSystemId ()=0
	Get the id number of this rank.

virtual bool	IsEnabled ()=0
	Returns enabled state of parallel environment.

virtual void	SendPacket (Ptr< Packet > p, const Time &rxTime, uint32_t node, uint32_t dev)=0
	Send a packet to a remote node.

Static Public Member Functions
static TypeId	GetTypeId ()
	Register this type.

Static Private Member Functions
static void	InitializeSendReceiveBuffers ()
	Initialize send and receive buffers.

static void	ReceiveMessages (bool blocking=false)
	Check for received messages complete.

static void	ReceiveMessagesBlocking ()
	Blocking message receive.

static void	ReceiveMessagesNonBlocking ()
	Non-blocking check for received messages complete.

static void	SendNullMessage (const Time &guaranteeUpdate, Ptr< RemoteChannelBundle > bundle)
	Send a Null Message to across the specified bundle.

static void	TestSendComplete ()
	Check for completed sends.

Static Private Member Functions inherited from ns3::Object
static TypeId	GetTypeId ()
	Register this type.

Static Private Member Functions inherited from ns3::ObjectBase
static TypeId	GetTypeId ()
	Get the type ID.

Static Private Attributes
static MPI_Comm	g_communicator = MPI_COMM_WORLD
	MPI communicator being used for ns-3 tasks.

static bool	g_enabled = false
	Has this interface been enabled.

static bool	g_freeCommunicator = false
	Did we create the communicator? Have to free it.

static bool	g_mpiInitCalled = false
	Has MPI Init been called by this interface.

static uint32_t	g_numNeighbors = 0
	Number of neighbor tasks, tasks that this task shares a link with.

static std::list< NullMessageSentBuffer >	g_pendingTx
	List of pending non-blocking sends.

static char **	g_pRxBuffers
	Data buffers for non-blocking receives.

static MPI_Request *	g_requests
	Pending non-blocking receives.

static uint32_t	g_sid = 0
	System ID (rank) for this task.

static uint32_t	g_size = 1
	Size of the MPI COM_WORLD group.

Additional Inherited Members
Private Member Functions inherited from ns3::Object
	Object ()
	Constructor.

	~Object () override
	Destructor.

void	AggregateObject (Ptr< Object > other)
	Aggregate two Objects together.

void	Dispose ()
	Dispose of this Object.

AggregateIterator	GetAggregateIterator () const
	Get an iterator to the Objects aggregated to this one.

TypeId	GetInstanceTypeId () const override
	Get the most derived TypeId for this Object.

template<typename T >
Ptr< T >	GetObject () const
	Get a pointer to the requested aggregated Object.

template<>
Ptr< Object >	GetObject () const
	Specialization of () for objects of type ns3::Object.

template<typename T >
Ptr< T >	GetObject (TypeId tid) const
	Get a pointer to the requested aggregated Object by TypeId.

template<>
Ptr< Object >	GetObject (TypeId tid) const
	Specialization of (TypeId tid) for objects of type ns3::Object.

void	Initialize ()
	Invoke DoInitialize on all Objects aggregated to this one.

bool	IsInitialized () const
	Check if the object has been initialized.

void	UnidirectionalAggregateObject (Ptr< Object > other)
	Aggregate an Object to another Object.

	Object (const Object &o)
	Copy an Object.

virtual void	DoDispose ()
	Destructor implementation.

virtual void	DoInitialize ()
	Initialize() implementation.

virtual void	NotifyNewAggregate ()
	Notify all Objects aggregated to this one of a new Object being aggregated.

Private Member Functions inherited from ns3::SimpleRefCount< Object, ObjectBase, ObjectDeleter >
	SimpleRefCount ()
	Default constructor.

	SimpleRefCount (const SimpleRefCount &o)
	Copy constructor.

uint32_t	GetReferenceCount () const
	Get the reference count of the object.

SimpleRefCount &	operator= (const SimpleRefCount &o)
	Assignment operator.

void	Ref () const
	Increment the reference count.

void	Unref () const
	Decrement the reference count.

Private Member Functions inherited from ns3::ObjectBase
virtual	~ObjectBase ()
	Virtual destructor.

void	GetAttribute (std::string name, AttributeValue &value) const
	Get the value of an attribute, raising fatal errors if unsuccessful.

bool	GetAttributeFailSafe (std::string name, AttributeValue &value) const
	Get the value of an attribute without raising errors.

virtual TypeId	GetInstanceTypeId () const =0
	Get the most derived TypeId for this Object.

void	SetAttribute (std::string name, const AttributeValue &value)
	Set a single attribute, raising fatal errors if unsuccessful.

bool	SetAttributeFailSafe (std::string name, const AttributeValue &value)
	Set a single attribute without raising errors.

bool	TraceConnect (std::string name, std::string context, const CallbackBase &cb)
	Connect a TraceSource to a Callback with a context.

bool	TraceConnectWithoutContext (std::string name, const CallbackBase &cb)
	Connect a TraceSource to a Callback without a context.

bool	TraceDisconnect (std::string name, std::string context, const CallbackBase &cb)
	Disconnect from a TraceSource a Callback previously connected with a context.

bool	TraceDisconnectWithoutContext (std::string name, const CallbackBase &cb)
	Disconnect from a TraceSource a Callback previously connected without a context.

void	ConstructSelf (const AttributeConstructionList &attributes)
	Complete construction of ObjectBase; invoked by derived classes.

virtual void	NotifyConstructionCompleted ()
	Notifier called once the ObjectBase is fully constructed.

Related Functions inherited from ns3::ObjectBase
static TypeId	GetObjectIid ()
	Ensure the TypeId for ObjectBase gets fully configured to anchor the inheritance tree properly.

Detailed Description

Interface between ns-3 and MPI for the Null Message distributed simulation implementation.

Definition at line 51 of file null-message-mpi-interface.h.

Constructor & Destructor Documentation

◆ NullMessageMpiInterface()

ns3::NullMessageMpiInterface::NullMessageMpiInterface ( )

Definition at line 147 of file null-message-mpi-interface.cc.

References NS_LOG_FUNCTION.

◆ ~NullMessageMpiInterface()

ns3::NullMessageMpiInterface::~NullMessageMpiInterface ( )

override

Definition at line 152 of file null-message-mpi-interface.cc.

References NS_LOG_FUNCTION.

Member Function Documentation

◆ Destroy()

void ns3::NullMessageMpiInterface::Destroy ( )

overridevirtual

Deletes storage used by the parallel environment.

Implements ns3::ParallelCommunicationInterface.

Definition at line 158 of file null-message-mpi-interface.cc.

References NS_LOG_FUNCTION.

◆ Disable()

void ns3::NullMessageMpiInterface::Disable ( )

overridevirtual

Clean up the ns-3 parallel communications interface.

MPI_Finalize will be called only if Enable (int* pargc, char*** pargv) was called.

Implements ns3::ParallelCommunicationInterface.

Definition at line 482 of file null-message-mpi-interface.cc.

References g_communicator, g_enabled, g_freeCommunicator, g_mpiInitCalled, g_numNeighbors, g_pendingTx, g_pRxBuffers, g_requests, NS_FATAL_ERROR, and NS_LOG_FUNCTION.

◆ Enable() [1/2]

void ns3::NullMessageMpiInterface::Enable	(	int *	pargc,
		char ***	pargv
	)

overridevirtual

Setup the parallel communication interface.

There are two ways to setup the communications interface. This Enable method is the easiest method and should be used in most situations.

Disable() must be invoked at end of an ns-3 application to properly cleanup the parallel communication interface.

This method will call MPI_Init and configure ns-3 to use the MPI_COMM_WORLD communicator.

For more complex situations, such as embedding ns-3 with other MPI simulators or libraries, the Enable(MPI_Comm communcicator) may be used if MPI is initialized externally or if ns-3 needs to be run unique communicator. For example if there are two parallel simulators and the goal is to run each simulator on a different set of ranks.

Note: The `SimulatorImplementationType attribute in ns3::GlobalValues must be set before calling Enable()

Parameters

pargc	number of command line arguments
pargv	command line arguments

Implements ns3::ParallelCommunicationInterface.

Definition at line 191 of file null-message-mpi-interface.cc.

References Enable(), g_enabled, g_mpiInitCalled, NS_ASSERT, and NS_LOG_FUNCTION.

Referenced by Enable().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ Enable() [2/2]

void ns3::NullMessageMpiInterface::Enable ( MPI_Comm communicator )

overridevirtual

Setup the parallel communication interface using the specified communicator.

See Enable (int* pargc, char*** pargv) for additional information.

Parameters

communicator MPI Communicator that should be used by ns-3

Implements ns3::ParallelCommunicationInterface.

Definition at line 204 of file null-message-mpi-interface.cc.

References g_communicator, g_enabled, g_freeCommunicator, g_sid, g_size, NS_ASSERT, and NS_LOG_FUNCTION.

◆ GetCommunicator()

MPI_Comm ns3::NullMessageMpiInterface::GetCommunicator ( )

overridevirtual

Return the communicator used to run ns-3.

The communicator returned will be MPI_COMM_WORLD if Enable (int* pargc, char*** pargv) is used to enable or the user specified communicator if Enable (MPI_Comm communicator) is used.

Returns: The MPI Communicator.

Implements ns3::ParallelCommunicationInterface.

Definition at line 178 of file null-message-mpi-interface.cc.

References g_communicator, g_enabled, and NS_ASSERT.

◆ GetSize()

uint32_t ns3::NullMessageMpiInterface::GetSize ( )

overridevirtual

Get the number of ranks used by ns-3.

Returns the size (number of MPI ranks) of the communicator used by ns-3. When running a sequential simulation this will return a size of 1.

Returns: number of parallel tasks

Implements ns3::ParallelCommunicationInterface.

Definition at line 171 of file null-message-mpi-interface.cc.

References g_enabled, g_size, and NS_ASSERT.

◆ GetSystemId()

uint32_t ns3::NullMessageMpiInterface::GetSystemId ( )

overridevirtual

Get the id number of this rank.

When running a sequential simulation this will return a systemID of 0.

Returns: system identification

Implements ns3::ParallelCommunicationInterface.

Definition at line 164 of file null-message-mpi-interface.cc.

References g_enabled, g_sid, and NS_ASSERT.

◆ GetTypeId()

TypeId ns3::NullMessageMpiInterface::GetTypeId ( )

static

Register this type.

Returns: The object TypeId.

Definition at line 140 of file null-message-mpi-interface.cc.

References ns3::TypeId::SetParent().

Here is the call graph for this function:

◆ InitializeSendReceiveBuffers()

void ns3::NullMessageMpiInterface::InitializeSendReceiveBuffers ( )

staticprivate

Initialize send and receive buffers.

This method should be called after all links have been added to the RemoteChannelBundle manager to setup any required send and receive buffers.

Definition at line 231 of file null-message-mpi-interface.cc.

References ns3::RemoteChannelBundleManager::Find(), g_communicator, g_enabled, g_numNeighbors, g_pRxBuffers, g_requests, g_size, NS_ASSERT, NS_LOG_FUNCTION_NOARGS, ns3::NULL_MESSAGE_MAX_MPI_MSG_SIZE, and ns3::RemoteChannelBundleManager::Size().

Referenced by ns3::NullMessageSimulatorImpl::CalculateLookAhead().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ IsEnabled()

bool ns3::NullMessageMpiInterface::IsEnabled ( )

overridevirtual

Returns enabled state of parallel environment.

Returns: true if parallel communication is enabled

Implements ns3::ParallelCommunicationInterface.

Definition at line 185 of file null-message-mpi-interface.cc.

References g_enabled.

◆ ReceiveMessages()

void ns3::NullMessageMpiInterface::ReceiveMessages ( bool blocking = false )

staticprivate

Check for received messages complete.

Will block until message has been received if blocking flag is true. When blocking will return after the first message is received. Non-blocking mode will only check for received messages complete, and return all messages that are queued up locally.

Parameters

[in] blocking Whether this call should block.

Definition at line 357 of file null-message-mpi-interface.cc.

References ns3::RemoteChannelBundleManager::Find(), g_communicator, g_enabled, g_numNeighbors, g_pRxBuffers, g_requests, ns3::NodeList::GetNode(), ns3::Simulator::Now(), NS_ASSERT, NS_LOG_FUNCTION, ns3::NULL_MESSAGE_MAX_MPI_MSG_SIZE, ns3::MpiReceiver::Receive(), and ns3::Simulator::ScheduleWithContext().

Referenced by ReceiveMessagesBlocking(), and ReceiveMessagesNonBlocking().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ReceiveMessagesBlocking()

void ns3::NullMessageMpiInterface::ReceiveMessagesBlocking ( )

staticprivate

Blocking message receive.

Will block until at least one message has been received.

Definition at line 341 of file null-message-mpi-interface.cc.

References NS_LOG_FUNCTION_NOARGS, and ReceiveMessages().

Referenced by ns3::NullMessageSimulatorImpl::HandleArrivingMessagesBlocking().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ReceiveMessagesNonBlocking()

void ns3::NullMessageMpiInterface::ReceiveMessagesNonBlocking ( )

staticprivate

Non-blocking check for received messages complete.

Will receive all messages that are queued up locally.

Definition at line 349 of file null-message-mpi-interface.cc.

References NS_LOG_FUNCTION_NOARGS, and ReceiveMessages().

Referenced by ns3::NullMessageSimulatorImpl::HandleArrivingMessagesNonBlocking().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ SendNullMessage()

void ns3::NullMessageMpiInterface::SendNullMessage	(	const Time &	guaranteeUpdate,
		Ptr< RemoteChannelBundle >	bundle
	)

staticprivate

Send a Null Message to across the specified bundle.

Null Messages are sent when a packet has not been sent across this bundle in order to allow time advancement on the remote MPI task.

Parameters

[in]	guaranteeUpdate	Lower bound time on the next possible event from this MPI task to the remote MPI task across the bundle. Remote task may execute events up to this time.
[in]	bundle	The bundle of links between two ranks.

Internal: The Null Message MPI buffer format uses the same packet: metadata format as sending a normal packet with the time, destination node, and destination device set to zero. Using the same packet metadata simplifies receive logic.

Definition at line 306 of file null-message-mpi-interface.cc.

References g_communicator, g_enabled, g_pendingTx, ns3::Time::GetInteger(), ns3::Time::GetTimeStep(), NS_ASSERT, and NS_LOG_FUNCTION.

Referenced by ns3::NullMessageSimulatorImpl::NullMessageEventHandler(), and ns3::RemoteChannelBundle::Send().

Here is the call graph for this function:

Here is the caller graph for this function: