Since this is a tutorial, I'm only going to develop for the Windows platform. This means that I don't have to be very careful about endianness. I've also chosen to use UDP because it's the fastest way to communicate and makes the most sense in a real-time game. I'm also going to design with the client/server (star) configuration in mind, because it is the most scalable and the most robust.
In all the online tutorials that I've read about creating multiplayer networked games, there's always one detail that's left out, and that detail is about the same size and level of danger as an out-of-control 18-wheel truck. The problem is multithreading.
Consider for a moment a simple single-thread game: In your main loop you read in from the keyboard, move things in the world, and then draw to the screen. So it would seem reasonable that in a network game you would read in from the keyboard, read in from the Internet, move things, send messages back out to the Internet, and then draw to the screen. Sadly, this is not the case. Oh sure, you can write a game to work like this (I learned the hard way), but it won't work the way you expect it to. Why? Let's say your computer can draw 20 frames per second. That means that most of 50ms is being spent drawing, which means that nearly 50ms go by between any two checks for new data from the Internet. So what? Anyone who's ever played a network game will tell you that 50ms can mean the difference between life and death. So when you send a message to another computer, that message could be waiting to be read for nearly 50ms and the reply could be waiting in your machine's hardware for an extra 50ms for an extra round trip time of 100ms!
Worse still is the realization that if you stay with a single-threaded app, there's nothing you can do to solve the problem; nothing will make that delay go away. Yes, it would be shorter if the frame rate were higher. But just try to tell people they can't play unless their frame rate is high enough—I bet good money they tar and feather you.
The solution is, of course, to write a multithreaded app. I'll admit the first time I had to write one I was pretty spooked. I thought it was going to be a huge pain. Please believe me when I say that as long as you write clean, careful code, you can get it right the first time and you won't have to debug a multithreaded app. And since everything you do from here on in will depend on that multithreading, let's start the MTUDP (multithreaded UDP) class there. First of all, be sure that you tell the compiler you're not designing a single-threaded app. In MSVC 6.0 the option to change is in Project|Settings|C/C++|Code Generation|Use Runtime Library.
Windows multithreading is, from what I hear, completely insane in its design. Fortunately, it's also really easy to get started. CreateThread() is a standard Windows function and, while I won't go into much detail about its inner working here (you have MSDN; look it up!) I will say that I call it as follows:
void cThread::Begin()
{
d_threadHandle = CreateThread( NULL,
0,
(LPTHREAD_START_ROUTINE)gsThreadProc,
this, 0, (LPDWORD)&d_threadID );
if( d_threadHandle == NULL )
throw cError( "cThread() - Thread creation failed." );
d_bIsRunning = true;
}
As you can tell, I've encapsulated all my thread stuff into a single class. This gives me a nice place to store d_threadHandle so I can kill the thread later, and it means I can use cThread as a base class for, oh, say, MTUDP, and I can reuse the cThread class in all my other applications without making any changes to it.
CreateThread() takes six parameters, the most important of which are parameters 3 and 4, gsThreadProc and this. this is a pointer to the instance of cThread and will be sent to gsThreadProc. This is crucial because gsThreadProc cannot be a class function because Windows doesn't like that. Instead, gsThreadProc is defined at the very beginning of cThread.cpp as follows:
static DWORD WINAPI cThreadProc( cThread *pThis )
{
return pThis->ThreadProc();
}
I don't know about you, but I think that's pretty sneaky. It also happens to work! Back in the cThread class ThreadProc is a virtual function that returns zero immediately. ThreadProc can return anything you like, but I've always liked to return zero when there is no problem and use every other number as an error code.
Sooner or later you're going to want to stop the thread. Again, this is pretty straightforward.
void cThread::End()
{
if( d_threadHandle != NULL )
{
d_bIsRunning = false;
WaitForSingleObject( d_threadHandle, INFINITE );
CloseHandle( d_threadHandle );
d_threadHandle = NULL;
}
}
The function cThread::End() is set up in such a way that you can't stop a thread more than once, but the real beauty is hidden. Notice d_bIsRunning? Well, you can use it for more than just telling the other threads that you're still working. Let's look at a simple version of a derived class's ThreadProc().
DWORD MTUDP::ThreadProc()
{
while( d_bIsRunning == true )
{
// Read and process network data here.
}
return 0;
}
This means that the moment d_bIsRunning is set to false, the thread will quit. Of course, we could get the thread to quit any time—if it detected an error, for example. This is an easy way for one thread to have start/stop control on another thread. In fact, if you didn't set d_bIsRunning to false, the first thread would stop running forever while it waited for WaitForSingleObject( d_threadHandle, INFINITE ). This is because d_threadHandle functions like a mutex.
Mutexes are crucial in multithreading because they protect data that is in a critical section. For example, let's say you have a linked list of information. One thread is adding to the linked list and the other thread is removing. What would happen if the two tried to access the linked list at the same time? There's a chance that one thread could walk through the linked list and then step off into "funny RAM" (some undefined location in RAM that is potentially dangerous to modify) because the other thread hadn't finished working with the linked list pointers.
Fortunately, C++ lets you set up a really nice little class to monitor these critical sections and make sure every thread plays nice.
class cMonitor
{
protected:
HANDLE d_mutex;
public:
cMonitor();
virtual ~cMonitor();
void MutexOn() const;
void MutexOff() const;
};
Again, this class is used as a base class for every class that has a critical section. In fact, I defined cThread as class cThread : public cMonitor. The four cMonitor functions are also very interesting.
cMonitor::cMonitor()
{
// This mutex will help the two threads share their toys.
d_mutex = CreateMutex( NULL, false, NULL );
if( d_mutex == NULL )
throw cError( "cMonitor() - Mutex creation failed." );
}
cMonitor::~cMonitor()
{
if( d_mutex != NULL )
{
CloseHandle( d_mutex );
d_mutex = NULL;
}
}
cMonitor will create a new mutex and clean up after itself.
void cMonitor::MutexOn() const
{
WaitForSingleObject( d_mutex, INFINITE );
}
void cMonitor::MutexOff() const
{
ReleaseMutex( d_mutex ); // To be safe...
}
Once again you see that WaitForSingleObject() will stall a thread forever if necessary. The big difference between this and d_threadHandle is that d_threadHandle was released by Windows. Here, control is left up to a thread. If WaitForSingleObject() is called, the thread will gain control of a mutex and every other thread will have to wait until that same thread calls ReleaseMutex() before they get a turn, and it's first come, first serve. This means you have to be very careful with how you handle your mutexes—if you don't match every WaitFor… with a ReleaseMutex(), threads will hang forever and you will soon find yourself turning your computer off to reboot it. I suppose I could have written a version of MutexOn() that would wait n milliseconds and return an error code, but I haven't found a need for it yet.
Try/throw/catch is a wonderful construction that can simplify your debugging. Unfortunately, it doesn't work very well inside other threads. Actually, it works, but it might surprise you. The following would work, but it would not catch anything thrown by the other thread.
// somewhere inside thread #1
try
{
cThreadDerived myNewThread;
mNewThread.Begin();
// Do other stuff.
}
catch( cError &err )
{
// No error would be reported.
}
}
// somewhere inside cThreadDerived::ThreadProc()
throw cError( "Gack!" );
The solution is to put an extra try/catch inside the ThreadProc of cThreadDerived and then store the error somewhere for the other thread to read it or process it right there and then.
You've seen the multithreading class and you've got a way to protect the critical sections. So here's how it's going to work: The main thread can read and send data with MTUDP whenever it can get the mutex. The rest of the time it can render, check the keyboard, play music, etc. MTUDP, meanwhile, will be constantly rechecking the network to see if there is data to be read in from the Internet and processing any that arrives.
Now you can start getting down to business!
class MTUDP : public cThread
{
protected:
SOCKET d_listenSocket,
d_sendSocket;
unsigned short d_localListenPort,
d_foreignListenPort;
bool d_bStarted,
d_bListening,
d_bSending;
// A list of all the data packets that have arrived.
public:
MTUDP();
virtual ~MTUDP();
virtual ThreadProc();
void Startup( unsigned short localListenPort,
unsigned short ForeignListenPort );
void Cleanup();
void StartListening();
void StartSending();
void StopListening();
void StopSending();
unsigned short GetReliableData( char * const pBuffer,
unsigned short maxLen );
void ReliableSendTo( const char * const pStr, unsigned short len );
};
Startup() and Cleanup() are the bookends of the class and are required to initialize and tidy up. StartListening() and StartSending() will create the d_listenSocket and d_sendSocket, respectively. One or more of these has to be called before ReliableSendTo() or GetReliableData() will do anything.
void MTUDP::Startup( unsigned short localListenPort,
unsigned short foreignListenPort )
{
Cleanup(); // just in case somebody messed up out there...
WSAData wsaData;
int error;
error = WSAStartup( MAKEWORD( 2, 2 ), &wsaData );
if( error == SOCKET_ERROR )
{
char errorBuffer[ 100 ];
error = WSAGetLastError();
if( error == WSAVERNOTSUPPORTED )
{
sprintf( errorBuffer,
"MTUDP::Startup() - WSAStartup() error.\nRequested v2.2, found only v%d.%d.",
LOBYTE( wsaData.wVersion ), HIBYTE( wsaData.wVersion ) );
WSACleanup();
}
else
sprintf( errorBuffer, "MTUDP::Startup() - WSAStartup() error %d",
WSAGetLastError() );
throw cError( errorBuffer );
}
d_localListenPort = localListenPort;
d_foreignListenPort = foreignListenPort;
d_bytesTransfered = 0;
d_bStarted = true;
}
Really the only mystery here is WSAStartup(). It takes two parameters: a word that describes what version of Winsock you'd like to use and a pointer to an instance of WSAData, which will contain all kinds of useful information regarding this machine's Winsock capabilities. I admire the way the Winsock programmers handle errors—just about everything will return SOCKET_ERROR, at which point you can call WSAGetLastError() to find out more information. The two variables passed to Startup (d_localListenPort and d_foreignListenPort) will be used a little later.
void MTUDP::Cleanup()
{
if( d_bStarted == false )
return;
d_bStarted = false;
StopListening();
StopSending();
// Clean up all data waiting to be read
WSACleanup();
}
An important note: WSAStartup() causes a DLL to be loaded, so be sure to match every call to WSAStartup() with exactly one call to WSACleanup().
All programming (and, as I've learned, all attempts to explain things to people) should follow the "method of least surprise." MTUDP's creation and destruction methods prove we've been able to stick to that rule.
At this point all the creation method does is initialize d_bStarted to false and the destruction method calls Cleanup().
Now we get to put d_localListenPort to use.
void MTUDP::StartListening()
{
if( d_bListening == true ||
d_bStarted == false )
return;
d_bListening = true;
Nothing special yet; this just prevents you from calling StartListening() twice.
d_listenSocket = socket( AF_INET, SOCK_DGRAM, 0 );
if( d_listenSocket == INVALID_SOCKET )
// throw an error here.
Socket() is a Winsock method that creates a socket. The three parameters are the address family, the socket type, and the protocol type (which modifies the socket type). The only parameter here you should ever mess with is SOCK_DGRAM, which could be changed to SOCK_STREAM if you wanted to work in TCP/IP.
SOCKADDR_IN localAddr;
int result;
memset( &localAddr, 0, sizeof( SOCKADDR_IN ) );
localAddr.sin_family = AF_INET;
localAddr.sin_addr.s_addr = htonl( INADDR_ANY );
localAddr.sin_port = htons( d_localListenPort );
result = bind( d_listenSocket,
(sockaddr *)&localAddr,
sizeof( SOCKADDR_IN ) );
if( result == SOCKET_ERROR )
{
closesocket( d_listenSocket );
// throw another error.
}
Bind() takes three parameters—the port number on which to open the new listening socket, some information about the type of socket (in the form of a SOCKADDR or SOCKADDR_IN structure), and the size of parameter 2. Every time you Bind() a socket you have to make sin_family equal to the same thing as the socket's address family. Since this is a listening socket, you want it to be on port d_localListenPort so that's what sin_port is set to. The last parameter, sin_addr.s_addr, is the address you would be sending data to. The listen socket will never send any data so set it to INADDR_ANY. Lastly, if the Bind() fails, be sure to close the socket. There's only one step left!
// We're go for go!
cThread::Begin();
}
The start of StartSending() is the same old deal—check that a send socket has not been opened (d_bSending == false) and create the send socket (which looks exactly the same as it did in StartListening()). The only significant change comes in the call to Bind().
SOCKADDR_IN localAddr;
int result;
memset( &localAddr, 0, sizeof( SOCKADDR_IN ) );
localAddr.sin_family = AF_INET;
localAddr.sin_addr.s_addr = htonl( INADDR_ANY );
localAddr.sin_port = htons( 0 );
result = bind( d_sendSocket, (sockaddr *)&localAddr, sizeof( SOCKADDR_IN ) );
if( result == SOCKET_ERROR )
// close the socket and throw an error.
I don't care what port the send socket is bound to, so sin_port is set to zero. Even though data is being sent, because UDP is being used, the sin_addr.s_addr is once again set to INADDR_ANY. This would have to change if you wanted to use TCP/IP, because once you open a TCP/IP socket it can only send to one address until it is closed or forced to change.
At the end of StartSending() you do not call cThread::Begin(). Thanks to the cThread class it wouldn't have an effect, so make sure to call Start-Listen() before StartSending(). Another good reason to call StartListening() first is because there's a very small chance that the random port Winsock binds your send socket to is the same port you want to use for listening.
Now to the real meat and potatoes. I'll explain the whole thing at the end.
DWORD MTUDP::ThreadProc()
{
if( d_bListening == false )
return 0; // Quit already?! It can happen...
char inBuffer[ MAX_UDPBUFFERSIZE ];
timeval waitTimeStr;
SOCKADDR_IN fromAddr;
int fromLen;
unsigned short result;
FD_SET set;
try
{
while( d_bListening == true )
{
// Listen to see if there is data waiting to be read.
FD_ZERO( &set );
FD_SET( d_listenSocket, &set );
waitTimeStr.tv_sec = 0; // Wait 0 seconds
waitTimeStr.tv_usec = 0; // Wait 0 microseconds (1/(1*10^6) seconds)
// Select tells us if there is data to be read.
result = select( FD_SETSIZE, &set, NULL, NULL, &waitTimeStr );
if( result == 0 )
continue;
if( result == SOCKET_ERROR )
// throw an error.
// Recvfrom gets the data and puts it in inBuffer.
fromLen = sizeof( SOCKADDR );
result = recvfrom( d_listenSocket,
inBuffer,
MAX_UDPBUFFERSIZE,
0,
(SOCKADDR *)&fromAddr,
&fromLen );
if( result == 0 )
continue;
if( result == SOCKET_ERROR )
// throw an error.
// Put the received data in a mutex-protected queue here.
ProcessIncomingData( inBuffer,
result,
ntohl( fromAddr.sin_addr.s_addr ),
GetTickCount() );
} // while
} // try
catch( cError &err )
{
// do something with err.d_text so that the
// other thread knows this thread borked.
}
// Returns 1 if the close was not graceful.
return d_bListening == true;
}
It may seem a little weird to put a check for d_bListening at the start of the thread proc. I added it because there is a short delay between when you call Begin() and when ThreadProc() is actually called, and even if you clean up properly when you're going to quit, it can make your debug output look a little funny.
MAX_UDPBUFFERSIZE is equal to the maximum size of a UDP packet, 4096 bytes. I seriously doubt you will ever send a UDP block this big, but it never hurts to play it safe. As you can see, try/catch/throw is here, just like I said. The next step is the while loop, which begins with a call to Select(). Select() will check any number of sockets to see if there is data waiting to be read, check if one of the sockets can send data, and/or check if an error occurred on one of the sockets. Select() can be made to wait for a state change as long as you want, but I set waitTimeStr to 0 milliseconds so that it would poll the sockets and return immediately. That way it's a little more thread friendly.
Some of you may have some experience with Winsock and are probably wondering why I didn't use something called "asynchronous event notification." Two reasons: First, it takes a lot of effort to get set up and then clean up again. Second, it makes MTUDP dependent on a window handle, which makes it dependent on the speed at which WndProc() messages can be parsed, and it would make MTUDP even more dependent on Windows functions, something we'd like to avoid, if possible.
The next steps only happen if there is data to be read. Recvfrom() will read in data from a given socket and return the number of bytes read, but no more than the MAX_UDPBUFFERSIZE limit. Recvfrom() will also supply some information on where the data came from in the fromAddr structure.
If some data was successfully read in to inBuffer, the final step in the while loop is called. This is a new MTUDP function called Process-IncomingData().
Well, I'm sorry to say that, for now, ProcessIncomingData() is virtually empty. However, it is the first opportunity to see mutexes in action.
void MTUDP::ProcessIncomingData( char * const pData,
unsigned short length,
DWORD address,
DWORD receiveTime )
{
cMonitor::MutexOn();
// Add the data to our list of received packets.
cMonitor::MutexOff();
}
GetReliableData() is one of the few methods that can be called by another thread. Because it also messes with the list of received packets, mutexes have to be used again.
unsigned short MTUDP::GetReliableData( char * const pBuffer,
unsigned short maxLen )
{
if( pBuffer == NULL )
throw cError( "MTUPD::GetReliableData() - Invalid parameters." );
if( maxLen == 0 )
return 0;
cMonitor::MutexOn();
// take one of the received packets off the list.
cMonitor::MutexOff();
// fill pBuffer with the contents of the packet.
// return the size of the packet we just read in.
}
TA DA! You've now got everything required to asynchronously read data from the Internet while the other thread renders, reads input, picks its nose, gives your hard drive a wedgie, you name it; it's coded. Of course, it doesn't really tell you who sent the information, and it's a long way from being reliable.
It's a good thing that I left this for the end because some of the code to get ReliableSendTo() working will help with reliable communications. In music circles this next bit would be called a bridge—the melody changes, maybe even enters a new key, but it gets you where you need to go.
You've probably had all sorts of ideas on how to store the incoming data packets. I'm going to describe my data packet format, which may be a little puzzling at first. Trust me, by the end it will all make perfect sense.
// this file is eventually inherited everywhere else, so this seemed
// like a good place to define it.
#define MAX_UDPBUFFERSIZE 4096
class cDataPacket
{
public:
char d_data[ MAX_UDPBUFFERSIZE ];
unsigned short d_length,
d_timesSent;
DWORD d_id,
d_firstTime,
d_lastTime;
cDataPacket();
virtual ~cDataPacket();
void Init( DWORD time,
DWORD id,
unsigned short len,
const char * const pData );
cDataPacket &operator=( const cDataPacket &otherPacket );
};
As always, it follows the K.I.S.S. (keep it simple, stupid) principle. Init sets d_firstTime and d_lastTime to time, d_id to id, and d_length to len, and copies len bytes from pData into d_data. The = operator copies one packet into another.
cQueueIn stores all the data packets in a nice, neat, orderly manner. In fact it keeps two lists—one for data packets that are in order and one for the rest (which are as ordered as can be, given that some may be missing from the list).
class cQueueIn : public cMonitor
{
protected:
list<cDataPacket *> d_packetsOrdered;
list<cDataPacket *> d_packetsUnordered;
DWORD d_currentPacketID,
d_count; // number of packets added to this queue.
public:
cQueueIn();
virtual ~cQueueIn();
void Clear();
void AddPacket( DWORD packetID,
const char * const pData,
unsigned short len,
DWORD receiveTime );
cDataPacket *GetPacket();
bool UnorderedPacketIsQueued( DWORD packetID );
DWORD GetHighestID();
inline DWORD GetCurrentID(); // returns d_currentPacketID.
inline DWORD GetCount(); // returns d_count.
};
d_currentPacketID is equal to the highest ordered packet ID plus 1. Clear() removes all packets from all lists. GetPacket() removes the first packet in the d_packetsOrdered list (if any) and returns it. UnorderedPacketIs-Queued() informs the caller if the packet is in the d_packetsUnordered list and returns true if packetID < d_currentPacketID. GetHighestID() returns the highest unordered packet ID plus 1 (or d_currentPacketID if d_packets-Unordered is empty). In fact, the only tricky part in this whole class is AddPacket().
void cQueueIn::AddPacket( DWORD packetID,
const char * const pData,
unsigned short len,
DWORD receiveTime )
{
if( pData == NULL ||
len == 0 ||
d_currentPacketID > packetID )
return;
// Create the packet.
cDataPacket *pPacket;
pPacket = new cDataPacket;
if( pPacket == NULL )
throw cError( "cQueueIn::AddPacket() - insufficient memory." );
pPacket->Init( receiveTime, packetID, len, pData );
// Add the packet to the queues.
cMonitor::MutexOn();
if( d_currentPacketID == pPacket->d_id )
{
// This packet is the next ordered packet. Add it to the ordered list
// and then move all unordered that can be moved to the ordered list.
d_packetsOrdered.push_back( pPacket );
d_currentPacketID++;
d_count++;
pPacket = *d_packetsUnordered.begin();
while( d_packetsUnordered.empty() == false &&
d_currentPacketID == pPacket->d_id )
{
d_packetsUnordered.pop_front();
d_packetsOrdered.push_back( pPacket );
d_currentPacketID++;
pPacket = *d_packetsUnordered.begin();
}
}
else // d_currentPacketID < pPacket->d_id
{
// Out of order. Sort into the list.
list<cDataPacket *>::iterator iPacket;
bool bExists;
bExists = false;
for( iPacket = d_packetsUnordered.begin();
iPacket != d_packetsUnordered.end(); ++iPacket )
{
// Already in list - get out now!
if( (*iPacket)->d_id == pPacket->d_id )
{
bExists = true;
break;
}
if( (*iPacket)->d_id > pPacket->d_id )
break;
}
if( bExists == true )
delete pPacket;
else
{
// We've gone 1 past the spot where pPacket belongs. Back up and insert.
d_packetsUnordered.insert( iPacket, pPacket );
d_count++;
}
}
cMonitor::MutexOff();
}
Now I could stop right here, add an instance of cQueueIn to MTUDP, and there would be almost everything needed for reliable communications, but that's not why I went off on this tangent. There is still no way of sending data to another computer and also no way of telling who the data came from.
Yes, this is another new class. Don't worry, there's only four more, but they won't be mentioned for quite some time. (I'm only telling you that to fill you with anticipation and dread in the same way Stephen King would start a chapter with "three weeks before the church steeple blew up" or Hitchcock would show you the ticking bomb hidden under a restaurant table. It's a spooky story and an education! More BANG! (aah!) for your buck.) The cHost class doesn't contain much yet, but it will be expanded later.
class cHost : public cMonitor
{
DWORD d_address;
unsigned short d_port;
cQueueIn d_inQueue;
public:
cHost();
virtual ~cHost();
unsigned short ProcessIncomingReliable( char * const pBuffer, unsigned
short len, DWORD receiveTime );
void SetPort( unsigned short port );
bool SetAddress( const char * const pAddress );
bool SetAddress( DWORD address );
DWORD GetAddress(); // returns d_address.
unsigned short GetPort(); // returns d_port.
cQueueIn &GetInQueue(); // returns d_inQueue. };
There are only two big mysteries here: SetAddress() and ProcessIncomingReliable().
bool cHost::SetAddress( const char * const pAddress )
{
if( pAddress == NULL )
return true;
IN_ADDR *pAddr;
HOSTENT *pHe;
pHe = gethostbyname( pAddress );
if( pHe == NULL )
return true;
pAddr = (in_addr *)pHe->h_addr_list[ 0 ];
d_address = ntohl( pAddr->s_addr );
return false;
}
The other SetAddress assumes you've already done the work, so it just sets d_address equal to address and returns.
As I said before, the cHost you're working with is a really simple version of the full cHost class. Even ProcessIncomingReliable(), which I'm about to show, is a simple version of the full ProcessIncomingReliable().
unsigned short cHost::ProcessIncomingReliable( char * const pBuffer,
unsigned short maxLen,
DWORD receiveTime )
{
DWORD packetID;
char *readPtr;
unsigned short length;
readPtr = pBuffer;
memcpy( &packetID, readPtr, sizeof( DWORD ) );
readPtr += sizeof( DWORD );
memcpy( &length, readPtr, sizeof( unsigned short ) );
readPtr += sizeof( unsigned short );
// If this message is a packet, queue the data
// to be dealt with by the application later.
d_inQueue.AddPacket( packetID, (char *)readPtr, length, receiveTime );
readPtr += length;
// d_inQueue::d_count will be used here at a much much later date.
return readPtr - pBuffer;
}
This might seem like overkill, but it will make the program a lot more robust and net-friendly in the near future.
Things are now going to start building on the layers that came before. To start with, MTUDP is going to store a list< > containing all the instances of cHost, so the definition of MTUDP has to be expanded.
// Used by classes that call MTUDP, rather than have MTUDP return a pointer.
typedef DWORD HOSTHANDLE;
class MTUDP : public cThread
{
private:
// purely internal shortcuts.
typedef map<HOSTHANDLE, cHost *> HOSTMAP;
typedef list<cHost *> HOSTLIST;
protected:
HOSTLIST d_hosts;
HOSTMAP d_hostMap;
HOSTHANDLE d_lastHandleID;
public:
HOSTHANDLE HostCreate( const char * const pAddress,
unsigned short port );
HOSTHANDLE HostCreate( DWORD address, unsigned short port );
void HostDestroy( HOSTHANDLE hostID );
unsigned short HostGetPort( HOSTHANDLE hostID );
DWORD HostGetAddress( HOSTHANDLE hostID );
So what exactly did I do here? Well, MTUDP returns a unique HOSTHANDLE for each host so that no one can do anything silly (like try to delete a host). It also means that because MTUDP has to be called for everything involving hosts, MTUDP can protect d_hostMap and d_hosts with the cThread::cMonitor.
Now, it may surprise you to know that MTUDP creates hosts at times other than when some outside class calls HostCreate(). In fact, this is a perfect time to also show you just what's going to happen to cHost::QueueIn() by revisiting MTUDP::ProcessIncomingData().
void MTUDP::ProcessIncomingData( char * const pData, unsigned short length,
DWORD address, DWORD receiveTime )
{
// Find the host that sent this data.
cHost *pHost;
HOSTLIST::iterator iHost;
cMonitor::MutexOn();
// search d_hosts to find a host with the same address.
if( iHost == d_hosts.end() )
{
// Host not found! Must be someone new sending data to this computer.
DWORD hostID;
hostID = HostCreate( address, d_foreignListenPort );
if( hostID == 0 )
// turn mutex off and throw an error, the host creation failed.
pHost = d_hostMap[ hostID ];
}
else
pHost = *iHost;
assert( pHost != NULL );
// This next part will get more complicated later.
pHost->ProcessIncomingReliable( pData, length, receiveTime );
}
Of course, that means you now have a list of hosts. Each host might contain some new data that arrived from the Internet, so you're going to have to tell the other thread about it somehow. That means you're going to have to make changes to MTUDP::GetReliableData().
unsigned short MTUDP::GetReliableData( char * const pBuffer,
unsigned short maxLen,
HOSTHANDLE * const pHostID )
{
if( pBuffer == NULL ||
pHostID == NULL )
throw cError( "MTUPD::GetReliableData() - Invalid parameters." );
if( maxLen == 0 )
return 0;
cDataPacket *pPacket;
HOSTLIST::iterator iHost;
pPacket = NULL;
cMonitor::MutexOn();
// Is there any queued, ordered data?
for( iHost = d_hosts.begin(); iHost != d_hosts.end(); ++iHost )
{
pPacket = (*iHost)->GetInQueue().GetPacket();
if( pPacket != NULL )
break;
}
cMonitor::MutexOff();
unsigned short length;
length = 0;
if( pPacket != NULL )
{
length = pPacket->d_length > maxLen ? maxLen : pPacket->d_length;
memcpy( pBuffer, pPacket->d_data, length );
delete pPacket;
*pHostID = (*iHost)->GetAddress();
}
return length;
}
See how I deal with pPacket copying into pBuffer after I release the mutex? This is an opportunity to reinforce a very important point: Hold on to a mutex for as little time as possible. A perfect example: Before I had a monitor class my network class had one mutex. Naturally, it was being held by one thread or another for vast periods of time (20ms!), and it was creating the same delay effect as when I was only using one thread. Boy, was my face black and blue (mostly from hitting it against my desk in frustration).
Finally! Code first, explanation later.
void MTUDP::ReliableSendTo( const char * const pStr, unsigned short length,
HOSTHANDLE hostID )
{
if( d_bSending == false )
throw cError( "MTUDP::ReliableSendTo() - Sending not initialized!" );
cHost *pHost;
cMonitor::MutexOn();
pHost = d_hostMap[ hostID ];
if( pHost == NULL )
throw cError( "MTUDP::ReliableSendTo() - Invalid parameters." );
char outBuffer[ MAX_UDPBUFFERSIZE ];
unsigned short count;
DWORD packetID;
count = 0;
memset( outBuffer, 0, MAX_UDPBUFFERSIZE );
// Attach the message data.
packetID = pHost->GetOutQueue().GetCurrentID();
if( pStr )
{
// Flag indicating this block is a message.
outBuffer[ count ] = MTUDPMSGTYPE_RELIABLE;
count++;
memcpy( &outBuffer[ count ], &packetID, sizeof( DWORD ) );
count += sizeof( DWORD );
memcpy( &outBuffer[ count ], &length, sizeof( unsigned short ) );
count += sizeof( unsigned short );
memcpy( &outBuffer[ count ], pStr, length );
count += length;
}
// Attach the previous message, just to ensure that it gets there.
cDataPacket secondPacket;
if( pHost->GetOutQueue().GetPreviousPacket( packetID, &secondPacket )
== true )
{
// Flag indicating this block is a message.
outBuffer[ count ] = MTUDPMSGTYPE_RELIABLE;
count++;
// Append the message
memcpy( &outBuffer[ count ], &secondPacket.d_id, sizeof( DWORD ) );
count += sizeof( DWORD );
memcpy( &outBuffer[ count ],
&secondPacket.d_length,
sizeof( unsigned short ) );
count += sizeof( unsigned short );
memcpy( &outBuffer[ count ], secondPacket.d_data, secondPacket.d_length );
count += secondPacket.d_length;
}
#if defined( _DEBUG_DROPTEST ) && _DEBUG_DROPTEST > 1
if( rand() % _DEBUG_DROPTEST != _DEBUG_DROPTEST-1)
{
#endif
// Send
SOCKADDR_IN remoteAddr;
unsigned short result;
memset( &remoteAddr, 0, sizeof( SOCKADDR_IN ) );
remoteAddr.sin_family = AF_INET;
remoteAddr.sin_addr.s_addr = htonl( pHost->GetAddress() );
remoteAddr.sin_port = htons( pHost->GetPort() );
// Send the data.
result = sendto( d_sendSocket,
outBuffer,
count,
0,
(SOCKADDR *)&remoteAddr,
sizeof( SOCKADDR ) );
if( result < count )
// turn off the mutex and throw an error - could not send all data.
if( result == SOCKET_ERROR )
// turn off the mutex and throw an error - sendto() failed.
#if defined( _DEBUG_DROPTEST )
}
#endif
if( pStr )
pHost->GetOutQueue().AddPacket( pStr, length );
cMonitor::MutexOff();
}
Since I've covered most of this before, there are only four new and interesting things.
The first is _DEBUG_DROPTEST. This function will cause a random packet to not be sent, which is equivalent to playing on a really bad network. If your game can still play on a LAN with a _DEBUG_DROPTEST as high as four, then you have done a really good job, because that's more than you would ever see in a real game.
The second new thing is sendto(). I think any logically minded person can look at the bind() code, look at the clearly named variables, and understand how sendto() works.
It may surprise you to see that the mutex is held for so long, directly contradicting what I said earlier. As you can see, pHost is still being used on the next-to-last line of the program, so the mutex has to be held in case the other thread calls MTUDP::HostDestroy(). Of course, the only reason it has to be held so long is because of HostDestroy().
The third new thing is MTUDPMSGTYPE_RELIABLE. I'll get to that a little later.
The last and most important new item is cHost::GetOutQueue(). Just like its counterpart, GetOutQueue provides access to an instance of cQueueOut, which is remarkably similar (but not identical) to cQueueIn.
class cQueueOut : public cMonitor
{
protected:
list<cDataPacket *> d_packets;
DWORD d_currentPacketID,
d_count; // number of packets added to this queue.
public:
cQueueOut();
virtual ~cQueueOut();
void Clear();
void AddPacket( const char * const pData, unsigned short len );
void RemovePacket( DWORD packetID );
bool GetPacketForResend( DWORD waitTime, cDataPacket *pPacket );
bool GetPreviousPacket( DWORD packetID, cDataPacket *pPacket );
cDataPacket *BorrowPacket( DWORD packetID );
void ReturnPacket();
DWORD GetLowestID();
bool IsEmpty();
inline DWORD GetCurrentID(); // returns d_currentPacketID.
inline DWORD GetCount(); // returns d_count.
};
There are several crucial differences between cQueueIn and cQueueOut: d_currentPacketID is the ID of the last packet sent/added to the queue; GetLowestID() returns the ID of the first packet in the list (which, incidentally, would also be the packet that has been in the list the longest); AddPacket() just adds a packet to the far end of the list and assigns it the next d_currentPacketID; and RemovePacket() removes the packet with d_id == packetID.
The four new functions are GetPacketForResend(), GetPrevious-Packet(), BorrowPacket(), and ReturnPacket(), of which the first two require a brief overview and the last two require a big warning. GetPacketForResend() checks if there are any packets that were last sent more than waitTime milliseconds ago. If there are, it copies that packet to pPacket and updates the original packet's d_lastTime. This way, if you know the ping to some other computer, then you know how long to wait before you can assume the packet was dropped. GetPreviousPacket() is far simpler; it returns the packet that was sent just before the packet with d_id == packetID. This is used by ReliableSendTo() to "piggyback" an old packet with a new one in the hopes that it will reduce the number of resends caused by packet drops.
BorrowPacket() and ReturnPacket() are evil incarnate. I say this because they really, really bend the unwritten mutex rule: Lock and release a mutex in the same function. I know I should have gotten rid of them, but when you see how they are used in the code (later), I hope you'll agree it was the most straightforward implementation. I put it to you as a challenge to remove them. Nevermore shall I mention the functions-that-cannot-be-named().
Now, about that MTUDPMSGTYPE_RELIABLE: The longer I think about MTUDPMSGTYPE_RELIABLE, the more I think I should have given an edited version of ReliableSendTo() and then gone back and introduced it later. But then a little voice says, "Hey! That's why they put ADVANCED on the cover!" The point of MTUDPMSGTYPE_RELIABLE is that it is an identifier that would be read by ProcessIncomingData(). When Process-IncomingData() sees MTUDPMSGTYPE_RELIABLE, it would call pHost->ProcessIncomingReliable(). The benefit of doing things this way is that it means I can send other stuff in the same message and piggyback it just like I did with the old messages and GetPreviousPacket(). In fact, I could send a message that had all kinds of data and no MTUDPMSGTYPE_RELIABLE (madness! utter madness!). Of course, in order to be able to process these different message types I'd better make some improvements, the first of which is to define all the different types.
enum eMTUDPMsgType
{
MTUDPMSGTYPE_ACKS = 0,
MTUDPMSGTYPE_RELIABLE = 1,
MTUDPMSGTYPE_UNRELIABLE = 2,
MTUDPMSGTYPE_CLOCK = 3,
MTUDPMSGTYPE_NUMMESSAGES = 4,
};
I defined this enum in MTUDP.cpp because it's a completely internal matter that no other class should be messing with.
Although you're not going to work with most of these types (just yet) here's a brief overview of what they're for:
MTUDPMSGTYPE_CLOCK is for a really cool clock I'm going to add later. "I'm sorry, did you say cool?" Well, okay, it's not cool in a Pulp Fiction/Fight Club kind of cool, but it is pretty neat when you consider that the clock will read almost exactly the same value on all clients and the server. This is a critical feature of real-time games because it makes sure that you can say "this thing happened at this time" and everyone can correctly duplicate the effect.
MTUDPMSGTYPE_UNRELIABLE is an unreliable message. When a computer sends an unreliable message it doesn't expect any kind of confirmation because it isn't very concerned if the message doesn't reach the intended destination. A good example of this would be the update messages in a game—if you're sending 20 messages a second, a packet drop here and a packet drop there is no reason to have a nervous breakdown. That's part of the reason we made _DEBUG_DROPTEST in the first place!
MTUDPMSGTYPE_ACKS is vital to reliable message transmission. If my computer sends a reliable message to your computer, I need to get a message back saying "yes, I got that message!" If I don't get that message, then I have to resend it after a certain amount of time (hence GetPacketForResend()).
Now, before I start implementing the stuff associated with eMTUDPMsgType, let me go back and improve MTUDP::ProcessIncomingData().
assert( pHost != NULL );
// Process the header for this packet.
bool bMessageArrived;
unsigned char code;
char *ptr;
bMessageArrived = false;
ptr = pData;
while( ptr < pData + length )
{
code = *ptr;
ptr++;
switch( code )
{
case MTUDPMSGTYPE_ACKS:
// Process any ACKs in the packet.
ptr += pHost->ProcessIncomingACKs( ptr,
pData + length - ptr,
receiveTime );
break;
case MTUDPMSGTYPE_RELIABLE:
bMessageArrived = true;
// Process reliable message in the packet.
ptr += pHost->ProcessIncomingReliable( ptr,
pData + length - ptr,
receiveTime );
break;
case MTUDPMSGTYPE_UNRELIABLE:
// Process UNreliable message in the packet.
ptr += pHost->ProcessIncomingUnreliable( ptr,
pData + length - ptr,
receiveTime );
break;
case MTUDPMSGTYPE_CLOCK:
ptr += ProcessIncomingClockData( ptr,
pData + length - ptr,
pHost,
receiveTime );
break;
default:
// turn mutex off, throw an error. something VERY BAD has happened,
// probably a write to bad memory (such as to an uninitialized
// pointer).
break;
}
}
cMonitor::MutexOff();
if( bMessageArrived == true )
{
// Send an ACK immediately. If this machine is the
// server, also send a timestamp of the server clock.
ReliableSendTo( NULL, 0, pHost->GetAddress() );
}
}
So ProcessIncomingData() reads in the message type then sends the remaining data off to be processed. It repeats this until there's no data left to be processed. At the end, if a new message arrived, it calls Reliable-SendTo() again. Why? Because I'm going to make more improvements to it!
// some code we've seen before
memset( outBuffer, 0, MAX_UDPBUFFERSIZE );
// Attach the ACKs.
if( pHost->GetInQueue().GetCount() != 0 )
{
// Flag indicating this block is a set of ACKs.
outBuffer[ count ] = MTUDPMSGTYPE_ACKS;
count++;
count += pHost->AddACKMessage( &outBuffer[ count ], MAX_UDPBUFFERSIZE );
}
count += AddClockData( &outBuffer[ count ],
MAX_UDPBUFFERSIZE - count,
pHost );
// some code we've seen before.
So now it is sending clock data, ACK messages, and as many as two reliable packets in every message sent out. Unfortunately, there are now a number of outstanding issues:
ProcessIncomingUnreliable() is all well and good, but how do you send unreliable data?
How do cHost::AddACKMessage() and cHost::ProcessingIncoming-ACKs() work?
Ok, so I ACK the messages. But you said I should only resend packets if I haven't received an ACK within a few milliseconds of the ping to that computer. So how do I calculate ping?
How do AddClockData() and ProcessIncomingClockData() work?
Unfortunately, most of those questions have answers that overlap, so I apologize in advance if things get a little confusing.
Remember how I said there were four more classes to be defined? The class cQueueOut was one and here come two more.
class cUnreliableQueueIn : public cMonitor
{
list<cDataPacket *> d_packets;
DWORD d_currentPacketID;
public:
cUnreliableQueueIn();
virtual ~cUnreliableQueueIn();
void Clear();
void AddPacket( DWORD packetID,
const char * const pData,
unsigned short len,
DWORD receiveTime );
cDataPacket *GetPacket();
};
class cUnreliableQueueOut : public cMonitor
{
list<cDataPacket *> d_packets;
DWORD d_currentPacketID;
unsigned char d_maxPackets,
d_numPackets;
public:
cUnreliableQueueOut();
virtual ~cUnreliableQueueOut();
void Clear();
void AddPacket( const char * const pData, unsigned short len );
bool GetPreviousPacket( DWORD packetID, cDataPacket *pPacket );
void SetMaxPackets( unsigned char maxPackets );
inline DWORD GetCurrentID(); // returns d_currentPacketID.
};
They certainly share a lot of traits with their reliable counterparts. The two differences are that I don't want to hang on to a huge number of outgoing packets, and I only have to sort incoming packets into one list. In fact, my unreliable packet sorting is really lazy—if the packets don't arrive in the right order, the packet with the lower ID gets deleted. As you can see, cQueueOut has a function called SetMaxPackets() so you can control how many packets are queued. Frankly, you'd only ever set it to 0, 1, or 2.
Now that that's been explained, let's look at MTUDP::Unreliable-SendTo(). UnreliableSendTo() is almost identical to ReliableSendTo(). The only two differences are that unreliable queues are used instead of the reliable ones and the previous packet (if any) is put into the outBuffer first, followed by the new packet. This is done so that if packet N is dropped, when packet N arrives with packet N+1, my lazy packet queuing won't destroy packet N.
Aside from these two functions, there's a few other things that have to be added to cHost with regard to ACKs.
#define ACK_MAXPERMSG 256
#define ACK_BUFFERLENGTH 48
class cHost : public cMonitor
{
protected:
// A buffer of the latest ACK message for this host
char d_ackBuffer[ ACK_BUFFERLENGTH ];
unsigned short d_ackLength; // amount of the buffer actually used.
void ACKPacket( DWORD packetID, DWORD receiveTime );
public:
unsigned short ProcessIncomingACKs( char * const pBuffer,
unsigned short len,
DWORD receiveTime );
unsigned short AddACKMessage( char * const pBuffer, unsigned short
maxLen );
}
The idea here is that I'll probably be sending more ACKs than receiving packets, so it only makes sense to save time by generating the ACK message when required and then using a cut and paste. In fact, that's what AddACKMessage() does—it copies d_ackLength bytes of d_ackBuffer into pBuffer. The actual ACK message is generated at the end of cHost::Process-IncomingReliable(). Now you'll finally learn what cQueueIn::d_count, cQueueIn::GetHighestID(), cQueueIn::GetCurrentID(), and cQueueIn:: UnorderedPacketIsQueued() are for.
// some code we've seen before.
d_inQueue.AddPacket( packetID, (char *)readPtr, length, receiveTime );
readPtr += length;
// Should we build an ACK message?
if( d_inQueue.GetCount() == 0 )
return ( readPtr - pBuffer );
// Build the new ACK message.
DWORD lowest, highest, ackID;
unsigned char mask, *ptr;
lowest = d_inQueue.GetCurrentID();
highest = d_inQueue.GetHighestID();
// Cap the highest so as not to overflow the ACK buffer
// (or spend too much time building ACK messages).
if( highest > lowest + ACK_MAXPERMSG )
highest = lowest + ACK_MAXPERMSG;
ptr = (unsigned char *)d_ackBuffer;
// Send the base packet ID, which is the
// ID of the last ordered packet received.
memcpy( ptr, &lowest, sizeof( DWORD ) );
ptr += sizeof( DWORD );
// Add the number of additional ACKs.
*ptr = highest - lowest;
ptr++;
ackID = lowest;
mask = 0x80;
while( ackID < highest )
{
if( mask == 0 )
{
mask = 0x80;
ptr++;
}
// Is there a packet with id 'i' ?
if( d_inQueue.UnorderedPacketIsQueued( ackID ) == true )
*ptr |= mask; // There is
else
*ptr &= ~mask; // There isn't
mask >>= 1;
ackID++;
}
// Record the amount of the ackBuffer used.
d_ackLength = ( ptr - (unsigned char *)d_ackBuffer)+( mask != 0 );
// return the number of bytes read from
return readPtr - pBuffer;
}
For those of you who don't dream in binary (wimps), here's how it works. First of all, you know the number of reliable packets that have arrived in the correct order. So telling the other computer about all the packets that have arrived since last time that are below that number is just a waste of bandwidth. For the rest of the packets, I could have sent the IDs of every packet that has been received (or not received), but think about it: Each ID requires 4 bytes, so storing, say, 64 IDs would take 256 bytes! Fortunately, I can show you a handy trick:
// pretend ackBuffer is actually 48 * 8 BITS long instead of 48 BYTES.
for(j=0;j< highest - lowest; j++ )
{
if( d_inQueue.UnorderedPacketIsQueued( j + lowest ) == true )
ackBuffer[j] == 1;
else
ackBuffer[j] == 0;
}
Even if you used a whole character to store a 1 or a 0 you'd still be using one-fourth the amount of space. As it is, you could store those original 64 IDs in 8 bytes, eight times less than originally planned.
The next important step is cHost::ProcessIncomingACKs(). I think you get the idea—read in the first DWORD and ACK every packet with a lower ID that's still in d_queueOut. Then go one bit at a time through the rest of the ACKs (if any) and if a bit is 1, ACK the corresponding packet. So I guess the only thing left to show is how to calculate the ping using the ACK information.
void cHost::ACKPacket( DWORD packetID, DWORD receiveTime )
{
cDataPacket *pPacket;
pPacket = d_outQueue.BorrowPacket( packetID );
if( pPacket == NULL )
return; // the mutex was not locked.
DWORD time;
time = receiveTime - pPacket->d_firstTime;
d_outQueue.ReturnPacket();
unsigned int i;
if( pPacket->d_timesSent == 1 )
{
for(i=0;i< PING_RECORDLENGTH - 1; i++ )
d_pingLink[i]= d_pingLink[i+1];
d_pingLink[i]= time;
}
for(i=0;i< PING_RECORDLENGTH - 1; i++ )
d_pingTrans[i]= d_pingTrans[i+1];
d_pingTrans[i]= time;
d_outQueue.RemovePacket( packetID );
}
In classic Hollywood style, I've finally finished one thing just as I open the door and introduce something else. If you take a good look at cHost::ACKPacket() you'll notice the only line that actually does anything to ACK the packet is the last one! Everything else helps with the next outstanding issue: ping calculation.
There are two kinds of ping: link ping and transmission latency ping. Link ping is the shortest possible time it takes a message to go from one computer and back, the kind of ping you would get from using a ping utility (open a DOS box, type "ping [some address]" and see for yourself). Transmission latency ping is the time it takes two programs to respond to each other. In this case, it's the average time that it takes a reliably sent packet to be ACKed, including all the attempts to resend it.
In order to calculate ping for each cHost, the following has to be added:
#define PING_RECORDLENGTH 64
#define PING_DEFAULTVALLINK 150
#define PING_DEFAULTVALTRANS 200
class cHost : public cMonitor
{
protected:
// Ping records
DWORD d_pingLink[ PING_RECORDLENGTH ],
d_pingTrans[ PING_RECORDLENGTH ];
public:
float GetAverageLinkPing( float percent );
float GetAverageTransPing( float percent );
}
As packets come in and are ACKed their round trip time is calculated and stored in the appropriate ping record (as previously described). Of course, the two ping records need to be initialized and that's what PING_DEFAULTVALLINK and PING_DEFAULTVALTRANS are for. This is done only once, when cHost is created. Picking good initial values is important for those first few seconds before a lot of messages have been transmitted back and forth. Too high or too low and GetAverage…Ping() will be wrong, which could temporarily mess things up.
Since both average ping calculators are the same (only using different lists), I'll only show the first, GetAverageLinkPing(). Remember how in the cThread class I showed you a little cheat with cThreadProc()? I'm going to do something like that again.
// This is defined at the start of cHost.cpp for qsort. static int sSortPing( const void *arg1, const void *arg2 ) { if( *(DWORD *)arg1 < *(DWORD *)arg2 ) return -1; if( *(DWORD *)arg1 > *(DWORD *)arg2 ) return 1; return 0; } float cHost::GetAverageLinkPing( float bestPercentage ) { if( bestPercentage <= 0.0f || bestPercentage > 100.0f ) bestPercentage = 100.0f; DWORD pings[ PING_RECORDLENGTH ]; float sum, worstFloat; int worst, i; // Recalculate the ping list memcpy( pings, &d_pingLink, PING_RECORDLENGTH * sizeof( DWORD ) ); qsort( pings, PING_RECORDLENGTH, sizeof( DWORD ), sSortPing ); // Average the first bestPercentage / 100. worstFloat = (float)PING_RECORDLENGTH * bestPercentage / 100.0f; worst = (int)worstFloat+(( worstFloat - (int)worstFloat ) != 0 ); sum = 0.0f; for(i=0;i< worst; i++ ) sum += pings[ i ]; return sum / (float)worst; }
The beauty of this seemingly overcomplicated system is that you can get an average of the best n percent of the pings. Want an average ping that ignores the three or four worst cases? Get the best 80%. Want super accurate best times? Get 30% or less. In fact, those super accurate link ping times will be vital when I answer the fourth question: How do AddClockData() and ProcessIncomingClockData() work?
There's only one class left to define and here it is.
class cNetClock : public cMonitor
{
protected:
struct cTimePair
{
public:
DWORD d_actual, // The actual time as reported by GetTickCount()
d_clock; // The clock time as determined by the server.
};
cTimePair d_start, // The first time set by the server.
d_lastUpdate; // the last updated time set by the server.
bool d_bInitialized; // first time has been received.
public:
cNetClock();
virtual ~cNetClock();
void Init();
void Synchronize( DWORD serverTime,
DWORD packetSendTime,
DWORD packetACKTime,
float ping );
DWORD GetTime() const;
DWORD TranslateTime( DWORD time ) const;
};
The class cTimePair consists of two values: d_actual (which is the time returned by the local clock) and d_clock (which is the estimated server clock time). The value d_start is the clock value the first time it is calculated and d_lastUpdate is the most recent clock value. Why keep both? Although I haven't written it here in the book, I was running an experiment to see if you could determine the rate at which the local clock and the server clock would drift apart and then compensate for that drift.
Anyhow, about the other methods. GetTime() returns the current server clock time. TranslateTime will take a local time value and convert it to server clock time. Init() will set up the initial values and that just leaves Synchronize().
void cNetClock::Synchronize( DWORD serverTime,
DWORD packetSendTime,
DWORD packetACKTime,
float ping )
{
cMonitor::MutexOn();
DWORD dt;
dt = packetACKTime - packetSendTime;
if( dt > 10000 )
// this synch attempt is too old. release mutex and return now.
if( d_bInitialized == true )
{
// if the packet ACK time was too long OR the clock is close enough
// then do not update the clock.
if( abs( serverTime+(dt/2)- GetTime() ) <= 5 )
// the clock is already very synched. release mutex and return now.
d_lastUpdate.d_actual = packetACKTime;
d_lastUpdate.d_clock = serverTime + (DWORD)( ping/2);
d_ratio = (double)( d_lastUpdate.d_clock - d_start.d_clock ) /
(double)( d_lastUpdate.d_actual - d_start.d_actual );
}
else // d_bInitialized == false
{
d_lastUpdate.d_actual = packetACKTime;
d_lastUpdate.d_clock = serverTime+(dt/2);
d_start.d_actual = d_lastUpdate.d_actual;
d_start.d_clock = d_lastUpdate.d_clock;
d_bInitialized = true;
}
cMonitor::MutexOff();
}
As you can see, Synchronize() requires three values: serverTime, packetSendTime, and packetACKTime. Two of the values seem to make good sense—the time a packet was sent out and the time that packet was ACKed. But how does serverTime fit into the picture? For that I have to add more code to MTUDP.
class MTUDP : public cThread
{
protected:
bool d_bIsServerOn,
d_bIsClientOn;
cNetClock d_clock;
unsigned short AddClockData( char * const pData,
unsigned short maxLen,
cHost * const pHost );
unsigned short ProcessIncomingClockData( char * const pData,
unsigned short len,
cHost * const pHost,
DWORD receiveTime );
public:
void StartServer();
void StopServer();
void StartClient();
void StopClient();
// GetClock returns d_clock and returns a const ptr so
// that no one can call Synchronize and screw things up.
inline const cNetClock &GetClock();
}
All the client/server stuff you see here is required for the clock and only for the clock. In essence, what it does is tell MTUDP who is in charge and has the final say about what the clock should read. When a client calls AddClockData() it sends the current time local to that client, not the server time according to the client. When the server receives a clock time from a client it stores that time in cHost. When a message is going to be sent back to the client, the server sends the last clock time it got from the client and the current server time. When the client gets a clock update from the server it now has three values: the time the message was originally sent (packetSendTime), the server time when a response was given (serverTime), and the current local time (packetACKTime). Based on these three values the current server time should be approximately cNetClock::d_lastUpdate.d_clock = serverTime + ( packetACKTime – packetSendTime)/2.
Of course, you'd only do this if the total round trip was extremely close to the actual ping time because it's the only way to minimize the difference between client net clock time and server net clock time.
As I said, the last client time has to be stored in cHost. That means one final addition to cHost.
class cHost : public cMonitor
{
protected:
// For clock synchronization
DWORD d_lastClockTime;
bool d_bClockTimeSet;
public:
DWORD GetLastClockTime(); // self-explanatory.
void SetLastClockTime( DWORD time ); // self-explanatory.
inline bool WasClockTimeSet(); // returns d_bClockTimeSet.
}
And that appears to be that. In just about 35 pages I've shown you how to set up all the harder parts of network game programming. In the next section I'll show you how to use the MTUDP class to achieve first-rate, super-smooth game play.