[SDL] [Framework] Event-based Networking

Jared Maddox absinthdraco at gmail.com
Wed Mar 20 22:34:03 PDT 2013

> Date: Mon, 18 Mar 2013 19:27:59 -0700
> From: "Nathaniel J Fries" <nfries88 at yahoo.com>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [Framework] Event-based Networking
> Message-ID: <1363660078.m2f.36189 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"

> Mutexes are the source of the bottleneck (using binary semaphores, critical
> sections, condition variables, or spinlocks will have the same effective
> result, and the same effective problem). Mutex is short for "mutual
> exclusion", so when I said "mutually exclusive code", I literally meant
> "code that uses mutexes".
> If you don't use mutexes, you wouldn't have a bottleneck, you'd have
> something way worse -- code that compiles just fine but doesn't work at
> all!

Other than quick locking implied in even "lockless" code, I'm
challenged to think of a situation where I would want MMO server-side
networking code to have any mutual exclusion that wasn't inate in
malloc() & co (because, really, what should those threads be doing
other than reading, writing, atomic swaps to and from queues, and
reference-counting?). Since I actually have some vaguely defined
future intentions down this road, could you throw out one or two
simple examples?

> Date: Tue, 19 Mar 2013 01:54:42 -0700
> From: "Glocke" <cgloeckner at freenet.de>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [Framework] Event-based Networking
> Message-ID: <1363683281.m2f.36194 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> /EDIT:
>> Please reply to your post instead, so the mailing list will see your
>> message.
> There are some questions in the context of SDL_net:
> 	 Do I have to consider Endianness?
> 	 Should I use fix-sized-types for my events (e.g.
> http://en.cppreference.com/w/cpp/types/integer, might be important for using
> 32-bit and 64-bit systems in one communication) ?
> 	 Or should I use serialization to avoid both possible issues?

> Date: Wed, 20 Mar 2013 08:07:16 -0700
> From: "Glocke" <cgloeckner at freenet.de>
> To: sdl at lists.libsdl.org
> Subject: [SDL] [SDL_net] Endianness
> Message-ID: <1363792035.m2f.36227 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> Hi, do I have to consider byte orders while working with TCPsocket in case
> of SDLNet_TCP_Recv and SDLNet_TCP_Send. If yes: what's the way to handle
> endianness in this case?
> Kind regards
> Glocke

I'd say yes to all three, but it actually has nothing to do with
SDL_net. In fact, it applies to ALL transfer of data outside of a
program (unless you're transfering it to a fork of the current
instance, in which case just do what makes sense in your case).

1) Proper networking code should (at least in my view) always convert
outgoing data to network-order, and incoming data to host-order: and
do the same if writing to disk. Hard drives can usually be swapped out
to different machines, and a problem that apparently used to pop up
between PCs and Macs (albeit, this is massively not the only problem
that came along) is that they often used different byte-orders. By
always converting out-going data to network order, and doing the
opposite to incoming data, you ensure that byte-order will never
interfer with data transmission.

2) There is currently the question of whether to choose 32-bit or
64-bit code. For a server the correct choice will presumably always be
64-bit, but for a client the correct choice will vary widely. By using
fixed-size types you ensure that even if you change your mind later
(or if you choose one size for the server and anther for the client)
you won't break anything, because the sizes were always explicitly
specified from the start. Once again, this applies to disk I/O as

3) The details of layout of elements in a structure vary depending on
the compiler, and what options it was given. Further, structures can
quite easily have unused space littered around themselves. Thus, by
using actual serialization code you can both reduce the chances of
accidentally introducing errors and incompatibilities, AND waste less
bandwidth (which is itself important with the rise of mobile
computing). This, too, applies to disk I/O. There are cases in major
projects such as Linux where this is done, but those projects tend to
use compiler-specific methods to specify the correct layout their data

As for the correct way to implement all of this, the serialization
code calls the endian-conversion functions, and from there you're set.

By the way, this applies to everything from games, to word-processors,
to databases. It's a specialization-agnostic rule-of-thumb.

> Date: Wed, 20 Mar 2013 12:49:38 -0700
> From: "Nathaniel J Fries" <nfries88 at yahoo.com>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [SDL_net] Endianness
> Message-ID: <1363808978.m2f.36233 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> Yes, SDL_net has no knowledge of the endianness of the buffers you send and
> receive.
> Fortunately, SDL provides some excellent, optimal functions for reversing
> the order of bytes in SDL_endian.h
> As a general rule, you have three options for transmission (either file or
> network) endianness:
> 1) Assume most machines using the transmitted data will be little-endian (a
> good general rule now that everyone is using x86)
> 2) Convert to network byte order, which is big-endian.
> 3) (Less practical): Have the client tell you its endianness, and convert to
> to client endianness before sending and to native endianness after
> receiving.

Obviously, I favor option 2 ;).

> Date: Tue, 19 Mar 2013 03:34:05 -0700
> From: "Glocke" <cgloeckner at freenet.de>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [Framework] Event-based Networking
> Message-ID: <1363689244.m2f.36196 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> /EDIT2:
> Jared Maddox wrote:
>> I suggest redesigning the thread api to
>> resemble the one provided by C++11, so that your code can be quickly
>> and easily used as a replacement for the C++11 library on platforms
>> that support SDL, but don't have C++11 thread support implemented.
> Well, I would prefer wrapping my Threading around C++11-Threading. At the
> moment my framework requires a lot C++11-stuff. So using C++11-Threading
> seems meanigful to me.

What level of C++11 stuff does it actually require? If it's library
support, well, the threading portion can be implemented with a wrapper
class for SDL's threading facilities, as the code I linked to
demonstrates. If C++11 features (I'm specifically thinking of variadic
templates) is required, then I strongly suggest wrapping it in some
preprocessor ifs that test for C++11 compliance. Some workplaces
require older compilers to be used (though I would at least HOPE not
in the gaming industry), so actually relying on new features could
cause you NEAR-TERM problems (long-term, those problems will hopefully
go away).

> Date: Wed, 20 Mar 2013 01:20:16 -0700
> From: "Glocke" <cgloeckner at freenet.de>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [Framework] Event-based Networking
> Message-ID: <1363767616.m2f.36221 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> Nathaniel J Fries wrote:
>> The thread-per-client model is a bad model for any highly-interactive
>> program. Like I said, synchronization bottlenecks. The key is to do as
>> little synchronizing as possible; which means using a specific thread for
>> a specific task and minimizing thread interaction.
> My current approach is:
> - n clients are handled by 2*n worker threads (one for sending, one for
> receiving) on the server

Wow, yikes. Listen to Nathaniel. If the OS supports hooks for green
threads/fibers/etc., then you MIGHT justify allocating THOSE in the
way you just described, but networks tend to be much slower than the
computers that access them, so unless you're targeting a specific OS
where you know that the OS will do x, y, and z optimal things if you
do things u, v, and w, then you should never provide more than one
networking thread per client in ANY server, whether gaming or

A bit of an expantion on Nathaniel's explanation, specialized to x86
but relevant elsewhere as well, is this:
Each process and/or thread has certain blocks of memory that it can
see, some data that ONLY it can see (such as registers), and some data
that it can't see which the OS uses to keep track of it. Each time
that the processor switches to a different process it muist change out
the register data, and change out the data that the OS uses to
describe the process and thread. When the processor changes threads it
doesn't have to swap out as much of the OS-visible data, but it does
have to change out some (and in the case of Linux, all) of it, and it
has to change out the registers. In addition to that, the OS may need
to move large blocks of memory out to the disk, and move others back
in, in order to actually run the "new" process/thread. All of this
takes up data-transfer resources, and therefor will ALMOST GUARANTEED
result in a decrease in execution speed each time that it's done. By
doing as much network I/O in one thread as possible you reduce this
speed cost by reducing the number of process/thread swaps required by
a given set of I/O operations, and also reduce the chances of
requiring disk swaps by simply reducing the amount of data that your
program needs.

> Date: Wed, 20 Mar 2013 13:34:44 -0700
> From: "Nathaniel J Fries" <nfries88 at yahoo.com>
> To: sdl at lists.libsdl.org
> Subject: Re: [SDL] [Framework] Event-based Networking
> Message-ID: <1363811683.m2f.36237 at forums.libsdl.org>
> Content-Type: text/plain; charset="iso-8859-1"
> Jonny D wrote:
>> This could be useful,
>> actually:?http://wiki.libsdl.org/moin.fcg/SDL_GetCPUCount
>> (http://wiki.libsdl.org/moin.fcg/SDL_GetCPUCount)
>> Jonny D
> I stand corrected!
> Still, if for whatever reason SDL cannot determine the number of threads,
> SDL_GetCPUCount will return a very conservative guess of 1. Which means, if
> SDL_GetCPUCount returns 1, you should again refer to my suggested
> assumptions on the server (since any modern server *will* have at least 4
> CPUs), while sticking with SDL's guess on the client (where one and two cpu
> machines are still the most common).

It is good to provide a command-line override, though! That way you
can both use a "crutch" if needed (let's say you're using a Haiku
port, but it isn't complete; or you have something else that you want
on one of thoe cores; whatever), and (possibly more importantly) so
that you can run some tests for correctness (has Intel cancelled it's
50-core chip plan, or is it still moving forward with it ;) ).

More information about the SDL mailing list