That is simply awesome. Would love to see the ghetto turok

Hopefully things get clear for you again really looking forward to this here and I got the source now thanks very much for that.

...
One of my next moves will be integrating Google Protocol buffers in order to handle the data exchanges...
With that I should be able to create deltas of the information per player and handle everything in much smaller packets reducing latency overall and increasing the ability to play even if it's in a 'ghetto' state of syncing at the moment

...

Hey, sorry to just pop in unannounced! I have been amassing a lot of curiosity and knowledge toward multiplayer game networking, even going so far as to reverse engineering some client/server games to create my own servers for education. Unfortunately I DON'T have a lot of knowledge reverse-engineering binary files or generally trying to figure out what the game is doing. However, if I could get in contact with you to explain things to me about how the dll works and modifies the game's behavior, I could probably help / contribute to this project in some way, if you're open to that.

Personally I don't see how protocol buffers would give you much benefit, especially not for a first person action game like turok. Protocol buffers might be a little too much luggage for something like this, I feel like a well built udp protocol with standardized read/write serialization through bitpacking seems like it would serve you better, but I don't have much experience with protobuf and you're probably more experienced than I am in this area.

Thanks for the input,
I probably don't have much more experience then you do with multiplayer networking and I feel as if protobuf would just make things easier in terms of transferring data I use it in another of my projects and from what I've read on gamedev.net it's actually kind of preferred as its fast and you can store data how ever you'd like.

The other project I'm working on (halo2) does use bitpacking and it's own custom read/write serialization, I've just never gotten too deeply into processing data like that in C/C++.

I've got stinkee on Skype feel free to add me, tddeadlydata
...

Yes, protocol buffers are fast, but I don't feel like they're quite fast enough for an action based game like a FPS. I'm not sure if the networking protocol is already defined for Turok or if you're just rolling your own in the mod, but if you're rolling your own you probably want to send player inputs at no less than 40 times a second to the server and state updates at least 30 times a second. Protocol buffers work pretty well toward serialization of data for use in things such as a small scale rpg MMO where state updates are less frequent but latency doesn't matter so much. Even at 30 state updates a second you're probably going to ultimately have at least 100ms latency for a client to see other player actions just from state interpolation(extrapolation is different but also not a good idea for an fps game) buffering before you even add in player ping.

I know of some great articles for network data serialization, you don't even have to create separate read/write serialization functions just a single "Serialize" function that reads or writes data from/to a bitpacker depending on an IsReading flag, that eliminates hard to track unaligned serialization function bugs, and it doesn't take much time to get running. I can provide you some easy to use classes to use if you like, that way you don't have to spend time setting up protocol buffers or learning a new library.

With this method your data layouts and definition would be represented via a regular ol' struct in your code, no crazy file generation or huge library to deal with.

Google protobuf isn't very hard to implement but I do understand what you're saying,
From the research I've done there's two goals whether it be FPS or MMO.

1) Deliver the data as fast as possible.
2) Deliver as small of chunks of data as possible.

That being said the idea is to calculate only the values which may have changed since the last send before sending the next packet in order to make sure the size of the packet is as small as possible which is why I agree with you on the bit packing, I'm not sure how efficient protobuf is in terms of 'compression' of the data, I just figured it'd be a decent choice considering I'm already familiar with it.

And I'd have to argue that even MMOs tend to send a shit ton of data, sometimes even more then action fps games do.
...

Yes, exactly, if you want to send very frequent small packets bit packing is your best bet, protocol buffers seem to be better for slightly larger less frequent packets, I don't know if it supports bit packing but I feel like it would just send byte values, which is a waste of bandwidth when you're sending things a lot. However, I know most MMO games don't bitpack at all due to the cpu overhead per client and that's why I said Protocol Buffers would be more suited toward an MMO type game, especially since they tend to send larger state updates less frequently.

What you're talking about is known as delta state updates, it's very common all the way back to the quake multiplayer source. It's the obvious choice for any action game to keep the packet sizes small, bandwidth usage must remain low especially on a p2p network setup. It's a bit of a challenge to make it work with the udp protocol though since if a delta state gets lost you have to send it again before processing the newer delta states to keep things in sync. With a game like turok using a client/server setup it's possible to send full entire states compressed with common compression methods and bit packing and still stay on very low bandwidth usage, avoiding the complexity overhead of delta states.

MMOs do send a lot more data, less frequently (normally only 20 times a second or even less). They also typically use the TCP protocol. In these types of games latency isn't as huge of a deal since most actions are target based and they do sometimes trust the client too much to hide it and avoid cpu costs on the server. It's okay though, they are very good at hiding this latency since it's not critical and they have a LOT of player and ai states to send around to all the clients, that's a lot of data.

With bitpacking on a udp protocol you could achieve a reliable delivery system of bitpacked full(not even delta) state updates for 16 players at 60 times a second and still remain under 256kbps per client easy(I'm way over estimating too), which is more than feasible for ANY modern or even most outdated network connections. If that somehow DID cause congestion you just crank it back to 30 times a second or so for that client until the way is clear again. If you plan on a p2p setup rather than hosted servers, things get MUCH more complicated since the upload bandwidth becomes EXTREMELY constrained and you have to fit client updates into WAY smaller packets than you otherwise would have to.

The average consumer internet connection might have high download speeds these days, but uploads are constrained a lot, typically you have less than 750kbps to work with on upload. This is bad because if you have to update 16 other players they each only get about 46kbps of data... that's terrible and you can't always expect even 750kbps since clients might have someone doing something else on their network using upload bandwidth, or have even worse upload speeds than that. The only solution here would be to lower the player count, I suspect Turok would only have 4 player multiplayer anyway, so probably not too much of a problem? Even if you wanted 16 players, it's easy in Turok's case to fit player states into 46kbps at 60 packets a second(800 bytes a packet, very easy).

My point here is that you probably want packets as small as possible, especially if it's going to be peer-to-peer games, protocol buffers might not bit pack or compress data, but if you do write your own bit packing and compressing(not as hard as it sounds) you can easily fit within the constraints.

I'll try to start communicating with you on skype to avoid clogging up this thread with technical jargon, honestly I don't think this project even requires as much thought into networking as I'm putting in to it.