New Network / Stream Room System v2

Get the latest news on new features, bug fixes and version updates right here!

New Network / Stream Room System v2

Postby Doidel » Fri Mar 14, 2014 9:02 am

Attention: This is not part of the official isogenic engine yet. For a fully stable stream version refer to the current master build.

Short list of stream changes:

  • allow an entity to be part of multiple stream rooms
  • add a component that provides an easy-to-use way of stream distance/level control (i.e. don't send stream updates to entities far away)
  • higher stream concatenation (larger packet sizes - read further down)
  • more stream data checks (= lower bandwidth usage)
  • other performance stuff


Why did you do that?

So far...
  • ... an entity could only be in one stream room
  • ... even with streams a network request was streamed for every entity, leading to clients * entities network emits each interval
  • ... stream data was gathered every single tick, leading to a large unnecessary server load
  • ... data comparison (should the stream send the data or not?) was only done after all the data was gathered, on the concatenated string (all the entity's data was concatenated into one single string), every tick (server load). And if you got tons of data and a single property was changed, you automatically streamed every single property to the client, even those which weren't changed.


The advantage of having higher stream concatenation

I don't quite know whether a character over websockets is transferred as ASCII or a UTF-16 char. I'll assume 2 bytes per character.
According to http://sd.wareonearth.com/~phil/net/overhead/ one packet has around 90 bytes overhead, plus the websocket overhead of 2 bytes. The MTU of usual ethernet is 1500 bytes.
We should take advantage of that, because every single packet you send over the websocket costs you 90 bytes. If you want to have many players this adds up extremely. You can do your own calculations with these numbers, but in my instance the performance changes + the level room system saved me a lot.


What are several stream rooms good for?

Whenever you want an entity to stream to some rooms and (therefore) to some clients, but not to all.

Examples:
  • Poker: Every player should see their own cards but not those of the other players. The spectator though can see all cards.
  • Invisibility: A player is buffed to be invisible for a short duration. The own team still sees him, the enemy team doesn't.
  • Distance streaming: Players too far away should not be streamed to the client (useless, just consumes resources). The IgeLevelRoomComponent can help you there.
  • Colors! Entities can have the rooms "red", "green", "blue", "purple", "yellow" and a player can only see two colors at once!
  • You get the idea. Tell me about other real-world examples if you know any.

Does the revamp break existing code?
Barely, you have to replace igeEntity's setStreamRoom to setStreamRooms ("s" at the end). And depending on your scenario you might have to add a third parameter "bypassTimeStream" to your "streamSectionData" function & prototype call. It's sometimes needed by the engine.
Yet be aware that by doing that alone you do not use the full potential of the stream room system, though it might save you a lot of bandwidth already.

What are the API changes?
  • Added class "IgeLevelRoomComponent"
  • IgeEntity.setStreamRooms(roomIds) is now plural ("s" at the end) and accepted parameters are a string (1 room id), a name (several room ids) or 'inherit' to inherit the roomIds from its parent
  • IgeEntity.setStreamSectionComparison(sectionId, funcOrFalse)
  • Call ige.network.clientJoinRoom(socket.id, 'yourRoom'); in the server's _onPlayerConnect


How do I use the the new stream room optimally?

We will use a custom class "Player.js" for this example. The player is one character on a map (in my case 3D, in your case maybe 2.5D). Player.js will be using the new stream features and the level room system.

In the player's constructor we set some values, check for streamCreateData, initialize the stream sections and add the IgeLevelRoomComponent.
That would already suffice to make the stream room system work. You do not need more than that!

But! We added something more!
We want to update the player's health on the client every now and then. Until now, for achieving this, you would probably have used the ige.network.send commands to communicate between server and client. That still works and is all fine, don't get me wrong! Yet one of the strenghts of the new stream room system is the concatenation and not having to transmit everything in single packages.
So!
Instead of using the "send-it-over-directly-no-matter-the-costs"-ige.network.send we will streeeeeaaammm the function call onto the player!
Be aware that you'll have the usual stream delay of the stream interval you set. Yet what you gain is a relieved network, and larger packet sizes!

For this purpose we introduce the streamActions. Those are like Remote Procedure Calls (RPC). We use these streamActions to update our player's health in _updateHealth. What it does step by step is:
  • On the server, call addStreamData with the id 'updateHealth' and the current health
  • On the server, when a stream interval happens the engine calls our "streamSectionData" with "updateHealth"
  • Since it's a stream action it is gathered for us by calling this._getJSONStreamActionData(sectionId)
  • On the client, the engine calls our "streamSectionData" with "updateHealth"!
  • For every "updateHealth" our "this._updateHealth(parseFloat(data));" is executed. We have just one though so it's called only once. (with addStreamData's "keepOld" we would've kept all previous "updateHealth"s which weren't streamed yet).
  • On the client, we set this.values.health to the health we got from "streamSectionData"!


Further we also want to stream runDirection. And we want the engine to handle that one, and just stream it whenever it changed. Easy enough, we added it to the "streamSections" and have some section logic for it in "streamSectionData". Aaaaand that's it. No "addStreamData" this time.

Code: Select all
var Player = IgeEntity.extend({
    classId: 'Player',

    init: function (id) {
        IgeEntity.prototype.init.call(this);

        var self = this;

      // initialize the runDirection state and the some health values
      this.states = {
            runDirection: 0
        };
      
        this.values = {
            health: 300,
            maxhealth: 300
        };
      
      
      // If we use streamCreate (a great thing to set some initial properties) we'll check it here

        if (!ige.isServer && typeof(id) == 'object') {
            var createData = id;
            id = id.id;
            this.values.health = createData.health;
        }

        if (id) {
            this.id(id);
        }

      
        //contains data and actions which have to be streamed to the client, e.g.
        this._streamActions = {};

        // Define the data sections that will be included in the stream
        this._streamActionSections = ['updateHealth'];
        this.streamSections(['transform', 'runDirection'].concat(this._streamActionSections));
      
        if (ige.isServer) {
         // On the server let's add the level room component!
         // We want to check every 500 miliseconds whether the player is in a new room,
         // and we want one room to have a size of 20x20.
         // Remember: The player is in the center room but also sees 8 rooms around him,
         // so in our case he/she'll see an are of 60x60
            self.addComponent(IgeLevelRoomComponent, {
                networkLevelRoomCheckInterval: 500,
                networkLevelRoomSize: 20
            // clientId: You can set the client id which correspons with this player manually.
            // However in my case the player's id is exactly the clientId. That's because I
            // do the following in ServerNetworkEvents.js (_onPlayerEntity):
            // new Player(clientId).streamMode(1).mount(ige.server.scene1);
            });
        }
    },

    /**
     * Override the default IgeEntity class streamSectionData() method
     * so that we can check for the custom1 section and handle how we deal
     * with it.
     * @param {String} sectionId A string identifying the section to
     * handle data get / set for.
     * @param {*=} data If present, this is the data that has been sent
     * from the server to the client for this entity.
    * @param {Boolean=} bypassTimeStream If true, will assign transform
    * directly to entity instead of adding the values to the time stream.
     * @return {*}
     */
    streamSectionData: function (sectionId, data, bypassTimeStream) {
        // Check if the section is one that we are handling
        if(sectionId == 'runDirection'){
            if(!data){
                if(ige.isServer){
               // If we're on the server and no data is given the server requests
               // data. We will grant this request.
               // With the new stream system this value is streamed ONLY if
               // it differs from the previously streamed value! If you do not
               // want that, call this.setStreamSectionComparison('runDirection', false);
               // and it will be streamed all the time.
               // Instead of "false" you can also add a function to compare the old and
               // new values yourself!
                    return this.states.runDirection;
                } else {
                    return;
                }
            } else {
            // since the data comes from JSON it's a string as of yet.
            data = parseInt(data);
            // now set the client's runDirection state to the one from the server.
            this.states.runDirection = data;
            }
        }
      // Is the section one of our streamActions?
      else if (this._streamActionSections.indexOf(sectionId) != -1) {
            if (!data) {
                if (ige.isServer) {
                    return this._getJSONStreamActionData(sectionId);
                } else {
                    return;
                }
            }
            var dataArr = JSON.parse(data);
            for (var dataId in dataArr) {
                data = dataArr[dataId];

                //execute section handlers
                if (sectionId == 'updateHealth') {
                        this._updateHealth(parseFloat(data));
                }
            else if (sectionId == 'anotherStreamAction') {
                    //do something with data
                }
            }
        } else {
            // The section was not one that we handle here, so pass this
            // to the super-class streamSectionData() method - it handles
            // the "transform" section by itself
            return IgeEntity.prototype.streamSectionData.call(this, sectionId, data, bypassTimeStream);
        }
    },

   /**
    * Return a stringified version of a streamAction property
    * @param {String} sectionId A string identifying the action section
    * @return {String=} JSON string or undefined
    */
    _getJSONStreamActionData: function(property) {
        if (this._streamActions.hasOwnProperty(property) && this._streamActions[property] != undefined) {
            var data = this._streamActions[property];
            delete this._streamActions[property];
            return JSON.stringify(data);
        }
    },

    //
   /**
    * Called when a player is first created on a client through the stream.
    *
    * From ige docs:
    * Override this method if your entity should send data through to
    * the client when it is being created on the client for the first
    * time through the network stream. The data will be provided as the
    * first argument in the constructor call to the entity class so
    * you should expect to receive it as per this example:
    *
    * Valid return values must not include circular references!
    */
    streamCreateData: function() {
        return {
            id: this.id(),
            health: this.values.health
        }
    },

   /**
    * Return a stringified version of a streamAction property
    * @param {String} id A string identifying the action section
    * @param {String} data Any sort of data. Can be string, array, object, whatever.
    * @param {String} keepOld Per default this method overrides previously set
    * data before the actual stream happened (the stream action data is emptied
    * everytime the data was gathered, i.e. after every data stream). If you want
    * the commands to stack up and not only one but all of them should be streamed,
    * use "keepOld".
    */
    addStreamData: function(id, data, keepOld) {
        if (keepOld === true && typeof(this._streamActions[id]) == 'object') {
            this._streamActions[id].push(data);
        } else {
            this._streamActions[id] = [data];
        }
    },
   
    /**
     * Can be called for manually synchronizing health.
     * @private
     */
    _updateHealth: function(health) {
        if (!ige.isServer) {
         this.values.health = health;
            // and maybe do some UI stuff
        }
        /* CEXCLUDE */
        else {
         //send update to all clients
         this.addStreamData('updateHealth', this.values.health);
        }
        /* CEXCLUDE */
    }
});

if (typeof(module) !== 'undefined' && typeof(module.exports) !== 'undefined') { module.exports = Player; }


Furthermore you can set the initial client rooms in _onPlayerConnect, you can set manual (or no) "old to new data"-comparisons per section, you can attach an entity to a player's rooms, etc. Look through the code, check out the API changes, figure out what suits you best :)
User avatar
Doidel
 
Posts: 59
Joined: Tue Jan 14, 2014 5:05 pm

Re: New Network / Stream Room System v2

Postby Smileypop » Sat Mar 15, 2014 11:21 pm

Hey Doidel,

Thanks for the update! I have pulled the branch and already it has improved things. The sample Player.js was really helpful.

I do have a few questions.

1) Is sending entity updates via stream better than using network.send? An example would be when a player clicks on a ship and damages it. Is streaming the ship._damage as a sectionId better than sending a network message to all players that the ship was damaged?

2) What is the real difference between a streamSection and streamAction? I looks like stream actions are called at intervals or possibly by user action. But since the new streamSections should always be checking for updated data before sending, I'm not sure why a streamAction is needed?

3) The room size and surrounding rooms connection is still a bit confusing. Does a room with size 20x20 refer to the size of the tilemap? Also, why does a player need to connect to surrounding rooms, instead of specific rooms? At least in my game, a player is always only in 1 room at a time, where 1 room = 1 level. As soon as the level finishes, they are booted from that room and connected to a new room / level.

Thanks again for the code update and sample. It's been really useful.
Smileypop
 
Posts: 28
Joined: Fri Dec 13, 2013 6:44 am

Re: New Network / Stream Room System v2

Postby Doidel » Sun Mar 16, 2014 12:16 pm

Hello Smileypop

1) For a ship getting damage there are two advantages with spreading the information over the stream system. Firstly, you can have it being done automatically e.g. with the "this.values.health". You would not have to implement any logic and the stream system would send an update to the clients only when the health has changed.
Secondly, with sending the update over network.send directly you send 1 update as a package, with all the overhead. With the stream tough it would send up to X updates, all which are happening in this specific time interval. So in your package you would not only have the health update, but also the transform data, the ammo, the energy status, and some stream actions. That can save you a lot of unnecessary bandwith cost.

2) Stream actions are remote procedure calls. In our "values.health" case the stream sends an update automatically when something has changed. With stream actions however something is sent only if you previously used "addStreamData".
That makes sense e.g. when:
  • You have data that is updated very frequently on the server but it's not necessary that the player is updated as often. (possible example: Energy regeneration)
  • You have a method you want to execute on the client which has a lot of parameters and is called seldomly. (possible example: Teleportation (where to? Energy drained? Damage received?)
  • You want to have several "data packages" with the same id per stream update. (The way we have it now e.g. our "values.health" is simply written over on the server, and the client won't know that he once had that non streamed health value)
In general, I tend to use the stream actions for seldom and individual method calls, and the usual stream sections for stats updates. This feels "cleanest" to me. You can solve either with only one of them though, it's up to you whether you want to use it.

3) My explanations will refer to the image below

Image

We have now 3 Player.js' in an open field. Or one level. Or whever, it doesn't matter, it's neither related to tiles nor game levels nor anything. Each one of those gray boxes relates to a "LevelRoom" with the dimensions we defined, in our case: 20x20.
Now let's say the whole image is one of your game levels. Currently 3 players are in there. We want the players to be able to see the players around him, but not really those far away. If every player could see every other one and we had, let's say, 100 players, then we would have to update every player about the 99 other players and himself. That makes 10'000 stream updates every update interval. Yet if the upper "see-only-neighbours" rule is in place that number drops dramatically.
In the upper image P1 streams to his LevelRoom and the 8 LevelRooms around him. P2 and P3 the same. This way P1 gets the stream updates from P3, and P3 gets the stream updates from P1. But neither P1 nor P3 share a neighbouring tile with P2, so none of them sees P2.

Why 9 rooms and not one?
P1 and P3 could then both stand in the very edges of their LevelRoom, literally 1 step away from each other, and wouldn't be able to see one another.


Hope this helps. Don't hesitate to tell me if something remains unclear :)
User avatar
Doidel
 
Posts: 59
Joined: Tue Jan 14, 2014 5:05 pm

Re: New Network / Stream Room System v2

Postby Smileypop » Sun Mar 16, 2014 11:19 pm

Those explanations are great, thanks.

My last question would be about the 9x9 room system then. Since the players will be moving around the map, the rooms they are attached to would constantly change? How do you update the "joined" rooms compared to the map location? If player 1 moves from room index (1,2) to (1,3) for example, he is now in rooms (0,2) (0,3) (0,4) (1,2) (1,3) (1,4) (2,2) (2,3) (2,4) for a total of 9 rooms. In your Player.js example, can you show how the leave/join of rooms should be handled when the player moves?

Thanks!
Smileypop
 
Posts: 28
Joined: Fri Dec 13, 2013 6:44 am

Re: New Network / Stream Room System v2

Postby Doidel » Mon Mar 17, 2014 5:10 pm

Yes, they constantly change.
The leaving and joining rooms is done automatically. In the LevelRoomComponent I have a behaviour (executed every tick) which checks whether your position changed, something like:
Code: Select all
var pos = [entity.position.x % 20, entity.position.z % 20];
if pos != previouslyStoredPos
        changeRooms

For the rooms that the player left a "ige.network.clientLeaveRoom" is executed. For those which he joined to it's "ige.network.clientJoinRoom".

You are welcome to have a look at ige/engine/components/network/stream/IgeLevelRoomComponent.js, the code is short and rather self-explanatory.

Hmm I didn't check this, but is "position.y" the height in a 2.5D environment? 'cause in 3D the xz-Plane is the "ground" and y is the height.
User avatar
Doidel
 
Posts: 59
Joined: Tue Jan 14, 2014 5:05 pm

Re: New Network / Stream Room System v2

Postby Smoozilla » Mon Aug 04, 2014 11:25 pm

Hi, thanks for all your hardwork!

When was this system last considered usable? I ask because the one under stream-revamp branch has some things that don't make sense in IgeLevelRoomComponent

_behavior() is trying to access this._options to get networkLevelRoomCheckInterval, but 'this' refers to the entity the component is attached to.

I got it working with a simple fix of
Code: Select all
this.levelRoom.networkLevelRoomCheckInterval


But what's stranger, and motivated my post is _clientLevelRoomChangedCheck()

Code: Select all
roomPosY = Math.floor(this._player._translate.z / this._options.networkLevelRoomSize);


Edit: Your most recent post makes more sense to me now. But are not all Isogenic Engine maps built around x,y with z as height?

I plan on taking what you've built as a base for what I need. I'm very grateful for this :-) I was prepared to figure it out on my own, but you've made it much simpler!

One other thought, when a second entity is created from the stream, they won't appear on the first entity's screen until they move. Maybe only streaming a changed value is the cause here?
User avatar
Smoozilla
 
Posts: 10
Joined: Wed Jul 30, 2014 6:37 am

Re: New Network / Stream Room System v2

Postby Doidel » Fri Aug 08, 2014 3:09 pm

Hello Smoozilla

I haven't used it in quite a while and I'm not actively developing with ige currently. However since the engine barely changed it should still work perfectly.

About the z-translate: I used the system in a 3D context where y is the height, not the usual ige maps. So feel free to change to whatever plane you use :)

There already exists a "fix" for that. Make sure you have the latest version, and I remember something you can call/do in the client connect events. Look through the example(s), I'm sure you'll find it.
User avatar
Doidel
 
Posts: 59
Joined: Tue Jan 14, 2014 5:05 pm

Re: New Network / Stream Room System v2

Postby Smoozilla » Fri Aug 08, 2014 8:05 pm

Thanks Doidel,

I'm not that familiar with github, so finding this:

https://github.com/Doidel/igeStream

Didn't happen until I was searching through the forums for something else.

After comparing that with what I was initially using (stream-revamp branch) it seems your forked repository is the one to go with. Hopefully this link can help anyone like me that had trouble finding this.
User avatar
Smoozilla
 
Posts: 10
Joined: Wed Jul 30, 2014 6:37 am

Re: New Network / Stream Room System v2

Postby Solib » Sun Jun 14, 2015 6:11 am

Error starts all examples. Stable version work.
Code: Select all
Client gameTiles command received from client id "2bd4b9240621640" with data: {}
c:\Winginx\home\g.local\public_html\game\gameClasses\CharacterContainer.js:73
         this._streamDir = this.path.currentDirection();
                                     ^
TypeError: undefined is not a function
    at IgeClass.IgeEntity.extend.update (c:\Winginx\home\g.local\public_html\game\gameClasses\CharacterContainer.js:73:32)
    at IgeClass.IgeEventingClass.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeObject.js:1541:21)
    at IgeClass.IgeObject.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeEntity.js:1620:30)
    at IgeClass.IgeEventingClass.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeObject.js:1541:21)
    at IgeClass.IgeObject.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeEntity.js:1620:30)
    at IgeClass.IgeEntity.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeScene2d.js:127:30)
    at IgeClass.IgeEventingClass.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeObject.js:1541:21)
    at IgeClass.IgeObject.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeEntity.js:1620:30)
    at IgeClass.IgeEntity.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeScene2d.js:127:30)
    at IgeClass.IgeEntity.extend.update (c:\Winginx\home\g.local\public_html\engine\core\IgeViewport.js:143:17)

Process finished with exit code 1
Solib
 
Posts: 17
Joined: Thu Oct 24, 2013 4:27 am


Return to New Features, Latest Versions & General Updates

Who is online

Users browsing this forum: No registered users and 1 guest
cron