Xiaoyu (game engine)

1

Hello everybody.

I am working on an engine entirely written in C++, and today i’ll show you the current result. There is still a lot of thing to do, but you know what it is, write a game engine is a never ending task.

First, a small video

I wrote it after months of works with JMonkeyEngine, and for this reason some aspects are similar. It’s the main reason of this topic: do you agree with that ? I wrote everything myself (except the Quaternion that i shamelessly pilfered). The similarity are: a scene graph, controls on elements, appstates and an assetsmanager. I didn’t say it in this video (cause i don’t know your opinion about it) but i’ll edit it (if i can do it. It’s my first video on youtube) to add a mention to JME if you agree. There’ll be no amiguity about your support (e.g. i’ll write something like “This engine is inspired from an other engine, JMonkeyEngine 3”).

Ok, now some tips about the engine itself:
The physic engine (you can jump etc) is made by me and is called BasicPhysicEngine for a reason: it’s not suppose to be used in production. It handle collision detection but there is no collision response (feedback torque etc).
This was the worst part of the engine. Now nice things:

Models are encoded in a home-made format, XSB (standing for eXtensible Statefull Builder) which a binary format inspired from the .obj one but improved to reduce the redunduncy of the information, make the loading easier, support local translation/rotation for every item and, in a near future, animations (i only need to find where animations are stored in blender). This will finalize the XSB+Vanilla. It the very end, it should look more like .bsa format, with blocks of informations that allow extensibility (each block announce its size, so if an implementation doesn’t support a block it can skip it). Extensibility means that i’ll not be the only one that will define block types and if artists need complexe shading with never-ever-except-once-in-a-life-used parameter i’ll not make my format dirt for that: it’ll become XSB+SpecialThingu, not related to the Xiaoyu Engine.
I plan to add script, images, sounds and some more things in it. For example it would be nice to have a “physic shape” entry.

The network is also home-made with a very political decision, but i can’t talk about it now, i need to finish it before. The main idea is: a human should stay the boss of its computer, always, and every message from the network should be treated as a hint. I hate the “server-client” approach and i think that a mailbox (physical one with two flag on it, one for “something new in the mail box”, one for “something to send in the mail box”) should be a good new approach. Also, push a “find near network” would be amazing, throwing away the mere idea of a central server who is the big boss and dumb clients etc.
Well, i told about it more than i expected …

For the gui, it’s a small thing: a nine-patch mesh and over it an other mesh with a texture containing the text. I’ll merge both later and cheat with the shader to obtain the same effect. Also, the engine support 3 fonts: ttf (with a sdl-ttf as backend) a F256 (a font that handle ascii encoding … not good, will likely disappear soon) and an other home-made format, CSF (Clear Sky Font, from an older project) that support UTF-8 encoding and it based on a picture + an xml file (you can imagine it).

The inputs are handled in a new way (home made :stuck_out_tongue: ) : in the idea, every device (mouse, keyboard, timer, in-game button, in-game door) is a statefulldevice, which is more or less … well, it’s hard to explain, but to make a long story short you can ask a statefulldevice question about its current state and about what happened since the last frame.
I also wrote a homemade handling of the gamepad, sdl wasn’t able to read it. So, i read datas from /something-i-forgot/js0 and give a meaning to it. It’s what i use in the video.

In the very end, Xiaoyu should be on its own linux distrib (XiaoyuX), specifically designed for game developpement. The idea is: too much project stay on the computer of the dev only because it’s hard to list all the library used, find absolute path, handle specific plateforms etc. This is also the reason why a console (the device) game crash less often, because devs know the target device. With in mind the idea that an OS should be an abstract layer between the hardware and the software it would be nice to have a linux distrib with a “burn image” button that will just burn the game on a playable distrib on a usb key or on a cd/dvd.

Well, there is a lot more to say, but it’s out of my mind right now.

Any comment is welcome. Also, everything rely on your validation (the projet is dead if you say that you want me to don’t use the idea of your game engine in mine).

6 Likes
2

Hi,
first, well done for all you did.
Then, i’m just an jme user but i’m wondering, why (re)build an engine who behave like jme when you can just improve jme directly ?

2 Likes
3

A reason is that the language is not the same: JME is in Java, this is in C++.
I want to build my own engine cause i want to have put in it a lot of concepts i thought about for years. For example the way inputs are handled in jme (and in sdl, qt, swing, xlib … ) is a very good one when it comes to “sleeping” software (like a text editor) but it’s not good when you do a 3D game. In a 3D game you already have a main loop and you should access elements of the scene graph only when you are in this loop. So, you can’t react to events when you receive them (through the callback of the inputs manager for example) and you have to use some boolean to store the information and it produces redunduncy in the information. Also, it means that when the game is in pause you need to not only “freeze” the update loop of these controls but you also need to prevent them from receiving inputs. And when they are deleted you need to make sure that the event emitter will no longer call them. And if you want have an item like an exploding bomb that you can trigger with a key, you can’t remove the bomb from the listeners in the callback (cause then you would be trying to remove an item while you iterate over the collection). Add to this that if a component can emit a lot of events it will have a lot of collections of listeners (look at the number of event a swing component can emit). Just for the potentiality, for the possibility to emit an event you pay the cost of several empty list in each components, making them heavy weighted for no reason.
Well, this is a quick lookup of the problems the current inputs system in jme (and in qt etc, again) has. You can solve most of these problems if only one or two element emits events, but when the number of emitter grows, the complexity grows with it quickly.

And this is just for the inputs part. I found equivalent problems in the network, the gui, the physic engine etc. So, i think that a new engine that embed not one but several brand new solutions can be interesting.

2 Likes
4

No, only when your game is in the prototype stage.

What you are talking about is polling which is extremely inefficient for things that don’t change often (like key state, for example). In a real game, 99% of the time a key press changes some velocity or acceleration. In that case there are no booleans, the key presses and releases are simply updating an acceleration vector that the main loop is using continuously.

See above… your physics/game loop would be frozen already so the acceleration wouldn’t be applied anymore. On the other hand, there really are events you’d want to continue processing… like the unpause button, or clicking on options menus, etc…

The alternative to callbacks is giant if/else statement blocks, anyway… so I have a suspicion that you will come back to callbacks at some point. And if you aren’t talking about getting rid of callbacks then it sounds like you are talking about JME’s analog listeners which call your code every frame already.

If you were a JME user and this were a JME game, I’d tell you that you need to use app states better. All of the rest of your issues can be nicely handled by implementing proper enable/disable in app states. (And using a more advanced input mapping like Lemur’s input mapper that lets you enable/disable entire event mapping sets while leaving listeners in place.)

Sorry if you weren’t looking for this kind of critical feedback… but then you did post to a ‘competing’ game engine forum about your alternative so I figure that’s what you wanted.

2 Likes
5

Hi bubuche.
Congratulation
Your enthusiasm to make your own game engine is really admirable. Keep it going . I hope a great success for you. If you have any idea that some thing in JME is wrongly implemented and if it would implemented in other way it would be better please help us to improve it in JME.
We will be glad if you let us to use your great knowledge.
Thanks.

6

While I applaud your efforts and enthusiasm; I fear your engine could be too much innovative (meaning that a lot of stuff is done “differently” than other engines). This could lead to a barrier to end users but at the same time it could be a competitive advantage.

Anyway, I think the point of it is “learning for everybody and from everybody”, rather than “competing with the likes of Unreal and Unity”, so I say go for it :+1:

There shouldn’t be any issue, as jme is BSD licensed.

1 Like
7

No problem, this is exactl what i expected. But i already thought about it: there is no need to optimize something is this thing doesn’t slow down the program. To say it differently: there is no need to optimize the inputs (with a callback system) when at most one or two hundred (and this would be A LOT) of components will listen to the inputs. It will cost one or two hundred of operation each frame, which is … hmm, ridiculous when it comes to a 3D game. I’ll put more effort in optimizing the scene graph, with a portal system for example (i.e. “if i can’t see a door of a room, i can’t see everything beyond that door”). Let’s take the keyboard as an example: when you press a key, only one component should react to this: if you are in game, the player will react to it, if you are in the menu, then the menu will react to it. There should be only afew components listening to the keyboard (i can imagine a hand-bomb listening for the “explode” action, a drone listener for movement actions and the controller of the character listening for jump/forward etc actions). Add to this that a keyboard can’t physically have moe than 4 or 5 key pressed at the same time. All of this is pretty small, so for each “is this key pressed” i have a lookup on a very small map with integers as key. Should be fast.

To make all of the above short: i disagree with your sentence “What you are talking about is polling which is extremely inefficient for things that don’t change often”.

If the system is not clear enough, a code of a control that use inputs will look like this:

void update(float tpf)
{
  if ( ! devices().keyboard().activatedThisFrame('r') )
   return;
  // do stuff
}

And if the element is itself a device (like a door), you can access its variable through methods like “realOf”, “intOf”, “setIntFor” etc
And the key is templatized in C++ way: if your device has less than 256 states, you can use a uint8_t as key.

I have more problems with the optimization of the physic engine :stuck_out_tongue:

And the main problem i have with jme is plugin stuff: when you create a game from A to Z and have control on everything, well, it’s fine, you can ensure that every state are in the good state when you do this or that. But when other people will create a plugin for your game and they will want to have their own in-game app states and in-menu app state, it can becomes a nightmare.

I don’t say that jme is bad, not at all, it’s pretty much awesome. Especially for everything made in its core, it’s wonderfull. But the bullet engine, the niftygui hud and the spider monkey is a no-no.

And about the bsd licence: well, i already knew that, the main question is: do they agree ? It’s not about hving the right, it’s about having the agrement.

EDIT: (i added yesterday a particle emitter and a wire view)

8

You know that jme has also RawInputListener right?
I’m not saying you shouldn’t make your own engine, but i am not sure that C++ is still the right choice for a game engine these days.

9

An input listener is still a “callback” approach. Of course i can build my system on the top of it, but it doesn’t change the fact that the built-in system isn’t good (and most of libraries developped for jme will rely on this idea of a callback).

10

mhh i see… instead of fire an event when a key is pressed. You listen for that event, you store something and you let the developer check if x key is pressed inside his code (ie. in mainloop). right?

11

Congratulations for your challenge.
But if you are doing this only for the reasons you said here, I sorry to say, but it will not improve anything.
First you need to innumerate why an c++ engine is better then an java engine in general, take good ideas from diverse other game engines, jm3 included, and then you have a goal.
For me, there is no reason to go to c++ for performance, things like cpu / i/o are not the bottleneck anymore, so there is no performance gaps, and java will give you advantages on mult-plataform compiling etc.
There is other reasons why I would prefer c++ thought, like assets protection, code protection, bigger user base, faster compilers ( if you know what you are doing ), access to more libs, more support, etc, etc, etc.
Personally, I dont use c++ just because I could not find any good c++ engine that works fine with non microsoft compilers… If you build one that works fine, for example with netbeans, I am in.

12

Yes, exactly. And the direct consequence is: when a controller “listen” a key event and the game is in pause, the “update” of this control is no longer called (it’s the default behaviour of the abstractcontrol, can be override though). So, the checking this event is also no longer called, and, smoothly, nothing happen.

In the same way: when you remove a spatial from the scene graph it is also “removed” from the event listeners. Smoothly, just because its update loop is no longer called.

@wagfellz: there is some things that you can do in C++ and you can’t do in java.
For example, let’s consider a gui element. A gui element is either a “no-child” element (like a label) of a “with-child” element (like a panel). Well, i already have this with Element (similar to Spatial in jme) and Node. However, i can’t have, in java, a code like this:

class Gui
{
  public:
    void setPixelSize(const Point2i & size);
}

template<typename T>
class A  : public Gui, public T // <3
{

}

//typedef ... well, no, but you understand what i mean.
typedef A<Node> Panel;
typedef A<Element> Label;

To this you can add all the “const” saviour: const methods is something amazing.
And for assets, yes :smile: : a “concept” in my engine is that an asset is something immutable. I’m not sure i’ll keep this, but i will try to. Anyway, the assetsManager has a “get” method (that return something cached) and a doLoad that load a fresh & new resource.

I use c++ cause i have a better control on types (unsigned int, structures etc) and i can check that i don’t have a memory leak (which is not so obvious in Java).

My code compile under linux (and likely under linux only). Most of it can compile under both windows and linux, only the gamepad integration rely solely on the /thing/js0 and is linux specific. It’s not my fault, neither the XInput method from windows nor the SDL gamepad integration could detect my old Thrustmaster 2 in 1 dual trigger. And as i said, Xiaoyu should come with a XiaoyuX linux distrib and be optimized for this distrib (and let the linux handle the hardware specificities).

I use kate as text editor (i know i could use a text editor with more advanced features, but this one is just light and oblige me to don’t over abuse of the ctrl+space) and a php script for compiling (it mainly find cpp files that are newer than their .o and compile them with some flags etc. I’ll replace it with a classic Makefile later).

13

In that way you preclude a lot of flexibility for no real performance boost.
Not to mention that, if you really need this, you can do the same thing with the Keyboard class in lwjgl.
There are a lot of ways to solve smoothly your problem with the pause, switching from even-driven to polling is imo one of the worst, not for performances degradation, because i guess jme does the same thing in the core, but for readability/flexibility/OOP

14

i edited the previous message. Also, the device thing goes way further than that: in game device (button, dorrs), abtract device (with a jump “key”, triggered from some other part that can bind an azerty or qwerty keyboard or a gamepad to it etc) and not-so-obvious devices like a Timer (with a “needFrame” key or a “timeToNextFrame” real etc).

15

Congrats and good luck, “you did what I didn’t”.

Sometime I also ask myself “how will I made my game engine (non java)?” and my current feeling is opposite to yours :

  • using Rust as language, more modern and cleaner (and maybe safer in memory management) than what I know from C++11 (I’m a noob), integrate concept I like (eg: pattern matching)
  • using events, I’m from the culture of Actor, Reactive Programming ,…
1 Like
16

I always disliked the jME3 input system (even though I wrote it…)

Adding a polling interface instead of the current event based one definitely sounds like a good idea, which would make a lot of things much easier.

Actually, not that many changes need to be done in InputManager to expose a polling-based interface, and the old one can be simply deprecated.

4 Likes
17

@david_bernard_31: well, i don’t use C++11. I alredy used it and it’s true that a lot of things are easier in this language. However, i learnt an important lesson in the past: when you develop a library ( opposite to "when you develop a “final” application) you should use as mush as possible an older version of the language (a stable one, of course). The main reason for this is: when someone will try to use your library in combination with others or in its program, if you library is newer than the one used in its program it will not work.
An other reason is that i experienced some bugs in the core of C++11 (memory corruption when using stl methods, leading to crash) and i think that the support is not completly stable.

@Momoko_Fan: i agree, it should be too hard. But an other less intrusive option would be to juste add an appstate and let it handle the mapping etc. For the current version, at least.
Jme is not designed for games, it’s designed for 3D apps. You could have a 3D editor, and then the callback thing wouldn’t be bad. But i think that it should be considered as “low level”, so not deprecated but not-so-easy (and not-so-hard, it’s still pretty handy as it is now) to use.

18

This is amazing! I myself am working on a game engine, but for an even bigger project: A frickin Pentium III based game console (I have an Instructable posted here)! The entire thing is going to be written in c/c++ and x86 assembly (how fun!), even though I don’t know jack about either. I will learn as I go. The engine itself will be written in c/c++, but the main OS and libraries will be the assembly part, since I don’t have much of a choice. It’s going to be a ***** to get working, but when it does it will be the biggest achievement in my life. The sad part is that I can’t use any standard drivers or libraries, since I’m making my own GPU and chipset. Maybe once I can get the hardware and OS / libraries working, you can help me with the engine? Like give me tips and explain things. I know a little bit about 3D game concepts, but not enough to write a fully functional engine.

19

My day is now ruined :scream:
Maybe I should go back and make games with a text editor and javac? :sweat:

3 Likes
20

Maybe she means a game like watch dog or call of duty , need for speed metal gears …
I am also curious about it. Is it really possible to make similar game. I know there should be
huge group of programmers ,artist , and … .
You suppose we have all of them so is it possible or not ?