Data-Oriented Programming

Hey, monkeys!

How do you think these techniques could be applied to game programming? The first thing I can think of is ECS, but where else could data-oriented programming be used? Is it worth it to write a game engine entirely based on a data-oriented approach?

2 Likes

ECS literally IS data-oriented programming.

I haven’t watched the video but I know the history of ECS. It is directly data-oriented programming at a fundamental level.

1 Like

I know, that’s why it was my first thought. I was wondering where else it could be applied.

I have watched the video now. Some interesting things… not sure when they will be in Java but it’s always nice to know these new language things so when a team mate starts using them incorrectly I will know it’s not a syntax error, just bad programming with the new hotness. :wink:

Define “entirely”. Even this talks says this works with OOP. I’d argue that ECS games are as close to “entirely” as you will get.

I find this talk interesting for another reason in that “data-oriented programming” as a term has come so full circle that a nice “side effect” is now touted as the main feature (re: duck typing).

The DOP approach originally got its start in constrained environments that needed to stream their memory. So data was organized into tight blocks of ‘source data’ that was only what was needed. Operations (systems) streamed over that data row by row performing operations and possibly writing to another tightly packed array of data.

Organizing entities as arrays of immutable components was a way to get source data. By selecting only the components required, they could be packed into memory tightly. The side effect was that instead of operating on big O objects of a specific type “WalkingCreature”, etc… you could grab all objects that had the needed components (Position, Velocity)… then the integrate operation would crunch each row and it didn’t matter what object type that was.

That’s essentially duck-typing (if it walks like a duck, talks like a duck, shaped like a duck… treat it like a duck).

I have indeed used this very concept in enterprise-level applications on unstructured data… which is about as far from an ECS and the origins of “data-oriented programming” that you can get. NoSQL databases like MongoDB even make this truly trivial… because the actual database queries can pull back just the parts you want.

That makes it interesting to see that the primary DOP example in the video is dealing with unstructured JSON data in a structured way… and in a very “duck typing” way. That ‘if’ block would deal with any JsonObject that had those fields, regardless of the original real type of the data.

Game take:
The useful part for games and ECS specifically would be the “record” stuff. 90% of my EntityComponents could be ‘record’ classes with a few extra convenience methods.

The “pattern matching” (horrible name, really) has nice idiomatic utility for day-to-day coding but not really a game-changer in the ECS realm. In an ECS, we aren’t throwing giant entities with all components in them at any operations… the entities already only have the components that we expect. There really isn’t any additional type resolution or casting to be done. I’m pretty sure that Zay-ES will work with these “out of the box”.

As for the ‘sealed’ class hierarchies thing… as an API writer, I foresee folks complaining about that even more than making my fields all private. “I would be able to add my own implementation of ‘x’ but you’ve needlessly locked it down!!!” I have trouble thinking of cases where polymorphism+flexibility wasn’t the better choice.

And the “better Future” example is supposed to look nicer but in the most common case will lead to required horribly bloated idioms. 9 times out of 10, a future.get(wait forever) is all that’s needed and any exception is just bubbled up… but now a giant switch with all possible outcomes is required. I can’t think of a single case where I’d want to specifically handle every possible outcome specifically.

Conclusion:
Yes, you should absolutely write your games using “data-oriented programming” (as an ECS). The new record classes will make this even easier.

It was nice to see “immutable data” being mentioned so clearly… since I get in that “why?!?” argument all the time.

5 Likes

I think the real benefit for ECS is the “Value Objects” classes that will come to java in the future (it started since 2014!). A class instance that does not have identity and is immutable. Those are treated like primitives in the memory (like int array).

“record” is still treated like a regular class with all its fields being “final” implicitly.

Yeah, but 90% of my EntityComponent implementations are boiler plate around a handful of fields and could directly be records with a few additional methods.

…though I’ve gotten so used to typing them by now that it’s almost automatic.

Right, though it will be even nicer when “with” is added to records. Then “additional methods” would not be required mostly. There is an ongoing JEP for that I think.

newPerson = person.with {
     name = "Ali";
}

it will create a clone by only changing the specified fields, kind of like groovy I guess, though they mentioned that this is not going to be a DSL.

1 Like

That would save a little boiler plate… really what I’m thinking of is all of my ID to String, String to ID, toString(EntityData) boilerplate that I add for components that have string fields that internally get converted to IDs.

Java Record Classes are very similar to Kotlin Data Classes, I used Kotlin data classes before on some of my android apps in the Adapter pattern for data to be rendered on GUI, so the idea is simple, they just serve as data structures for relevant data with a lot of boilerplate code removed, I think you could use them also in rendering models as separate pieces or loading materials into models or even breaking scenes into multiple pieces, for example, a Car is a record, a Car Record has a Chassis, Tires, and Nitrous…

So, a record could be used as Data Structure for the adapters at least in my opinion, adapter patterns can be fit into GUI frameworks, scene renderers, states, and more…

Pattern matching is a great feature as long as developers don’t do extensive or nested matching, that is a bad practice.

As for Sealed Classes, I don’t think they will serve a lot besides annoying the developers, it is annoying to restrict inheritance to some classes, if you want to force Good Practice, use documentation and code examples properly on your project, so I don’t see good use for this feature.

EDIT:
Fun Fact, the C programming language has the Record feature hidden for so many years since invented :grinning:.

typedef struct record {
    const int x;
    const int y;
} Point;
....
Point point = {
    10,
    20
};
2 Likes

Now do point.equals() or point.hashCode() and add some alternate constructors and convenience methods.

struct and record are very different… even fundamentally so.

2 Likes

Could be done using pointers to functions in C way, as OOP will abuse it.

All of this has been part of Scala for years already in a much more thoroughly researched and even soundness-proven fashion, with much cleaner syntax compared to Java and very high performance. I am currently working on a pet project adapting functional programming with immutable data in Scala for the state machine and asynchronous simulation bits, while the real time part is still realized with mutable classes which are simply Scala extensions of the JME base classes.

So far it is very nice to have deep pattern matching, stack-less recursion (tail-recursion) and the concept of Futures available for implementing AI, level generation and rules.

What I really hate though, are the many null initializers required when extending the JME3 base classes, because of the clumsy side-effecting way. Almost all bugs I am producing in this code are caused by this mutable state mess. I hope to find the time to refactor later for better isolation of initialization code.

case class Point(x:Int, y:Int)
val p1 = Point(0,1)
val p2 = Point(0,2)
val p3 = Point(0,1)
println(p1 == p2) // false
println(p1 == p3) // true, yep no equals(p3) required

myPoint match {
   case Point(0, myY) => println(s"first column at row $myY")
   case Point(otherX, 10) => println(s"somewhere at row 10: $otherX")
   case p : Point => println(s"Some other $p")
}
2 Likes