In dev log #2 we spoke about the audience luxe is intended for, and in this third post we'll take a closer look at bringing all the effort spent exploring design into a cohesive whole on the road to v1.0.0.
We can look at some of the earlier assumptions made and how they worked, or how they failed to live up to their goals - and the steps taken to make the design complete.
foundations
The goal of building a core that facilitates building stuff on top of it was a long standing one for luxe, and in exploring I initially wanted people to be able to opt out of almost all of the foundations available. I think you can guess the problem with that, but I'll continue!
This adds a lot of flexibility in some cases, but I feel like it's possible to have too much flexibility. This echoes what I was saying about %, that if something only caters to a tiny percentage of users in practice what cost does it carry? It is a pretty heavy cost if it impacts a key design goal negatively.
The approach of "everything is optional" taken too far breaks the design goal intended by the core being a foundation to build upon. It's difficult to build above a foundation that has a lot of holes in it...
In practice, it doesn't make it easy to make modules for my games, and for other people to rely on. It can lead to inter-module dependencies for foundational elements. An easy example: the concept of a Scene. If the idea of a scene COULD be different in every game, how do you write a pathfinding module that people can use reliably? It is likely going to end in a mess, could impose arbitrary limitations and discourage building cohesive (or complete) solutions. I can imagine various issues of fragmentation, none of which I like.
Another issue with this approach is surprises. The assumptions and expectations that users bring with at times can be useful and necessary in them understanding the engine (what is a sprite?). If your idea of what sprite means changes radically every time you look at a different project it will lead to unhelpful surprises and confusion.
This is the benefit of exploring right? We can see the ideas that sound nice on paper but turn out to be counter to our goals. Since we're finalizing the design now, it's time to fix these issues. The core must contain all the foundations to build on, and they must be unified to work together as as whole. They must be first class to the engine for the design goals to be achieved.
on convergence
Instead of a 10000 word post, I will be posting more details on each aspect soon, and some details are intentionally glossed over below. Questions until then are welcome in the feedback thread of course.
missing pieces: world
As touched on above, the idea of a Scene was too loose before.
In luxe terms this now called a world.
We need a concept of a world to exist, for things to exist in this world (entities), and everything spatial or visual (on the high level!) would exist within a world, attached to these entities. This is normal entity system stuff in game engines, no surprises there.
There is a significant amount of integration that comes from a world concept though, everything from rendering to physics, gameplay, editors, even networking will hinge on a consistent description of the game world. It is something that modules need to rely on to exist and to remain consistent, well versioned alongside the engine, and fit the design philosophy well.
In the final design, luxe has a well defined world system.
missing pieces: data driven content
In exploring, I tried out various approaches to data driven content and most of them required a fairly manual process to implement in your game. Most API's provided a way to feed it json, but it wasn't necessarily a part of the engine. This was something I had intended to resolve, but it needs to be considered at every stage and it needs to be in the foundation in order to fulfill this design goal.
Rapid iteration, reloading content, text based comparing for version control, avenues for better tooling and much more come from the ability to control everything from data. It opens the engine up to people with no or low programming experience, which can be especially powerful in team settings like game jams or studios. It doesn't remove the code focused workflow, it compliments it.
The final luxe design incorporates it in the foundation.
missing pieces: UI
I really like what mint offered me when I made it, and I like that I managed to make it agnostic from the engine without sacrificing much from workflow. It's still delivering on it's purpose. But in luxe specifically, it lacks certain things that need to be corrected in design and workflow.
It's designed to allow easy custom control rendering but there is more friction than I had wanted using it that way. What I really want is the ability to customize the rendering of a specific button (or control) in little to no time, almost inline. A good example is something like a player facing statistics graph - a blank Control but the custom rendering draws a nice line, some text, and some grid lines. Doing this is easy enough but due to the nature of mint being agnostic it wasn't possible to make this as fluid as I hoped.
There's more: the fact that the UI clipping wasn't well behaved in the luxe backend for mint didn't allow camera rotation, or rendering it in 3D, without first rendering it to a texture. This made it a little more manual work to do anything like that, such as applying a shader effect to the canvas.
So, the final design of the UI fulfills these needs properly. It must be a part of the core engine so that it is completely cohesive. It must be trivial to render custom controls, it must handle clipping properly regardless of the camera, and it should be renderable into a target (the default) or to the screen. These are all in place in the final design (and 99% implemented).
missing pieces: collision + physics
The concepts explored in providing physics as a sort of plugin-like-thing was a nice choice, it allows users to throw together a simple simulation for their game or to use a full fledged provider like nape. You could drop it in and use it. This is a bit tricky though given the first missing piece - the world - and the lack of a consistent approach to giving entities physical properties.
The collision api in current also offers a nice and easy to use collision only queries which I like to use. I wanted to have 3D primitive tests as well. The goals of this collision only api and user implemented physics provider remain, and will be cohesive with the world system.
However, in keeping with the design we have to consider %. There is no physics or collision by default that "just work". You always have to wire them together, which can be a lot of work for a simple game if you're learning. There are more than enough games that use physics and collision to (simple or otherwise) to justify having a consistent, well integrated physics provider as a part of the world system available to use.
For the "full fledged" provider that will be available in luxe, I have opted for the excellent liquidfun.
missing pieces: assets + parcels
The parcel design in luxe extends a lot deeper than the current implementation does. The original version of the parcels (in my first version even before luxe) took them too far, and the current version of luxe takes them just far enough to be useful but still has a lot to be desired. The final design will bring all of the design goal requirements into the implementation.
The parcel design includes a wide range of aspects skipped for this version - like a packed binary format, post-release content changes, world streaming usage and more. Other pieces left out for now relate to before they become parcels at all - in engine terms this is typically called the "asset conditioning pipeline", asset processing, asset building (it has various names).
The intent being to take a bunch of input assets, a bunch of configuration data, and to generate the parcel which is optimized and ready to use by the engine. This allows for many benefits like great font handling, data validation, format conversions, and so much more to happen before you run the game, freeing your source assets up to be source assets.
These kind of parcels are usually "built" for specific platforms, build variants, languages, and a variety of other selections that the developer decides. In luxe a parcel is not platform specific in terms of hardware or literal platform but rather up to you. An iOS build of your game won't include Steam specific build data if you don't want it to.
I opted to not implement the full extent of this to save the best aspects for the final design. It requires a deep integration with the engine and requires decent tooling to be fluid to work with.
luxe now has most of this in place, and will continue to crystallize as 1.0.0 approaches. It fixes many issues with parcel workflow, below is some cropped/edited example debug output:
missing pieces: contextual input
Input has been rather annoying in my design exploring :p
I have been trying to solve certain design challenges but it always ends up being problematic because the idea of input is somewhat contextual. A UI that pops up is a context that receives input.
The issue though is that the context might not be exclusive, like you might show a tutorial dialog that the user can dismiss above the game. You might also disable portions of another context, and it might disable portions of it's own context. The user still needs to interact with the game context, tutorial context, and more; there are all sorts of conflicts that can arise.
What you really don't want usually is the game code being littered with tutorial UI code and your UI code being littered with game code.
Input is also technically global state that needs contextual filtering and direction, but if you isolate it to a context as "local" then you lose that ability to meaningfully handle anything nuanced without backflips. Let's say you had a debug console that the developer can type into, if the game context doesn't know about your debug console context, you're going to have a rough time since having to account for all possible input contexts in the game and vice versa isn't fun.
The input also needed better input mapping, and easier ways to manage that from the developer <-> user perspective.
In the final design, I finally have a system that seems to make sense, and is integrated into every foundation system where input is relevant so that it can be consistently controlled.
missing pieces: debug
Games and engines are much easier to use when you can see their insides. The debug console in the current iteration was quick and useful but far from the intended design.
The ideal approach is usually able to debug a running application over the network, like when running on a device. The engine debug layer should provide comprehensive state visualization and in some cases modification of the runtime data. It should be resilient to crashes, have decent logging and filtering and more.
This is a foundational aspect that requires deep roots, and in the final design they have been placed. Since the effort required for great tooling is significant, immediate term will provide the fundamentals but will grow to cover all aspects of the runtime.
missing pieces: rendering
There are too many things to mention in this headline, and rendering is a huge topic but some of the important missing factors were:
More control over the rendering of individual geometry, more control over the geometry itself (i.e attributes), more control over the rendering of the frame, more control over rendering state, better shader support and myriads of other proper rendering™ features.
The renderer is also a full abstraction over a backend, designed against modern API's, which makes porting to different rendering backends much better. This type of large scale system is large by definition, and while it will be relatively full featured at first, some aspects will take some time to grow.
It is expected there will be continual growth on this level to iron out usage in practice and finalize features and some higher concept stuff (like compute shaders) will probably land after 1.0 instead to ensure scope is attainable. The renderer is designed to have support for all modern features though, I'd just prefer to get the 90% in use while we connect all the dots.
For users that don't need any of that control, the rendering just offers much easier ways to do stuff like post processing effects, and higher level game specifics. The underlying support for materials opens up options for better tooling, easier sharing of fancy shaders, and more efficient rendering - while giving more control over specifics like the stencil buffer or depth buffer on a per geometry level.
There's a solution for the shaders also being portable - which will be expanded on in the next dev log as part of a long term trajectory.
missing pieces: high level scripting
Script languages have always been a staple for making games and continue to play a role in teams all the way to solo developers because there are many upsides. The rapid iteration, fluidity, accessibility, moddability and engine<->language cohesion isn't taken for granted. Being able to have a precompiled binary can remove platform specific build steps, which makes it possible to deploy to other platforms from your own.
I had intended to bring this type of workflow back to the forefront of the user experience in luxe, but I wasn't exactly sure how at the time. In the final design with all the other missing pieces in place, I arrived at the solution I feel fits the engine wholly immediate term, and grows solidly into the future (facilitated by the core design).
In our final design, luxe has a user facing workflow empowered by high level scripting. Rest assured though, the best immediate and long term decisions are being made for both the engine and it's audience.
There's a lot of details on exactly how this looks in practice,
but it deserves it's own full post to elaborate; we'll cover it in detail soon.
structure
The above is a handful of examples to demonstrate the significant differences a finalized design and specification is meant to bring. In many cases whole systems and workflows would be unrecognizable, which is the point of exploring! This was what I meant in dev log #1 regarding backwards incompatibility - expect something new but in final form.
It also leaves a lot of out, like designing for parallel hardware and more - all of which relate to the long term trajectory covered in the next post. The engine remains small, light weight and agile but by converging all of the design goals into a full picture makes the picture appear complete.
I've been working with the (almost) complete picture the last while and I'm honestly itching to start making games with it. I'm sticking my focus to finishing the last systems though and I will continue expanding details on the road to v1.0.0.
implementation
To wrap up this post on convergence of the design I'll list some core systems that make up the engine foundation. I'll will expand on these in detail later since some systems include multiple high level systems (like the world). The engine still remains minimal, it's just that this is what the full picture looks when we bring it all together.
- input
- audio
- debug
- plugin
- assets
- scripting
- rendering
- animation
- physics
- world
- text
- ui
A "specification" for each of these is the goals, needs, and requirements. The missing pieces show some examples of that, where the specification states necessity by design.
These are tackled one by one, a system at a time. Each system is implemented to the fullest extent possible (some systems have to wait for others, and there is back and forth like rendering needs assets but assets needs but ....).
They are progressing very well and are almost all complete. There is plenty of work still, especially on the higher level, but having the systems complete to specification means the foundation can solidify, rough edges can be removed, and stability can set in.
Each system is also designed to be decoupled from each other, and it's purpose is primarily to fulfill the requirements of the design however it wants to. Since the design doesn't change, the system is then free to optimize and adapt over time without changing the high level.
This gives a lot of space for continuous improvement and growth, say if the parcel compiler improved it's performance tomorrow, nothing for the user would be affected. Changes that do affect the user get tied to the engine version, which allows rigid stability/silo development for releases.
Once an implementation (or part of) is complete the system tests are written for each design goal and aspect, in an isolated and easy to read example. You've seen one of the the animation ones in dev log #2, here is an example of a system test for the input system:
Next time - long term trajectory
In the next post I want to talk about the decisions that are designed to mature over time, while still providing a great experience now. As the road to 1.0.0 is filled with infinity choices, my intent has always been and will continue to be to solidify a foundation for the long term for me (and others) to build games on.
To make this happen some decisions are made that affect the immediate term, but are made for the medium and long term sake. With the design of the core being a foundation - there is only upward to build.
I'll cover a number of this type of key choice in the next post.
As always feel free to ask questions and share your thoughts,
either on the feedback link below or directly to me!
Feedback on this post can be found below or shared in this discussion.