Future City: Update 6 - A Balancing Act

Industrial Zone

Using the previous steel structure procedures and some new bits and pieces I've been building a few elements for populating the industrial areas of the city.  For now we have:

• Warehouse or factory building, with roof-top windows/solar panels, and chimneys.
• Round storage silo or tank, with and without steel support structure around it.
• Simple tower based on the steel-work structures.
• Simple chimney of a couple of different styles.

Structures for first-pass industrial zone

During the creation of these, I've been regularly checking the look of the city.  I was pleased with the initial appearance and the variety and layout was working well giving a good view.

Structures in use throughout the industrial districts.

A Recurring Problem

As more elements and more detail were added, holes in the industrial blocks often appeared.

A hole in the industrial zone

This is an indicator of a procedure synthesis running out of buffer space, due to either too much geometry (triangles/lines) or too much calculation (procedure depth/complexity).  To find the problem procedure a bit of digging using the view diagnostics panel is required thus:

1. Set render depth to show only the tier with the problem in.
2. Turn on octree node display.
3. Set octree node depth to match render depth selected above.
4. Set the node highlight to the node surrounding the problem area.

First stage of diagnosis; what is the error?

This should show up the actual error message (stat shows first error message under node).  To see exactly what happened during the synthesis of this model we can perform these further steps:

1. Open the engine dev panel to show the synthesis monitoring panel.
2. Switch on node model filtering to only render models under this node.
3. Set the model select to only render the problem model.
4. Request synth analysis graph generation (triggers re-synth).

Second stage of diagnosis; which exact model?

This will re-synthesise the model (shown in the synth panel) so we can look at the output stats in more detail.

Third stage of diagnosis; what sort of memory did we run out of?

From this we can see that we are running out of parameter buffer (abbreviated to 'Control' here), the memory used for all the information flowing between and through the procedures as the calculations are performed.  This will be because the city block sub-division is quite demanding due to there being quite a depth to the process between the top level of the city at which we are starting and where we actually start to generate geometry.

Turning Up The Heat

Hitting these problems has left me feeling a bit in the dark about how well the system is performing in general, how close we are to having them happen again else-where, and where else we may need to improve the procedures building our scene.  It is a balancing act fitting the design and requirements into the resources available (as is often the case), but this is going to be a common workflow occurrence and we need more tools to help us monitor the situation as we work.

A visualisation tool often used for assessing the overall level of something and getting a good feel of how some value varies across a space is a heat-map.

An example of a heat-map

There are many metrics we could benefit from being able to visualise such as:

• Rendering cost (triangle count, draw calls, materials)
• Memory use (system memory, GPU memory, synthesis buffer)
• Synthesis cost (time/cycles taken, buffer use)
• Measures of complexity (synth depth, stack use, passes generated, analysis, refinement level)
• Scene complexity (octree nodes, model counts, mesh counts, node status)

All of these help us assess the overall cost and effectiveness of the procedures creating our world.

Some support work was required to capture the statistics needed from the synthesis process, and propagate it through to the rendering stage (with the meshes). Plus, because the output of synthesis is often merged into single meshes, we also needed a way to render parts of meshes separately to give per-procedure colourising.

Various heat-map modes: Triangles, Cost, Time, Stack, Depth

Each new display mode has its own scale configuration setting so that the range of values being visualised in the scene can be seen in the most detail by spreading them across the full colour gradient.

Gradient scaling settings for each heat-map mode

By also having a distinct colour for a 'maxed-out' value, i.e. on or above the scale setting we can interactively measure the value on a particular element in the scene by dialling the value up and down until we are on the changeover point between being the top end of the gradient and the highlight colour.

I'm not 100% happy with the choice of colour gradient, but full spectrum gradients (which I assumed would be best) seem more suited to continuous functions and not the more random patterns we get here.  Instead I'm trying a simpler black-blue-cyan-white 'icy' gradient, with red for out-of-range values (above the chosen scale setting).

Hot In The City

Testing these display modes out on Future City as a whole shows that we are actually in a pretty good position with fairly low quantities of geometry and no hot-spots that really stand out as problematic.

Triangle counts visualised across detail levels.  Only a few hot-spots at around 7,500 triangles.

It looks like we can actually afford a fair bit more geometry and detail, but at the moment we can't afford synthesis cost (hence the holes).

Typical synth stats for chunk of the city.  Scope for a re-balance I think.

In the upper (distant) tiers most models are in 5,000 to 10,000 triangle range, and the lower tiers are at about 1,500 to 3,000 triangle range, which is pretty low for assets these days, with the overall GPU memory at about 100MB with the viewpoint in the centre of the city.  The draw-calls are moderately high at about 1,000 but this shouldn't change much as more detail is added and can be improved in other ways later on (I'm not doing a frustum cull yet for example!).

So, we need to reduce the synthesis cost.  There are a couple of ways we could go here:

1. Reduce the complexity of some of the more complex procedures, for example we could convert our super-splitter procedure (see last weeks post [link]) into an operator.
2. We can re-balance the way each synthesiser's buffer is allocated so there is more parameter space available (and less geometry space).
3. We can look at one of the several planned optimisation passes on the synthesis processing and evaluation itself.
4. Other ways?  (there are always other ways).

At the moment I think it is a bit premature to bake the Splitter proc into a procedure as we lose the ability to live edit and continue to improve it (much slower turnaround).  Some investigation of the usage of buffer memory would be helpful in deciding on a new parameter/geometry split for the buffers, perhaps with a buffer use heat-map mode?  I didn't want to get into optimising the synth system yet as I think I can get more elsewhere first and it's tricky work.

Hot-line

I did do some work simplifying the steel tower procedure, which seemed particularly bad, using line primitives (which I have so-far not needed).  In the far distance, the steel beams reduce down to 8 triangles each, but this seemed too much so I added an extra detail switch to a single line segment.  This helped the geometric/visual detail level, but didn't  help the synthesis complexity as we were still having to sub-divide down a lot to get to the individual girders.  Instead of relying on these line based beams I remembered I had a line grid primitive (originally used to draw the ground-plane grid) and so switched this in at a higher level for the steel-work side sections.  Thus each tower is now made of four grid primitives in the distance.  I took this a stage further by double the grid cell size progressively too as I noticed that the fixed width (screen space) line rendering becomes a solid block when the tower is really small.  This helps keep it looking thinned out and again reduces the render cost.

Reduction in tower complexity at lower detail levels by use of line (grid) primitive.

Late Night Ideas

One night last week I couldn't sleep for thinking about these problems and an enticing thought occurred to me about how I can neatly improve this synthesis depth/complexity problem by an order of magnitude and allow lots more detail for the distant city.  This will help a lot for aerial views of the city where you should be able to see a huge amount of small detail when at a considerable distance (probably the worst case scenario).  I think I'm going to explore this next after playing with the synthesis memory balancing.  More details next time… maybe…

What is project "Apparance"?

Videogames

I have been making videogames for over twenty years, and the process always seems to go something like this:

1. Design game
2. Create assets
3. Write code
4. Test game
5. Repeat above in various combinations until done (ship or bust)

These steps are quite separate, mostly performed by different people (or groups of people), and the bigger the game the more of all this you need.

Some time ago I came to the conclusion that above a certain project size this model isn't very efficient for a variety of reasons. Firstly, because assets need to be worked on by multiple people, the friction associated with moving them around (conversion, source control, management, communication, builds, deployment, and loading) can mean lots of additional effort and add up to long and frustrating iteration times.  Secondly, as projects have become more ambitious, with larger scopes and game-play areas where the work involved in creating them increases exponentially (with area/volume), the cost also ramps up this way because the work a team puts in can only, at best, grow linearly with size.  In addition to this, a natural consequence of growing teams is the need for increasing communication, management, and support.  All of these pressures usually mean compromises being made to the game's detail, richness, variety, polish, quality, or seemingly low priority support work is neglected, in order to get it to fit within the budget.

Pondering this problem for several years I have formed a vision of how I think we can improve this construction and authoring process, and what the technology and tools it requires would look like.  I'm fed up with conventional approaches, the waste, the sluggish pace, the repetition, the tighter squeeze on the more fuzzy aspects of creativity like experimentation, play, fun, emergence, and instinct. I want to try working a different way, using novel and interesting techniques, powerful technology and tools, to empower smaller teams to create bigger things.  I believe we can do better, and whilst it won't be everyone's cup of tea, it's something I believe passionately in.

"I'm fed up with conventional approaches, the waste, the sluggish pace, the repetition, the tighter squeeze on the more fuzzy aspects of creativity like experimentation, play, fun, emergence, and instinct."

I've been working on this project in my spare time for around five years, with the original concept rolling around my brain for about five years before that.  As I described in my previous post I am now in the fortunate position to be able to invest all my time in this project.  So, what exactly is it I am proposing?

Vision

I want to enable rapid creation of rich and varied worlds through fully dynamic procedural generation.  I want to encourage play and exploration through fully interactive live editing of content and behaviours.  I want to make games in creative and exciting ways:

• Rich worlds full of the variety that parameterisation and procedural techniques can enable.
• Vast structures with intricate detail from highly scalable dynamic modelling.
• Endless continuous worlds with no loading and no transitions required.
• Don't repeat to save time, every instance can be unique in design and appearance.
• Instant feedback and fast iteration from rapid and responsive tools.
• The play and exploration that real-time control systems with live editing allows.
• Freedom from bulk assets and offline asset pipelines by going fully run-time procedural.
• Increasing the efficiency of creating content through asset modularity and easy reuse.
• Making extensive use of visual editing paradigms (data-flow, node-graph).
• Removing hard coded content, everything data driven.
• Small footprint applications yet with all the richness of today's larger titles.

There is nothing out there that does all this, the technology just don't exist. I see this project as an adventure where we are specialising in procedural generation and real-time interactive experiences that can be applied to a variety of applications; especially games, education, and interactive visualisations. If we aspire to travel to new lands but the vehicles to do so do not exist, we must first build them.

"If we aspire to travel to new lands but the vehicles to do so do not exist, we must first build them."

What it is

This is an authoring environment in which to create your worlds, populate them, and bring them to life. The real-time synthesis engine uses procedural definitions, their input parameters, and the surrounding environment to dynamically fill out the world in more and more detail as you explore.

Apparance Editor - Procedure graph for roof section of an old house

The editor provides authoring and testing of the world through a visual flow based editing model (nodes and wires) to create the procedure definitions describing the geometry, texturing, effects, physics, behaviour, logic, and much, much more.

A standalone player application provides a non-editable view of world, for test, play, and release.
The underlying engine contains the technology driving both editor and game, using highly concurrent operation, with dynamic detail management, largely stateless data-flow control and AI systems, and all with live editing/monitoring during authoring and play.

What it is not

It is not a silver bullet and won't solve everyone's problems; Some games won't fit the development model and technology being created here.  It may be that a game relies heavily on areas that are a particular weakness for this technology. Some games are based on very tightly controlled and designed environments and behaviours. But I believe that working toward its strengths and away from its weaknesses is no different to any existing games development, more so historically as hardware and software capabilities have applied their own unique constraints on possibility.

I don't see it as a plugin for other engines; using a lot of unique techniques to achieve its aims means it may not work well in more conventional engine scene graphs and rendering.  The lack of scalability in many existing systems can cause problems too as non-scaling systems 'crowd-out' the scalable ones as a project grows.  This is a disruptive technology, and relies a lot on the interaction between the various novel techniques at play.

It is not finished yet.  This is a project that I've been working on in my spare time, and whilst showing great promise already, there is still a lot to do to.  I see many stages to its development and I will be building demonstrations and even games to show-case it at various points in the future.

Direction

This is an ongoing project and I want to involve people as it develops.  Through blogging and sharing the results of my work I hope to involve like-minded and creative people to help shape the tech and tools into something that many developers could benefit from.  I am working towards releasing versions of the tools for you all to play with to see what we can create together.  If you are interested in these ideas and the prospect of trying something different please get in touch via email, via my Twitter feed, or just follow progress here on my blog.

The Name

Logo design idea - featuring one of the first models built with the system.

Apparance (or often Apparence) refers to the outward manifestation of a person or thing, the act of being visible, what others see, the process of creating this external impression. The original inspiration for this comes from "Against A Dark Background" by Iain M Banks; The Solipsists don't subscribe to the concept of the individual with no use of the word 'me', referring to, and in a way justifying, the existence of other members of their group as 'apparences'.

Next

Next time I will bring you a full project status report, showing the current state of play and what I'm working towards at the moment.  It should be a very visual post as there is lots to see.

An Open Book

I'm not sure this is the right thing to do. I don't know what is. Here are my journal and technical design notes:

• VICE Design Journal - my original design ideas, thoughts, notes and journal.


• VICE Technical Design - the latest structural overview of the main concepts.

At least it should help save some blog time. It's quite a lot to absorb, but dig in, let me know what you think, ask questions.

The longest journal starts with a single word

Welcome to my journal. I hope for this to become a place for me to record, reason, and reflect on thoughts and ideas for an ongoing project of mine. I'd like to have a go at describing, divulging and developing these ideas somewhat to see if it helps build them into a working project, or at least help maintain some forward momentum.
I've always admired great scientists and inventors for the doggedness with which they maintain their notes, but it's something I've never been able to do for very long. I've kept a small journal before for this project, and it did help, but eventually petered out. It took the form of a cross between rambling, lists of ideas, and conversations with myself. This sounds to me like perfect blog material so I thought I'd give it a go.
I'm not sure how public this really needs to be, or if I need any external feedback at all. I think just being able to talk things through with myself will be invaluable, and moments of re-reading my words may give me that external perspective. If people do read this, and are interested, then maybe that will help spur me on to put more work into the project and ultimately fulfil my ambition.
I'll talk about the nature of the project in my next post.
Wish me luck.