Guildford Game Jam 2016

Jam

This weekend I attended the Guildford Game Jam organised by @Kilo_bytes and hosted at the new offices of the lovely @RocketdeskUK. There were about 15+ people in teams of one to four with a few floating jammers creating audio and art assets for multiple teams.  I took my creaky old laptop and managed to build a game using Apparance in the two (and a bit) days.  

Guildford Game Jam: 2 Choices

Here's how it went...

Idea

Given the limitations of the Apparance tech at this stage in its development I constrained to something that was non-interactive, i.e. with no moving parts.  I couldn't even rely on collision to build any form of physical puzzles.

The main theme for the event was 'two choices' so I was thinking about exploration of a branching space where each choice led you towards one of many final destinations.  Each split would have some clues as to the kind of outcome you were heading towards.  You would have to fly through the rooms and doors pretending that you couldn't collide with anything.  At the very least I needed my stand-alone player app to support camera controls (mouse/keyboard, and ideally game-pad) for this to be possible.

Pre-production

In anticipation of the jam I worked on tidying up the Apparance Player a bit and adding support for the following that I was going to or probably going to need:

• Command line option support - to select the level of control (I still wanted the house viewer to work, but also free-cam).
• Mouse and keyboard control of camera - for flying round environments (this would also be good for demonstrating the Future City scene).
• Game-pad support - I really wanted to be able to try navigating using a game-pad, it should be much more accessible.

Once I had all this up and running it meant I could run the Old House demo, fly around Future City, and support the game jam all with one player, each using different command line options to launch.

Command line options now supported by the standalone Apparace Player application.

Day 1

Setup

The first day (just Friday evening really) was an opportunity to meet the jammers and catch up with some friends.  We all set up, and started discussing ideas.  I was open to the idea of working in a team, but within the short time-scale I don't think it would have been possible to get someone up to speed using the Apparance tools (particularly as they aren't 'consumer ready' yet).  As it was I worked solo.

Dungeon Reuse

I had intended to start pretty-much from scratch, building an new world to navigate, but once I got playing around with some of my old test scenes I realised I could get a long way with the room-subdivision work I'd already done.

Early dungeon generation experiments.  A good starting point for a maze.

Playing with this and re-thinking my goal I decided that a simple maze of rooms with clues in to some final decision would work well and I had the room network to support it.  I was thinking that the puzzle should be finding your way through the maze and clues leading you towards the correct exit (one bad, one good), for example, bones or treasure scattered on the floor.

To Collide or Not To Collide

At this point I was starting to think that having to manually fly round the level with user-implemented collision was really going to spoil the experience.  I decided that, back at home, I'd try to implement some simple physics for the player controller.

Day 2

Early Hacking

Up early, I started hacking in some collision systems before I headed back to the Rocketdesk office for the 10 am start.  I got to the point where I could tag generated triangles with a material the engine picked up as collision and passing off to the camera controller each frame.  A bit messy but it would work, as long as I could actually do the collision and simple motion sim.  I found a neatly encapsulated bit of sphere-triangle sweep collider code that I was going to use.  But first, a 'day at the office'.

Beginning & End

Back at the Jam I set about building start and exit rooms that bolted onto the generated 'dungeon' space.

Basic level structure test, start room (blue), maze, exit rooms; bad (red) and good (green).

This meant that I had a basic level design that I could fly through from start to (a choice of) finish.  A good start.

Doors

The original dungeon work just had open doorways so I quickly stuck the doors from the Old House project in them to divide up the rooms a bit more.

We can use the door procedure from the old house experiment

Doors in place, double for wider opening.

As it stands Apparance doesn't have an moving object support so there was no way to animate the doors opening.  Instead I had the cheeky idea of switching between open and closed doors based on the detail level and letting the detail blend system provide the transition.

Door opening hack in action; high LOD open, low LOD closed.

This is very hacky, but just the sort of things you try and get away with at a jam.  It adds a lot to the feel of the rooms.

Late Hacking

Back at home (after an evening out at the local Nov 5th fireworks display) I set about integrating the collision test code into the engine.  It's a piece of code I found written by Igor Kravtchenko, which does exactly what I wanted.  For simple environment collision and movement simulation, all you need is sphere-triangle swept collision testing.  Basically, the player (camera) sits on top of a ball which is moved around by the controller.  Each frame you do a sweep test as you attempt to move the ball a bit and adjust the new position according to the collision results.  There are lots of edge cases that would break, but this should have been enough for me to move round a 'boxy' level.  After much coding late into the night I managed to get a rough version working.  Things were looking promising.

Collision testing against tagged triangles.

This wasn't to last though...

Day 3

In the cold light of day, my physics experiments from the previous night started unravelling (too many edge cases, falling through the world, etc) and I decided to abandon the idea as I didn't have enough time to sort them out.  Back to fly-cam.
The game at this stage wasn't much of a game as there were no clues to help you choose your final room.  Puzzles were needed, so I headed out to the jam room again for the final day.

Puzzles

Mulling over the puzzle mechanics the previous night I came up with the idea of mysterious symbols in each room providing clues to the final rooms outcome.  It would work like this:

1. A set of symbol design are available.
2. This set is split into three sub-sets 'good', 'evil', and 'neutral' symbols.
3. Each room in the maze will be clearly distinguishable as either a good clue room or an evil clue room.
4. Each clue room has a few symbols in it, some selected from the good/evil symbols according to the room type, and some from the neutral set.
5. The two final rooms will have a similar set of clues, from which the player must infer the exit type.

Difficulty

The difficulty of the game can be adjusted by varying a number of elements of the game:

• Number of symbols.
• Number of symbols shown in clue rooms.
• Proportion of neutral symbols shown in clue rooms.
• Size of maze.

To simplify the level authoring, the difficulty level is chosen at random based on the game seed in effect.  This means that you choose difficulty by changing the seed until you find a difficulty you are happy with (more on this later).  Most of the difficulty modifiers above were easy to hook up, but unfortunately I didn't have time to implement maze size changing (it would be simple to add though).

Symbols

So I set about implementing the symbol set manipulation (as a bit-field) for generating the sets, and extracting sub-sets, driving symbol generation, etc.  As I worked I quickly set up unit tests to make sure each step worked properly.  I needed to be sure it was correct as any inconsistency would break the puzzles.

Test rigs for bit-field operations: Pick, Split, and Count

Symbol (glyph) generation was a quick mixture of base shapes, colour, scaling, distribution, and variation.

Sample glyph family.

Combining these gave a generation step where from a game seed we could generate a consistent set of symbols for use through a given level.

Glyph sets and choosing example

From these it is possible to then generate a set of symbols for use in each room, good+neutral and evil+neutral, the choice randomised by a varity seed (different for each room).

Example glyph panel for clue rooms

I chose to always show four symbols in each room, with each room being designated good clue/evil clue from the room seed.  This could be varied by difficulty if desired.

Glyphs on the walls of good/evil clue rooms.

I intended for the symbols to be more subtly placed around the room and built into the decoration of objects, but this was far too much given the time constraints and I ended up just plastering them on the walls.

Exits

The two exit rooms needed to be clearly different to the other rooms and also the way you make the final choice needed to be 'no return'.  By colouring the room a neutral colour and making the final exit a hole in the floor it nicely fulfilled both of these requirements.  Under the two exit rooms (which by the way are switched at random) I constructed suitably rewarding and punishing environments to indicate whether the player chose the right room or not.  You will have to play the game if you want to see what they are.  :O]

Start Screen

I managed to squeeze in game seed and difficulty indicators above the starting door to provide a bit of a start screen/menu/front-end to the game.

Game start room with seed/difficulty indicator.

The controller support I added earlier included a key/button to select the seed value going into the top-level procedure.  By driving the difficulty off this seed (at random) it doubles as a difficulty select too.

Postpartum

The final 'game' was playable and enjoyed by a few of the fellow jammers so I'm happy to have met that criteria at least.  It did have the 'two choices' element, and maybe even the 'the player cannot win' modifier as you can't actually escape the dungeon.  I was a bit disappointed that I didn't manage to get the physics working as this would make the game easier to play and overall experience more claustrophobic, as well as the ending drop to doom/glory more dramatic.  I had an ongoing issue with the rendering engine struggling on laptops which really got me down and is something I really need to diagnose.

This week I'm going to try and resolve the rendering issues, finish the physics, and wrap it all up in a package for release on itch.io so you can have a play.  It's quite exciting that this will be the first release using the Apparance technology, and I look forward to your feedback.

Coincidentally this week is also Procedural Generation Jam 2016 week and since my game meets the requirements for that I will look to release it into the proc jam circuit too.  Watch this space for an update!

ProcJam is also going on this week.

As for the jam experience, I found the environment an odd one working solo and would have maybe got more done at home in my own office, but I would have missed out on the creative and collaborative atmosphere.  It's good to meet and interact with people, especially those with similar interests and passions, and I endeavour to continue doing things like this as I work to get Apparance more exposure.  At this stage the tools and tech isn't ready for collaboration at such a short notice, and I continue to realise that it's still a very technical process to build procedures.  It strikes me as very much 'geometry programming' than sculpting of assets, which may or may not be a problem long-term.  I think it's 'just different'.  Thanks to everyone involved and it was great to see everyone's projects played at the end.

Me pointing as a willing victim plays through my game.

Future City: Update 9 - Apartment Construction

Apartments

I've been thinking about how to build interesting residential apartment buildings, that look more futuristic than just boxes with windows.  A few ideas come to mind:

• Surface objects  - balconies, air-con units, ducting, etc.
• Surface projections - bits that stick out, and in-turn have windows/etc.
• Multi-part buildings - intersecting parts in a cluster, at different heights.
• Non-right-angle intersections - should be do-able.
• Buildings on buildings - large rooftops can recursively have smaller/similar buildings on top.
• Styling - design seed can keep same style among a cluster but different between

We'll see how far this gets us.

Building Units

I started on the apartment buildings by properly subdividing a building block into obvious parts: windows, faces, friezes, separators, corners, parapets, roof, etc.

Procedure injection points on a general building structure.

This can later have a variety of proper alternatives and parts switched in and out to give them their final look.  I wanted the buildings to support intersection, so I added 'opening' parameters on each side, spending some time getting the various combinations of opening needed working.

Rig to test support for openings in the sides of buildings. 'Unit testing for procedures' in a way.

Two main ways to intersect building faces occurred to me:

1. Clipping - Intersect, working out where the join is and just filling each part with the usual facade (window grid, etc), making some changes to the edges to fit better.
2. Culling - Intersect, but this time just cull any windows that would become split where the join should bee (replacing them with basic wall).

Clipping has the limitation that you need to intersect at quantised positions otherwise the windows beside and above the intersecting building won't line up, but Culling means you will get areas with no windows.

I thought I'd try clipping first, and with some calculation I was able to get the intersection working.

The clipping approach in action.  Windows scale to fit space.

To support arbitrary angle intersections though the quantising requirement is going to be really hard to meet, so I also implemented the culling approach for comparison.

The culling approach in action.  Windows are removed when not enough space.

Seeing these in operation show's up the main pro's and con's.

The main differences between the clipping and culling approaches to building intersection.

On balance I think the culling approach will be less messy, especially with proper surface materials and objects.

Proper Facade

Switching the placeholder geometry for more interesting design produced some promising results:

Basic design variation examples

Next

 Now I need to look at arranging these together in interesting ways for use in the city:

• Generating building clusters to fill the city blocks.
• Populating the surfaces with proper features.
• Adding some LOD support for full city use.

Future City: Update 8 - A Mixed Bag

I haven't managed to write anything this last week (or two) due to other things going on in my life (shock!), and not really feeling like I've been focused on one particular thing.  Instead of a themed post let's just have a look through what's been going on.

Refinement Fixes

Spent some time tidying up the city a bit.  There are a few places where detail refinement was being blocked so I investigated those.

• Empty Stacker recursion procedure leaf nodes were not ignored.

Stacker Merge appears as leaf when no further content needed, but should be ignored as not a geometry producer itself.
(excuse low-res image, Cairo scaled it down because the GraphViz output was too large)

• Temporary general triangle fill added for rooftops didn't support refinement.

Roof triangle refinement - before and after

• Bolts had bounds that were too large, needed to be pre-shrunk.

Bolt bounds blocking refinement - additional re-size in procedure to fix

• Mesh primitive operators were not being counted as geometry producing (which broke the fix for the Stacker recursion issue above).

Post load checks to classify operators by whether they modify/produce geometry or not.

• Unnecessary deep refinement of surfaces meant a deluge of synthesis when you strayed very close to a surface. This was slowing things down a little so I added a limit check in the sub-division process.

Adding a refinement limit test, whilst also abstracting out the recursion check to make it more re-usable.

To help track down some of these I added a bit more error diagnostics information:

• Capture errors show in view panel for selected node
• Error filter slider for when too many to display at once
• Any bounds information in the error messages is visualised in red

Additional error diagnostics - better messages, error select, and bounds rendering

More Girders

Couldn't resist more girder improvements as this has become a bit of an 'extreme detail' test case.

• Washers under nuts
• Proper facets on bolts
• Threads on the bolts (hacky, but fine for now)

Washers, faceted nuts, and threads.

• Added cross bracing to girders
• Fillet pieces for cross bracing

Cross-bracing and attachment fillets

• Added rolling fillets to I-beam section
• Added welds to fillets and fixing plates

Welds and I-beam section forming fillets

All a bit excessive, but some good tests of the detail management refinement systems.  Several issues and bugs were surfaced from this work.

The Wilderness

To make the city more interesting, and provide more of a 'setting' for it I decided to surround it with a Rural district type.  Currently just green/grass, this breaks up the city tone and gives it a rounder appearance.

Rural district type now surrounds the city.

This will provide something to have as the backdrop, or vista, when height support is added and we can have hills and mountains in the background.

This was implemented by re-purposing the commercial district type and giving it a negative gradient along it's radius instead of a positive one like the central districts.  I also parameterised the location, sizes and weights of the district types so we get much greater city layout variance as we vary the city seed values.

Some parameterised city examples

Demo Planning

I'm aiming to have my first-pass city demo-able by the end of the year, and so far I feel like I'm on track for that.  I thought I'd better make a clearer plan though so I spent a morning breaking down the things I have left to do and thinking about the timings.  Here's the to-do list:

• Finish filling out the placeholder zones (1) - I really need to fill in the residential and leisure areas as they will lend a 'more human' aspect to the city (business and industrial so far).  I'm postponing commercial and spaceport as there are more important things to focus on.
• Implement first-pass landscape support (1) - A big thing this city doesn't have yet is elevation.  It was always something I wanted to do to add richness and interest to the city.  So we need support for a height function (of some sort), blending (to interface with flat parts), and placing of blocks on landscape (giving them the appropriate elevation).
• Height integration (1) - Once supported, the natural areas (leisure, surround, and business in-fill) and the interfaces between adjacent elevations need content to provide pleasant and realistic transitions.  This will need to take into account block boundaries, height, and importantly; access ways (or lack thereof) between blocks.
• Detail pass (2) - Another pass is needed to add more detail to the low-level city, that you will see when 'walking around' at street-level. This includes features like doors, windows, pavement, as well as assets such as bins, streetlights, benches, planters, and maybe trees.
• Navigation/viewing (1) - To demo this city properly I need people to be able to control the camera in the standalone player.  This would start with basic is keyboard/mouse (FPS) support, with controller support coming later, and then maybe an attract-mode camera mode if there is time.  This would be quite involved as it will need to 'explore' the city to find good camera positions and paths (for some definition of 'good').
• Lighting (4) - I'd really like to try out my lighting ideas, they should really bring the city alive.  This would also include some fogging/haze/sky work, and maybe some simple clouds for fun.
• Bonus: Transparency and reflection - If I get time or want a break for something more adventurous I'll try and get some transparency and reflection support in to allow water and glass effects to be added.

Timings (in brackets) are estimated weeks.  We'll see how it goes as I am already a little behind and am still due to have some time off.

Next

I need to get the residential parts populated as soon as I can, so I'll be looking at more 'house-like' buildings and how to create them.

Future City: Update 7 - Building a Better Skyscraper

Sorry for the late post, a few mad things going on at the moment.  One of which was the Rocketjump event in Bournemouth which @dannyt of @Moov2 invited me to, to showcase Apparance.  A lovely little event with some great speakers and games to see, where I got to show Apparance to a few more people.

Skyscrapers

Last week I decided that the stand-in skyscrapers were too dull and not nearly futuristic enough.

Temporary, non-futuristic, skyscrapers.

I was looking for things like these:

The sort of futuristic skyscrapers I was after
(Sources: source 1, source 2, source 3, source 4)

Spaceships

I knew I wanted interesting angular surfaces and chamfering so as a starting point I imported some old hull surface triangulation procedures I made for my spaceship construction experiments.

Ship hull built by connecting triangulated pieces.

The premise behind these is that we can build up large and interesting building hulls out of simpler parts that connect rectangular and angled edges together.  However we also want to make sure we can use shapes that map well to the frame construct used to model within.  Rectangular sections are easy, and right-angled triangles aren't too much of a stretch either.  In most of the cases I wanted these were sufficient to build interesting shapes.

Hull triangulation procedures

Cross Section

To limit the complexity of the models built I decided a cross-sectional form where each side can be in-set and corners chamfered independently would still provide sufficient flexibility to give interesting models.

Example of a skyscraper cross-section (two matching chamfered corners and an inset side)

This information is passed around encoded in Vector3's (like the connectivity information), one for each side, defined as follows:

Side inset/chamfer encoding for skyscraper cross-sections.

This will be used to describe the outline of the building at various elevations, with hull triangulation providing us with frames to build the surface geometry to fit.  The cross section will be mutated at random along the way to make the shape change and evolve as the evaluation proceeds skywards.  The changes we can make are:

1. Leave it the same - a straight vertical sided section
2. Chamfer any un-chamfered corners - introduce triangular sections.
3. Inset un-chamfered sides - introduce slopes and overhangs.
4. Remove any chamfered corners - return to square corners.

These can be applied in various combinations and quantities, some examples shown here:

Example cross-section mutations

The requirement for this building section triangulation exceeded the original hull construction primitives so I created a new one specifically for this, based on the various techniques employed before.

Diagnostics display for building section surface triangulation

Building It Up

By iteratively segmenting the skyscrapers into sections, potentially mutating the cross-section at each intersection we get all sorts of building shapes.

Example skyscraper shapes

Surface Noise

Initially I filled in the surface spaces with the general purpose smooth gradient rectangle and triangle procedures I had to hand.  These however had consistency problems at their edges.  Even though the sub-division they perform to add detail does maintain coherency internally between new triangles and rectangles, there was no way to maintain this between adjacent instances of the procedures.

Incoherent detailing, particularly noticeable on triangular sections.

A change of approach was needed to fix this; instead of choosing corner variations at random and then using midpoint noise to sub-divide, a simpler approach is turns out is to use the 3D noise distortion algorithm I was already using for landscape and wobbly building experiments.

Experiments with 3D noise mesh distortion

Adding an operator to do single samples in space for this allowed me to use the output to modulate the surface detail.  This had the advantage that the noise function was based in world space and so any two evaluations that happened to be at a coincident vertex, and hence the same position in space, would result in the same modulation effect.  This produced the effect I was after and improved the look of the buildings no-end.

Moving from a mid-point algorithm to world-space noise algorithm for surface detail sub-division.

Roof

A quick triangulation hack later and I had the geometry to cap off the buildings too.

Temporary roof tops for the skyscrapers

I will re-visit this later as it will be great to add intermediate roof pieces between building sections so we get stepped in parts and mid-level terracing.  This complicates the roof calculation though as we need to in-fill between two cross-sections, as well as potentially generating railing frames around the perimeters too.

Windows

For some extra finesse I also knocked up a 'window' filler for the rectangular sections, giving a much better sense of scale as you 'walk' around them. Oh, and I've been playing with grass for the plaza areas that break up the city space a bit.  Some level of success was obtained, more work to do on that though ;)

Simple surface window detail increases sense of scale

The City

These new buildings, with a bit more tweaking and polish give a very pleasing look to the business district and are far more in-keeping with the direction I was intending the city to go.

Business District, now with added futuristicness

Design choices can alternate up through sections

Bigger and better business buildings

One Million Bytes

A slight aside today while I play with futuristic skyscraper shapes.  I'm not going to talk about cities, instead this post is about an interesting technical aspect of the synthesis engine; the use of the call stack.

The Stack

Typically the stack is something that most programs and programmers don't need to worry about, ever.  Programs tend to rarely have a call graph that goes even remotely near the stack depth limit.  Of course we have all hit stack-overflow situations, but these are usually due to either:

1. A function accidentally calling itself (either directly or indirectly).
2. A recursive algorithm not terminating properly.

Both of these are just bugs you would hit during development, fix, and move on.  Sometimes though you may be explicitly using recursion to solve a problem and need to be wary of how deep the recursion goes.  Usually in this case you would just tweak the code to stop it running out of stack space, or perhaps your data-set is of limited complexity and it just works, so you can ignore it and again just move on.  With Apparance however we have to be a bit more careful.

Tree Evaluation

The synthesis process in Apparance is implemented as an operator-tree evaluation.

Simple tree evaluation example

This means that starting at some root procedure we evaluate its output, triggering this sequence of events:

1. The graph inside the procedure is instantiated and connected up.
2. We find what the procedure output is wired to, and evaluate its output (recurse to step 1).
3. At some point we reach a fundamental operator instead of a procedure.  Evaluating its output results in some code being executed to perform its intended function.  In order for it to be able to do this though it needs input data, which it requests (another function call) as required until it can return with the calculated output result.
4. We find out what the operator input is wired up to (this happened in step 1 for its containing procedure).
1. Usually this will be another output and we will evaluate that (step 2).
2. If our input is actually wired to a procedure input, then in a similar way to 4 we have to see what that is wired up to and recursively follow it until we either reach another operator output (step 2 again), or we reach a constant value.
5. On reaching a constant value we now have data to pass back up the call graph and we can return.  There are a couple of ways constant values can appear in a procedure:

1. Embedded in a procedure itself as a constant input.
2. As an input parameter supplied to the root procedure.
3. Output-only operators that obtain the value from outside systems, e.g. mathematical constants like Pi, or useful synthesis state like the current block size.

The upshot of all this is that as we dig deeper into our procedure graph, and at every step towards some constant data value, we are making deeper and deeper function calls.

Data Driven Code

Normally the execution call graph is largely hard-coded into a program; conditional branching changes the exact shape of this graph, and recursion can cause chains of similar shaped sub-branches to push it deeper, but it is still largely predetermined in behaviour.  During development, the stack usage may increase, but this maximum will largely not change during the execution of the program.  As a consequence, for most programs it is impossible to blow the stack in every-day use.

For a data-driven situation however, you can no-longer make this assertion; we don't know how deep our procedures will end up going when we are building them in the editor.  It is very simple to accidentally create a procedure that recurses for-ever and instantly overflows the stack.

Simplest procedure that overflows the program stack.

This is a problem since these crashes happen at edit time, when the application is in the hands of the end user, and not during its development where a debugger and developer are at hand.

A Walk In The Dark

In some senses, every program is a ticking time-bomb.  Are you really aware of how close your program runs to the stack limit?  Do you just ignore the problem since you've never hit it?  Do you even know how much stack space is available to your program?  It's a bit like taking a night-time stroll on the cliff-tops, everything is fine, until it suddenly really isn't.

It's a bit of an odd situation because languages don't really acknowledge the existence of a stack limit and it seems a little talked about point.  There don't seem to be any ways to query the stack, or work out the current depth, using standard library calls.  This may be because there are ways to hand code this and it's such a rare use case that it's just not worth the effort supporting it, or alternatively it is acknowledging what should be entirely an implementation detail that the programmer should make no assumptions about.  Anyway, I need to know the limit and I need to monitor the depth as procedure evaluation is performed and so in the absence of any library support for this it's down to a hand-rolled approach. 

Get A Tape Measure

First, how deep are we?  A few searches online reveal what I suspected; that the simplest way to measure how deep you are in function calls is to take the address of a local variable (or function parameter) at the start of the program and compare it with the address of a local variable in your current function.  I created a macro to do this:

 #define CURRENT_STACK_LOCATION(a) ((uint64_t)(void*)&(a))

Used like this:

 int a;

 uint64_t stack_pos = CURRENT_STACK_LOCATION(a);

During synthesis this is evaluated each time we dig deeper, and compared to the initial stack position.  If we go too deep, we flag the error and abort the synthesis, unwinding the stack with returns triggered by an error check.

How High Can You Get?

The other important question is "how big is the stack?".  There doesn't appear to be a way to measure this in code and some investigation reveals that this is something you can set in your EXE project settings (for C++ project), or via some EXE fiddling (with EditBin.exe) for C#.  In both cases though, the default seems to be one megabyte.

Assuming this 1 MB is actually 1,048,576 bytes (2^20) I decided a 1,000,000 byte threshold would leave a nice safety margin after triggering an abort of the synthesis run.  This seems to work fine, and you can see the results of this with the simple test case (proposed earlier).

Results of simple stack overflow test procedure running.

We managed to get to an evaluation depth of 15,624 and it was a final stack frame of (at least) 96 bytes that pushed us over the million byte limit.  This is plenty of space for very complex procedure evaluation, but with the safety net of knowing you aren't going to crash.

Engineering

It's quite likely that I have missed some useful API call or technique that could have saved me some time or prevented various assumptions here, but I guess this is an engineer's approach to the problem; measure, try stuff, get something that works, and test it.  It seems to work, and until I find a platform/machine/OS/scenario where it doesn't (or a suggestion to improve it)  I will consider it solved for now.  If you have any suggestions, or concerns, please chip in below.

Thanks

A quick thank-you to those that came up for a chat and see me demo some of Apparance at last weeks Tuesday Night Gamedev Pub gathering in Guildford.  There were many good questions and interesting conversation, and I could see that the whole project was making people think, hard (either that or the beer was making it hard to think).  Your questions and chatter prompted the writing of this post.  I hope to demo again next time for those that missed it.

Future City: Update 6 - A Balancing Act

Industrial Zone

Using the previous steel structure procedures and some new bits and pieces I've been building a few elements for populating the industrial areas of the city.  For now we have:

• Warehouse or factory building, with roof-top windows/solar panels, and chimneys.
• Round storage silo or tank, with and without steel support structure around it.
• Simple tower based on the steel-work structures.
• Simple chimney of a couple of different styles.

Structures for first-pass industrial zone

During the creation of these, I've been regularly checking the look of the city.  I was pleased with the initial appearance and the variety and layout was working well giving a good view.

Structures in use throughout the industrial districts.

A Recurring Problem

As more elements and more detail were added, holes in the industrial blocks often appeared.

A hole in the industrial zone

This is an indicator of a procedure synthesis running out of buffer space, due to either too much geometry (triangles/lines) or too much calculation (procedure depth/complexity).  To find the problem procedure a bit of digging using the view diagnostics panel is required thus:

1. Set render depth to show only the tier with the problem in.
2. Turn on octree node display.
3. Set octree node depth to match render depth selected above.
4. Set the node highlight to the node surrounding the problem area.

First stage of diagnosis; what is the error?

This should show up the actual error message (stat shows first error message under node).  To see exactly what happened during the synthesis of this model we can perform these further steps:

1. Open the engine dev panel to show the synthesis monitoring panel.
2. Switch on node model filtering to only render models under this node.
3. Set the model select to only render the problem model.
4. Request synth analysis graph generation (triggers re-synth).

Second stage of diagnosis; which exact model?

This will re-synthesise the model (shown in the synth panel) so we can look at the output stats in more detail.

Third stage of diagnosis; what sort of memory did we run out of?

From this we can see that we are running out of parameter buffer (abbreviated to 'Control' here), the memory used for all the information flowing between and through the procedures as the calculations are performed.  This will be because the city block sub-division is quite demanding due to there being quite a depth to the process between the top level of the city at which we are starting and where we actually start to generate geometry.

Turning Up The Heat

Hitting these problems has left me feeling a bit in the dark about how well the system is performing in general, how close we are to having them happen again else-where, and where else we may need to improve the procedures building our scene.  It is a balancing act fitting the design and requirements into the resources available (as is often the case), but this is going to be a common workflow occurrence and we need more tools to help us monitor the situation as we work.

A visualisation tool often used for assessing the overall level of something and getting a good feel of how some value varies across a space is a heat-map.

An example of a heat-map

There are many metrics we could benefit from being able to visualise such as:

• Rendering cost (triangle count, draw calls, materials)
• Memory use (system memory, GPU memory, synthesis buffer)
• Synthesis cost (time/cycles taken, buffer use)
• Measures of complexity (synth depth, stack use, passes generated, analysis, refinement level)
• Scene complexity (octree nodes, model counts, mesh counts, node status)

All of these help us assess the overall cost and effectiveness of the procedures creating our world.

Some support work was required to capture the statistics needed from the synthesis process, and propagate it through to the rendering stage (with the meshes). Plus, because the output of synthesis is often merged into single meshes, we also needed a way to render parts of meshes separately to give per-procedure colourising.

Various heat-map modes: Triangles, Cost, Time, Stack, Depth

Each new display mode has its own scale configuration setting so that the range of values being visualised in the scene can be seen in the most detail by spreading them across the full colour gradient.

Gradient scaling settings for each heat-map mode

By also having a distinct colour for a 'maxed-out' value, i.e. on or above the scale setting we can interactively measure the value on a particular element in the scene by dialling the value up and down until we are on the changeover point between being the top end of the gradient and the highlight colour.

I'm not 100% happy with the choice of colour gradient, but full spectrum gradients (which I assumed would be best) seem more suited to continuous functions and not the more random patterns we get here.  Instead I'm trying a simpler black-blue-cyan-white 'icy' gradient, with red for out-of-range values (above the chosen scale setting).

Hot In The City

Testing these display modes out on Future City as a whole shows that we are actually in a pretty good position with fairly low quantities of geometry and no hot-spots that really stand out as problematic.

Triangle counts visualised across detail levels.  Only a few hot-spots at around 7,500 triangles.

It looks like we can actually afford a fair bit more geometry and detail, but at the moment we can't afford synthesis cost (hence the holes).

Typical synth stats for chunk of the city.  Scope for a re-balance I think.

In the upper (distant) tiers most models are in 5,000 to 10,000 triangle range, and the lower tiers are at about 1,500 to 3,000 triangle range, which is pretty low for assets these days, with the overall GPU memory at about 100MB with the viewpoint in the centre of the city.  The draw-calls are moderately high at about 1,000 but this shouldn't change much as more detail is added and can be improved in other ways later on (I'm not doing a frustum cull yet for example!).

So, we need to reduce the synthesis cost.  There are a couple of ways we could go here:

1. Reduce the complexity of some of the more complex procedures, for example we could convert our super-splitter procedure (see last weeks post [link]) into an operator.
2. We can re-balance the way each synthesiser's buffer is allocated so there is more parameter space available (and less geometry space).
3. We can look at one of the several planned optimisation passes on the synthesis processing and evaluation itself.
4. Other ways?  (there are always other ways).

At the moment I think it is a bit premature to bake the Splitter proc into a procedure as we lose the ability to live edit and continue to improve it (much slower turnaround).  Some investigation of the usage of buffer memory would be helpful in deciding on a new parameter/geometry split for the buffers, perhaps with a buffer use heat-map mode?  I didn't want to get into optimising the synth system yet as I think I can get more elsewhere first and it's tricky work.

Hot-line

I did do some work simplifying the steel tower procedure, which seemed particularly bad, using line primitives (which I have so-far not needed).  In the far distance, the steel beams reduce down to 8 triangles each, but this seemed too much so I added an extra detail switch to a single line segment.  This helped the geometric/visual detail level, but didn't  help the synthesis complexity as we were still having to sub-divide down a lot to get to the individual girders.  Instead of relying on these line based beams I remembered I had a line grid primitive (originally used to draw the ground-plane grid) and so switched this in at a higher level for the steel-work side sections.  Thus each tower is now made of four grid primitives in the distance.  I took this a stage further by double the grid cell size progressively too as I noticed that the fixed width (screen space) line rendering becomes a solid block when the tower is really small.  This helps keep it looking thinned out and again reduces the render cost.

Reduction in tower complexity at lower detail levels by use of line (grid) primitive.

Late Night Ideas

One night last week I couldn't sleep for thinking about these problems and an enticing thought occurred to me about how I can neatly improve this synthesis depth/complexity problem by an order of magnitude and allow lots more detail for the distant city.  This will help a lot for aerial views of the city where you should be able to see a huge amount of small detail when at a considerable distance (probably the worst case scenario).  I think I'm going to explore this next after playing with the synthesis memory balancing.  More details next time… maybe…