Posts Tagged ‘programming’

« Older Entries

A quick rant about consistency

Friday, November 13th, 2009

So, I’m doing lots of exciting gathering of statistics and drawing of graphs at work at the moment, and have been making use of the excellent Matplotlib and NumPy Python libraries to help me. They’re both really handy bits of code, but there’s some awful examples of inconsistent design in there. For example:

import numpy
data = numpy.array([1,2,3])
mean = data.mean()
median = numpy.median(data)

– that is, mean() is a method on array(), which makes sense, but median() is a global function that takes an array as a parameter. I’m sure someone, somewhere, has a really good reason for that. But I’m also sure they’re pretty much wrong.

Tags: , , , ,
Posted in Geeky | Comments Off on A quick rant about consistency

Cityscape – update 14

Sunday, July 12th, 2009

So, we’ve got a road layout – now we just need to add some buildings to it. For a first pass, I just add simple blocky buildings; one per lot, with a height equal to the minimum of the width or height of that lot, and it looks like this:

…which isn’t bad, I guess, but there’s these enormous monolithic slab like buildings all over the place, and it looks really homogeneous again. So, let’s add the other building types back in and see what that looks like:

Well, that’s a bit better, but it still looks a bit Bladerunner-ish – the big buildings are still far too big, and I’m getting scene-dominating and unrealistic massive wedges still. It just doesn’t look realistic – I need to reduce the size of some of the buildings.

So, the next step is that once I’ve done my initial load of lot-splitting, as documented in the previous entry, I then loop over all the lots I’ve created, and if any of them are bigger than a certain maximum size, or have a length:width ratio of greater than 5, then the lot is split again, and this repeats until all the lots are a sensible size/ratio. And the final result of that is this:

…which I’m much happier with. I’ve also played around with the fogging a bit, which means the buildings don’t start to fade out until they’re much farther away; I’m not totally convinced it helps – especially on the NC10, which these screenshots were taken on – it just makes the scene busier and adds more visual noise. I’m also a bit unhappy that the buildings still all seem to blend into one – it looks better in motion, but perhaps I need to increase the amount of colour modulation that happens with the windows? The other this is that there’s no street lights or roads yet – I’m hoping that these might help to break the city up a bit more, and that’s the next step.

(note that the framerate in all these screenshots is ridiculously poor – this is because I took the screenshots on my little Atom-powered Intel-integrated-graphics netbook – on a machine with a sane graphics card, we’re still getting close to 60fps – although not close enough that I’m happy to leave it without future optimisations)

In the meantime, you can, as ever, check the code out of the bzr repo – we’re up to revision 45 now.

Tags: , , , ,
Posted in Cityscape | 9 Comments »

Cityscape – Hello, internet!

Thursday, July 9th, 2009

So, Shamus Young, the lovely man whose PixelCity project this entire developer diary is inspired by a tribute to a total rip-off of, has linked to it from his blog. Which means my pages views for the last couple of days have totally exploded.

I left a comment on his blog, but I should perhaps address a couple of the same points here:

  • I’m doing this because I can, and because I wanted something to do in the evenings that my wife is out of the house. I’m on a hiatus from World of Warcraft at the moment, so figured a programming project might be fun, and as an ex-games developer with a lingering interest in 3D graphics, PixelCity provided a perfect inspiration.
  • I’m focusing far more on the technical aspects of my project than Shamus did for his – this is because, as I said above, this is a total rip-off of his original project, so all the interesting stuff about what looks right and making things look realistic has all already been written about. This isn’t a “How to draw a realistic city” diary – it’s a “How I did it in C#/XNA using more modern techniques” diary. As things progress, I might head in a different direction on some aspects (like the city planning entry) – I’ll discuss what I’m doing there in a bit more detail.
  • The C#/XNA “ZOMG MS ARE TEH EVILS” thing is not something I’m interested in discussing. I just can’t get worked up about it, I’m afraid.
  • I make no promises about how often this will update. It was pretty rapid at the start but, as with many things, it’s sort of tailed off a bit lately. It’s got sunny, I’ve been busy at work, my chilli plants need tending – I fully intend to carry this through to… well, some poorly-defined and probably still distant end-point, but I have no idea when that might be yet.
  • I apologise for inconsistently switching between first-person singular (“I did [x]”) and first-person plural (“We need [x]”). I’m a programmer, not a writer.

I think that’s about it. Thanks for dropping by; I hope you enjoyed what you read.

Tags: , , , , , ,
Posted in Cityscape | 4 Comments »

Cityscape – update 13

Monday, July 6th, 2009

So, I’ve been busy trying to work out algorithms for generating interesting city layouts. I spent a good long time chasing down alleys that turned out to not work too well: I’d decided that I wanted something a bit more elaborate than the straightforward grid layout I’d been using (and that TwentySided also seemed to use), but making things look interesting and realistic, but not also horrifyingly complex to code turned out to be more difficult than I thought. I also caught myself in the classic case of refusing to throw away code I’d already written and piling more bad code on top of bad code.

One approach I initially tried was to start with an empty city, and then plant lots on it, starting with large lots clustered around the centre, and then gradually filling in the gaps with decreasing size lots. Each lot would be bordered by roads on each of the four sides, and then I’d add random roads afterwards to divide up any remaining space. However, this often ended up with a load of roads running right next to each other – and also, I wanted the ability to have variable width roads, and this approach didn’t really lend itself to that. So, after far too long, I gave up and threw it away, and came up with the approach that I’m working with now.

Instead of starting with an empty space and adding lots, I decided to do exactly the opposite – start with a space filled by a single large lot, and then carve it up: so, the initial lot gets carved into two, and then each of those gets carved into two, and so on. Each split creates a road – the initial road width is wide, but as the lots get smaller, so do the roads – thus areas with a large number of small lots will tend to have small roads, whereas areas with larger buildings will tend to have bigger roads.

I also added a number of heuristics to make it look more “city”-like: when splitting a lot, the split point is random, and may be horizontal or vertical – if the lot is taller than it is wide, the split is more likely to be horizontal than vertical, and vice-versa. This ensures that buildings are mainly roughly square – long, thing buildings are rare in real life. Also, not every lot is split: in order to create a more built-up CBD, for each lot, the probability of a split taking place is increased the further away from the center of the city the lot is, but in order to make sure the lots aren’t too big, the probability of a split is also increased as the size of a lot increases.

The result can be seen below as an animation – the road colours correspond to different “classes” of road, that may end up being rendered differently at some point.

The next job is to actually plant some buildings in the lots – there’s still some work to do here, because a lot of my code tends to work best with base sizes that are roughly square, and I’ve still got a fair few lots being generated that are of a size ratio 2:1 or more – but anyway. You can check out the latest revision – 44 – from the bzr repository.

Tags: , , , , ,
Posted in Cityscape | 3 Comments »

Cityscape – Update 12

Sunday, June 21st, 2009

Finally got round to adding some lights to the buildings to stop those pesky aeroplanes from crashing into them all the time:

Again, as with so many things, these are not so simple as they appear. Several steps were needed to get these little static lights in place – but hopefully, it should mean later things like streetlights, cars and, hey, maybe fireworks of something, should be much easier to implement.

First of all, we generate another texture – a simple circular fade from white to transparent that can be used as the basis for all lights. At the moment, the fade is linear, and does look a little artificial, but it’ll do for now.

Secondly, we need some way of rendering these: I’m using a billboarding technique that’s fairly widely used in particle systems in most games. Essentially, you render a quad that always points towards the camera, giving the illusion of a solid 3D object with rotational symmetry. The effect – especially at a distance – is quite convincing. The trouble is, we need to recalculate the position of these quads every frame, to ensure they’re always pointed towards the camera – and doing this on the CPU can get expensive. So, the obvious solution is to use a vertex shader: rather than pass in the pre-transformed coordinates, for each vertex in the quad, we pass in the location of the centre of the quad along with an offset to indicate which corner we’re talking about (handily, this also corresponds with the texture coordinates, so we can re-purpose the texture component of the vertex data for this). Then, in the vertex shader, we use this information along with the position of the camera to calculate the real quad vertices.

Lastly, we need something to manage all the particles, so I’ve added a ParticleBatch class, analogous to the BuildingBatch, that looks after all the particles, calculates the vert buffer, renders them and that sort of thing. It provides an IParticleService for other game components to grab and dump their particles into: The naming of some of the interfaces should give a hint as to one of the current restrictions – that is, we only support static particles at the moment. This is handy from a speed perspective, but means that nothing in our scene can move: this is something we’ll have to address a bit more when I want to add moving vehicles and things.

We’re up to revision 40 in the repository now. Next step – I’m going to have to delve into the murky world of city planning, and hopefully make things look a bit less like a random collection of buildings, and a bit more like a real city…

Tags: , , , , , ,
Posted in Cityscape | Comments Off on Cityscape – Update 12

Cityscape – update 11.1

Tuesday, June 16th, 2009

It occurs to me that the screenshot attached to the last post doesn’t show my classic buildings off terribly well, so here’s a grab of just one of them:

Tags: , , , ,
Posted in Cityscape | Comments Off on Cityscape – update 11.1

Cityscape – update 11

Tuesday, June 16th, 2009

It’s been a while. It feels longer, as I turned 30 since my last update, so happy birthday me. It’s all downhill from here. I’ve still been working on this darned city, though, and here’s a screenshot to prove it:

So, what do we have since last time? Well, building on update 10, where I clarified the box-building code by assembling boxes out of 5 panels (I ignore the base, as my intent is to always have the camera high enough that you can’t see the bottom of a box), I’ve actually introduced columned boxes. To see what I mean by this, see the following diagram:

There’s two ways of creating a columned panel; either way, we can specify the width of the windowed panel, and the width of the spacer columns. Then we either specify the desired number of windowed panels (easy to do, harder to use), or the desired width of the whole panel (which is harder to do, as I discovered, and obviously means that some compromise has to be reached if the desired width is not exactly achievable with the panel widths supplied). The algorithm I use essentially ensures the whole panel is symmetrical about the mid-point, and won’t ever have two blank column panels next to each other. The implementation is a bit yucky, but you can check it out in BuildingBuilderPanels.cs in AddColumnedPanel() if you’re interested.

That done, I set about making panelled boxes, with exactly the same sizing options as before, and then from that made a “Classic” tiered building design, as you can see in the screenshot above. This is an extremely customisable building type, and takes a ton of parameters on construction. Basically, each tier is slightly smaller in width and height than the one below it, and is separated by a black spacer block, which can overhang the blocks slightly. All the box sizes are rounded to integer numbers of stories and window-panels, and (for the moment) stretched textures aren’t allowed, although I might bring these back in if I can figure out a clean way of doing it. I’ve also added a simple routine to add some rooftop furniture, but I’m not totally happy with this yet – it doesn’t really stand out enough for me.

If you check out the latest revision, you’ll also see the very, very early phases of some particle code that I’m going to be using for things like streetlights, cars and lights on top of buildings – these will just be simple billboarded particles, not actual light sources (although maybe I’ll change to deferred rendering later and have them as actual light sources… or, maybe not!) and I hope to have them in some time in the next few days.

I’ve also spaced out the buildings a bit more – it means there’s blue gaps in the world now but I think the city looks a bit more natural spaced out like that; my city planning code is still extremely rudimentary, so needs quite some work doing on it, anyway.

The latest revision in the Bazaar repository is 37 now – check it out and see what you think.

Tags: , , , ,
Posted in Cityscape | Comments Off on Cityscape – update 11

Cityscape – update 10

Sunday, June 7th, 2009

Again, no pretty pictures this time, because there’s not actually anything to see. I’ve spent a few days refactoring things in Cityscape, and having completely broken and then unbroken pretty much everything, I’m about back to the point I was a week ago, but with (hopefully) a somewhat tidier and more sensible bunch of geometry generation functions. I’m still a bit unhappy about the number of parameters these things take, but it’s a tradeoff between how much the caller needs to figure out for itself and how much flexibility the callee offers.

I’ve essentially ripped out the entire box generation function, and replaced it with a far more sensible (and hopefully) simpler set of routines that builds a box out of a set of panels. The idea is that now I’ve got a function to build panels in a useful way, I can use it to build more interesting looking buildings with blank spaces on their faces; this should hopefully make the city look less uniform.

One thing that keeps biting me in the ass is XNA’s peculiar insistence on using a right-handed coordinate system; I spent a good while trying to get the panels to line up on my boxes because my brain basically just can’t deal with the idea that positive-Z comes out from the screen rather than going into it. I really, really don’t get why MS did that, and if anyone knows the story behind it, I’d love to know.

Anyway, you can pull down revision 27 from the repo if you fancy having a look at the new building generation stuff.

Tags: , , , , , ,
Posted in Cityscape | 1 Comment »

Cityscape – update 9

Thursday, June 4th, 2009

Not a terribly coherent entry, this one. And no pretty screenshot for you to look at either, I’m afraid – in fact, it’s not going to be very exciting at all. I’d skip it if I were you.

However, if you’re determined to know what I’ve been up to lately, you’ll find the following bits and bobs in the latest revision.

  • Cylindrical buildings – and therefore, also, a new cylindrical primitive type. I tried to get this working in such a way that I could carve off sections to give a more modern, building-ish look, but I had trouble getting the surface normals to behave. The way I’m doing it at the moment, I generate vertices around the central point, with the normal for each vertex pointing outwards, and then generate the polygons by sharing the vertices. The problem comes when you slice a bit off, and the normals become interpolated across the face, thus:


    The led to some weird lighting artefacts and the only solution I could see that would eliminate the problem simply would be to stop sharing vertices and rework the whole cylinder geometry generation code. Which is a job for another day, I think.

  • Manchester “Beetham Tower” style buildings – This is quite an iconic building in Manchester and a simple enough one that I figure it’s worth putting in – it’s basically a simple block skyscraper with an overhang halfway up:

    Simple as it may be, though, it took me quite a while to get my textures to line up, and it’s still not right – also, the tower has quite characteristic window patterning that I should perhaps attempt to replicate. Still some work to be done here, I think.
  • Lastly, I’ve factored out a lot of the building builder classes into separate files – the Building.cs file was getting unwieldy and awkward to navigate, so now it’s much simpler.

I’m still not happy with the buildings: the big problem at the moment is a lack of detail: the buildings are all too uniform and the flat walls-of-windows lack any sort of visual interest. I’ve got a couple of ideas to add more interest – adding black columns to break up the walls of windows, a bit of rooftop furniture and more detailed buildings, that sort of thing – and hopefully I’ll find some time to add that sort of thing soon; it requires quite a bit of reworking of the BuildingBuilder, though, so it might be a while before we see any real results from it…

We’re up to revision 25 in the repo now. I wouldn’t check it out, though, as I seem to have forgotten to add a file to the commit. Bugger. I’ll sort that out when I get home tonight.

Tags: , , , ,
Posted in Cityscape | Comments Off on Cityscape – update 9

Cityscape – Update 8

Wednesday, May 27th, 2009

Unfortunately, today’s minimised screenshot doesn’t look so hot (I guess it’s something to do with Flickr’s contrast enhancing algorithm), so if you want to get an idea of the latest developments, you should probably click through to the original image on Flickr to see it properly.

The big news this time is that I’ve added lighting attentuation and fogging – cities are big, nasty polluted things, and it’s rare that you can see more than a couple of blocks anyway – plus, it means that when I add in distance culling in a while, I we hopefully shouldn’t see things pop in and out too obviously 🙂

Lighting attenuation is just the light falloff as things get further away from the light – I’m not 100% convinced it’s worthwhile at the moment, though. Fogging is very similar: a gradual blend towards the fogging colour (in this case, it’s the same colour as the sky) as distance from the viewer increases.

Both of them are very simple to implement in our shaders – a couple of parameters to control falloff and fog colour, and a little bit of extra code to calculate the light falloff/fogging amount – basically, 1/(distance * k), where k is a more-or-less arbitrary fudge factor. The actual fog blending has to be done in the pixel shader, as we need to interpolate between the final resultant colour, lighting, texture and all, and the fog colour. This unfortunately utterly kills framerate on Intel integrated graphics chipsets; their poor little pixel pipelines just can’t take it. There’s actually a better way to do fogging than this on shader model 1.1 chipsets – the shader model actually supports fogging parameters – so I might implement a version for Intel chipsets that uses this mechanism and see if it improves matters.

I’ve also changed the building generation code: I’ve factored out an IBuilding interface and a BaseBuilding that can be simply inherited from to create different building types – I’ve renamed my existing building class UglyModernBuilding, and added a SimpleBuilding (a straightforward single block building). I’ve also factored out the actual geometry generation methods into another class, BuildingBuilder. I’m beginning to think Building.cs is getting a bit unweildy now, so there might be more changes/tidyups to come.

So, you’ll notice the polys/sec is well down: about 8 million polys per second now. I’m not entirely sure what’s causing this – turning any one of my individual effects off doesn’t seem to recover the framerate so it’s probably something more subtle. I’m not that worried about this right now, as 8 million polys per second is still pretty respectable, but I’ve got plans for more complex buildings pretty soon, which is going to eat into that budget pretty quickly, and necessitate either finding some more polys/sec, or implementing some sort of distance culling or LOD twiddling fairly soon.

Anyway, the latest bzr revision is 23; have a look if you like.

Tags: , , , , , ,
Posted in Cityscape | Comments Off on Cityscape – Update 8

« Older Entries