Interacting With Spaces

This project was part of my interaction design education at Malmö University. Built with Simon Ulvsand

There's a kind of magic in a room where the air changes when people move through it. The acoustics shift, the energy shifts, and mood transforms. For this project, we asked a simple question: how might we enable spaces to respond to people?

Constraints as creative fuel

Our ideation session started with a camera, some body-tracking code, and a set of examples to pull from. On paper, that sounds limiting, but as we learned through creative projects, constraints can be a creativity enhancer.

When you can't wander too far into abstraction, you're forced to commit. We reframed the camera to be the eyes of the space, body movements and interactions between people became inputs, and Sound became the output. We envisioned the concept before we'd written a line of code, and that clarity turned out to be generative.

I've noticed this pattern before, in design, in writing, in almost any creative work. The blank canvas is often the hardest surface to work on. Give me a constraint and I know where to push.

What we were building

The concept we landed on: a space where your body, and your relationship to other bodies in the room, generates sound in real time.

Not a fixed mapping where left means this note and right means that one. Instead, a dynamic sound that responds to pose strength, how expressively, how fully, you're using your body. And where in the room one is moving, and how close/far they are from other people.

The goal was to dissolve the boundary between the person and the space. You enter, you move, the room responds. Someone else enters, and the relationship between your sounds begins to shift. The space communicates the atmosphere and mood of the people in it, without any explicit coordination or clear awareness of how people are influencing this output.

Jens, our supervisor, framed it through a concept from Janlert & Stolterman's work on impression and expression: every interaction involves something received (an impression) and something produced (an expression). We were building a system that would make those two things indistinguishable, where your body receiving the space and your body expressing through it were the same act.

The geometry problem

The tracking library we used gave us X and Y coordinates. Not depth. A flat read of a three-dimensional space.

This matters because distance, real distance, the kind you feel when someone walks toward you, includes Z. We spent time trying to estimate depth through shoulder-width changes: if someone's shoulders appear narrower in the frame, they're further away. It worked, barely, for one person. With multiple people moving simultaneously, it became unreliable fast.

We almost abandoned it. Then I wanted to try one more thing.

If you can't measure depth directly, you can sometimes approximate it through geometry. Specifically: if you treat the positions of two people in the frame as two vertices of a right triangle, the hypotenuse gives you a distance estimate that encodes both horizontal and vertical separation simultaneously. Pythagorean theorem, applied to camera coordinates, used to stand in for something the camera couldn't see.

I have a soft spot for moments like this, where a mathematical idea you learned years ago turns out to be exactly the right tool for a problem you're facing now. It's always surprised me when students ask what the point of learning math is. The honest answer is that it's less about the specific formulas and more about the framing of problems. The ability to look at an ambiguous situation, find a structure in it, and reason through it. That's what programming teaches too, which is part of why I think it belongs in schools alongside mathematics, not as a vocational skill, but as a way of thinking.

The geometry worked. The values it produced were accurate enough that we could use them cleanly as inputs. My teammate Simon started building sounds.

What happened when people walked in

Once we had a working system, testing it was the best part. Each person who enters the room get assigned a unique tone. The tone will change its volume, frequency, and velocity depending on the person's posture, interaction with the space and with other individuals.

People would enter the space without being told what to do. Within a few seconds, they'd notice something, some ambient shift, a tone that seemed to respond. They'd test it. Move toward someone. Step back. Reach their arms out. The system wasn't obvious enough to be immediately legible, which turned out to be a feature. The ambiguity made people curious, and curiosity made them move more expressively.

We'd also added a background ambient sound, a layer beneath the individual tones that gave the space its own texture. It helped the sounds blend as new people entered, but it also did something subtler: it made the room feel like it had an identity before you arrived in it. You were stepping into something, not switching something on.

One thing we noticed we couldn't fully test: the camera. Everyone in the room knew it was there, knew it was tracking them. That awareness pulled attention toward it, toward the system itself and away from the space and the people in it. The most interesting version of this experience probably only happens when the technology disappears. When someone walks into a room and just starts feeling it, without knowing why.

That's hard to prototype in a class of interaction design students who are actively watching each other's technical decisions. But it's the right goal.

What it points toward

The interaction paradigm we're used to is still deeply screen-centric, even as devices migrate onto our bodies and into our environments. We look at things. Things show us things back. Information is predominantly visual, delivered in discrete moments, requiring attention.

What this project was reaching toward is an ambient mode. Interaction that doesn't demand your focus because it's already woven into the texture of being somewhere. Your presence in a space becomes the input. The space becomes the output. No interface required.

This is, if you squint at it, close to what Janlert and Stolterman describe as faceless interaction, the idea that as interfaces mature, they recede. The most sophisticated systems are the ones you stop noticing. They just respond.

The prototype wasn't close to the level of smoothness and invisibility we wanted, but it made the idea real enough to communicate to others, which is what prototypes are for.