Project Overview

Houselights is a self-directed concept for a hands-only spatial concert app on Meta Quest 3. There are no controllers and no voice input: you look at something to aim and pinch to select. Users browse by genre, open an artist, unlock a show, and watch it on a floating screen with the setlist at their side.

I designed the spatial layout, the interaction model, the component system, and the brand. I directed an AI model — Claude, connected to my Figma file through the Figma connector — to do the building. The result is a fully playable prototype of about 90 boards, a 49-component library whose states double as the engineering spec, a rendered logo animation, and a complete record of how the direction happened.

Role & Responsibilities

Art director and designer. I set the visual language, the spatial layout and the interaction rules, chose the content, and made every call on what stayed and what went. Claude worked inside my Figma file as the builder — created boards and components, wired the prototype, and rendered video from code — all to my direction.

The point of this case study isn’t the prototype — it’s how it got made. What follows is how I worked with AI as a co-worker while keeping every design decision in my hands.

Key Contributions

• Set a gaze-tuned interaction contract (90ms dwell, 120ms arrival, 220ms release) applied to every hover-reactive element so cards you glance across don’t light up
• Built a 49-component library where each component carries its rest, gaze, and pressed states as real prototype reactions — the library doubles as the engineering spec
• Directed the show view over three rounds, cutting two viewing modes down to one and adding explicit hide / show controls
• Directed the logo and intro animation through six font-only options and twenty-plus revisions, with the last video frame matching the sign-in board pixel for pixel
• Oversaw ~90 boards, 2,300 component instances, and ~950 prototype links — every design decision and final judgement mine

Working With AI

Claude was connected to the Figma file through Figma’s connector. That gave it the ability to read the file’s structure, run scripts against it (create frames, components and variants; set prototype links and timings; rename, move and restyle layers), and take screenshots of boards to check its own work. It also had a separate workspace where it could write code to render video.

I worked in Figma and in the prototype at the same time, reviewing each change, and gave direction in plain language. Nothing went into the file that I didn’t ask for, review, and approve. When something missed, I rejected it — sometimes after it had been built out at scale — and asked for the reversal.

What I did

• Set the concept, product scope and user flow
• Defined the visual language — glass panels, dark room, one accent colour, spectrum motif
• Wrote the interaction rules for gaze and pinch, including the 90 / 120 / 220 ms hover timing
• Chose between options by trying them in the live prototype — not from static mockups
• Made every content decision: artists, shows, setlists, what got featured
• Directed the brand and logo: picked the typeface direction, directed the animation revision by revision
• Gathered reference, judged what was worth borrowing and what wasn’t
• Had the final say on every change — including rejecting and reversing work that missed
• Placed every image and video into Figma by hand
• Recorded the prototype myself

What Claude did

• Built boards and overlays to the layout I described
• Applied the visual language consistently across ~90 boards
• Encoded the gaze timing as real prototype reactions on every component
• Built the options I needed to compare as live, hoverable components — not static comps
• Populated boards with content and corrected them when I flagged errors
• Proposed logo directions, drew every option, and rendered each animation revision from code
• Analysed reference material I provided and proposed layouts for me to react to
• Reworked or reverted on instruction, at any scale
• Edited the recording I made into the walkthrough video and looping clips

Direction Log

The evidence that the direction came from me. Real moments from the working session, in my own words.

“These need to be live components I can hover in the prototype. Right now they’re just static frames.”

Hover-depth options rebuilt as live, hoverable components inside the prototype.

“The hover timing is tuned for a cursor, not a gaze. That’s the whole problem.”

Hover timing retuned for gaze — short dwell, fast in, slow out.

“Revert the recede treatment across the prototype. Keep float only.”

A combined depth effect that read well on one board but too busy across the app — reverted everywhere.

“The angled side panels aren’t working. Set them straight, facing the viewer, as we had before.”

First build of the show view scrapped; panels set straight.

“Remove the 180° immersive mode entirely. We don’t need two viewing modes — one well-executed show view is the right call.”

Mode removed, switch removed, 111 prototype links repointed in one pass.

Where AI helped

• Scale and repetition. Changing the show view meant repointing 111 prototype links across the file — one instruction, one pass, links read back to confirm none were missed.
• Options as working things. Hover treatments and logo directions arrived as live components, so choices were made by trying them.
• Consistency. A timing rule or a colour rule, once stated, was applied to every instance without drift.
• Rendering from code. The logo animation was drawn frame by frame from the same letter geometry as the Figma file, which is why the last video frame matches the first prototype board exactly.

Where AI didn’t help

• Taste. First attempts were often rejected — the first logo, the angled side panels, the combined depth effect. The AI proposed; it couldn’t tell which proposals were good.
• Feel. It couldn’t play video inside the prototype or judge how a hover feels under a hand or an eye. I did that by using the prototype.
• Media. It couldn’t place images or video into Figma. I placed every one by hand.
• Accuracy. Content errors — wrong show details, a stray label — were caught by me, not by the AI.

Principles

• I direct, AI executes. Every design decision, every revision call, and the final sign-off were mine. The AI built what I art-directed. When it produced something I didn’t like, it was changed or removed.
• Open disclosure. I say plainly where AI was used and what it did, including on this page. The walkthrough video ends with the credit: “Designed and directed by John Lomax. Built in Figma with Claude.”

This prototype was built in Figma with Claude, an AI model, working to my direction through the Figma connector. The concept, the design decisions and the final judgement are mine.

Design Language

The interface is glass on a dark house. Panels sit at a fixed distance and behave like objects, not screens. One accent colour carries one meaning, so a pink element is always the thing you have selected or the thing that is live. A spectrum of colour ties the brand to the content: it sweeps around an artist when you look at them and it lives inside the logo.

Interaction Model — Eyes and a Pinch

Eyes dart and lock. Cursors glide. A hover tuned for a mouse lags the eye and lights up everything you glance across. I set a simple contract for every gaze-reactive element: wait 90 milliseconds before reacting, arrive in 120, release in 220. The target shouldn’t move under a fixating eye, so the lift is small and the depth cue comes from the shadow and the ring.

Component System — States Are the Spec

Every component holds its own states and the motion between them, so the library is the spec. When I wanted to choose a hover treatment I didn’t compare stills — I had six options built as live components on one board, hovered each of them, shipped one, tried a second across the whole prototype and took it back out when it was too much.

The prototype grew to about 88 live boards, a 49-component library, roughly 2,300 component instances, and ~950 navigation links — all consistent because a rule, once stated, was applied everywhere.

The Show View — One Decision at a Time

The show view took three rounds. It started as two viewing modes. I cut it to one: the show on a floating screen, now playing on the left, the setlist on the right, and a single control that clears everything but the music. An early build tilted the side panels toward the viewer — it looked clever and read badly, so I had them set straight.

Brand & Motion

The logo is two weights of one typeface and a strip of concert footage. HOUSE in a light weight, LIGHTS in a black weight, with the footage playing inside the heavy letters. The intro animation was rendered from code frame by frame, and its last frame matches the sign-in board in the prototype pixel for pixel — so the video hands off to the interactive prototype without a visible seam.