Microsoft
MR Studios
Creating Mixed Reality Apps for Customers
Role: Research, Concept, Tech-Art, Code
Since 2016, I've been designing and building Mixed Reality experiences for exhibitions, training, visualisation and more.
Engaging in end-to-end projects from identifying needs, through to ideation, prototyping, implementation and deployment.
Touching
Holograms
Exploring more immersive Mixed Reality interactions
Role: Concept, Code, Writing
What if you could treat holograms just like real objects?
I attempted to answer this question, through prototyping interactions on HoloLens 2, based around physics engines combined with articulated hand tracking and eye-gaze tracking.
The design process and findings are documented in an article for the Microsoft Design blog on Medium.
Kin
Explore your heritage by building a 3D family tree
Role: Concept, Code
One summer, on a family holiday, we attempted to map our extended family tree. After many hours, our creation was an unreadable, 3 page wide scribble, in which close relations, with whom we grew up, appeared far away from us. The traditional tree structure became difficult when depicting multiple marriages and the children of these marriages. The scribble tree also bore no relation to time as it was based around generations which were different in length.
This attempt inspired the design of a more engaging and inclusive family tree, where space is used more efficiently and multiple marriages are treated as a fact of life, not an edge case.
People are positioned vertically by their birth year, and people act as repelling magnets, held together by elastic family connections. The structure of the tree emerges from these constraints. Each person has a colour and these colours are mixed together in their children to create a vibrant nest of people in the family.
In this cross-platform app, people can manipulate and edit their family tree using refined multi-touch controls and simple UI. The trees can be shared between family and friends by sending a .kin file.
Blobjects
3D Modelling in AR, with real-time ray-tracing
Role: Concept, Code
An investigation into how 3D modelling and rendering could be made more intuitive in AR.
Use a 3D brush to paint in mid-air without needing to learn complex 3D camera controls.
Pose with your creation in high quality photo renders.
Use the phone's camera to capture materials from the real world and apply them on your model.
Add or remove material from a solid model without needing to learn about 3D mesh topology. In fact, there is no mesh! Blobjects uses SDF sphere marching instead.
The models are shaded to blend with the real world, casting soft shadows, reflections and self-reflections with ray-tracing, running in real-time on a Google Pixel 3.
BOTSU
Multiplayer, Physics-Based Team Sport Game
Role: Concept, Code, Music
Solo project to create a video game based on emergent interactions from procedural, physics-based animations.
Players get to grab, punch and throw each other while trying to score a goal for your team.
Endless unique and funny moments arise as anything can happen.
CodeJumper
Programming for Children with Visual Disabilities
Role: Concept, Industrial Design, Prototyping, CAD
A tangible programming language, inclusive of visually impaired children, developed at Microsoft Research.
Each bead on the chain is a function call with buttons and knobs to pass parameters. The physicality helps to make complex programming concepts intuitive, such as loops and threading.
It is designed to be explored either visually or by touch, enabling collaborative play between students with different levels of impairment.
Zanzibar
Flexible Sensing Mat
Role: Industrial Design, Interaction, Prototyping, CAD
Designed for tangible interactions, Zanzibar tracks the identity and location of objects placed upon it, as well as touch and gestures with the objects and the sensing surface.
This was a research project to create the device as a platform for applications, then build prototype apps and games on top.
Fantastic
Flocking
Real-Time Mass Group Behaviour Simulation
Role: Code, Animation
Simulation of group behaviour for birds, fish and crowds of humans.
Fantastic Flocking was created for Mixed Reality apps and is optimised for mobile with thousands of agents using a K-D Tree for neighbour searching.
Nebula
Painter
Paint Emissive Clouds in VR
Role: Concept, Code
Creating 3D content is usually done by manipulating surface polygons or programmatic expressions. But what if you want to depict something that has no surface, and cannot be easily expressed in code?
Nebula painter is an investigation into painting non-solid volumetric geometry such as nebulae.
It uses a custom ray-marching shader together with a compute shader to paint and render lit, emissive volumetric clouds in Unity Engine.
Real-Time
Path-Traced
Modelling
Solid Modelling with Photo-Realistic Lighting
Role: Concept, Code
An investigation into alternative modelling methods and geometry. This method makes ray collisions inexpensive compared to polygon modelling.
On top of this, I built a ray tracer inside a Unity shader that is able to render at real time thanks to the performance unlocked by this modelling method.
Evolution
Simulator
Simple Creatures Mutate to Survive by Natural Selection
Role: Concept, Code
Each creature has a food chain level and a set of attributes that govern its behaviour and reproduction. Energy is used when moving and when the energy is used up, the creature dies. The ones that seek out food survive to reproduce, gradually evolving the species.
Creatures mutate when they reproduce and even the mutation rate mutates. Eventually, some move up the food chain and eat others to survive and become much larger.
Created with Processing.
BBC: The Big Life Fix
Improving Quality of Life for People with Specific Needs
Role: Concept, Code
Helped to build technology to assist people with their disabilities through series 1 & 2. Supporting Haiyan Zhang's projects by creating software and hardware.
These included:
Breath-controlled games to help a boy keep to his cystic fibrosis physiotherapy. Ongoing trials at Great Ormond Street Hospital.
Classroom companion app for a girl with memory loss to help her remember and learn.
Leg-tracking sports games for a boy with muscle difficulties to encourage physiotherapy and help him learn to walk.
Loop
Navigation Device for People with Visual Impairments
Role: Concept, Industrial Design, CAD, Electronics, Code
How could our interactions with a navigation app change when it does not need a display?
Designed as a final year university project to be the hardware counterpart to Microsoft Soundscape, this fully working prototype connects to a smartphone to provide a more intuitive user experience for visually impaired users.
The touch strip and back button allow users to navigate lists with haptic feedback.
The orientate button on the top will tell the user their location and call out nearby landmarks in 3D audio.
Information is delivered to the user through 3D audio, localised by the compass in the ear.
To prevent the 3D audio from blocking the user's hearing, each earphone has a microphone to pass through sound as if the earphone was not there. Users can cross-fade between real and virtual sounds.
The device is designed to be inconspicuous yet always within reach, hanging as a pendant around the user's neck.
