Your submission was sent successfully! Close

Jump to main content

Getting physical

Canonical

on 14 September 2010

Introduction

With the ingress in the market of products like Nintendo Wii, Apple iPhone and Microsoft Kinect, developers finally started realizing that there are several ways a person can control a computer besides keyboards, mice and touch screens. These days there are many alternatives, obviously based on hardware sensors, and the main difference is the dependency on software computation. In facts solutions based on computer-vision (like Microsoft Kinect) rely on state of the art software to analyze pictures captured by one or more cameras.
If you are interested on the technical side of it, I recommend to have a look to the following projects: Arduino, Processing and OpenFrameworks.

Usage with Ubuntu

During a small exploration we did internally few months ago, we thought about how Ubuntu could behave if it was more aware of its physical context. Not only detecting the tilt of the device (like iPhone apps) but also analysing the user’s presence.
This wasn’t really a new concept for me, in 2006 I experimented with a user proximity sensitive billboard idea. I reckon there is a value on adapting the content of the screen based on the distance with who is watching it.

We came up with few scenarios which are far to be developed and specified, hopefully will just open some discussions or, even better, help to start some initiatives.

Lean back fullscreen

If the user moves further from the screen while a video is playing on the focused window, the video will go automatically to fullscreen.

Fullscreen notifications

If the user is not in front of the screen, the notifications could be shown at fullscreen so the user can still read them from a different location.

Windows parallax

Since this is the year of 3D screens, we couldn’t omit a parallax effect with the windows. A gesture of the user could also trigger the launcher appearance (see prototype below).

Prototype

With few hours available, I mock up something very quickly in Processing using a face recognition library (computer-vision).
Despite it could be hard to detect the horizontal position of the user’s head without a camera, we are in no way defining the technology required. The proximity could be in-facts detected with infra-red or ultra-sound sensors.

Parallax and fullscreen interaction via webcam from Canonical Design on Vimeo.

Newsletter signup

Select topics you're
interested in

In submitting this form, I confirm that I have read and agree to Canonical's Privacy Notice and Privacy Policy.

Related posts

Ubuntu 22.10 on the Raspberry Pi delivers new display support and the full MicroPython stack

Ubuntu 22.10 delivers support for a range of embedded displays, microcontroller development and more for the Raspberry Pi.

Canonical releases Ubuntu 22.10 Kinetic Kudu

Ubuntu 22.10 improves the experience of IT administrators and enterprise developers with a particular focus on the IoT ecosystem.

How Ubuntu Pro delivers enhanced security and manageability for Linux Desktop users

This week Ubuntu Pro entered general availability, giving Ubuntu users access to expanded security coverage on top of key enterprise management features.