MTouch Multi-touch
SDK
The mTouch is the next generation of the world's most innovative
multi-touch devices. Its unique capabilities and powerful technologies will change
your tech lifestyle forever.
|
What is Tuio?
|
TUIO is an open framework that defines a common protocol and API for tangible multitouch surfaces.
The TUIO protocol allows the transmission of an abstract description of
interactive surfaces, including touch events and tangible object states. This
protocol encodes control data from a tracker application (e.g. based on
computer vision) and sends it to any client application that is capable of
decoding the protocol. There exists a growing number of TUIO enabled tracker applications and
TUIO client libraries for various programming environments, as well as
applications that support the protocol. This combination of TUIO trackers,
protocol and client implementations allow the rapid development of table based
tangible multitouch interfaces. TUIO has been mainly designed as an abstraction
for interactive surfaces, but also has been used in many other related
application areas. Technically TUIO is based on Open Sound Control - an emerging
standard for interactive environments not only limited to musical instrument
control - and can be therefore easily implemented on any platform that supports
OSC.
TUIO (Table-Top User Interfaces
Objects) is a simple yet versatile protocol designed specifically to meet the
requirements of table-top tangible user interfaces. Inspired by the idea of
interconnecting various existing interactive surface interfaces such as the reacTable or AudioTouch. The TUIO
protocol has been implemented using OpenSound Control and
is therefore usable on any platform supporting this protocol.
Simply put: TUIO is just a standardized way of formatting OSC data.
TUIO
Documentation (pdf)
Message Syntax:
Since TUIO is implemented using OSC it follows its general syntax:
/
tuio
/
[profileName] set sessionID [parameterList]
/
tuio
/
[profileName] alive [list of active sessionIDs]
/
tuio
/
[profileName] fseq int32
Parameters:
The parameters defined in this section reflect the object properties we
considered important for an interactive surface interface. A session ID number
is assigned to each object. This is necessary to uniquely identify untagged
objects across successive frames, and in the case where multiple objects tagged
with the same classID are simultaneously present on the surface.
Some of these parameters (id, position and angle) are retrieved directly by the
sensor. Others (speed, acceleration) are derived from these primary parameters
using timing information. Computing these parameters on the low level side of
an tangible user interface system allows a more efficient computation, since
the necessary timing information does not need to be transferred to clients.
The parameters allowed in a set message:
s sessionID
,
temporary ob ject ID
,
int32
i classID
,
fiducial ID number
,
int32
x
,
y
,
z position
,
float32
,
range 0.
.
.1
a
,
b
,
c angle
,
float32
,
range 0..2PI
X
,
Y
,
Z movement vector
(
motion speed
&
amp
;
direction
),
float32
A
,
B
,
C rotation vector
(
rotation speed
&
amp
;
direction
),
float32
m motion acceleration
,
float32
r rotation acceleration
,
float32
P free parameter
,
type defined by OSC packet header
Profiles:
We define a set of profiles, which apply to most table-style tangible user
interfaces. This allows the tracking of objects and cursors on two dimensional
surfaces and in special cases also in the 3D space above the table surface. If
one of these predefined profiles does not meet a system’s requirements we also
allow so-called raw profiles that send the raw sensor data, as well as free
form profiles, which allow a user defined set of parameters to be transmitted.
2D Interactive Surface:
/
tuio
/
2Dobj set s i x y a X Y A m r
/
tuio
/
2Dcur set s x y m r
3D Interactive Surfaces:
/
tuio
/
3Dobj set s i x y z a X Y Z A m r
/
tuio
/
3Dcur set s x y z m r
Custom Profile:
/
tuio
/
_[formatString]
/
tuio
/
_ixyP set s i x y 0.57
TUIO Homepage
OSC Homepage
Multitouch TUIO Simulator
TouchLib
project
Building
Applications for the mTouch
To build applications
that function with the mTouch, you need to use a framework that is compatible
with the mTouch System. The mTouch can launch and manage any .NET and/or Flash /Adobe Air applications
.Net Applications
Breeze Framework: (Download) (Help File)
Breeze for WPF 3.5 is a
full core framework for multi-touch showing all the internals. Unlike Windows
7, you can see how the rotate, move and scale actions are interpreted, how the
centre of rotation is calculated and change it all to improve and extend.
Breeze
has a load of cool features, some simple docs, a load of source code and a
pretty example photo browser. Breeze for WPF 3.5 is ideal as a core for
building multi-touch applications in WPF for Windows XP/Vista and 7, or as a
reference implementation to create a new multi-touch application from scratch.
Getting Started
The Download link for
the Breeze Framework provided above gives you a sample application called
“TouchExample” that shows you how to scale, rotate, move, remove and add
objects with the multitouch framework right out of the box. This is a great sample and headstart to
learning how to use the Breeze Framework.
Breeze Framework
Sample Code
//Starts the breeze
Framework
framework = TrackingHelper.GetTracking(canvas1,
currentTrackingType);
framework.OnProcessUpdates
+= new FrameworkControl.ProcessUpdatesDelegate(this.DisplayPoints);
framework.Start();
//Registering an item
for multitouch
ElementProperties sb_prop = new ElementProperties();
sb_prop.ElementSupport.AddSupport(TouchFramework.TouchAction.Slide | TouchFramework.TouchAction.Tap);
MTSmoothContainerRev sb_cont = new MTSmoothContainerRev(SLIDING_CLOSE_BUTTON,
canvas1, sb_prop);
framework.RegisterElement(sb_cont);
SLIDING_CLOSE_BUTTON.ButtonDragged
+= new
//Add an event
SlidingButton.ButtonDraggedHandler(SLIDING_CLOSE_BUTTON_ButtonDragged);
//Event Function
private void SLIDING_CLOSE_BUTTON_ButtonDragged()
{
//Event fired here
}
|
Adobe Flash/Air:
|
Requirements:
Flash Player 9+ (Download)
Java Runtime 6+ (Download)
Adobe Air (Download)
1) Download
the attached file and extract to your desktop.
2) Open Flosc.jar (then click start)
3) Open Simulator.jar
4) Open Test.exe
5) Go back to Simulator.jar and draw within the circle, you will see
your actions replicated into Test.exe
5a) Use SHIFT+Click for multi-touch events. (See Simulator Help for more
info)
Explanation:
Step by Step:
1) The TUIO (OSC) data is SENT from the Simulator OR Touchlib’s “osc.exe”.
2) The FLOSC gateway CONVERTS the OSC (UDP) data into XML (TCP)
3) The Flash client uses an XMLSocket to RECEIVE and parse the XML data then renders the cursors within your
application.
Notes:
- Host: localhost ---- Ports: 3000/3333
- Remember TUIO and OSC are referring to the SAME data.
- The reason we need Step 2 is because Flash CANNOT read
UDP data, thus we convert it to TCP
- The Direct Binary method is not covered in this tutorial.
But you can read more about it here:
Binary Socket Connection in Flash.
Flash Tuio Libraries
AS3: alternative
Flash AS3 TUIO/FlashTCP library (by Dean North)
AS3: alternative
Flash AS3 TUIO library (by
Ignacio Delgado)
AS3: alternative
Flash AS3 TUIO library (by Johannes
Luderschmidt)
Sample Applications and Sources
Getting Started with Flash Development and AS3
Digital Music Volvelles
Mapstouch (A Google Maps Multi-touch Application)
Flickr multi-touch photo
viewing application
Allowing users to update application settings
online
Before your application can communicate, and get
user defined information from Mereltechnologies.com such as background color,
images, videos, audio and other types of uploaded content and application
settings, you will need to specify the parameters your application will accept
when submitting your app for approval in the mStore. Upon approval of your
application the mTouch team will set up your user defined variables for your
application. When the application is downloaded and a user logs in or registers
an account with Mereltechnologies.com, the user will be able to configure each
one of those settings online. When your application is launched on the user’s
mTouch and the user is logged in to their mStore Account all setting and files
uploaded to the users account from the online portal will be downloaded and the
application will start with the new settings and files.
Receiving Application Setting
When your application is launched and the
user is also logged in to their mStore Account. The application settings are
sent to your app via arguments. You can retrieve these arguments like this
Sample Arguments: "LOGGED_IN=true MAX_IMAGES=5 MAX_VIDEOS=1 MAX_MAGAZINES=1 ENABLE_KEYBOARD=yes
BACKGROUND_COLOR=#000000 BROWSER_HOMEPAGE=http://www.yahoo.com";
C#
Code
public Boolean LOGGED_IN = false;
private String BACKGROUND_COLOR = "";
void Window_Loaded(object sender, RoutedEventArgs e)
{
string[] args = Environment.GetCommandLineArgs();
if (args.Length > 1)
{
//Process the new configuration and save to
configuration file.
foreach (string arg in args)
{
String[] s = arg.Split('=');
if (s[0] == "LOGGED_IN")
{
if (s[1] == "true")
{
LOGGED_IN = true;
}else{
LOGGED_IN = false;
}
}
}
if (s[0] == "BACKGROUND_COLOR")
{
BACKGROUND_COLOR = "#" +
s[1];
}
}
}
Simulators
SimTouch
Instructions:
1) Download and install
SimTouch from http://code.google.com/p/simtouch/
2) Watch the video here on how to use SimTouch http://www.youtube.com/watch?v=MkRXr2Y85c0
3) Use SHIFT+Click for
multi-touch events. (See Simulator Help for more info)
SimTouch is an Adobe
Air application that simulates native touch events on the Flash platform.
SimTouch is a transparent app that sits on top of the application that you are
developing and dispatches, TouchEvents to your application through a connection
class.
Currently
SimTouch only dispatches TOUCH_BEGIN, TOUCH_MOVE, and TOUCH_END events.
TUIO Simulator: (Download)