Smart home controlled with an iPhone
Here is a demo showing how an iPhone can help turn on/off a bedroom light. It can
be extended to various scenarios like turning on the computer/air conditioner/garage
door, with more perception ability provided by sensors.
A linux server is deployed to accept http requests from iPhone. A signal with then
be triggered to the serial port telling the Arduino chip driving the servos to turn
on/off the light switch. It would be easier with relays but I'm not qualified to
operate on the home electricity. Ethenet shield or Raspberry Pi would help too.
The following easier example shows how to control a servo with an iPhone. Just like
the "guiding in 3D space with gyroscope", the gyroscope data is first sent to remote
PC server with Javascript. The asp.net server receives the data, preprocesses it,
and then sends to an Arduino chip with serial port. The Arduino chip then controls
the servo to the specified angle.
User clustering with spatial-temporal data
Spatial-temporal data like GPS trajectories or check-in data from facebook contains
valuable info and has the potential to build various applications. However, due
to the intrinsic complexity and rich context, it's also not easy to incorporate.
Here we've made exploratory attempts on designing features for GPS trajectories,
and clustering users based on the extracted features.
Besides, as a Data Mining course project, we also demonstrate how data visualization
and algorithm tracking help build a solid algorithm as well as implementation in
the real world.
Slides are available here.
Beamer Outline Generator
Beamer is excellent for academic presentation, but endless \begin{frame}, \end{frame},
\begin{itemize} and \end{itemize} drive users crazy, especially considering there
are few editors really (and easily) supporting auto-completion of beamer. This makes
writing beamer code rather inefficient.
Beamer Outline Generator is a tool for fast beamer writing. This tool enables you
to express your outlines with indents and generate elegant beamer code automatically.
The aim/motivation is to write slides as fast as you can in Microsoft Powerpoint
with LaTeX. .
Enter Beamer Outline Generator.
Graph-cut based foreground/background segmentation
A demo showing how to do foreground/background segmentation based on Markov Random
Field and graph-cut. For more info, please consult this blog post.
Line-based image warping
A demo showing how to do image warping based on several marker lines.
Ref: Feature-Based
Image Metamorphosis
Viewpoint Invariant Feature based on RGB-D images (Kinect-style images)
Traditional local features already achieve resistance toward changes in illumination,
translation, rotation, and scale variance, thus play an important role in visual
content retrieval. However, they are not stable to 3D rotation or viewpoint changes.
This report aims to solve this problem by introducing depth sensor, a new kind of
sensor which can capture depth with regular optical information, in this problem
and proposing a novel descriptor to achieve viewpoint invariance. We first transform
input RGB-Depth images to 3D point cloud in Cartesian space, estimate normal vector
and calibrate viewpoint for each local patch, and then project the cloud to 2D plane
and calculate the descriptors. Some preliminary experiment results also support
the effectiveness of our descriptor.
For detailed report, please click here.
Manipulate a 3D cube with Kinect, gyroscope or multi-touch
How to guide in a 3D space (manipulating a 3D cube here) in a user-friendly way
is interesting as well as important. Here we demonstrate several natural ways with
new sensors like Kinect, gyroscope (embedded in an iPhone) and multi-touch screens.
More technical details below the videos.
With Kinect:
With gyroscope (the delay in the end is caused by the 3G network):
With multi-touch screen:
More technical details:
Our demo using Kinect was released before Microsoft's official Kinect SDK. Therefore
that involves getting Kinect recognized by our PC with OpenNI and writing our own tracking algorithm.
The iPhone demo uses JavaScript APIs for gyroscope signals and use an ASP.NET backend
to receive the sensor information thus update the rendering parameters of the 3D
cube.
The multi-touch version may be useful in tablet PCs or Microsoft Surface (the huge
device with a 70-inch screen, not the Win8 tablet), and is running our own multi-touch
intrepretation algorithm.
[more info]
Stock selection with machine learning
Build a portfolio from stock market to achieve the best return rate with machine
learning. Tested on stocks in S&P 500 Index, and the average annual return rate
is 21.72%, while that of the random stock selector is -1.76%. (Trading fees unconsidered
in this test, and buying a fraction of share is permitted)
For a technical report, click here.
Mobile animal language classifier
Trying to introduce the approaches of image classification into the field of animal
sound classification. We are using SVM and MFCC features, and have built an iPhone
web app for demonstration.
(A joint work of Felix and grapeot)
HTML app using camera in iPhone
To develop programs in iOS devices, we need to hand $99/year to Apple. Although there are tools like php, java, and python in jailbroken iOS devices, it is still not feasible to invoke native components like camera or audio recorder without an iPhone SDK. (There are Objective-C bridges like PyObjC, but currently I don't see any samples being able to use cameras.)
But! There are already camera and voice memo utilities in iPhone, why shall we repeat ourselves? Why not just invoke these utilities and fetch the files they saved?
After some investigation, I found it seems impossible to invoke such utility apps from php or python. So here we take the strategy that first activate the camera app manually, and then launch the image processing app written in html. The data flow is like:
Camera App: take a photo and save it in a folder.
Our php page : list the files in that folder, take the latest one -> redirect to
a html5 page, with the file name of the latest photo as parameter.
The html page: read in and render the photo, and respond to user's interactions.
After some simple implementation, we got this app:
iPhone text message falsification
The text messages of iPhones are stores as SQLite database files, and even mobile user can modify it. This provides possibility to falsificate text messages. This project aims to figure out how to do this in PC client and will possibly provide a command-line tool to manipulate text messages in iPhone.
The experiment result is like the image above. A related article can be found Here (in Chinese, dead link for now).
Reverse engineering of the iPhone game Gold Miner
Gold miner is a popular game in iOS (especially in China). But some stages are too difficult to accomplish. Therefore I downloaded the map files of the game, observed patterns in them, figured out the meaning of each byte, and wrote a utility to generate maps according to users' design. The final result is shown as following figures. The game will accept and recognize the generated map as if it's genuine. For the detailed reasoning process, please consult this post (in Chinese).
