-- Now some fun stuff

Smart home controlled with an iPhone

Here is a demo showing how an iPhone can help turn on/off a bedroom light. It can be extended to various scenarios like turning on the computer/air conditioner/garage door, with more perception ability provided by sensors.

A linux server is deployed to accept http requests from iPhone. A signal with then be triggered to the serial port telling the Arduino chip driving the servos to turn on/off the light switch. It would be easier with relays but I'm not qualified to operate on the home electricity. Ethenet shield or Raspberry Pi would help too.

The servos and the Arduino Uno borard

The following easier example shows how to control a servo with an iPhone. Just like the "guiding in 3D space with gyroscope", the gyroscope data is first sent to remote PC server with Javascript. The server receives the data, preprocesses it, and then sends to an Arduino chip with serial port. The Arduino chip then controls the servo to the specified angle.

User clustering with spatial-temporal data

Spatial-temporal data like GPS trajectories or check-in data from facebook contains valuable info and has the potential to build various applications. However, due to the intrinsic complexity and rich context, it's also not easy to incorporate. Here we've made exploratory attempts on designing features for GPS trajectories, and clustering users based on the extracted features.
Besides, as a Data Mining course project, we also demonstrate how data visualization and algorithm tracking help build a solid algorithm as well as implementation in the real world.
Slides are available here.

Beamer Outline Generator

Beamer is excellent for academic presentation, but endless \begin{frame}, \end{frame}, \begin{itemize} and \end{itemize} drive users crazy, especially considering there are few editors really (and easily) supporting auto-completion of beamer. This makes writing beamer code rather inefficient.
Beamer Outline Generator is a tool for fast beamer writing. This tool enables you to express your outlines with indents and generate elegant beamer code automatically. The aim/motivation is to write slides as fast as you can in Microsoft Powerpoint with LaTeX. .
Enter Beamer Outline Generator.

Graph-cut based foreground/background segmentation

A demo showing how to do foreground/background segmentation based on Markov Random Field and graph-cut. For more info, please consult this blog post.

Line-based image warping

A demo showing how to do image warping based on several marker lines.
Ref: Feature-Based Image Metamorphosis

Viewpoint Invariant Feature based on RGB-D images (Kinect-style images)

Traditional local features already achieve resistance toward changes in illumination, translation, rotation, and scale variance, thus play an important role in visual content retrieval. However, they are not stable to 3D rotation or viewpoint changes. This report aims to solve this problem by introducing depth sensor, a new kind of sensor which can capture depth with regular optical information, in this problem and proposing a novel descriptor to achieve viewpoint invariance. We first transform input RGB-Depth images to 3D point cloud in Cartesian space, estimate normal vector and calibrate viewpoint for each local patch, and then project the cloud to 2D plane and calculate the descriptors. Some preliminary experiment results also support the effectiveness of our descriptor.
For detailed report, please click here.
Sample input

Manipulate a 3D cube with Kinect, gyroscope or multi-touch

How to guide in a 3D space (manipulating a 3D cube here) in a user-friendly way is interesting as well as important. Here we demonstrate several natural ways with new sensors like Kinect, gyroscope (embedded in an iPhone) and multi-touch screens. More technical details below the videos.
With Kinect:

With gyroscope (the delay in the end is caused by the 3G network):

With multi-touch screen:

More technical details:
Our demo using Kinect was released before Microsoft's official Kinect SDK. Therefore that involves getting Kinect recognized by our PC with OpenNI and writing our own tracking algorithm.
The iPhone demo uses JavaScript APIs for gyroscope signals and use an ASP.NET backend to receive the sensor information thus update the rendering parameters of the 3D cube.
The multi-touch version may be useful in tablet PCs or Microsoft Surface (the huge device with a 70-inch screen, not the Win8 tablet), and is running our own multi-touch intrepretation algorithm. [more info]

Stock selection with machine learning

Build a portfolio from stock market to achieve the best return rate with machine learning. Tested on stocks in S&P 500 Index, and the average annual return rate is 21.72%, while that of the random stock selector is -1.76%. (Trading fees unconsidered in this test, and buying a fraction of share is permitted)
For a technical report, click here.

Mobile animal language classifier

Trying to introduce the approaches of image classification into the field of animal sound classification. We are using SVM and MFCC features, and have built an iPhone web app for demonstration.
(A joint work of Felix and grapeot)

HTML app using camera in iPhone

To develop programs in iOS devices, we need to hand $99/year to Apple. Although there are tools like php, java, and python in jailbroken iOS devices, it is still not feasible to invoke native components like camera or audio recorder without an iPhone SDK. (There are Objective-C bridges like PyObjC, but currently I don't see any samples being able to use cameras.)

But! There are already camera and voice memo utilities in iPhone, why shall we repeat ourselves? Why not just invoke these utilities and fetch the files they saved?

After some investigation, I found it seems impossible to invoke such utility apps from php or python. So here we take the strategy that first activate the camera app manually, and then launch the image processing app written in html. The data flow is like:

Camera App: take a photo and save it in a folder.
Our php page : list the files in that folder, take the latest one -> redirect to a html5 page, with the file name of the latest photo as parameter.
The html page: read in and render the photo, and respond to user's interactions.

After some simple implementation, we got this app:

iPhone text message falsification

The text messages of iPhones are stores as SQLite database files, and even mobile user can modify it. This provides possibility to falsificate text messages. This project aims to figure out how to do this in PC client and will possibly provide a command-line tool to manipulate text messages in iPhone.

The experiment result is like the image above. A related article can be found Here (in Chinese, dead link for now).

Reverse engineering of the iPhone game Gold Miner

Gold miner is a popular game in iOS (especially in China). But some stages are too difficult to accomplish. Therefore I downloaded the map files of the game, observed patterns in them, figured out the meaning of each byte, and wrote a utility to generate maps according to users' design. The final result is shown as following figures. The game will accept and recognize the generated map as if it's genuine. For the detailed reasoning process, please consult this post (in Chinese).

Map producer interface GoldMiner customized map

Visitors' comments