mobile image processing part2

Mobile image processing – Part 2„ Mobile application development on Android „ Project 1: “Hello Image”, reading/writing/modifying images „ Project 2: “Viewfinder EE368”, processing viewf

Mobile image processing – Part 2

„ Mobile application development on Android

„ Project 1: “Hello Image”, reading/writing/modifying images

„ Project 2: “Viewfinder EE368”, processing viewfinder frames

„ Project 3: “CVCamera”, utilizing OpenCV functions

„ Mobile application examples

Recognizing video at a glance

(1) User snaps a photo of screen.

(2) Our system identifies video and

(3) User resumes video on the phone.

Recognizing video at a glance

Mobile device versus personal computer

Screen size 4.6 in × 2.4 in 25 in × 16 in

Permanent storage 133 MB internal

8 GB external flash 500 GB disk

Internet access Typically 3G or

WiFi at hotspots WiFi or wired Ethernet Telephony Core feature Supplementary feature

embedded External webcam

“Viewfinder EE368” project

„ Goals of this project

z Learn how to access frames from the viewfinder

z Learn how to modify camera parameters

z Learn how to augment the viewfinder frames

„ Step-by-step instructions available in Tutorial #1:

z http://ee368.stanford.edu/Android

Android manifest file

<intent-filter>

<action android:name=" android.intent.action.MAIN " />

<category android:name=" android.intent.category.LAUNCHER " />

</intent-filter>

</activity>

</application>

<uses-sdk android:minSdkVersion=“ 7 " />

<uses-permission android:name=" android.permission.CAMERA "/>

Set the proper hardware permissions

Landscape mode without title bar

Main Activity

Class hierarchy for “Viewfinder EE368”

Viewfinder EE368 (Activity)

Preview (View)

Draw on Top (View)

Manage two views and manage program

open/close

Handle incoming viewfinder frames and adjust camera parameters

Augment viewfinder frames with a new layer of information

on top

Viewfinder class: full screen mode

Viewfinder EE368

Icon and title bars visible Icon and title bars hidden

public void onCreate(Bundle savedInstanceState) {

getWindow().setFlags(

WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN );

requestWindowFeature(Window.FEATURE_NO_TITLE);

Class hierarchy for “Viewfinder EE368”

Viewfinder EE368 (Activity)

Preview (View)

Draw on Top (View)

Manage two views and supervise program

open/close

Handle incoming viewfinder frames and adjust camera parameters

Augment viewfinder frames with a new layer of information

on top

Timeline of events on the mobile device

Preview class: frames go down two paths

Preview (View)

Viewfinder frames

…

Display on phone screen

Forward to custom callback function

Preview class: set camera parameters

Preview (View)

Camera.Parameters parameters = myCamera.getParameters();

parameters.setPreviewSize( 640 , 480 );

parameters.setPreviewFrameRate( 15 );

parameters.setSceneMode(Camera.Parameters.SCENE_MODE_NIGHT); parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO); myCamera.setParameters(parameters);

Class hierarchy for “Viewfinder EE368”

Viewfinder EE368 (Activity)

Preview (View)

Draw on Top (View)

Manage two views and manage program

open/close

Handle incoming viewfinder frames and adjust camera parameters

Augment viewfinder frames with a new layer of information

on top

DrawOnTop class: YCbCr to RGB conversion

84

276 256

41 516 256

08 298

58

135 256

12 208 256

29 100 256

08 298

921

222 256

58 408 256

08 298

−

⋅ +

⋅

=

Cb

Y B

Cr Cb

Y G

Cr

Y R

DrawOnTop class: create some paint brushes

myPaintRed = new Paint();

myPaintRed.setStyle(Paint.Style.FILL); myPaintRed.setColor( Color.RED );

myPaintRed.setTextSize(25);

myPaintGreen = new Paint();

myPaintGreen.setStyle(Paint.Style.FILL); myPaintGreen.setColor( Color.GREEN );

myPaintGreen.setTextSize(25);

myPaintBlue = new Paint();

myPaintBlue.setStyle(Paint.Style.FILL); myPaintBlue.setColor( Color.BLUE );

myPaintBlue.setTextSize(25);

DrawOnTop class: draw histogram bars

Bin 0 Bin 1 Bin 2 Bin 3 Bin 4

// Draw red rectangle

Rect rect = new Rect(left x, top y,

right x, bottom y); myCanvas.drawRect(rect, myPaintRed);

Viewfinder EE368 demo

http://ee368.stanford.edu/Android

Real-time debugging using DDMS (1)

Real-time debugging using DDMS (2)

Taking a screenshot of the phone (1)

0123456789ABC

Taking a screenshot of the phone (2)

Class Project: Mobile Image Completion

H Wang and A Lin, Spring 2010

Class Project: Mobile HDR Imaging

J Mathe and T Wong, Spring 2010

Mobile image processing – Part 2

„ Mobile application development on Android

„ Project 1: “Hello Image”, reading/writing/modifying images

„ Project 2: “Viewfinder EE368”, processing viewfinder frames

„ Project 3: “CVCamera”, utilizing OpenCV functions

„ Mobile application examples