20 Video Browsing on Handheld Devices 449KEYFRAMES Single frames from a scene that represent its content STORYBOARD Spatial arrangement of thumbnails representing temporally ordered sc
Trang 1capabilities of handheld devices is a difficult task that is yet unsolved In theremainder of the article, we summarize our ongoing work in developing better in-terfaces that offer a richer browsing experience and therefore better usability ofmobile video.
A Short Review of Video Browsing Techniques
for Larger Displays
When browsing video, for example, to get an overview of the content of a file or
to localize a specific position in order to answer some information need, there arenormally two major problems First, video is a continuous medium that changesover time With a static medium such as text, people always see some context at atime and can decide themselves at which speed they look at it In contrast to this,for video only a single frame of a sequence of time-ordered frames is shown for atime slot that depends on the playback speed (e.g 1=25 sec for a typical video play-back rate) Second, there is often not much meta-information available to supportusers in their search and browsing tasks Again, think about browsing the pages of
a book Besides the actual content, there is meta-information encoded, for ple, in the header and footer Spatial information such as the separation in differentparagraphs illustrates related parts with regards to content Headlines give a shortsummary of the following content Different font styles, such as bold face or italic,are used to highlight important information, and so on In addition, higher levelmeta-information exists such as an abstract printed on the cover of the book, thecontent list at its beginning, etc All of this meta-information supports users in vari-ous browsing task For video however, comparable information is usually missing.Not surprisingly, most existing research in digital video browsing tries to make
exam-up for this lack of meta-information by automatically extracting comparable mation from videos and representing it in an appropriate way that supports users intheir browsing tasks (cf Figure1) For example, automatic segmentation techniquesare often used to identify content-related parts of a video [13,17] This structure in-formation can be displayed and used for navigation (e.g jumping from scene toscene using dedicated buttons) in order to make up of the missing structure infor-mation encoded in paragraphs and spaces between them in printed text Single keyframes can be extracted from a scene and represented as a storyboard, that is, avisual arrangement of thumbnails containing the key frames where the spatial orderrepresents the temporal alignment in the video [4,16] This static representationcan be used to get a rough idea of the video’s content, similarly to the content list in
infor-a book One vinfor-ariinfor-ation, so cinfor-alled Video Minfor-anginfor-as, represent different scenes in infor-a comicbook style where thumbnail sizes depend on the relevance of the related scene, thusresembling the hierarchical structure of a content list [2,18] Another variation ofstoryboards, so called video skims or moving storyboards pay tribute to the dy-namic nature of video Here, the static thumbnail representation is replaced with ashort video clip that offers a glimpse into the related scene [3] On a higher level,
Trang 220 Video Browsing on Handheld Devices 449
KEYFRAMES Single frames from a scene that represent its content
STORYBOARD Spatial arrangement
of thumbnails representing temporally ordered scenes
VIDEO MANGA representation of thumbnails indicating scenes with different relevance
TRAILER Automatically generated video summary
PARAGRAPHS AND SPACES Indicate structure and units that are related in terms of content
CONTENT LIST Gives high level info about content and structure of the book
HEADLINES AND PAGE HEADERS Give high level information
SPECIAL FONT STYLES Highlight words of particular interest
ABSTRACT ON BOOK COVER Gives high level content description
TEXT BROWSING
Fig 1 Comparing content-based video browsing approaches with text skimming
automatically generated trailers offer a high level overview of a video’s content andcan thus be compared to the abstract often found on the back side of a book’s cover[11] Because all of these approaches are based on the actual structure or content of a
file, we will subsequently refer to them as content-based approaches Figure1marizes how they relate to text browsing thus illustrating the initial claim that most
sum-of the work on video browsing aims at making up for the missing meta-informationcommonly available for text
The usefulness of such content-based approaches for video browsing has beenconfirmed by various evaluations and user studies However, when browsing text,people do not only look at meta-data, but also skim the actual content at differentspeeds and levels of detail For example, when grabbing a new book, they often skim
it in flip-book style in order to get a rough overview They browse a single page byquickly moving their eyes over it and catch a glimpse of a few single words allowingthem to get a rough idea of the content If they run over something that might be ofparticular interest, they quickly move their eyes back, read a few sentences, and so
on Hence, they skim text by moving their eyes over the content at different speedsand in random directions Their visual perception allows them to make sense of thesnatches of information they are picking up by filtering out irrelevant informationand identifying parts of major interest
Unfortunately, such intuitive and flexible ways for data browsing are not possiblefor a dynamic medium such as video Due to its continuous nature, people can not
Trang 3VIDEO BROWSING For video, only one
information unit (frame) is visible per time unit Its context (i.e information encoded in consecutive frames making up a scene) arises when users modify playback speed (top) or directly manipulate the currently visible part of the video by directly accessing the related position along the timeline using a slider
2x 2x
Modification of playback speed (e.g fast forward)
Slider for scrolling along the timeline
FLIP-BOOK STYLE SKIMMING Getting a quick overview of the content by flipping through the pages
TEXT BROWSING When looking at printed
text, people always see some context (spatially arranged words and meta-information) and decide by themselves at which speed they process the visual information
CROSS-READING Moving your eyes at various speeds and in random direction over the static arrangement of words
Fig 2 Comparing timeline-based video browsing approaches with text skimming
move their eyes spatially over a video However, comparable to how readers areable to make sense of the snatches of information they grasp when moving theireyes quickly over a printed text, the visual perception of the human brain is able toclassify certain parts of the content of a video even if played back at higher speeds
or in reverse direction We call video browsing approaches that try to take advantage
of this characteristic subsequently timeline-based approaches In related techniques,
users control what part of the video they see at a particular time by manipulating thecurrent position on the timeline This is comparable to implicitly specifying whichpart of a text is currently seen by moving ones eyes over the printed content.Figure2illustrates how such temporal movements along the timeline when skim-ming a video relate to spatial movements of your eyes over printed text The mostobvious approach to achieve something like this is to enable users to manipulateplayback speed This technique is well known from analog VCRs where fast for-ward and backward buttons are provided to skim forward or backward Since digitalvideo is not limited by the physical characteristics of an actual tape, but only by thetime it takes to decode the encoded signal, we are usually able to provide users with
a much larger variety of different browsing speeds Alternatively to manipulation
of playback speed, people can often also navigate a video by dragging the thumb
of a slider representing the video’s timeline If visual feedback from the file is vided in real-time, such an approach can be used to quickly skim larger parts of afile, abruptly stop and change scrolling speed and direction, and so on, thus offeringmore flexibility than modification of replay speed On the other hand, increasing
pro-or decreasing playback speed seems to be a better and mpro-ore intuitive choice whenusers want to continuously browse larger parts of a document at a constant speed or
if the information they are looking for is encoded into the temporal changes of anobject in the video
Both approaches enable users to perceive visual information from a video in acomparably flexible way to moving their eyes over text when browsing the con-tent of a book It should also be noted that in both cases, browsing of static mediasuch as text as well as dynamic media such as video, the content-based browsingapproaches summarized in Figure1also differ from the timeline-based ones illus-trated in Figure2in a way that for content-based approaches, users generally browsesome meta-data that was preprocessed by the system (e.g headlines or extracted
Trang 420 Video Browsing on Handheld Devices 451key frames), whereas for timeline-based approaches, they usually manipulate them-selves what part of the content they see at a particular point in time (either by movingtheir eyes over text at random speed or by using interface elements to manipulatethe timeline of a video) Hence, none of the two concepts is superior to the other butthey both complement each other and it depends on the actual browsing task as well
as personal preference which approach is preferred in a particular situation
Mobile Video Usage and Need for Browsing
Even though screen sizes are obviously a limiting factor for mobile video, ments in image quality and resolution have recently led to a viewing experiencethat in many situations seems reasonable and acceptable for users In addition, tech-niques for automatic panning and scanning [12] and adequate zooming [10] offergreat potential for video viewing on handhelds although they have not made it tothe market yet Recent reports claim that mobile video usage, although still beingsmall, is facing considerable rates of growth with “continued year-over-year growth
improve-of mobile video consumption”1
Observing that mobile video finally seems to take of, it is interesting to noticethat so far, most mobile video players only offer very limited browsing function-ality, if supported at all Given that we can assume that established future usagepatterns for mobile video will differ from watching traditional TV (a claim shared
by Belt et al [1]), one might wonder if intensive mobile video browsing might not
be needed or required by the users Indeed, a real-life study on the usage of bile TV presented by Belt at al [1] indicated little interest in interactive services.However, the authors themselves claim that this might also be true do to a lack offamiliarity with such advanced functions In addition, the study focused on live TVwhere people obviously have different expectations for its consumption on mobiles
mo-In contrast to this, the study on the usage of mobile video on handheld devicespresented by O’Hara et al [14] did report several mobile usage scenarios and sit-uations that already included massive video browsing or would most likely profitfrom improved navigation functionality For example, in one case, a group of fourkids gathered around on PSP (Sony’s PlayStationR
talk about the scenes of their favorite movie that each of them liked the most Such
an activity does not only require massive interaction to find the related scene, butalso continuously going backwards in order to replay and watch particular partsagain to discuss them or because they have not been well perceived by some of the
1 The quote was taken from an online article from November 4, 2008, that was posted at http:// www.cmswire.com/cms/mobile/mobile-video-growing-but-still-niche-003453.php (accessed Feb
1, 2009) and discussed a related report by comScore On January 8, 2009, MediaWeek reported comparable arguments from a report issued by the Nielsen Company, cf http://www.mediaweek com/mw/content display/news/media-agencies-research/e3i746 3e6c2968d742bad51c7faf7439 adc (accessed Feb 1, 2009).
Trang 5participants due to the small screen size Ojala et al [15] present a study in whichseveral users experimented with multimedia content delivered to their device in astadium during hockey games According to their user study, the “most desired con-tent was video footage from the ongoing match” Reasonable applications of suchdata streams would be to get a different view of the game (e.g., a close up of theplayer closest to the puck that complements the overview sight of the hockey fieldthey have from their seat) but also the opportunity to re-watch interesting scenes(e.g replays of goals or critical referee decisions) – a scenario that would requiresignificant interaction and video browsing activity.
At this rather early stage of video usage on handhelds, we can only speculatewhat kind of browsing activities users would be willing and interested to really do
on their mobiles once given the opportunity However, the examples given abovedemonstrate that there are indeed lots of scenarios where users in a mobile contextwould be able to take advantage of advanced browsing functionality, or which wouldonly be feasible if their system offers such technologies in an intuitive and usefulway In the following section, we present an example that is related to the study in
a hockey stadium done by Ojala et al [15] but extends the described scenario to afictional case illustrating the possibilities advanced browsing functionalities couldoffer in order to increase the mobile video user experience
Timeline-Based Mobile Video Browsing and Related Problems
In order to motivate the following interface designs and illustrate the most ical problems for timeline-based mobile video browsing, let’s look at a simpleexample Assume you are watching a live game in a soccer stadium The game
crit-is also transmitted via mobile TV onto your mobile phone In addition, live streamsfrom different cameras placed in the stadium are provided Having a large stor-age space (common newer multimedia smart phones already offer storage of up to16GB, for example), you can store all these live streams locally and then have instantaccess to all videos on your local device The study related to hockey games pre-sented by Ojala et al [15] (cf previous section) confirmed that such a service might
be useful and would most likely be appreciated and used intensively by many sportsfans But what kind of browsing functionality would be necessary? What could andwould many people like to do (i.e search or browse for)? We can think of many in-teresting and useful scenarios For example, it would be good to have some systemgenerated labels indicating important scenes, goals, etc that users might want tobrowse during halftime During the game, people might want to quickly go back in
a video stream in order to review a particular situation, such as a clever tactical moveleading to a goal or an offside situation, a foul, a critical decision from the referee,etc In the latter case, it can be useful to be able to navigate through the video at avery fine level of detail – even frame by frame, for example to identify the one singleframe that best illustrates if a ball was indeed outside or not Such a scenario wouldrequire easy and intuitive but yet powerful and ambitious browsing functionality
Trang 620 Video Browsing on Handheld Devices 453For example, people should be able to quickly switch between browsing on a largerscale (e.g to locate a scene before the ball went outside of the playfield) and verysensitive navigation along the timeline (e.g to locate a single frame that illustratesbest which player was the last to touch it) It is also important to keep in mind thatthe related interactions are done by a user who is standing in a soccer stadium (andprobably quite excited about the game or a questionable decision by the referee) andthus neither willing nor able to fully concentrate on a rather complex and sensitiveinteraction task Given the small form factor and the limited interaction possibilities
of handheld devices this clearly makes high demands on the interface design andthe integration of the offered browsing functionality
Obviously, the content-based approaches known from traditional video ing could be quite useful for some higher level semantic browsing, for examplewhen users want to view all goals or fouls during halftime For a more advancedinteraction, for example to check if a ball was outside of the field or not, timeline-based approaches seem to be a good choice For example, by moving a slider thumbquickly backwards along the timeline, a user can identify a critical scene (e.g anoffside) that is then further explored in more detail (e.g by moving the slider thumbback and forth in a small range in order to identify a frame confirming that it wasindeed an offside)
brows-However, one significant problem with this approach is that sliders do not scale
to large document files Due to the limited space that is available on the screen,not every position from a long video can be mapped onto a position on the slider.Thus, even the smallest possible movement of a slider’s thumb (i.e one pixel on thescreen) will result in a larger jump in the file, making it impossible to do a detailednavigation and access individual frames In addition, grabbing and manipulating thetiny icon of a slider’s thumb on a mobile is often considered hard and unpleasant.Interfaces that allow users to browse a video by manipulating its playback speedoften provide a slider-like interaction element as well in order to let users select from
a continuous range of speed values Although the abovementioned scaling problem
of sliders might appear here as well, it is usually less critical because normally, notthat many values, that is, levels of playback speed need to be mapped to the slider’slength However, the second problem, that is, targeting and operating a very tinyicon during interaction remains (and becomes especially critical in situations such
as standing in a crowded soccer stadium)
In the following, we will present different interface designs for handheld vices that deal with these problems by providing an interaction experience that isexplicitly optimized for touch screen based input on mobiles The first four ap-proaches realize timeline-based approaches – both navigation along the timeline atdifferent levels of granularity and skimming the file by using different playbackrates (cf Fig 2) – whereas the fifth approach presents a content-based approachthat also takes into account another important characteristic we often observe inmobile scenarios: that often, people only have one hand available for operating thedevice Research on interfaces for mobile video browsing is just at its beginning and
de-an area of active investigation The question of how both interaction concepts cde-anseamlessly be integrated into one single interface is yet unanswered and thus part ofour ongoing and future research
Trang 7All interfaces presented in the next two sections are optimized for pen-based action with a touch sensitive display Touch screen based interaction has become animportant trend in mobile computing due to the tremendous success of the iPhone
inter-So far, we restricted ourselves to pen-based operation in our research, althoughsome of the designs presented below might be useful for finger-based interaction aswell All proposed solutions have been implemented on a Dell AXIMTMX51v PDAwhich was one of the high end devices at the time we started the related projects.Meanwhile, there are various other mobile devices (PDAs as well as cell phones)offering similar performance Our Dell PDA features an Intel XScal, PXA 270,
624 MHz processor, 64 MB SDRAM, 256 MB FlashROM, and an Intel 2700g processor for hardware-side video encoding The latter one is particularly importantfor advanced video processing as it is required by our browsing applications The de-vice has a 3.7-inch screen with a resolution of 640 480 pixels and a touch sensitivesurface for pen-based operation Our interfaces have been implemented in C C C onthe Windows Mobile 5 platform on top of TCPMP (The Core Pocket Media Player)which is a high-performance open source video player The implementation wasbased on the Win32 API using the Graphics Device Interface for rendering.For all approaches we present below, audio feedback is paused when users startbrowsing the visual information of a video stream We believe that there are lots ofsituations where approaches to browse the audio stream are equally or sometimesmaybe even more important than navigation in the visual part of a video However,
co-at the time we started these projects, technical limitco-ations of the available devicesprevented us from addressing related issues With newer, next generation models,this issue certainly becomes interesting and therefore should be addressed as part offuture work (cf the outlook at the end of this article) All our implementations havebeen evaluated in different user studies In the following, we will only summarizethe most important and interesting observations For a detailed description of therelated experiments as well as further implementation details and design decisions
we refer to the articles that are cited in the respective sections
Flicking vs Elastic Interfaces
As already mentioned in the introduction, the iPhone uses a technique called
flick-ing to enable users to skim large lists of text, for example all entries of your music
collection For flicking, users touch the screen and move their finger in the directionthey want to navigate as if they want to push the list upwards or downwards Uponreleasing the finger from the screen, the list keeps scrolling with a speed that slowlydecreases till it comes to a complete stop The underlying metaphor can be explainedwith two rolls each of which holding one end of the list (cf Figure3) Pushing therolls faster increases scrolling speed in the respective direction Releasing the finger
Trang 820 Video Browsing on Handheld Devices 455
Left: Flicking your finger over the touch screen starts scrolling of the content in the same direction After a while, scrolling slows down and comes to a complete stop simulating the frictional loss of two rolls that wind the document
Right: Moving you finger over the screen without flicking it results in a similar movement of the document’s content
However, instead of scrolling automatically, the content is not “pushed” but directly follows the movements of your finger
FLICKING AND RELATED METAPHOR
Fig 3 Scrolling text lists on the iPhone by flicking
By flicking their fingers over the touch screen, users can “push” the video along the timeline
Fig 4 Applying flicking to video browsing
causes scrolling to slow down due to frictional loss If the user does not push the tent but the finger rests on the screen while moving it, the list can be moved directlythus allowing some fine adjustment By modifying how often and how fast the fin-ger is flicking over the touch screen or by changing between flicking and continuousmoving users can achieve different scrolling speeds thus giving them a certain vari-ety for fast and slow navigation in a list Transferring this concept to video browsing
con-is straightforward if we assume the metaphor illustrated in Figure4 Although thebasic idea is identical, it should be noted that it is by no means clear that we canachieve the same level of usability when transferring such an interaction approach
to another medium, that is, from text to video With text, we always see a certaincontext during browsing, allowing us, for example, to identify paragraph bordersand new sections easily even at higher scrolling speeds With video on the otherhand, scene changes are pretty much unpredictable in such a browsing approach.This might turn out to be critical for certain browsing tasks Based on an initialevaluation that to some degree confirmed these concerns, we introduced an indica-tion of scrolling speed that is visualized at the top of the screen during browsing
In a subsequent user study it turned out that such information can be quite useful inorder to provide the users a certain feeling for the scrolling speed which is otherwiselost because of the missing contextual information Figure5shows a snapshot of theactual implementation on our PDA
Trang 9Fig 5 Implementation of flicking for video browsing on a PDA The bar at the top of the display illustrates the current scrolling speed during forward and backward scrolling
Our second interface design, which also enables users to navigate and thusbrowse through a video at different scrolling speeds, is based on the concept ofelastic interfaces For elastic interfaces, a slider’s thumb is not dragged directly butinstead pulled along the timeline using a virtual rubber band that is stretched be-tween the slider thumb and the mouse pointer (or pen, in our case) The slider’sthumb follows the pointer’s movements at a speed that is proportional to the length
of the virtual rubber band A long rubber band has a high tension, thus resulting in
a faster scrolling speed Shortening the band’s length decreases the tension and thusscrolling slows down Using a clever mapping from band length to scrolling speed,such interfaces allow users to scroll the content of an associated file at differentlevels of granularity The concept is illustrated in Figure6(left and center) Simi-larly to flicking, transferring this approach from navigation in static data to scrollingalong the timeline of a video is straightforward However, being forced to hit thetimeline in order to drag the slider’s thumb can be critical on the small screen of ahandheld device In addition, the full screen mode used per default on such devicesprevents us from modifying the rubber band’s length at the beginning and the end
of a file when scrolling backward and forward, respectively Hence, we introduced
the concept of elastic panning [5] which is a generalization of an elastic slider thatworks without explicit interface elements Here, scrolling functionality is evoked
by simply clicking anywhere on the screen, that is, in our case, the video This tial clicking position is associated with the current position in the file Scrollingalong the timeline is done by moving the pointer left or right for backward and for-ward navigation, respectively Vertical movements of the pointer are ignored The(virtual) slider thumb and the rubber band are visualized by small icons in orderprovide maximum feedback without interfering with the actual content Figure6
ini-(right) illustrates the elastic panning approach Photos from the actual interface onthe PDA can be found in Figure7 For implementation details of this approach werefer to [5,9]
With both implementations we did an initial heuristic evaluation in order toidentify design flaws and optimize some parameters such as appropriate levels forfrictional loss and a reasonable mapping of rubber band length to scrolling speed.With the resulting interfaces, we did a comparative evaluation with 24 users Aftermaking themselves familiar with the interface, each participant had to solve three
Trang 1020 Video Browsing on Handheld Devices 457
Virtual slider thumb
Pen position
Large rubber band: fast scrolling
Short rubber band: slow scrolling
Scrolling speed
Length of rubber band
ELASTIC SLIDER INTERFACE Mapping rubber band ELASTIC PANNING
length to scrolling speed
Fig 6 Elastic interface concepts: slider (left) and panning (right)
Fig 7 Implementation of elastic panning for video browsing on a PDA
browsing tasks that required navigation in the file at different levels of granularity:First, on a rather high level (getting an overview by identifying the first four newsmessages in a new show recording), second, a more specific navigation (findingthe approximate beginning of one particular news message), and finally, a very finegranular navigation (finding one of the very few frames showing the map with thetemperature overview in the weather forecast)
Flicking and elastic panning are comparable interaction approaches insofar asboth can be explained with a physical metaphor – the list or tape on two rolls inone case vs the rubber band metaphor in the other case Both allow users to skim
a file at different granularity levels by modifying the scrolling or playback speed –
in the first case by flicking your finger over the screen with different speeds, in thesecond case by modifying the length of the virtual rubber band In both cases it ishard, however, to keep scrolling the file at a constant playback speed similar to thefast forward mode of a traditional VCR due to the frictional loss and the effect of aslowing down slider thumb in result of a shorter rubber band Despite these similar-ities, both concepts also have important differences Dragging the slider thumb bypulling the rubber band usually gives people more control over the scrolling speedthan flicking because the can, for example, immediately slow down once they seesomething interesting In contrast to this, flicking always requires a user to stop firstand then push the file again with a lower momentum However, being able to do afine adjustment by resting the finger on the screen is much more flexible, for ex-ample, to access single frames than using the slow motion like behavior that resultsfrom a very short rubber band The most interesting and surprising result in the
Trang 11evaluation was therefore that we were not able to identify a significant difference inthe average time it took for the users to solve the three browsing tasks Similarly,average grades calculated from the subjective user ratings given after the evalua-tion also showed minimum differences However, when looking at the distribution,
it turned out that the ratings for elastic panning were mostly centered around theaverage whereas for flicking, they were much more distributed, that is, many peoplerated it as much better or much worse Given that both interfaces performed equallywell in the objective measure, that is, the time to solve the browsing tasks, we canassume that personal preference and pleasure of use played an important role forusers when giving their subjective ratings In addition, flicking is often associatedwith the iPhone and thus, personal likes or dislikes of the Apple brand might haveinfluenced these ratings as well
Linear vs Circular Interaction Patterns
When comparing the flicking and elastic panning approaches from the previous tion, it becomes clear that the latter only supports manipulation of playback speed
sec-In contrast to this, flicking also allows a user to modify the actual position of a file,similar to moving a slider’s thumb along the timeline of a video, by resting andmoving a finger over the screen However, this kind of position-based navigationalong the timeline is only possible in a very short range due to the small size of thedevice’s screen In the following, we present two approaches that enable users toscroll along the timeline and offer more control over the scrolling granularity, that
is, the resolution of the timeline
Similarly to flicking and elastic panning, scrolling functionality in both cases isevoked without the explicit use of any widget but by doing direct interactions ontop of the video In the first case, clicking anywhere on the screen creates a virtualhorizontal timeline Moving the pointer to the left or right results in backward andforward navigation along the timeline in a similar way as if the slider thumb icon isgrabbed and moved along the actual timeline widget However, the resolution of thevirtual timeline on the screen depends on the vertical position of the pen At the bot-tom, close to the original slider widget, the timeline has the same coarse resolution
At the very top of the screen, the virtual timeline has the smallest resolution ported by the system, for example, one pixel is mapped to one frame in the video.The resolutions of the virtual timelines in between are linearly interpolated as illus-trated in Figure8 Hence, users have a large variety of different timeline resolutionsfrom which to choose from by moving the pen horizontally at an appropriate verti-cal level The resulting scrolling effect is similar to zooming in or out of the originaltimeline in order to do a finer or coarser, respectively, navigation Hence, we called
sup-this approach the Mobile ZoomSlider.
Navigation along the timeline offers certain advantages over manipulation ofplayback speed in certain situations For example, many users consider it easier
to access individual frames by moving along a fine granular timeline in contrast to
Trang 1220 Video Browsing on Handheld Devices 459
Finest scale (1 pixel = 1 frame)
Coarsest scale (1 pixel = no of frames
in video / screen width)
Linearly interpolated scale
Fig 8 Mobile ZoomSlider design for timeline scrolling at different granularities
0.5x
4.0x
AREA FOR TIMELINE SCROLLING
“SPEED BORDER” FOR MANIPULATION OF PLAYBACK RATE
Slow motion
Linear interpolation
of playback speed
Fast forward
Fig 9 Modification of playback speed in the Mobile ZoomSlider design
using a slow motion like approach However, there are also cases where playbackspeed manipulation might be more useful, for example, when users want to skim awhole file at a constant speed In the Mobile ZoomSlider design this kind of naviga-tion is supported at the left and right screen border If the user clicks on the right side
of the screen, constant scrolling starts with a playback speed that is proportional tothe vertical position of the pen At the bottom, you get a fast forward like feedback
At the top, video is played back in slow motion In between, the level of playbackspeed is linearly interpolated between these two extremes On the left screen border,you get a similar behavior for backward scrolling Figure9illustrates this behavior
It should be noted that in both cases – the navigation along the timeline in the center
of the screen and the modification of playback speed on the screen borders – finernavigation is achieved at the top of the screen whereas the fastest scrolling is donewhen the pen is located at its bottom Therefore, users can smoothly switch betweenboth interaction styles by moving the pen horizontally, for example, from the rightregion supporting playback speed based navigation to the position-based navigation
in the center of the screen
An initial evaluation with 20 users that verified the usability and usefulness ofthis design can be found in [6] Figure10shows the actual implementation of thisinterface on our PDA Similarly to the flicking and elastic panning approaches de-scribed above, visualization of additional widgets is kept to a minimum in order tonot interfere with the actual content of the video
Trang 13Fig 10 Implementation of the Mobile ZoomSlider design on a PDA.
Mapping frames from
the timeline of the
video onto a circle
Larger circles enable mapping of more frames onto the same time interval
Fig 11 Basic idea of the ScrollWheel design: mapping timeline onto a circle
The second approach is called the ScrollWheel design Its basic idea is to map
the timeline onto a circle Despite being an intuitive concept due to the similarity
to the face of an analog clock, a circle shaped timeline as an important advantageover a linear timeline representation: a circle has no beginning or end and thus, ar-bitrarily file lengths can be mapped onto it Not surprisingly, using hardware withknob-like interfaces is very popular for video editing In our case, we implemented
a software version of the circular timeline that can be operated via the PDA’s touchscreen Once a user clicks on the screen, the center of the circle is visualized by asmall icon in the center of the screen A specific interval of the video’s timeline, forexample, five minutes, is then mapped to one full rotation Compared to a hardwaresolution, such an implementation has the additional advantage that users can im-plicitly manipulate the resolution of the timeline and thus scrolling granularity bymodifying the radius of their circular movements when navigating the file Largercircles result in slower movements along a finer timeline whereas smaller circles can
be done to quickly skim larger parts of the file as illustrated in Figure11 The ing behavior is somehow comparable to the functionality in the center of the MobileZoomSlider Here, users can get a finer scrolling granularity by increasing the dis-tance from the center With the Mobile ZoomSlider, a similar effect is achieved byincreasing the distance between the bottom of the screen and the pen position
Trang 14result-20 Video Browsing on Handheld Devices 461
Area for timeline scrolling
Area for speed modification
VARIANT 2:
COMBINATION OF BOTH CONCEPTS
Fig 12 Different variants of the ScrollWheel design
In an initial heuristic evaluation we compared the ScrollWheel implementationdescribed above with two variations which are illustrated in Figure 12 In the firstoption, we did not map the actual timeline on the circle but different values for play-back speed Turning the virtual scroll wheel on the screen clockwise results in anincrease in scrolling speed Turning it counterclockwise results in a decrease Oncethe initial clicking point is reached, scrolling switches from forward to backwardnavigation and vice versa The second variant combines both approaches The twothirds of the circle around the initial clicking position on the screen are associatedwith the timeline before and after the current position in the file, thus supportingslider-like navigation in a certain range of the file The remaining part of the circle
is reserved for playback speed manipulation Depending on from which side thisarea is entered, playback speed in forward and backward direction, respectively, isincreased It should be noted that users have to actively make circular movements inorder to navigate along the timeline whereas for the second variant and the part ofthe circle in the third version that supports playback speed manipulation they have
to stay at a fixed point on the circle in order to keep scrolling with the associatedplayback rate
Since our initial heuristic evaluation indicated that it might be to confusing forusers to integrate two different interaction styles in one interface (variant 3) andthat just playback speed manipulation without navigation along the timeline (vari-ant 2) might not be powerful enough compared to the functionality offered by theMobile ZoomSlider design, we decided to provide both interaction styles separatelyfrom each other In the final implementation, the ScrollWheel represents a continu-ous timeline as illustrated in Figure11 Playback speed manipulation is achieved bygrabbing the icon in the center of the screen and moving it horizontally Pen move-ments to the right result in forward scrolling, movements to the left in backwardsnavigation Playback speed is proportional to the distance between pen and center
of the screen with longer distances resulting in faster replay rates This final concept
is illustrated in Figure13 Figure14shows the actual implementation
In a user study with 16 participants we compared the Mobile ZoomSlider withthe ScrollWheel design All users had to solve tree browsing tasks with each of thetwo interfaces The tasks were similar to the ones used in the comparative evaluation
Trang 15By grabbing the
icon, users can
modify playback rate
Moving the pen
to the right increases playback rate
Playback rate is proportional to the distance between pointer and icon Fig 13 Integration of playback speed modification into the ScrollWheel design
Fig 14 Implementation of the ScrollWheel design on a PDA
of flicking with elastic panning described in the previous section They included oneoverview task, one scene searching task, and one exact positioning task However,they were formulated more informally and thus we did not do any qualitative timemeasurement in this experiment but solely relied on the users’ feedback and ourobservation of their behavior during the studies Therefore the results of these ex-periments should not be considered as final truth but more as general trends whichare nevertheless quite interesting and informative Both interfaces had a very goodreception by the users and allowed them to solve the browsing talks in an easyand successful way One important observation with both interfaces was a tendency
by many participants to use different interaction styles for more complex browsingtasks thus confirming our initial assumption that it is indeed useful for a systemdesigner to support, for example, navigation along the timeline and playback speedmanipulation in one interface Considering the direct comparison between the twointerface designs, there was no clear result However, for the navigation along thetimeline we could identify a slight trend for people often preferring the ScrollWheelapproach compared to the horizontal navigation in the screen center supported bythe Mobile ZoomSlider However, for manipulation of playback speed, the situationwas reversed, that is, more people preferred to modify the replay rate by moving thepen along the left and right border of the screen in contrast to grabbing the icon inthe screen’s center as required in the ScrollWheel implementation In contrast to ourexpectations, the seamless switch between both interaction styles provided by theMobile ZoomSlider implementation did not play an important role for the majority
of users In contrast, we had the impression that most preferred a strict separation ofboth interaction styles Another major reason for the common preference towards