solution for sound playback delay on android devices

Examination of possible problems, application of audio playback latency recording and measuring methods and audio playback latency decreasing methods are used to achieve a smallest possi

Trang 1

Procedia Computer Science 104 ( 2017 ) 413 – 420

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Peer-review under responsibility of organizing committee of the scientific committee of the international conference; ICTE 2016

doi: 10.1016/j.procs.2017.01.154

ScienceDirect

ICTE 2016, December 2016, Riga, Latvia Solution for Sound Playback Delay on Android Devices

Dmitrijs Zigunovsa, Jekaterina Smirnovaa,*, Gatis Vitolsa, Gintautas Stonysb

a

Latvia University of Agriculture, 2 Liela Street, Jelgava, LV-3001, Latvia

b

Alytaus kolegija, 17 Studentu Street, Alytus, 62252, Lithuania

Abstract

Recent reports show that there is a 40-200 millisecond delay for sound playback in Android devices At the same time there is no such reports and identified problem for other platforms Research focuses on unresolved audio playback latency problems in devices with Android operating systems Examination of possible problems, application of audio playback latency recording and measuring methods and audio playback latency decreasing methods are used to achieve a smallest possible audio playback delay

As a result with a proposed solution audio playback delay was reduced to 9ms using OpenSL EU API application and uncoded audio data with a device that support digitization and most high performance

Keywords: Audio playback latency; Android devices; Mobile application development

1 Introduction

Audio playback impact all layers of application processing, initially with giving the command in the software application framework to the audio playback, for example in Linux kernel Use of Android Software Development Kit (SDK) is appropriate way to implement audio file playback to devices with the Android Operation System (OS)1 Literature2,3 confirm that there is on average between 40-200 milisecond (ms) sound delay on Android devices Sometimes delay exceeds 200ms However such problem has not been identified in devices with other OS4 It could

be that this issue is Android OS related Usually a person feels sound delay around 10-15 ms and above5 Therefore any developed product sound playback delay should not exceed this level

* Corresponding author Tel.: +371 63005701; fax: +371 63023095

E-mail address: jekaterina.smirnova@llu.lv

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Trang 2

Android OS has the following playback methods6:

x Android.media.MediaPlayer object through the Android SDK (from Application framework layer)

x EU OpenSL API via Android Native Development Kit (NDK) (Android service layer)

To playback a simple audio file, for example when developing audio players, MediaPlayer object typically is used This object provides easy control to playback process using recommended parameters, such as the size of the buffer and the decoding algorithm

Programmers usually need to configure playback process in a local Android environment using Application Programming Interface (API) such as Open Sound Library for Embedded Systems (OpenSL ES) This gives more control to playback process, but also increases development complexity Use of this API is based on multimedia application development requiring the smallest possible sound delay time, or necessity to create cross-platform code The aim of this research is to ensure a smallest possible audio playback delay in devices with Android operating system The experimental part of the research includes testing audio playback during application development with various audio data types, audio playback delay measurement and data analysis

2 Materials and methods

There are several measurement options with different levels of measuring accuracy and application complexity But all of these measurement methods are based on the idea that it is necessary to register time of the command giving moment and the time of the moment when the sound starts to play Moreover, these time moments must be registered in one time metering system Therefore, commands giving process must be conveyed as observation result

For this research light-emitting diode can be used The smallest delay (less than 0.5 milliseconds) between giving command and monitoring can be recorded with the switch of the LED indicator If simultaneously commands of switching on LED and audio file playback through dynamics or TRS connector are given, delay between LED indicator and sound playback can be recorded

Signal must be passed to dual-channel oscilloscope that records LED indicator state and the playback state LED indicator activation can be converted into a signal using a photoelectric cell, which must be placed on opposite side to the LED indicator

The photoelectric signal channel must be connected to one of the oscilloscope channels LED start-up time takes about 2 ms This time depends on the LED specification7 Also additional photoelectric response time must be taken into consideration which usually is between 5 ms and 100 ms This time have correlation with light brightness When light is brighter, photoelectric response time is faster

As modern LED devices are able to produce up to 100 lux and more, the photoelectric response time will be minimal However, for this experimental setting hardware combination adds totally to about 7 ms to playback command signal fixing time It should be taken into consideration when processing the recordings If there is an access to the Printed Circuit Board (PCB) it is possible to use it to connect LED to the oscilloscope channel This will remove the created delay and will increase the accuracy of the result

Trang 3

Photoelectric cell voltage is observed using voltmeter (see Fig 1(a).) where R 1 – resistor, R f – a photoelectric cell,

R V – voltmeter resistance, U– voltage source, V– voltmeter, GND – grounding As circuit elements are connected in series, R 1 and U are constants, then the voltage of the photoelectric cell varies with the resistance, which in turn varies with the incident light intensity This voltage (V f) is calculated using the formula (1)

1

R R

R U V

f

f f

(1)

In reality, the voltage measuring devices have their constant resistance Therefore, the real light switching on

observation scheme is showed in Fig 1(b) With the known voltmeter resistance voltage (V f) is calculated using the formula (2)

1

R R R

R R

R U U

V

f v

f v f

Knowing the photoelectric cell resistance dependence on light intensity and tested device LED light intensity, voltage to the photoelectric cell can be calculated when the LED is turned on and off Using these values, the indicator, which indicates when LED is turned on, can be identified An indicator can take a situation where the voltage of the photoelectric cell falls in range fromʹ ͵Τ ή ሺ୫ୟ୶െ ܸ௠௜௡ሻ and ͳ ͵Τ ή ሺܸ௠௔௫െ ܸ௠௜௡ሻ having background voltage into consideration The voltage in circuit should not be too low

3 Experimental setting and testing application

To get the experimental results it was necessary to connect oscilloscope channel to the sound TRS audio output channel The developed scheme was used for final testing (see Fig 2)

Fig 2 Common playback delay observation scheme

To get objective results of audio playback delay, measurements were performed on multiple devices: Sony Xperia V, LG G3, Just5 Blaster 2 These devices vary with Android OS version, branding, and technical support and cover majority of Android OS modifications at the moment of the experiment

An oscilloscope with a time digitization and functionality of measuring time delay in milliseconds is needed to delay time measuring in testing smartphones Therefore, selected oscilloscope must be able to perform measurements with a frequency of 1 kHz or faster The requirement is also two-channel availability and the ability

to record the direct current SEINTEK S2405 digital oscilloscope was used for delay time measuring in testing smartphones

Trang 4

For testing purpose Android application was developed Following audio file playback methods were implemented into the developed application:

x Android.media.MediaPlayer object via Android SDK

x OpenSL ES via Android NDK

x Superpowered Audio API via Android NDK

Immediately after application launcher the necessary objects are initialized, configured and prepared for work The application provides the control using graphical interface (see Fig 3) Each method complies with a button, which provides playback start command In this way there is no activity related to the playback start command giving and finally no linked playback delay

Fig 3 Developed application interface for research purposes

Immediately after the sound playback, application calls the LED command Both these commands are given in a separate threads

4 Choosing formats for the sound testing

MP3 is the most popular and more distributed file format for sounds storage Audio data in this format is encoded with losses The data must be encoded at first to playback MP3 type audio files In this situation playback delay occurs due to decoding process

To observe the audio data storage format impact on the playback delay time, it was necessary to make tests using open-label uncompressed audio data formats The way to play such data with all the methods is use of Waveform Audio File Format (WAV) container with Pulse-code modulation (PCM) format data

To know exactly the audio data storage format impact on the play amount of delay, the third format was used, which is supported by all the used testing methods As the third format Advanced Audio Coding (AAC) format was chosen because it is the modernized version of the standard MP3 format This audio data encoding compresses them over the same MP3 sound quality which in result produces ACC format sound with better quality than MP3 with the same size of data file

Trang 5

5 Results and discussion

The results (see Table 1) were grouped by playbacks methods, devices, and audio data

Table 1 Playback delay measurements (milliseconds)

LGG3

Playback

method

Data

format

44.1kHzfrequency 48kHzfrequency

MediaPlayer AAC 50,520 37,406 47,032 52,744 60,482 23,222 48,338 49,275 49,210 57,664

MP3 89,476 43,086 42,180 47,351 45,174 37,484 44,824 47,400 99,758 24,741 PCM 37,940 44,693 49,478 54,898 50,262 19,870 22,850 12,407 54,939 54,334 OpenSLES AAC 114,697 72,160 69,408 105,153 74,201 87,186 49,856 57,843 47,267 72,054

MP3 104,881 47,596 60,379 49,935 50,224 51,831 47,904 48,222 49,137 47,532 PCM 55,331 49,435 17,592 42,233 39,931 11,977 12,517 9,722 62,674 60,141 Superpowered AAC 91,660 97,369 107,431 107,961 107,408 102,802 100,761 94,804 61,731 94,358

MP3 99,411 98,255 92,090 99,723 98,084 47,068 45,041 99,317 34,989 72,280 PCM 36,832 45,170 87,571 62,896 62,765 44,775 47,544 47,501 50,050 48,335

Just5Blaster2

Playback

method

Data

format

MediaPlayer AAC 179,262 180,890 180,657 175,183 185,006 194,880 187,522 200,207 197,478 203,003

MP3 396,587 402,905 385,593 407,501 422,529 412,185 400,079 385,011 402,853 424,950 PCM 327,069 323,266 306,958 342,307 347,202 330,404 353,262 345,459 351,866 355,504

SonyXperiaV

Playback

method

Data

format

MediaPlayer AAC 344,937 345,528 349,963 239,148 350,154 194,384 344,442 350,226 210,266 374,209

MP3 510,070 480,146 479,481 499,809 489,322 470,017 499,686 499,905 489,546 480,226 PCM 430,132 440,744 440,734 420,485 430,132 440,248 439,735 419,542 409,376 440,005

Trang 6

Because all measurements were made directly, measurement error was calculated using formula (3) where Ɂ – relative mean error, %; – measuring device absolute error, V; ୶ – the mean value of the root mean square error (4), V; – measurements count; Ƚ – importance rate, Ƚ א ሾͲǢ ͳሿ; ஑ǡ୬ିଵ – student's t-distribution; ୟ୴୥ – arithmetic average of the n measurements, V ) 12,13

ߜ ൌ ටሺௌೣ௧ഀǡ೙షభሻ

మ ା஺మ

ܵ௫ ൌ ටσ ሺ௫ೌೡ೒ି௫೔ሻమ

೙

೔సభ

Generally relative measurement error is accepted as broad, because each research which measure different data, allow special maximum relative error rate This rate permits to determine the reliability of measurement results Performed playback delay measurement can raise a subjective agreement that the results with a relative error of more than 50% are not reliable The maximum relative error is accepted with the agreement, that the used method of measurement is accurate But background processes which may be affected to the result of the obtained values are included in solution with tested devices

Fig 4 Relative errors measurement

As a result, measurement errors are different (see Fig 4) There is no assurance that the results reflect the real situation One factor could be that device loading is impacted by the background running software In addition, the device load increases with use of Wi-Fi, Global System for Mobile Communications (GSM), Near-field communication (NFC), Global Positioning System (GPS), Bluetooth The greater is the device loading level, the more load it takes between sound processing cycles Since the test device LG G3 was under increased load measurements, it could show a greater distinction than measurements of other devices In addition, testing using Just5 Blaster 2 device gave lowest relative error measurement because it was used only by default included software

Trang 7

Playback delay average values by used data encoding type are different in test devices (see Fig 5) Device specification impact to the audio playback delay is undeniable When comparing average playback delay value used

by the data encoding, it can be concluded that the PCM-format data is played back the fastest This speed has been triggered with no need to decode the data MP3 playback data decoding due delay to increases for 27.64 ms comparing with PCM audio data playback testing Tested AAC data decoding takes about 17.56 ms more time than the MP3 data decoding because AAC encoding algorithm execution takes more time

Fig 5 Data format impact to audio playback delay

Fig 6 Supported and unsupported audio digitization playback delay difference

Sounds discretization also impacts performance time and computing resource This factor also increases playback delay time This conclusion is illustrated in the test results (see Fig 6)

Trang 8

It can be observed that test devices playback delay difference is small for data with different discretization

6 Conclusion

Audio playback delay was reduced using OpenSL EU API application, uncoded audio data in the device supported digitization

Using developed application it is possible to reduce the sound delay time to 9ms that does not interfere to human perception

Uncoded data playback is faster than the coded data playback Data is processed faster using digitization that is supported by devices rather than using other digitization method High performance devices have smallest playback delay time

Testing process can be improved by reducing measurement errors using factory setting Android operation system which is free from additionally installed and running software, turned off communication modules and full battery connected to the charging network Such conditions are close to the ideal test conditions, however real living situation such conditions are not present

References

1 Android Open Source Project Audio Available: https://source.android.com/devices/audio/

2 BlackBerry Developer Blog Tone Generator ௗ: How to add Low Latency Audio to your Native or Cascades App July 2013 Available: http://devblog.blackberry.com/2013/07/tone-generator/

3 Superpowered.com Android Audio Latency and iOS Audio Latency Test App Available: http://superpowered.com/latency/#table

4 Lobao M Android Audio Latency In-Depth: It’s Getting Better, Especially With The Nexus 5X And 6P

5 International Telecommunication Union Recommendation BT.1359: Relative Timing of Sound and Vision for Broadcasting Geneva, Switzerland; 1998

6. Huang CC, Nie R Design and realization of music player based on android system In: Int Conf Mechatronics Inf Technol Vol 462;

2013 p 1123-1126

1 Khan LU Visible light communication: Applications, architecture, standardization and research challenges Digit Commun Networks;

2016

2 Zheng H, Chen J, Yu C, Gurusamy M Inverse design of LED arrangement for visible light communication systems Opt Commun.;

2017

3 Bolton JR University of Western Ontario Photochemistry Unit., Boston University Department of Chemistry Solar Power and Fuelsௗ: In: Proceedings of the First International Conference on the Photochemical Conversion and Storage of Solar Energy London, Canada Academic Press; 1977

4 Henderson R Samsung’s new LED components could improve Galaxy S5 flash and hint at April launch; 2014 Available:

http://www.pocket-lint.com/news/127328-samsung-s-new-led-components-could-improve-galaxy-s5-flash-and-hint-at-april-launch

5 Rumsey F, McCormick T Sound and Recording Elsevier/Focal; 2009

6. Myttenaere A, Golden B, Le Grand B, Rossi F Mean Absolute Percentage Error for regression models Neurocomputing 192; 2016

p 38-48

7 Nakajima J, Omori Y Stochastic volatility model with leverage and asymmetrically heavy-tailed error using GH skew Student’s

t-distribution Comput Stat Data Anal 56(11); 2012 p 3690-3704

Gatis Vitols received PhD in Information technologies at Latvia University of Agriculture Primary research fields include cross-cultural information systems development, usability and database technologies Author works as a leading researcher and associate professor at Faculty

of Information Technologies in Latvia University of Agriculture Contact him at gatis.vitols@llu.lv

Tiêu đề	Solution for sound playback delay on Android devices
Tác giả	Dmitrijs Zigunovsa, Jekaterina Smirnovaa, Gatis Vitolsa, Gintautas Stonysb
Trường học	Latvia University of Agriculture
Chuyên ngành	Computer Science
Thể loại	Conference paper
Năm xuất bản	2017
Thành phố	Riga

Định dạng
Số trang	8
Dung lượng	370,17 KB