OpenMAX AL for Android

This article describes the Android native multimedia APIs based on the Khronos Group OpenMAX AL™ 1.0.1 standard, as of Android API level 14 (Android platform version 4.0) and higher.

OpenMAX AL is a companion API to OpenSL ES, but for multimedia (video and audio) rather than audio only.

Android 4.0 provides a direct, efficient path for low-level streaming multimedia. The new path is ideal for applications that need to maintain complete control over media data before passing it to the platform for presentation. For example, media applications can now retrieve data from any source, apply proprietary encryption/decryption, and then send the data to the platform for display.

Applications can now send processed data to the platform as a multiplexed stream of audio/video content in MPEG-2 transport stream format. The platform de-muxes, decodes, and renders the content. The audio track is rendered to the active audio device, while the video track is rendered to either a Surface or a SurfaceTexture. When rendering to a SurfaceTexture, the application can apply subsequent graphics effects to each frame using OpenGL.

OpenMAX AL provides a C language interface that is also callable from C++, and exposes features similar to these Android APIs callable from Java programming language code:

As with all of the Android Native Development Kit (NDK), the primary purpose of OpenMAX AL for Android is to facilitate the implementation of shared libraries to be called from Java programming language code via Java Native Interface (JNI). NDK is not intended for writing pure C/C++ applications.

Note: though based on OpenMAX AL, the Android native multimedia API is not a conforming implementation of either OpenMAX AL 1.0.1 profile (media player or media player / recorder). This is because Android does not implement all of the features required by either of the profiles. Any known cases where Android behaves differently than the specification are described in section "Android extensions" below. The Android OpenMAX AL implementation has limited features, and is intended primarily for certain performance-sensitive native streaming multimedia applications such as video players.

The major feature is the ability to play an MPEG-2 transport stream containing a single program stream made up of one H.264 video elementary stream and one AAC audio elementary stream. The application provides the stream via an Android buffer queue data source, which is based on the OpenSL ES buffer queue concept and Android-specific extensions.

The video sink is an ANativeWindow * abstract handle, derived from an android.view.Surface ("surface"). A Surface from SurfaceHolder.getSurface() should be used when displaying an unaltered video within a fixed SurfaceView frame. A Surface from new Surface(SurfaceTexture) allows streaming the decoded video frames to an OpenGL ES 2.0 texture, where the frames can be used as input to a shader algorithm in the Graphics Processing Unit (GPU). Be sure to release() the Surface as soon as possible after calling setSurface or ANativeWindow_fromSurface.

The audio sink is always an output mix, a device-independent mixer object similar to that of OpenSL ES.

Getting started

Example code


Supported and tested example code, usable as a model for your own code, is located in NDK folder platforms/android-14/samples/native-media/.

Not recommended

The OpenMAX AL 1.0.1 specification contains example code in the appendices (see section "References" below for the link to this specification). However, the examples in Appendix D: Sample Code use features not supported by Android. Some examples also contain typographical errors, or use APIs that are likely to change. Proceed with caution in referring to these; though the code may be helpful in understanding the full OpenMAX AL standard, it should not be used as is with Android.

Adding OpenMAX AL to your application source code

OpenMAX AL is a C API, but is callable from both C and C++ code.

Add the following lines to your code:

#include <OMXAL/OpenMAXAL.h>
#include <OMXAL/OpenMAXAL_Android.h>


Modify your as follows:

Multimedia content

The only supported way to supply multimedia content is via an MPEG-2 transport stream.

Finding or creating useful multimedia content for your application is beyond the scope of this article.

Note that it is your responsibility to ensure that you are legally permitted to play the content.


For robustness, we recommend that you examine the XAresult value which is returned by most APIs. Use of assert vs. more advanced error handling logic is a matter of coding style and the particular API; see the Wikipedia article on assert for more information. In the supplied example, we have used assert for "impossible" conditions which would indicate a coding error, and explicit error handling for others which are more likely to occur in production.

Many API errors result in a log entry, in addition to the non-zero result code. These log entries provide additional detail which can be especially useful for the more complex APIs such as Engine::CreateMediaPlayer.

Use adb logcat, the Eclipse ADT plugin LogCat pane, or ddms logcat to see the log.

Supported features from OpenMAX AL 1.0.1

This section summarizes available features. In some cases, there are limitations which are described in the next sub-section.

Global entry points

Supported global entry points:

Objects and interfaces

The following figure indicates objects and interfaces supported by Android's OpenMAX AL implementation. A green cell means the feature is supported.

Supported objects and interfaces


This section details limitations with respect to the supported objects and interfaces from the previous section.


The audio stream type cannot be configured; it is always AudioManager.STREAM_MUSIC.

Effects are not supported.

Dynamic interface management

RemoveInterface and ResumeInterface are not supported.


Supported: Not supported: For CreateMediaPlayer, these restrictions apply:

MIME data format

In the current Android implementation of OpenMAX AL, a media player receives its source data as an MPEG-2 transport stream via a buffer queue.

The source data locator must be XA_DATALOCATOR_ANDROIDBUFFERQUEUE (see "Android extensions" below).

The source data format must be XADataFormat_MIME. Initialize mimeType to XA_ANDROID_MIME_MP2TS, and containerType to XA_CONTAINERTYPE_MPEG_TS.

The contained transport stream must have a single program with one H.264 video elementary stream and one AAC audio elementary stream.


Resume, RegisterCallback, AbortAsyncOperation, SetPriority, GetPriority, and SetLossOfControlInterfaces are not supported.


Use the StreamInformation interface on a media player object to discover when the video metrics (height/width or aspect ratio) are available or changed, and to then get the sizes.


Not supported:


This interface on the engine object reports video decoder capabilities without interpretation, exactly as claimed by the underlying OpenMAX IL implementation.

These fields in XAVideoCodecDescriptor are filled:

The other fields are not filled and should be ignored.

Applications should rely on the capabilities documented at Android Supported Media Formats, not the information reported by this interface.

Data structures

Android API level 14 supports these OpenMAX AL 1.0.1 data structures:


The native display data locator is used to specify the video sink:
typedef struct XADataLocator_NativeDisplay_ {
    XAuint32 locatorType;      // XA_DATALOCATOR_NATIVEDISPLAY
    XANativeHandle hWindow;    // ANativeWindow *
    XANativeHandle hDisplay;   // NULL
} XADataLocator_NativeDisplay;
Set the hWindow field to an ANativeWindow * and set hDisplay to NULL. You can get a ANativeWindow * handle from an android.view.Surface, using this NDK function:
#include <android/native_window_jni.h>

ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface);
Don't forget to free this handle in your shutdown code with ANativeWindow_release.

Platform configuration

OpenMAX AL for Android is designed for multi-threaded applications, and is thread-safe.

OpenMAX AL for Android supports a single engine per application, and up to 32 objects. Available device memory and CPU may further restrict the usable number of objects.

xaCreateEngine recognizes, but ignores, these engine options:

OpenMAX AL and OpenSL ES may be used together in the same application. In this case, there is internally a single shared engine object, and the 32 object limit is shared between OpenMAX AL and OpenSL ES. The application should first create both engines, then use both engines, and finally destroy both engines. The implementation maintains a reference count on the shared engine, so that it is correctly destroyed at the second destroy.

Planning for future versions of OpenMAX AL

The Android native multimedia APIs at level 14 are based on Khronos Group OpenMAX AL 1.0.1 (see section "References" below). As of the time of this writing, Khronos has recently released a revised version 1.1 of the standard. The revised version includes new features, clarifications, correction of typographical errors, and some incompatibilities. Most of the incompatibilities are relatively minor, or are in areas of OpenMAX AL not supported by Android. However, even a small change can be significant for an application developer, so it is important to prepare for this.

The Android team is committed to preserving future API binary compatibility for developers to the extent feasible. It is our intention to continue to support future binary compatibility of the 1.0.1-based API, even as we add support for later versions of the standard. An application developed with this version should work on future versions of the Android platform, provided that you follow the guidelines listed in section "Planning for binary compatibility" below.

Note that future source compatibility will not be a goal. That is, if you upgrade to a newer version of the NDK, you may need to modify your application source code to conform to the new API. We expect that most such changes will be minor; see details below.

Planning for binary compatibility

We recommend that your application follow these guidelines, to improve future binary compatibility:

Planning for source compatibility

As mentioned, source code incompatibilities are expected in the next version of OpenMAX AL from Khronos Group. Likely areas of change include:

Android extensions

The API for Android extensions is defined in OMXAL/OpenMAXAL_Android.h . Consult that file for details on these extensions. Unless otherwise noted, all interfaces are "explicit".

Note that use these extensions will limit your application's portability to other OpenMAX AL implementations. If this is a concern, we advise that you avoid using them, or isolate your use of these with #ifdef etc.

The following figure shows which Android-specific interfaces and data locators are available for each object type.

Android extensions

Android buffer queue data locator and interface

Comparison with OpenSL ES buffer queue

The Android buffer queue data locator and interface are based on similar concepts from OpenSL ES 1.0.1, with these differences: The data locator type code is XA_DATALOCATOR_ANDROIDBUFFERQUEUE and the associated structure is XADataLocator_AndroidBufferQueue.



A typical buffer queue configuration is 8 buffers of 1880 bytes each.

The application enqueues filled buffers of data in MPEG-2 transport stream format. The buffer size must be a multiple of 188 bytes, the size of an MPEG-2 transport stream packet. The buffer data must be properly aligned on a packet boundary, and formatted per the MPEG-2 Part 1 specification.

An application may supply zero or one of these item codes (command key/value pairs) at Enqueue:

End of stream. Informs the decode and rendering components that playback is complete. The application must not call Enqueue again. There is no associated value, so itemSize must be zero. There must be no data buffer alongside the EOS command.
Discontinuity. This and following buffers have a new presentation time. The new presentation time may be optionally specified as a parameter, expressed in itemData as a 64-bit unsigned integer in units of 90 kHz clock ticks. The itemSize should be either zero or 8. The discontinuity command is intended for seeking to a new point in the stream. The application should flush its internal data, then send the discontinuity command prior to, or alongside of, the first buffer corresponding to the new stream position. The initial packets in the video elementary stream should describe an IDR (Instantaneous Decoding Refresh) frame. Note that the discontinuity command is not intended for stream configuration / format changes; for these use XA_ANDROID_ITEMKEY_FORMAT_CHANGE.
Format change. This and following buffers have a new format, for example for MBR (Multiple Bit Rate) or resolution switching.

Upon notification of completion via a registered callback, the now consumed buffer is available for the application to re-fill.

The implementation may supply zero or more of these item codes (status key/value pairs) to the callback handler:

Buffer queue event mask. The itemSize is 4, and itemData contains the bit-wise "or" of zero or more of the XA_ANDROIDBUFFERQUEUEEVENT_* symbols. This event mask explains the reason(s) why the callback handler was called.

Reporting of extensions

Engine::QueryNumSupportedExtensions, Engine::QuerySupportedExtension, Engine::IsExtensionSupported report these extensions:

Programming notes

These notes supplement the OpenMAX AL 1.0.1 specification, available in the "References" section below.

Objects and interface initialization

Two aspects of the OpenMAX AL programming model that may be unfamiliar to new developers are the distinction between objects and interfaces, and the initialization sequence.

Briefly, an OpenMAX AL object is similar to the object concept in programming languages such as Java and C++, except an OpenMAX AL object is only visible via its associated interfaces. This includes the initial interface for all objects, called XAObjectItf. There is no handle for an object itself, only a handle to the XAObjectItf interface of the object.

An OpenMAX AL object is first "created", which returns an XAObjectItf, then "realized". This is similar to the common programming pattern of first constructing an object (which should never fail other than for lack of memory or invalid parameters), and then completing initialization (which may fail due to lack of resources). The realize step gives the implementation a logical place to allocate additional resources if needed.

As part of the API to create an object, an application specifies an array of desired interfaces that it plans to acquire later. Note that this array does not automatically acquire the interfaces; it merely indicates a future intention to acquire them. Interfaces are distinguished as "implicit" or "explicit". An explicit interface must be listed in the array if it will be acquired later. An implicit interface need not be listed in the object create array, but there is no harm in listing it there. OpenMAX AL has one more kind of interface called "dynamic", which does not need to be specified in the object create array, and can be added later after the object is created. The Android implementation provides a convenience feature to avoid this complexity; see section "Dynamic interfaces at object creation" above.

After the object is created and realized, the application should acquire interfaces for each feature it needs, using GetInterface on the initial XAObjectItf.

Finally, the object is available for use via its interfaces, though note that some objects require further setup. Some use cases needs a bit more preparation in order to detect connection errors. See the next section "Media player prefetch" for details.

After your application is done with the object, you should explicitly destroy it; see section "Destroy" below.

Media player prefetch

For a media player, Object::Realize allocates resources but does not connect to the data source (i.e. "prepare") or begin pre-fetching data. These occur once the player state is set to either XA_PLAYSTATE_PAUSED or XA_PLAYSTATE_PLAYING.

The prefetch status interface is useful for detecting errors. Register a callback and enable at least the XA_PREFETCHEVENT_FILLLEVELCHANGE and XA_PREFETCHEVENT_STATUSCHANGE events. If both of these events are delivered simultaneously, and PrefetchStatus::GetFillLevel reports a zero level, and PrefetchStatus::GetPrefetchStatus reports XA_PREFETCHSTATUS_UNDERFLOW, then this indicates a non-recoverable error in the data source or in rendering to the video sink.

The next version of OpenMAX AL is expected to add more explicit support for handling errors in the data source. However, for future binary compatibility, we intend to continue to support the current method for reporting a non-recoverable error.

In summary, a recommended code sequence is:


Be sure to destroy all objects on exit from your application. Objects should be destroyed in reverse order of their creation, as it is not safe to destroy an object that has any dependent objects. For example, destroy in this order: audio players and recorders, output mix, then finally the engine.

OpenMAX AL does not support automatic garbage collection or reference counting of interfaces. After you call Object::Destroy, all extant interfaces derived from the associated object become undefined.

The Android OpenMAX AL implementation does not detect the incorrect use of such interfaces. Continuing to use such interfaces after the object is destroyed will cause your application to crash or behave in unpredictable ways.

We recommend that you explicitly set both the primary object interface and all associated interfaces to NULL as part of your object destruction sequence, to prevent the accidental misuse of a stale interface handle.

Callbacks and threads

Callback handlers are generally called synchronously with respect to the event, that is, at the moment and location where the event is detected by the implementation. But this point is asynchronous with respect to the application. Thus you should use a mutex or other synchronization mechanism to control access to any variables shared between the application and the callback handler. In the example code, such as for buffer queues, we have omitted this synchronization in the interest of simplicity. However, proper mutual exclusion would be critical for any production code.

Callback handlers are called from internal non-application thread(s) which are not attached to the Dalvik virtual machine and thus are ineligible to use JNI. Because these internal threads are critical to the integrity of the OpenMAX AL implementation, a callback handler should also not block or perform excessive work. Therefore, if your callback handler needs to use JNI or do anything significant (e.g. beyond an Enqueue or something else simple such as the "Get" family), the handler should instead post an event for another thread to process.

Note that the converse is safe: a Dalvik application thread which has entered JNI is allowed to directly call OpenMAX AL APIs, including those which block. However, blocking calls are not recommended from the main thread, as they may result in the dreaded "Application Not Responding" (ANR).


As OpenMAX AL is a native C API, non-Dalvik application threads which call OpenMAX AL have no Dalvik-related overhead such as garbage collection pauses. However, there is no additional performance benefit to the use of OpenMAX AL other than this. In particular, use of OpenMAX AL does not result in lower audio or video latency, higher scheduling priority, etc. than what the platform generally provides. On the other hand, as the Android platform and specific device implementations continue to evolve, an OpenMAX AL application can expect to benefit from any future system performance improvements.

Security and permissions

As far as who can do what, security in Android is done at the process level. Java programming language code can't do anything more than native code, nor can native code do anything more than Java programming language code. The only differences between them are what APIs are available that provide functionality that the platform promises to support in the future and across different devices.

Applications using OpenMAX AL must request whatever permissions they would need for similar non-native APIs. Applications that play network resources need android.permission.NETWORK. Note that the current Android implementation does not directly access the network, but many applications that play multimedia receive their data via the network.

Platform issues

This section describes known issues in the initial platform release which supports these APIs.

Dynamic interface management

DynamicInterfaceManagement::AddInterface does not work. Instead, specify the interface in the array passed to Create.

References and resources

Android: Khronos Group: For convenience, we have included a copy of the OpenMAX AL 1.0.1 specification with the NDK in docs/openmaxal/OpenMAX_AL_1_0_1_Specification.pdf.