Graphic APIs Available to Windows at a Glance

This is a quick run through of some of the Windows Graphic APIs currently available. Some of these are available on Windows Phone (in complete or reduced form) others are not. Here I run through a description of each one.

GDI

GDI is a foundational Windows Mobile API. It is the most frequently used API though many times it not be apparent then it is being used.  The Windows controls (text box, buttons, and dropdown list, so on) are all rendered using GDI.  GDI functionality includes drawing basic shapes (lines, rectangles, ellipses, text, and so on) and rendering bitmaps.  GDI functions take into account differences in pixel format. If you try to render 24-bit image to a 16-bit surface GDI will be able to take care of converting the image to the appropriate format without the need of any additional effort from the developer.

GDI+

GDI is a vector based GDI library found on desktops and unsupported on Windows Phones. While Windows Phones have a GDI+ library on them it is undocumented. Undocumented means more than there is just a lack of documentation. Undocumented also means that there is no guarantee that the library will continue to work. As an example if one installs Office 2010 on their Windows Phone the GDP+  dll  is replace with a newer version that doesn’t implement all of the same functions as the default library and it may break dependent applications.

Imaging Codec API

The Imaging Codecs API is a COM based API for manipulating and creating images.  It supports JPEG, PNG, and TIFF images and has support for manipulation of per pixel transparency and Exif tags.  The Imaging code API is good for loading and saving images but its rendering speed is slow compared to the other APIs available for rendering bitmaps.

DirectX

DirectX is a name applied to a group of COM based APIs Windows Mobile has a subset of the DirectX APIs (there are a several more available on the desktop).  DirectDraw is for manipulating bitmaps and sprites.  Direct3D is for rendering 3D graphics.  And DirectShow is for manipulating video and sound streams. There are several other members  of the DirectX family. The latest version of DirectX available to the desktop at the time of this writing is DirectX 11. The Xbox 360 uses a variant of DirectX 9 with shaders.  The Windows Mobile implementation of DirectX  comes closest to DirectX 9.

DirectDraw

The goal of DirectDraw is to provide a fast device independent way to manipulate display surfaces.  DirectDraw will take advantage of hardware acceleration whenever possible, though when hardware support is not available it will fall back on software emulation. Basic functionality (ex: copying a bitmap from one surface to another) will always be available through either a hardware accelerated form of software implemented form. Other functionality (ex: per pixel transparency) is only available if there is hardware support. In general functionality that requires hardware support is functionality that would perform poorly if implemented in software.  I’ve written a DirectDraw code sample and have made it available on CodeProject.com with at http://www.codeproject.com/KB/mobile/WiMoBubblePuzzle.aspx.

Direct3D

Direct3D is the Windows Native API for 3D graphic operations. Like DirectDraw Direct3D was designed to support either hardware accelerated operations and software emulated operations; if a feature were not implemented in hardware the software version was automatically used.  This is where the pain points begin.

As an API Direct3D API targets a lot of 3D functionality. But the last hardware accelerated implementation was found in the Windows Mobile 5 Dell Axiam. In what seems to be common practice a reference implementation for Direct3D is included on the device.  The reference implementation (d3dmref.dll) is a pure software implementation for Direct3D that wasn’t made with performance in mind. It was made so that OEMs implementing a device could have a reference as to the output that graphic operations should produce to aid them in developing their own implementation.  The emphasis in this driver’s implementation is on the output, not the performance.  Attempts to use this driver in a game will result in frame rates that are better measured in seconds per frame instead of frames per seconds. 

The other driver type found on Windows phones is called a software optimized driver. These drivers make no use of hardware acceleration but are made to minimize the time it takes to perform 3D operations. While these drivers tend to perform much better than the reference implementation they don’t come anywhere near the performance of a device with a dedicated graphics processing unit.

Direct2D

Direct2D is the newest of the DirectX APIs. It isn’t found on Windows Mobile at all.  I mention it because its name has been in use longer than the API has been in existence. In reading through several forums over the years I’ve seen developers mistakenly refer to DirectDraw as Direct2D. The Direct2D API was introduced in Windows 7 and Windows 2008 Server R2 (though it is available to Vista and Windows 2008 Server with an update). The API is for rendering 2D vector based graphics and takes advantage of hardware acceleration (if available).

DirectWrite

DirectWrite is an API that makes use of Direct2D.  DirectWrite is for rendering glyphs for rendering text with exceptional clarity. Internet Explorer 9 makes use of DirectWrite for its text rendering.

DirectShow

DirectShow is the DirectX library made for handling media streams.  The foundational object type in DirectShow is a Filter. In general a filter will accept a data stream of one type (audio or video), perform some operation on it, and output the stream. There are some filters are producers only (create and output date) and some filters are consumers only.

OpenGL ES

Despite being able to find implementations in some Windows devices OpenGL ES is not a Windows API. Microsoft was once a member of the OpenGL architecture review board and in 1997 began working with SGI to combine the OpenGL and Direct3D APIs (known as the Fahrenheit project).   In early 2003 Microsoft withdrew membership on the board to concentrate on its own 3D API (Direct3D).  If you search the Microsoft Developer’s Network site for OpenGL references (http://msdn.microsoft.com) the Microsoft published references that you find  target Windows 95/98.

On Windows Mobile devices some OEMs have included implementations of OpenGL ES (a version of OpenGL for embedded devices) on their devices. Some of HTCs devices have implementations of OpenGL ES 1.x. The Samsung Omnia II has support for a version of OpenGL ES 2.0 (but no support for 1.x).  A developer making a Windows phone program that requires OpenGL  would either need to target a specific set of implementations (in which case devices with other implementations or no implementation would be incompatible) or include a software implementation with their program to use if no other implementation is present.  Either solution results in a user experience that isn’t consistent.

XNA

Whereas DirectX is a native family of APIs XNA is a managed version built on top of DirectX. (note: there was a managed version of DirectX for the desktop that is no deprecated). XNA is not an acronym for anything (reiterated by some documentation saying that XNA stands for “XNA is Not an Acronym”).  XNA contains functionality for rendering 2D sprites, 3D objects, sounds, and handling input, and communication among devices running an XNA game. These applications can target the PC, the Xbox 360, or any one of the Zunes (code example: http://www.codeproject.com/KB/game/XNALevel.aspx ).  

The primary development tools needed for XNA development are Visual Studio 2008 or the free Visual Studio Express and the development language used is C# (XNA currently does not support VB.NET). “XNA Game Studio” must be downloaded along with the compiler that you select to enable XNA targets (http://creators.xna.com/en-US/downloads ).  A popular free modeling tool for creating 3D objects is Blender (http://www.blender.org/download/get-blender/ ). While the development tools or free the ability to deploy ones programs to the Xbox 360 hardware requires a registration fee of 99 USD/year. (Available for free to the academic community).

Project types available for XNA targets.

XNA programs targeting the Zune currently have no supported distribution channel. Programs  targeting the Xbox 360 can be made available  in the  Xbox Marketplace and are assigned a price of 80, 240, and 400 points (80 points = 1 USD). The developer receives 70% of the price of the program.

The Xbox 360 and PC versions of XNA have support for both 2D (sprite based) and 3D graphics. At the time of this writing the Zune’s only support 2D XNA applications. An update to the Zune HD several months ago introduced 3G games on the Zune. There’s not yet been anything stating that the 3D functionality of the Zune HD will be made available to the Zune.  XNA is currently not available on any Windows Phones but leaked documentation shows that Windows Phone 7 Series will have XNA.

Windows Forms

Most of the Windows Mobile managed applications that you see are based on Windows Forms. With Windows Forms a developer can build a user interface by either dragging UI elements from a control palette into a design area, the UI can be built completely using code, or some combination of the two.  The default look of the controls is plain. To change this look one would either need to download/purchase their own control set or write code to change the appearance of the controls.

The Forms Designer in Visual Studio 2008.

UI Built by Bertoneri Luigi for Windows Mobile.

XAML

XAML (Extensible Application Markup Language) is a declarative language from Microsoft. The language can be used to describe UI layouts, bind those layouts to underlying data, describe shapes, animations, transitions, and declare other resources that a program may need. The language is used by Microsoft’s WPF and Silverlight technologies. XAML can be manipulated from a text editor or from Microsoft Expressions Blend.

A productivity advantage that Blend has over some of Microsoft’s other technologies for creating UI is that the construction of the UI is not exclusive to someone with a developer skill set.  Conventionally when effort is separated among developers and designers the designers may create a user interface in a paint tool and a developer would then take that image and try to reconstruct it in his or her developer tools. When creating a XAML based application the designer can create the UI within Expressions Blend and the developer can attach his or her code to it.

XAML code for a page with a button, textblock, and textbox displayed in Expressions Blend

Expressions Design can be used to draw graphics and output them to XAML

WPF

WPF ( Windows Presentation Foundation) is a XAML derived technology for the user interface for Windows applications.  The look of a XAML application is subject to the styling applied by the operating system.  In addition to creating controls for input or display of information (text box, list box, so on) WPF also supports manipulating and rendering 3D objects. WPF applications run with the same set of privileges that any other .Net application has. So they can run with a full set of permissions or be granted partial trust to restrict the resources to which they have access.  WPF applications only target the Windows operating system.

Silverlight

Silverlight is another XAML derive technology. At the time of this writing Silverlight 3 is the latest version and the release of Silverlight 4 is forthcoming.  In Silverlight 1 while a developer could declare layout and animations from within Blend programmatic control of the Silverlight object had to be performed from JavaScript embedded within the hosting page. In Silverlight 2 the objects became programmable with C# and VB.Net. While Silverlight is a .Net technology it is derived from a subset of the .Net common language runtime. So a computer does not need to have the full .Net runtime installed to run a Silverlight application.  It only needs the Silverlight Plug-in. In the forthcoming Silverlight 4 the runtimes for Silverlight and Windows applications will be combined.  These applications usually run within a browser but can be configured to run outside of a browser too.

While WPF is Windows  centric Silverlight was designed to be operating system agnostic.  A Silverlight 3 application can be run on OS X or Windows.  A Silverlight 2 application can be run on Windows, OS X, and Linux (through Moonlight). Support for Silverlight on Series 60 and Windows Phone devices (Windows Phone 7 Series only) is also coming. The styling of a Silverlight interface is independent of the operating system on which it is run, so the look and feel of the interface will be the same across operating systems.

Silverlight applications do not support 3D graphics, though 3D transformations can be applied to 2D panes.

WPF applications run with full trust unless configured to do otherwise. Silverlight 3  applications always run in a sandbox. They do not have full access to a user’s file system. IsolatedStorage may be available to a Silverlight application (the user can disable isolated storage all together or limit the amount of data available to a Silverlight application). Silverlight applications also cannot arbitrarily open connections to other network resources.  It can open a connection on the server that it came from and open connections to other servers if those other servers have a policy that allow the application to have access through its cross domain settings (example: you can see twitter’s cross domain policy at http://twitter.com/crossdomain.xml  or Bin’s policy at http://bing.com/clientaccesspolicy.xml ). 

Trusted Silverlight 4 applications have the ability to have access to resources outside of the sandbox and have increased isolated storage.