Punny photo credit: Saigon by Regime Management
Unity is an awesome way to make an affordable game available on many platforms quickly. It also has an amazing ecosystem of developer tools (ie: the Unity Asset Store). Unity assets are a great way for middleware providers to sell to developers. If these assets are files natively supported by multiple platforms (ie: videos, models, mono scripts, and images) than little to no work is required to sell them for all platforms. Native plugins (ie: libraries), however, are not so easy.
The Unity team has created documentation and a video that makes native plugins appear to be easy to create. Mike Geig did a good job with the video, but it’s far more complicated than he can cover in an hour (and that means it’s probably too complicated for me to cover in a blog as well). In this article I’m going to describe the issues (like this and this) I ran into and how to solve them. I’ll also give you a more real-world example of how to create a native plugin.
My experience with writing native plugins is from my work at Affectiva to create an emotion-sensing Unity plugin. Briefly, the plugin allows Unity games to detect emotions based on facial expressions via a webcam. We already had SDKs for various platforms and had to decide if we wanted to port the core science code to mono or write a wrapper. For various reasons, we decided to write a wrapper.
Affectiva already had a C# wrapper, so I naively thought I could use that. Unity uses C#, so why wouldn’t it work? Well, the version of Mono that Unity uses is equivalent to .NET 2.0. Our C# SDK was built for .NET 4.5. This meant we had to write a wrapper for our native libraries.
The video tutorial makes this appear trivial, and it is, if you are only calling functions from native libraries. Classes, however, are not so easy. Here is what just the constructor and destructor look like for the Windows wrapper:
CAffdexNativeWrapper::CAffdexNativeWrapper()
{
detector = new FrameDetector(30);
faceFoundListener = NULL;
faceLostListener = NULL;
imageResultsListener = NULL;
return;
}
CAffdexNativeWrapper::~CAffdexNativeWrapper()
{
detector->stop();
delete detector;
}
To be clear, this is a wrapper for the SDK (which is already a wrapper). This wrapper exposes the SDK functionality to Unity. If you are as naive as I was, you probably think this code could easily be used for other platforms. Here is the OS X version of these methods:
CAffdexNativeWrapper::CAffdexNativeWrapper()
{
detector = std::make_shared<FrameDetector>(30);
faceFoundListener = NULL;
faceLostListener = NULL;
imageResultsListener = NULL;
return;
}
CAffdexNativeWrapper::~CAffdexNativeWrapper()
{
detector->stop();
detector = NULL;
}
The Windows code is in a cpp file that is part of a Visual Studio project that builds a dll. The OS X wrapper is in an mm file that is part of an XCode project that builds a bundle. There is some common code for the Windows and OS X wrapper, which is encompassed in a 139 line header file. For perspective, the mm file is 312 lines and the cpp file is 468 lines.
For Windows, the library is in the form of a DLL file. The Windows wrapper is also a DLL file. To support 32 and 64 bit games on Windows, you need corresponding 64 and 32 bit DLLs.
For OS X, the library must be a dylib file. You cannot use an OS X static library or framework! The OS X wrapper is a bundle that contains the dylib. For OS X, you need a universal bundle that preferably supports i386, x86_64 and x86. Here is what the file structure looks like:
For each platform, I have a native platform script. Here is an example method for Windows from Assets / Affdex / Plugins / Scripts / WindowsNativePlatform.cs:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void ImageResults(IntPtr i);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void FaceResults(Int32 i);
[DllImport("AffdexNativeWrapper")]
private static extern int initialize(int discrete, string affdexDataPath);
public void Initialize(Detector detector, int discrete)
{
WindowsNativePlatform.detector = detector;
//load our lib!
String adP = Application.streamingAssetsPath;
String affdexDataPath = Path.Combine(adP, "affdex-data"); // Application.streamingAssetsPath + "/affdex-data";
//String affdexDataPath = Application.dataPath + "/affdex-data";
affdexDataPath = affdexDataPath.Replace('/', '\\');
int status = initialize(discrete, affdexDataPath);
Debug.Log("Initialized detector: " + status);
FaceResults faceFound = new FaceResults(this.onFaceFound);
FaceResults faceLost = new FaceResults(this.onFaceLost);
ImageResults imageResults = new ImageResults(this.onImageResults);
h1 = GCHandle.Alloc(faceFound, GCHandleType.Pinned);
h2 = GCHandle.Alloc(faceLost, GCHandleType.Pinned);
h3 = GCHandle.Alloc(imageResults, GCHandleType.Pinned);
status = registerListeners(imageResults, faceFound, faceLost);
Debug.Log("Registered listeners: " + status);
}
Here is how it looks for OS X in Assets / Affdex / Plugins / Scripts / OSXNativePlatform.cs :
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void ImageResults(IntPtr i);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void FaceResults(Int32 i);
[DllImport("affdex-native")]
private static extern IntPtr initialize(int discrete, string affdexDataPath);
public void Initialize(Detector detector, int discrete)
{
OSXNativePlatform.detector = detector;
String adP = Application.streamingAssetsPath;
String affdexDataPath = Path.Combine(adP, "affdex-data-osx");
int status = 0;
try
{
cWrapperHandle = initialize(discrete, affdexDataPath);
}
catch (Exception e)
{
Debug.LogException(e);
}
Debug.Log("Initialized detector: " + status);
FaceResults faceFound = new FaceResults(this.onFaceFound);
FaceResults faceLost = new FaceResults(this.onFaceLost);
ImageResults imageResults = new ImageResults(this.onImageResults);
h1 = GCHandle.Alloc(faceFound, GCHandleType.Pinned);
h2 = GCHandle.Alloc(faceLost, GCHandleType.Pinned);
h3 = GCHandle.Alloc(imageResults, GCHandleType.Pinned);
status = registerListeners(cWrapperHandle, imageResults, faceFound, faceLost);
Debug.Log("Registered listeners: " + status);
}
As you can see, there is some redundancy here, which I might be able to consolidate. An interface (INativePlatform.cs) is used to make it so that people can use the asset without specifying a platform:
using System;
namespace Affdex
{
internal enum NativeEventType
{
ImageResults,
FaceFound,
FaceLost
}
internal struct NativeEvent
{
public NativeEventType type;
public object eventData;
public NativeEvent(NativeEventType t, object data)
{
type = t;
eventData = data;
}
}
internal interface INativePlatform
{
/// <summary>
/// Initialize the detector. Creates the instance for later calls
/// </summary>
/// <param name="discrete"></param>
/// <param name="detector">Core detector object. Handles all communicatoin with the native APIs.</param>
void Initialize(Detector detector, int discrete);
/// <summary>
/// Start the detector
/// </summary>
/// <returns>Non-zero error code</returns>
int Start();
void Stop();
/// <summary>
/// Enable or disable an expression
/// </summary>
/// <param name="expression">ID of the expression to set the state of</param>
/// <param name="state">ON/OFF state for the expression</param>
void SetExpressionState(int expression, bool state);
/// <summary>
/// Get the ON/OFF state of the expression
/// </summary>
/// <param name="expression">ID of the expression</param>
/// <returns>0/1 for OFF/ON state</returns>
bool GetExpressionState(int expression);
/// <summary>
/// Enable or disable an emotion
/// </summary>
/// <param name="emotion">ID of the emotion to set the state of</param>
/// <param name="state">ON/OFF state for the emotion</param>
void SetEmotionState(int emotion, bool state);
/// <summary>
/// Get the ON/OFF state of the emotion
/// </summary>
/// <param name="emotion">emotion id to get the state of</param>
/// <returns>0/1 for OFF/ON state</returns>
bool GetEmotionState(int emotion);
/// <summary>
&n
