Using libpd with Unity3D on mobile devices (part 2)

Today’s blog post comes with an announcement that a lot of people have been waiting for for some time: We’ve just released the source code for kalimba, our libpd to unity3d-mobile binding on github. Check it out and enjoy using it for your projects - we’re excited to see what you’re going to create with it:

https://github.com/hagish/kalimba

And now here’s some more information about recent improvements in the pd workflow and some other kalimba puredata news by our lead developer Sebastian.

Due to Android being our first target platform I improved deploying files for this platform. The old workflow consisted of manually copying the pd files to a fixed path on the sdcard and loading them from there. Because the Android app is embedded in a apk file which is a zip file it is not possible to simple fopen the assets content from there. But that is the way pd opens its files. So what I wanted was to simply put files into the StreamingAssets folder in Unity, click build and that is it - o manual copying stuff around.

All the files in the Unity StreamingAsset folder are accessible via simple “fopens” on desktop and iOS but not on Android. One can load the file content via the WWW class but that is no option for pd (fopen). So I need to extract the files from the apk during runtime. On each start the Android version lists all files in the folder StreamingAssets/pd (embedded in the android apk in the assets folder) and extracts it to to Application.persistentDataPath. Now pd can run with plain fopens.

This lists the apk content form the StreamingAssets using the AssetManager:

public static ArrayList listAssetsContent(String path)
{
	ArrayList l = new ArrayList();
		
	try {
		AssetManager am = instance.getResources().getAssets();	// instance is a reference to the activity
			
		String [] list = am.list(path);
		for (String s : list)
		{
			String subPath = path + "/" + s;
			if (subPath.charAt(0) == '/')
			{
				subPath = subPath.substring(1);
			}
				
			ArrayList subList = listAssetsContent(subPath);
				
			if (subList.size() > 0)
			{
				// directory
				l.addAll(subList);
			}
			else
			{
				// file
				l.add(subPath);
			}
		}
			
	} catch (IOException e) {
		e.printStackTrace();
	}
		
	return l;
}

And this is how its used (from the kalimba android binding):

private int _openFile (string baseName, string pathName)
{
	// spawn files from path
	foreach(string path in listAssetsContent(pathName))
	{
		string newFilePath = copyFromStreamingAssetsIntoPersistant(path);
			
		if (newFilePath.Substring(newFilePath.Length - 3) == ".pd")
		{
			// oggread path hack
			Regex p = new Regex(@"#X msg ([0-9]+) ([0-9]+) open (.*[/\\])?([^/\\;]+);", RegexOptions.Multiline);
			string t = File.ReadAllText(newFilePath);
			t = p.Replace(t, "#X msg $1 $2 open " + Application.persistentDataPath + "/pd/$4;");
			File.WriteAllText(newFilePath, t);
		}
	}
		
	string file = Application.persistentDataPath + "/" + pathName + "/" + baseName;
		
	AndroidJavaClass jc = new AndroidJavaClass("org.puredata.core.PdBase");
	return jc.CallStatic("openPatch", file);
}

// based on something found on stackoverflow
public string copyFromStreamingAssetsIntoPersistant(string p)
{
	string filepath = Application.persistentDataPath + "/" + p;

	// this is the path to your StreamingAssets in android
	WWW f = new WWW ("jar:file://" + Application.dataPath + "!/assets/" + p);  
		
	// CAREFUL here, for safety reasons you shouldn't let this while loop unattended, place a timer and error check
	while (!f.isDone) {
	}  

	// then save to Application.persistentDataPath		
	var fileDirectoryParts = filepath.Split('/');
	if (fileDirectoryParts.Length > 1)
	{
		string directory = string.Join("/", fileDirectoryParts.Take(fileDirectoryParts.Length - 1).ToArray());
		Directory.CreateDirectory(directory);
	}
		
	File.WriteAllBytes (filepath, f.bytes);

	return filepath;
}

Currently I copy everything everytime a patch file gets opened. Not perfect but perfectly simple :). As long as the pd files and dependencies (wav, ogg) stay small it is not worth the effort to implement a complex version-checking-system that only extracts new or changed files. But perhaps this will become necessary in the future.

As one can see there is a ugly “Regex p = new Regex(@”#X msg ([0-…“ kind of hack in the _openFile method.  We encountered some problems with paths using the oggread module (that we recently integrated). oggread does not respect pd’s search paths and simply opens ogg files from the current working directory or by absolute path. A consistent working directory on every platform including our composer’s workstation was nearly impossible :) so we went for the absolute path way. Fillipo (our composer) opens the ogg files with a absolute path (oggread’s open message) to StreamingAssets/pd and put the ogg file there. On the device the _openFile method searches for open messages ”#X msg ([0-9]+) ([0-9]+) open (.*[/\])?([^/\;]+);“ in the pd files and strips the absolute path. ”/my/lala/music.ogg" turns into “music.ogg”. Not very elegant but at least no manual adjusting/copying stuff.

I also integrated the seq module from cyclone for parsing midi files. Before this Filippo converted the midi files using pd-extended into some text representation and copy & pastes this text into the pd files used on the device. So again less manually copying stuff ;) yeah.

Source code: https://github.com/hagish/kalimba