Auto-rotation in iOS with openFrameworks

I don’t think that I’ve mentioned this before, but WaveShaper is a mixture of openFrameworks, UIKit, and Minim for C++. One of the major difficulties I’ve had in developing the app is getting the UIKit views and controllers to play nice. It’s not hard to render UIKit widgets on top of the EAGLView, you simply add your UIKit widget as a subview of the UIWindow or the EAGLView that openFrameworks creates, but it does not work to directly use openFrameworks coordinates to set the position and size of UIKit widgets.

The reason it doesn’t work is because (I’m pretty sure) openGL’s coordinate system is not the same as Apple’s. Part of what openFrameworks does is translate the coordinates of incoming touches from Apple’s coordinate system to openGL’s before sending touch events. This allows you to always think about the drawing surface in the “natural” way for doing openGL apps: upper left corner is always (0,0), positive X is to the right, positive Y is down. You can use this knowledge to simply transform your openGL coordinates to Apple coordinates when setting the position and size of UIKit widgets, but if you are using ofxiPhoneSetOrientation to reorient your openGL surface, the mapping between openGL coordinates and UIKit coordinates changes depending on the orientation. This means you have to reposition all of your UIKit objects every time the device is reoriented. Even worse, what I discovered is that in one of the landscape modes, even though the UIKit widgets looked correct, I actually had the device orientation set to the opposite of what it should have been, so when I displayed popovers, they were upside-down. When I presented modal view controllers, they would appear in the correct orientation, but then not rotate when device orientation changed.

After much hair-pulling, I finally came up with a solution that I think is pretty solid. Essentially, what I have done is embed the EAGLView in a view hierarchy that is well behaved with respect to Apple’s system and I do not call ofxiPhoneSetOrientation to change the orientation of the app. What this means is that from openFrameworks’ perspective the app never changes orientation, which is true in some sense because the EAGLView is added as a subview to a vanilla UIView that is strictly responsible for handling the rotation.

Ok, onto some actual code. Before setting up any UI, I set the openFrameworks orientation to landscape:

C
1
ofxiPhoneSetOrientation( OF_ORIENTATION_90_RIGHT );

Then I rearrange the view hierarchy that openFrameworks creates so that I can set the position and size of UIKit widgets using openFrameworks coordinates. Included in this is creating a root view controller for the window, so that I can take advantage of the auto-rotation behavior they provide (and because Apple expects your app to have a root view controller):

Objective-C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// remove the glview from the UIWindow, which is where openFrameworks puts it
[ofxiPhoneGetGLView() removeFromSuperview];
// create a plain old UIView that is the same size as the app
// just use OF sizes as our reference, everything should work out
CGRect mainFrame = CGRectMake(0, 0, ofGetWidth(), ofGetHeight());
mMainView = [[UIView alloc] initWithFrame:mainFrame];
// grab the glView and rotate/position it so it looks correct
UIView* glView = ofxiPhoneGetGLView();
glView.transform = CGAffineTransformRotate(CGAffineTransformIdentity, M_PI_2);
glView.center = mMainView.center;
// glView goes on our main view to prevent wonky resizing
[mMainView addSubview:glView];
// root controller has main view as its view
mRootViewController = [[LandscapeViewController alloc] init];
mRootViewController.view = mMainView;
// and finally we set the root view controller
// which will make mMainView the window's primary view
// all other views are then added as subviews of mMainView
// NOT as subviews of the UIWindow or the EAGLView
ofxiPhoneGetUIWindow().rootViewController = mRootViewController;

Here’s the LandscapeViewController class, which limits supported orientations to landscape orientations and also turns off animations during rotation so that the orientation snaps. If you like the animated orientation change, you can simply remove those lines.

Objective-C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// barebones UIViewController so we can control allowed orientations of SoundCloud interface.
@interface LandscapeViewController : UIViewController
{
}
@end
@implementation LandscapeViewController
- (BOOL) shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)toInterfaceOrientation
{
if ( UIInterfaceOrientationIsLandscape(toInterfaceOrientation) )
{
return YES;
}
return NO;
}
- (void) willRotateToInterfaceOrientation:(UIInterfaceOrientation)toInterfaceOrientation duration:(NSTimeInterval)duration
{
// snap to new rotation
[UIView setAnimationsEnabled:NO];
}
- (void)didRotateFromInterfaceOrientation:(UIInterfaceOrientation)fromInterfaceOrientation
{
// animations back on so we get the nice slide-ons for the keyboard and so forth
[UIView setAnimationsEnabled:YES];
}
@end

Finally, the class that registers to receive ofxiPhoneAlerts needs to handle device orientation changes like this:

Objective-C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
void App::deviceOrientationChanged(int newOrientation)
{
UIApplication * app = [UIApplication sharedApplication];    
// status bar orientations are reversed from device orientations
// so that we can use OF coordinates to layout UIKit widgets.
    if ( newOrientation == UIDeviceOrientationLandscapeLeft )
    {
[app setStatusBarOrientation: UIInterfaceOrientationLandscapeRight
animated:NO];
    }
    else if ( newOrientation == UIDeviceOrientationLandscapeRight )
    {
[app setStatusBarOrientation: UIInterfaceOrientationLandscapeLeft
animated:NO];
    }
}

And again, WaveShaper only displays in landscape, which is why those are the only orientations I handle.

Once all of this is in place, you can happily position buttons, sliders, popovers, and whatever else using the openFrameworks coordinate system and they will all look correct and rotate with the screen.

WaveShaper 1.1: bug fix update coming soon!

I’ve been seeing several reports of WaveShaper crashing a lot and after spending some time looking into it, I’m pretty sure I’ve fixed the problem. It wasn’t a crash, per se, but actually a lock up in the audio system, which caused the OS to kill the app. Sometimes this could happen on start up and sometimes when attempting to preview files in the file list.

The update should be available on December 30th. This is later than I’d like it to be available, but the iTunes Connect holiday shutdown began today and lasts through the 29th.

My sincerest apologies to all who have bought the app and experienced these crashes.

WaveShaper for iPad Released!

Today I released my first sound making app for iPad!

WaveShaper is an app for iPad that allows you to load up any audio file and make crazy cool sounds with it. It’s a lot like record scratching, but totally maxed out. This video will do a much better job of explaining it than words ever will:

Visit waveshaperapp.com for more info, sound samples, and screenshots. Or just go buy it right now!

Minim without Processing

I’ve been asked many times to remove the PApplet dependency from Minim and tonight I have done it. If you’d like to try out Minim in your not-Processing Java app, you can grab the latest from the repo to do so. I’ve put the details in the Javadoc, but I’ll lay out the basics here, as well.

Processing provides two key methods that the JavaSound implementation of Minim uses when dealing with sound files. The sketchPath method is used by createRecorder to generate an absolute path from the file name provided to that method. The createInput method is used to get InputStreams for reading audio files. In order to remove the dependency on PApplet, the Minim constructor that required a PApplet as an argument has been replaced with one that takes an Object. This Object is then passed to the JavaSound implementation, which uses reflection to try to locate sketchPath and createInput methods on that Object.

What it boils down to is that if you are building a not-Processing Java app, you must simply write sketchPath and createInput methods for one of your application classes and then pass an instance of that class to the Minim constructor. The exact method signatures are:

String sketchPath(String fileName);
InputStream createInput(String fileName);

Entre 2 Mondes at Elektra

For the last month, I’ve been working on the audio portion of an installation for the Elektra Festival in Montréal along with Mickaël Lafontaine. The installation, designed by Mickaël, is an interactive touchscreen experience that is a meta-installation about Cycloïd-E, which is a “kinetic polyphonic installation” also being shown at Elektra. Mickaël had elementary and high school students watch video and listen to recordings of Cycloïd-E and then write short poems about it. These poems have been split into four “thematic spaces” where people can reconstruct the poems by dragging words into place. Each “thematic space” has a theme and a unique soundscape that evokes different aspects of Cycloïd-E. The soundscapes are constructed out of bits of recordings of Cycloïd-E, recordings of the kids reading their poems, and procedurally generated audio. I used the new UGen Framework in Minim to create all of the effects and do real-time mixing and parameter control, some of which is tied directly to touchscreen input.

If you are in Montréal, you can see the installation at the Elektra Festival for free. It’s currently installed in the lobby of Usine C and can be viewed starting from 5pm, through May 7. More details are available at the Elektra site: http://www.elektramontreal.ca/2011/#/program/ENTRE_2_MONDES/

Sound Byte: Granulizer Patch

This Sound Byte gives you control over a reasonably complex UGen chain. The meat of it is a UGen I’ve called Granulizer, which takes sample data and, based on some parameters, randomly chooses short sections of the data to loop before jumping to a different short section. This sketch gives you mouse control over the size of the sections that are looped, as well as how many times they are looped. There are also keyboard commands for controlling some effects that the Granulizer is being patched through, such as a double delay, a resonant filter, a sample repeater, a bit crusher, and a playback rate controller. The controls are outlined on the applet page, so check it out!

I never really got into using max/msp or pd, but now that I’ve got these UGens to play around with in Minim, I’m finally discovering the joy of patching!