What the heck is SUBQUERY?

One of the lesser known bits of NSPredicate is the SUBQUERY() function. The documentation for a subquery expression explains a little bit about what’s going on, but it takes a while to understand when it’s really useful.

Let’s break it down.

The Syntax

A subquery expression takes 3 arguments:

  1. A collection
  2. A variable name
  3. A predicate

The Collection

The collection can be one of the two standard Cocoa collection: NSArray and NSSet (or some subclass of them). This collection can be hard-coded in, or it can be the result of another expression (like a keyPath expression or the like). A hard-coded collection would look like:

SUBQUERY(%@, ...

A keyPath expression would look like:

SUBQUERY(contents, ...

With the stipulation that [self contents] (or self.contents if you prefer) must return an NSArray or NSSet.

The Variable

The SUBQUERY is going to iterate over the collection, gathering certain objects. We need a way to represent what each item in the collection is, and for that we use the variable.

Variables in an NSExpression (or NSPredicate format string) take the form $identifier, where identifier is a valid C-style identifier. Most examples of SUBQUERY generally use $x as the variable. It’s short and to-the-point.

It’s the second argument in the expression:

SUBQUERY(contents, $x, ...

The Predicate

A predicate, as we know, is a statement that evaluates to true or false. In the case of the subquery, the predicate will be evaluated for each object (represented by the variable) in the collection. If the predicate returns true, then that object will be included as part of the resulting collection.

SUBQUERY(contents, $x, $x.property = 'foo' and $x.number = 42)

TL;DR

A SUBQUERY() expression is the functional equivalent of doing this:

NSPredicate * p = [NSPredicate predicateWithFormat:@"property = 'foo' and number = 4"];
NSArray * results = [[self contents] filteredArrayUsingPredicate:p];

If this were expressed as a subquery in a predicate, it would be:

NSPredicate * p = [NSPredicate predicateWithFormat:@"SUBQUERY(contents, $x, $x.property = 'foo' and $x.number = 42) ..."];  
//with some operation to use the resulting collection in a comparison or something

So what?

Now that we get what the various bits of a subquery expression are, let’s ask the real question: when is this ever useful?

To be honest, the answer to this is “not often”. However, when you need it, it’s incredibly useful.

Rule of thumb

The general rule of thumb on when you should consider using SUBQUERY is this:

If you have a collection (A) of objects, and each object has a collection (B) of other objects, and you’re trying to filter A based on some varying attributes (at least 2) of the objects in B, then you should probably be using SUBQUERY.

Example

Let’s say you have a bunch of Project objects, and each Project has a bunch of ToDo items. A ToDo item has a completionDate (an NSDate) and a user (a name). You want to find all projects that have a todo item that was completed by Joey (so completionDate is not nil and user is “Joey”). We’re going to display these in a “Joey’s Recent Projects” group (or something).

Our first reaction might be a predicate that uses ANY in there, like:

ANY todos.completionDate != nil AND ANY todos.user == joey

Unfortunately, that would give us projects that have at least one completed ToDo and that has a ToDo whose user is Joey. However, they don’t have to be the same ToDo.

The proper predicate is:

SUBQUERY(todos, $todo, $todo.completionDate != nil AND $todo.user = 'Joey').@count > 0

This predicate will be evaluated against each Project. First, we’ll get the collection of ToDo objects by evaluating the todo keyPath on the Project. Then for each item ($todo) in the array of ToDo objects, we’re going to check and see if that object’s completionDate is non-nil and if that object’s user is "Joey". If that’s true, then it’ll be added to the resulting collection.

When the SUBQUERY completes, we’ll have an array of ToDo items that were completed by Joey. At this point, we retrieve the number of items in that collection (via the @count keyPath) and see if it’s greater than 0. If it is, then the corresponding Project will be added to the final array. In this manner, we can retrieve all Projects that have ToDos completed by Joey.

Creating an advanced NSPredicateEditorRowTemplate

In a recent post I covered how to make a simple NSPredicateEditorRowTemplate. In this post we’re going to go a bit deeper.

The Setup

In our last example, we did some simple date comparisons, where we were comparing an NSTimeInterval against a property that represented a duration. For example, if we had an array of Song objects, then we could use that row template to construct a predicate to find all songs that were longer than 5 minutes (duration > 300). But what if our objects don’t have a duration property? What if they have a date property, such as the date this song was added? Well, we can use the built-in initializers of NSPredicateEditorRowTemplate to create a template where the right expression type is NSDateAttributeType. This is all well and good, but it has a fundamental flaw: It can only compare against static dates.

In other words, I can easily create a row template for predicates of the form <a date> <is, is not, is before, is on or before, is after, is on or after> <a specific date>. But what if I want to do something like <a date> <is in the last, is in the next> <user-entered number> <seconds, minutes, hours, days, weeks>? Is there a way to do relative date comparison? The answer is “yes”, but this is going to take a lot of explanation. So buckle up, and let’s dive right in!

The Problems

There are two major problems to overcome when doing relative date comparison and integrating that with NSPredicateEditorRowTemplate:

  1. NSPredicateEditor compacts left expressions into a unique list. Where a left expression supports multiple operators, those are also uniqued. In other words, you can’t do both myKeyPath = <a value form a textfield> and myKeyPath = <a value from a popup> in the same NSPredicateEditor. It doesn’t matter if these are defined in different row templates. NSPredicateEditor will still find both of them, and won’t know what to display when the user chooses myKeyPath from the first popup and = from the second popup. What should it display? A textfield? A popupmenu? We care about this because if we want to support both myDate <is after> <a specific date> and myDate <is in the last> <number> <time unit>, these both boil down to the same thing (after doing dynamic date resolution): find if one date is after another date. In other words, the same left expression, the same operator, but different right expressions. NSPredicateEditor doesn’t support this. We need to find a way around this.

  2. NSPredicate doesn’t have any obvious date addition or date subtraction features. We need to figure out how to add and subtract arbitrary time intervals from arbitrary dates.

The Solutions

While these may seem like road-blocker problems, they’re really not too bad.

Solving the operator problem

The key to tricking NSPredicateEditor into showing different right expressions for two operators that are fundamentally the same is to not use the same operator. These are the operators that NSPredicateEditor recognizes:

typedef enum {
   NSLessThanPredicateOperatorType = 0,
   NSLessThanOrEqualToPredicateOperatorType,
   NSGreaterThanPredicateOperatorType,
   NSGreaterThanOrEqualToPredicateOperatorType,
   NSEqualToPredicateOperatorType,
   NSNotEqualToPredicateOperatorType,
   NSMatchesPredicateOperatorType,
   NSLikePredicateOperatorType,
   NSBeginsWithPredicateOperatorType,
   NSEndsWithPredicateOperatorType,
   NSInPredicateOperatorType,
   NSCustomSelectorPredicateOperatorType,
   NSContainsPredicateOperatorType,
   NSBetweenPredicateOperatorType
} NSPredicateOperatorType;

You’ll notice that only six of 14 operators are for numeric comparisons (<, <=, >, >=, =, !=). The rest are for string comparisons. This means we have 7 or 8 operators that are sitting around unused when we’re doing a numeric comparison (and date comparisons are really numeric comparisons).

The other key to tricking NSPredicateEditor into showing different right expressions is to realize that NSPredicateEditor is just a UI. It doesn’t care what the actual underlying NSPredicate is. It’s just going to show whatever it’s given, even if that is different from the actual NSPredicate.

So the solution:

#define NSInTheLastPredicateOperatorType NSMatchesPredicateOperatorType
#define NSInTheNextPredicateOperatorType NSLikePredicateOperatorType

That’s it. Any time we want to show an “in the last”-style right expression, we’re going to tell the editor “this is the ‘matches’” operator. Then we’re just going to do a bit of handling in code to not actually use the matches operator, but the proper comparator instead. Simple!

Solving the relative date problem

Have you ever looked at how dates are represented in predicates? It’s quite interesting:

NSDate * d = [NSDate date];
NSLog(@"%@", [NSPredicate predicateWithFormat:@"%@ != 0", d]);
//logs "CAST(312333057.230785, "NSDate") != 0"

Well now, that’s curious. We’re taking some number and “casting” it as an NSDate. But what’s that number? Let’s do:

NSLog(@"%f", [d timeIntervalSinceReferenceDate]);
//logs "312333057.230785"

So this number is a time interval! This is very convenient. But what about all this casting? If we capture the left expression of this predicate, we will find that it’s an NSExpression of type NSFunctionExpressionType, and that the -function of this expression is @"castObject:toType:". Further introspection reveals that the -arguments to this function are two NSExpressions: The first is a constant value expression holding an NSNumber, whose -doubleValue is 312333057.230785, and the second is a constant value expression holding the string @"NSDate".

This is very important, because we learn 3 major things:

  1. There is an undocumented function for NSExpression: castObject:toType:.
  2. This function wants a number as its first argument
  3. This function wants a string (that represents a class) as its second argument.

Armed with this information, we can surmise:

  1. We can create predicates that have the string CAST() in them. We don’t have to rely on predicateWithFormat: to insert that for us.
  2. Any number, or any function that results in a number, can be used as the first parameter to a CAST() function and be cast to an NSDate. That number represents a time interval since the first instant of 1 January 2001, GMT (the reference date).
  3. We can cast objects to other kinds of classes

Let’s test each one of these things:

NSPredicate * p = [NSPredicate predicateWithFormat:@"%@ > CAST(0, 'NSDate')", [NSDate date]];
NSLog(@"%d", [p evaluateWithObject:nil]);  //logs "1".  right now is after our reference date!

p = [NSPredicate predicateWithFormat:@"%@ > CAST(1+2+3+4+5+6, 'NSDate')", [NSDate date]];
NSLog(@"%d", [p evaluateWithObject:nil]);  //logs "1".  right now is after 21 seconds after the reference date!

p = [NSPredicate predicateWithFormat:@"CAST(%@, 'NSNumber') > 0", [NSDate date]];
NSLog(@"%d", [p evaluateWithObject:nil]);  //logs "1". We can cast dates to their numerical equivalent!

So all three of our assumptions appear to be correct! The last one is the most interesting, however. If we pull out the leftExpression of the predicate and evaluate it, we get 312333057.230785. In other words, casting a date to a number simply converts it to its reference date time interval! Since we could cast a number to a date, it makes perfect sense that we can cast a date to a number.

The last trick is “how do we get the current date without having to substitute it in every time?”. Fortunately, there’s a very simple answer: now(). now() is one of the built-in functions supported by NSExpression, and it (conveniently enough) returns an NSDate representing the current date and time.

Now that we know this, we can construct relative date predicates! So for example, let’s do “someDate is in the last 30 days”. If a date is in the last 30 days, that means that someDate is after now - 30 days:

[NSPrediate predicateWithFormat:@"someDate > CAST(CAST(now(), 'NSNumber') - %d, 'NSDate')", (30*86400)];

Working from the inside-out, we have:

  • now() - retrieve the current NSDate
  • CAST(now(), 'NSNumber') - convert the current date into an NSNumber
  • CAST(now(), 'NSNumber') - %d - subtract a time interval from the current timestamp. This difference must be in seconds. 30*86400 is 30 days in seconds (there are 86,400 seconds in a day)
  • CAST(CAST(now(), 'NSNumber') - %d, 'NSDate') - once we’ve done our subtraction, cast the time interval back to an NSDate

At this point, we can compare it against our target date. We could’ve done the date subtraction on the value returned by the left expression, but there are some circumstances where that’s not a good idea (primarily when integrating with CoreData), so we’ll keep it in the right.

VoilĂ ! Date subtraction in an NSPredicate! Doing something like “<some date> is during <this week, this month, this year>" would be slightly more complex (it would involve a BETWEEN comparison [or something functionally identical]), but still do-able. Even more fun would be a predicate of the form “<some date> is during the <day, week, month, year> of <another date>”. Also very difficult, but again not impossible. Maybe we’ll do those in another post. Anyway…

The Row Template

Our row template is going to be very similar to the template in my earlier post, because (again) we’re doing NSTimeInterval manipulations. Let’s get started:

The Interface

@interface DDRelativeDateRowTemplate : NSPredicateEditorRowTemplate {
    NSPopUpButton * unitPopUpButton;
}

- (id) initWithLeftExpressions:(NSArray *)leftExpressions;

@end

Nothing new to see here.

The Implementation

#define NSInTheLastPredicateOperatorType NSMatchesPredicateOperatorType
#define NSInTheNextPredicateOperatorType NSLikePredicateOperatorType

Some #defines to make things a bit more readable in our code

@implementation DDRelativeDateRowTemplate

- (id) initWithLeftExpressions:(NSArray *)leftExpressions {
    NSAttributeType rightType = NSDoubleAttributeType; //NSTimeInterval is a typedef'd double
    NSComparisonPredicateModifier modifier = NSDirectPredicateModifier; //don't need "ANY" or "ALL"
    NSArray * operators = [NSArray arrayWithObjects:
                           [NSNumber numberWithUnsignedInteger:NSInTheLastPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSInTheNextPredicateOperatorType],
                           nil];
    NSUInteger options = 0;
    return [super initWithLeftExpressions:leftExpressions
             rightExpressionAttributeType:rightType
                                 modifier:modifier
                                operators:operators
                                  options:options];
}

Our init method is almost identical to the init method in our previous template, with the exception that this time we’re only supporting 2 operators. Notice that we’re telling the predicate editor that these are the “matches” and “like” operators. Again, this is only to get around problem #1 (see above).

static NSString * unitNames[] = {@"seconds", @"minutes", @"hours", @"days", @"weeks"};
static NSInteger unitIntervals[] = {1, 60, 3600, 86400, 604800};
#define numberOfUnits() (sizeof(unitNames)/sizeof(unitNames[0]))

- (NSPopUpButton *) unitPopUpButton {
    if (unitPopUpButton == nil) {
        unitPopUpButton = [[NSPopUpButton alloc] initWithFrame:NSZeroRect pullsDown:NO];

        NSMenu * unitMenu = [unitPopUpButton menu];
        for (int i = 0; i < numberOfUnits(); ++i) {
            [unitMenu addItemWithTitle:unitNames[i] action:NULL keyEquivalent:@""];
        }
    }
    return unitPopUpButton;
}

- (void) dealloc {
    [unitPopUpButton release];
    [super dealloc];
}

- (void) getBase:(NSTimeInterval *)base unit:(NSInteger *)unit fromTimeInterval:(NSTimeInterval)timeInterval {
    if (base == nil || unit == nil) {
        [NSException raise:NSInvalidArgumentException format:@"base and unit cannot be nil"];
    }

    for (NSInteger unitIndex = numberOfUnits() - 1; unitIndex >= 0; unitIndex--) {
        if (timeInterval >= unitIntervals[unitIndex]) {
            *base = timeInterval / unitIntervals[unitIndex];
            *unit = unitIndex;
            return;
        }
    }
}

- (NSTimeInterval) timeIntervalFromBase:(NSTimeInterval)base unit:(NSInteger)unitIndex {
    if (unitIndex >= numberOfUnits()) {
        [NSException raise:NSInvalidArgumentException format:@"unitIndex beyond max allowed bounds (%d)", numberOfUnits()];
        return 0;
    }

    return (base * unitIntervals[unitIndex]);
}

This code is identical to the code that was in our previous template, so I won’t explain it here.

- (NSArray *) templateViews {
    NSMutableArray * views = [[super templateViews] mutableCopy];

    [views addObject:[self unitPopUpButton]];

    NSPopUpButton * operatorView = [views objectAtIndex:1];
    [[operatorView itemAtIndex:0] setTitle:@"is in the last"];
    [[operatorView itemAtIndex:1] setTitle:@"is in the next"];

    return [views autorelease];
}

Our -templateViews method is almost the same, except that we’re altering the title of of the operators. If we were to leave out that bit, then we would see “matches” and “is like” in the interface. By changing the title here, we’re providing a new default value for the operator title. Note that this can be changed via localization, and more importantly, your NSLocalizedString macros for generating the strings file should use the values listed here, not the “matches” and “is like” values. Also note that operators show up in the interface in the same order in which they’re given in the -init method.

- (double) matchForPredicate:(NSPredicate *)predicate {
    if (![predicate isKindOfClass:[NSComparisonPredicate class]]) { goto errorExit; }

    NSComparisonPredicate * comparison = (NSComparisonPredicate *)predicate;

    NSExpression * left = [comparison leftExpression];
    if (![[self leftExpressions] containsObject:left]) { goto errorExit; }

    NSPredicateOperatorType operator = [comparison predicateOperatorType];
    if (operator != NSGreaterThanPredicateOperatorType && operator != NSLessThanPredicateOperatorType) { goto errorExit; }

    NSExpression * right = [comparison rightExpression];
    if ([right expressionType] != NSFunctionExpressionType) { goto errorExit; }
    if (![[right function] isEqual:@"castObject:toType:"]) { goto errorExit; }

    NSArray * outerCastArguments = [right arguments];
    //we can pull these out without bounds checking because a CAST() function *must* have 2 arguments

    NSExpression * firstArgument = [outerCastArguments objectAtIndex:0];
    {
        //the first argument must be either an addition or subtraction function
        if ([firstArgument expressionType] != NSFunctionExpressionType) { goto errorExit; }
        if (![[firstArgument function] isEqual:@"add:to:"] && ![[firstArgument function] isEqual:@"from:subtract:"]) { goto errorExit; }
        NSArray * relativeArguments = [firstArgument arguments];
        NSExpression * firstRelativeArgument = [relativeArguments objectAtIndex:0];
        {
            if ([firstRelativeArgument expressionType] != NSFunctionExpressionType) { goto errorExit; }
            if (![[firstRelativeArgument function] isEqual:@"castObject:toType:"]) { goto errorExit; }
            NSArray * innerCastArguments = [firstRelativeArgument arguments];
            NSExpression * firstInnerArgument = [innerCastArguments objectAtIndex:0];
            if ([firstInnerArgument expressionType] != NSFunctionExpressionType) { goto errorExit; }
            if (![[firstInnerArgument function] isEqual:@"now"]) { goto errorExit; }

            NSExpression * secondInnerArgument = [innerCastArguments objectAtIndex:1];
            if ([secondInnerArgument expressionType] != NSConstantValueExpressionType) { goto errorExit; }
            if (![[secondInnerArgument constantValue] isEqual:@"NSNumber"]) { goto errorExit; }
        }

        NSExpression * secondRelativeArgument = [relativeArguments objectAtIndex:1];
        {
            //the second relative argument is the actual relative difference, so it must be an NSNumber
            if ([secondRelativeArgument expressionType] != NSConstantValueExpressionType) { goto errorExit; }
            if (![[secondRelativeArgument constantValue] isKindOfClass:[NSNumber class]]) { goto errorExit; }
        }
    }

    NSExpression * secondArgument = [outerCastArguments objectAtIndex:1];
    {
        if ([secondArgument expressionType] != NSConstantValueExpressionType) { goto errorExit; }
        if (![[secondArgument constantValue] isEqual:@"NSDate"]) { goto errorExit; }
    }

    return DBL_MAX;

errorExit:
    return 0.0;
}

This is a beast of a method. This is how we determine if this row template recognizes a predicate or not. Basically what we’re doing here is making sure that the predicate has a extremely specific structure. If every requirement is met, then we return DBL_MAX, which is NSPredicateEditorRowTemplate's way of saying “I match this predicate better than anyone else ever could”. When NSPredicateEditor finds that multiple row templates match a predicate, it uses the row template that returns the highest match value. DBL_MAX is just a cheap way of ensuring that no other template can handle this predicate.

The expected structure is as follows: (each box is an NSExpression, and downward arrows indicate arguments to a function expression)

The NSExpression structure for relative dates

And yes, I use goto all over the place in that code. Deal with it.

- (NSPredicate *) predicateWithSubpredicates:(NSArray *)subpredicates {
    NSPredicate * predicate = [super predicateWithSubpredicates:subpredicates];
    if ([predicate isKindOfClass:[NSComparisonPredicate class]]) {
        NSComparisonPredicate * comparison = (NSComparisonPredicate *)predicate;

        NSExpression * right = [comparison rightExpression];
        NSNumber * value = [right constantValue];

        NSInteger unit = [[self unitPopUpButton] indexOfSelectedItem];

        NSTimeInterval newInterval = [self timeIntervalFromBase:[value doubleValue] unit:unit];
        value = [NSNumber numberWithDouble:newInterval];
        right = [NSExpression expressionForConstantValue:value];

        //default values are for NSInTheLastPredicateOperatorType
        NSString * function = @"from:subtract:";
        NSPredicateOperatorType newOperator = NSGreaterThanPredicateOperatorType;

        if ([comparison predicateOperatorType] == NSInTheNextPredicateOperatorType) {
            function = @"add:to:";
            newOperator = NSLessThanPredicateOperatorType;
        }

        //"now()" is a function and takes no arguments
        NSExpression * now = [NSExpression expressionForFunction:@"now" arguments:[NSArray array]];

        //CAST(now(), 'NSNumber')
        NSArray * castNowArguments = [NSArray arrayWithObjects:now, [NSExpression expressionForConstantValue:@"NSNumber"], nil];
        NSExpression * castNow = [NSExpression expressionForFunction:@"castObject:toType:" arguments:castNowArguments];

        //CAST(now(), 'NSNumber') [+/-] {a time interval}
        NSArray * relativeTimestampArguments = [NSArray arrayWithObjects:castNow, right, nil];
        NSExpression * relativeTimestamp = [NSExpression expressionForFunction:function arguments:relativeTimestampArguments];

        //CAST(CAST(now(), 'NSNumber') [+/-] {a time interval}, 'NSDate')
        NSArray * castToDateArguments = [NSArray arrayWithObjects:relativeTimestamp, [NSExpression expressionForConstantValue:@"NSDate"], nil];
        NSExpression * castToDate = [NSExpression expressionForFunction:@"castObject:toType:" arguments:castToDateArguments];

        predicate = [NSComparisonPredicate predicateWithLeftExpression:[comparison leftExpression] 
                                                       rightExpression:castToDate 
                                                              modifier:[comparison comparisonPredicateModifier] 
                                                                  type:newOperator 
                                                               options:[comparison options]];
    }
    return predicate;
}

This method is similar in principle to its counterpart in our earlier row template. Here we’re letting super give us a predicate that contains what the user entered into the interval field, and then we extract that number, convert it to an absolute time interval, and then re-create the right expression of the comparison predicate. We create the expression manually to ensure that the structure returned here exactly matches the structure expected by -matchForPredicate:. Notice that we’re changing our matches and like operators to the expected less than and greater than versions.

- (void) setPredicate:(NSPredicate *)predicate {
    if ([predicate isKindOfClass:[NSComparisonPredicate class]]) {
        NSComparisonPredicate * comparison = (NSComparisonPredicate *)predicate;
        /**
         I know that to get here, I must have matched the predicate.
         Therefore I can drill into the comparison predicate structure with reckless abandon!
         **/
        NSExpression * right = [comparison rightExpression];
        NSArray * arguments = [right arguments];

        //this expression is our addition or subtraction function
        NSExpression * relative = [arguments objectAtIndex:0];

        //the second argument to our relative function is our absolute time interval
        NSExpression * intervalExpression = [[relative arguments] objectAtIndex:1];
        NSNumber * interval = [intervalExpression constantValue];

        NSTimeInterval base = 0;
        NSInteger unit = 0;
        [self getBase:&base unit:&unit fromTimeInterval:[interval doubleValue]];

        [[self unitPopUpButton] selectItemAtIndex:unit];

        NSExpression * newRight = [NSExpression expressionForConstantValue:[NSNumber numberWithDouble:base]];
        NSPredicateOperatorType newOperator = NSInTheLastPredicateOperatorType;
        if ([comparison predicateOperatorType] == NSLessThanPredicateOperatorType) {
            newOperator = NSInTheNextPredicateOperatorType;
        }

        predicate = [NSComparisonPredicate predicateWithLeftExpression:[comparison leftExpression] 
                                                       rightExpression:newRight
                                                              modifier:[comparison comparisonPredicateModifier] 
                                                                  type:newOperator 
                                                               options:[comparison options]];
    }
    [super setPredicate:predicate];
}

@end

And finally! When given a predicate, we need to reflect those values in the UI. Here we’re simply going to extract the absolute time interval that’s buried in the rightExpression and use it to figure out what our base and unit should be. We also change the operator from greater than or less than to likes or matches, simply so that the operator popup shows the right value. After that, we re-package things, and send it on up to super.

Wrap-up

Not too bad, eh? Grokking this involves a little bit of mental gymnastics, but if you’re familiar with predicates and their internal structure, most of this code should be really straight-forward. This was mainly to help connect all those disparate dots and form a coherent picture. The more you play with these row templates, the more you’ll truly understand how they interact with each other and their parent predicate editor. They’re quite clever little classes, and they can really make our apps that much more “Mac-like”.

Mathematically evaluating NSStrings

I’ve recently posted the code to a project I’ve written called DDMathParser. It’s a way whereby you can take an NSString and evaluate it as a mathematical expression. For example, you can do:

NSLog(@"%@", [@"1 + 3 * 4 - 5" numberByEvaluatingString]); //logs "8"

There are several other ways to evaluate strings, including methods that allow you to use variables (1 + $a, and then substitute $a for a value later). You can even define custom functions, like mySuperSecretFunction(42).

This was a really fun project, because there were some really interesting challenges to work around (like how can I parse a left associative expression using a recursive descent parser without getting caught in an infinite recursion?). I’ll save those for another post.

For now, go check out the source on github and let me know what you think!

http://github.com/davedelong/DDMathParser

Creating a simple NSPredicateEditorRowTemplate

NSPredicateEditor is a great class, provided you figure out how it works. There’s a fair amount of documentation as to what’s going on, but if you ever want to do something beyond what’s possible through configuration in Interface Builder, you’re probably going to spend several hours playing with things until you get it right.

Hopefully, I can help cut down that time.

What’s going on under the hood

NSPredicateEditor is a subclass of NSRuleEditor. However, they really don’t behave the same way at all, so knowing that doesn’t really give you much insight (that I’ve found).

NSPredicateEditor uses a template mechanism for displaying an NSPredicate. These templates are instances of NSPredicateEditorRowTemplate (go figure). Each row template (with one exception) displays a single NSComparisonPredicate.

The exception to this is the built-in row template for handling NSCompoundPredicates. We’re going to ignore this template in this post and focus on customizing templates for comparison predicates.

When the NSPredicateEditor gets a new predicate object (via -setObjectValue:), it has to figure out what row templates it needs. It’s a pretty simple process: The editor simply asks each one of its row templates whether it recognizes the NSPredicate. (-matchForPredicate:) If the template responds affirmatively, then the editor will -copy the template (this is why row templates conform to <NSCopying>) and invoke -setPredicate:.

-setPredicate: is simply going to extract the predicate’s leftExpression, predicateOperatorType, and rightExpression. These values will then be set into the UI.

The UI

One of the interesting things about row templates is that they do not inherit from NSView. In other words, the actual NSPredicateEditorRowTemplate object is never (indeed cannot be) displayed to the user. So what is it the user is seeing? It’s the row template’s -templateViews.

By default, an NSPredicateEditorRowTemplate provides 3 template views: an NSPopUpButton, a second NSPopUpButton, and either a third NSPopUpButton or some sort of user-interactable view, like an NSDatePicker or an NSTextField. The first popup lists all of the possible left-hand NSExpressions, the second holds the operators, and the third view holds the right-hand NSExpressions.

These views are received by the NSPredicateEditor and shown in the UI. Their styles are manipulated to make them have the “rounded rect” appearance, and their frames are changed to make them fit in the row. The main attribute that’s useful to configure is the frame width of the right-hand view. If the third view is an NSDatePicker, you may alter the width of the view to allow room for specifying a time in addition to the date. (By default, an NSDatePicker only allows date selection)

Customizing the UI is a simple matter of overriding the -templateViews method:

- (NSArray *) templateViews {
  NSMutableArray * templateViews = [[super templateViews] mutableCopy];

  //at this point, you have a mutable array of views.
  //you can add views, remove views, alter appropriate properties of existing views, etc.

  return [templateViews autorelease];
}

However, you need to be careful with this. -templateViews is called more than once, so if you’re instantiating a new view each time this is invoked, you may end up allocating too many views and not know which one is actually being shown on the UI. For this reason I prefer a “lazy allocation” method:

- (NSView *) myExtraView {
  if (myExtraView == nil) {
    myExtraView = [[MyExtraView alloc] init...];
    ...
  }
  return myExtraView;
}

- (NSArray *) templateViews {
  NSMutableArray * templateViews = [[super templateViews] mutableCopy];

  [templateViews addObject:[self myExtraView]];

  return [templateViews autorelease];
}

Now I know that the myExtraView object will only be allocated once per instance of my row template, and I can always go access the view directly to retrieve information about what the user has manipulated.

<caution>

If you’re localizing this NSPredicateEditor, then you must consider this:

  • When specifying the translated value of a row template, the number of positional specifiers in the translated value must be equal to the number of views returned from -templateViews. If not, the editor will fail to localize that row.

In other words, if you have a custom -templateViews method that adds a fourth view, then your translated string for this row must have %1$@, %2$@, %3$@, and %4$@ (with the translated values interspersed appropriately). %1$@ will be replaced with the first view in the array returned from -templateViews, %2$@ will be replaced with the second view, etc.

</caution>

On a final point regarding -templateViews, I’ve found that I rarely have an occasion to alter the first or second views returned by super. Your mileage may vary, but I’ve generally found that it’s best to leave them alone.

Building a custom row template

Now that we understand the basics of what’s going on, let’s build a simple NSPredicateEditorRowTemplate. To keep things simple, we’re going to build a row template that allows us to compare against an NSTimeInterval.

The Interface

@interface DDTimeIntervalRowTemplate : NSPredicateEditorRowTemplate {
    NSPopUpButton * unitPopUpButton;
}

- (id) initWithLeftExpressions:(NSArray *)leftExpressions;

@end

It’s pretty simple. We inherit from NSPredicateEditorRowTemplate, we provide a custom initializer (since we’ll be instantiating the template programmatically), and we have one instance variable for holding on to our extra NSPopUpButton.

The Implementation

@implementation DDTimeIntervalRowTemplate

- (id) initWithLeftExpressions:(NSArray *)leftExpressions {
    NSAttributeType rightType = NSDoubleAttributeType; //NSTimeInterval is a typedef'd double
    NSComparisonPredicateModifier modifier = NSDirectPredicateModifier; //don't need "ANY" or "ALL"
    NSArray * operators = [NSArray arrayWithObjects:
                           [NSNumber numberWithUnsignedInteger:NSLessThanPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSLessThanOrEqualToPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSGreaterThanPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSGreaterThanOrEqualToPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSEqualToPredicateOperatorType],
                           [NSNumber numberWithUnsignedInteger:NSNotEqualToPredicateOperatorType],
                           nil];
    NSUInteger options = 0;
    return [super initWithLeftExpressions:leftExpressions
             rightExpressionAttributeType:rightType
                                 modifier:modifier
                                operators:operators
                                  options:options];
}

Again, this should be fairly straight-forward. We’re creating a row template with the left expressions given to the initializer. These left expressions will be compared against a user-entered double value. We’ll be doing a direct comparison (simply put: <left expression> <comparison operator> <user-entered value>), and we’re supporting the standard numeric operators: <, <=, >, >=, =, and !=. The options bit is only useful for string comparison (it’s how you specific case insensitivity or diacritic insensitivity, etc). Once we’ve got everything specified, we’ll let super's initializer finish everything off for us.

Continuing…

static NSString * unitNames[] = {@"seconds", @"minutes", @"hours", @"days", @"weeks"};
static NSInteger unitIntervals[] = {1, 60, 3600, 86400, 604800};
#define numberOfUnits() (sizeof(unitNames)/sizeof(unitNames[0]))

- (NSPopUpButton *) unitPopUpButton {
    if (unitPopUpButton == nil) {
        unitPopUpButton = [[NSPopUpButton alloc] initWithFrame:NSZeroRect pullsDown:NO];

        NSMenu * unitMenu = [unitPopUpButton menu];
        for (int i = 0; i < numberOfUnits(); ++i) {
            [unitMenu addItemWithTitle:unitNames[i] action:NULL keyEquivalent:@""];
        }
    }
    return unitPopUpButton;
}

- (void) dealloc {
    [unitPopUpButton release];
    [super dealloc];
}

This method will be our lazy initializer for our unit popup button (to ensure that we only allocate one button per instance of the row template). The button has 5 items in its menu, each representing a different unit of time measurement. We also have a static NSInteger array that reflects the number of seconds in each unit (ie, one day is 86,400 seconds).

One thing that’s interesting here is that we’re initializing our NSPopUpButton with NSZeroRect. We can do this because NSPredicateEditor will adjust the origin of the view to be in the appropriate spot relative to the other views and the size to fit the content. In other words, the frame will be taken care of for us.

And of course, since we’ve allocated an object, we must be sure to release it in our -dealloc method.

- (void) getBase:(NSTimeInterval *)base unit:(NSInteger *)unit fromTimeInterval:(NSTimeInterval)timeInterval {
    if (base == nil || unit == nil) {
        [NSException raise:NSInvalidArgumentException format:@"base and unit cannot be nil"];
    }

    for (NSInteger unitIndex = numberOfUnits() - 1; unitIndex >= 0; unitIndex--) {
        if (timeInterval >= unitIntervals[unitIndex]) {
            *base = timeInterval / unitIntervals[unitIndex];
            *unit = unitIndex;
            return;
        }
    }
}

- (NSTimeInterval) timeIntervalFromBase:(NSTimeInterval)base unit:(NSInteger)unitIndex {
    if (unitIndex >= numberOfUnits()) {
        [NSException raise:NSInvalidArgumentException format:@"unitIndex beyond max allowed bounds (%d)", numberOfUnits()];
        return 0;
    }

    return (base * unitIntervals[unitIndex]);
}

These 2 methods are simply convenience methods. The first method will convert an NSTimeInterval into a unit and a base. For example, if given 172800 as the interval, it will return a base of 2 and a unit of 3, to represent 2 days.

The second method is simply the inverse. Given a base and a unit, it will turn it into an absolute NSTimeInterval.

Now for the fun stuff!

- (NSArray *) templateViews {
    NSMutableArray * views = [[super templateViews] mutableCopy];

    [views addObject:[self unitPopUpButton]];

    return [views autorelease];
}

This is about as simple as it gets. When we’re asked for what views to display in the interface, we’re going to return the three views provided by super (two popup buttons and an NSTextField) and add a fourth button of our own (the popup button denoting a unit of time).

- (NSPredicate *) predicateWithSubpredicates:(NSArray *)subpredicates {
    NSPredicate * p = [super predicateWithSubpredicates:subpredicates];
    if ([p isKindOfClass:[NSComparisonPredicate class]]) {
        NSComparisonPredicate * comparison = (NSComparisonPredicate *)p;

        NSExpression * right = [comparison rightExpression];
        NSNumber * value = [right constantValue];

        NSInteger unit = [[self unitPopUpButton] indexOfSelectedItem];

        NSTimeInterval newInterval = [self timeIntervalFromBase:[value doubleValue] unit:unit];
        value = [NSNumber numberWithDouble:newInterval];
        right = [NSExpression expressionForConstantValue:value];

        p = [NSComparisonPredicate predicateWithLeftExpression:[comparison leftExpression] 
                                               rightExpression:right 
                                                      modifier:[comparison comparisonPredicateModifier] 
                                                          type:[comparison predicateOperatorType] 
                                                       options:[comparison options]];
    }
    return p;
}

This method is one of the two at the heart of our custom template. Via this method, we’re translating what’s on the UI into an NSPredicate. For the most part, we’re going to let super handle the implementation, with one minor alteration. By invoking super on this, we’re going to get back an NSPredicate of the form <left expression> <operator> <value of textfield>. We need to extract the rightExpression, who’s constantValue will be an NSNumber representing what was in the textfield. It is the doubleValue of this NSNumber that becomes the base, and the index of our unitPopUpButton is our unit. Once we have those two values, we can run them through our conversion method, then rebuild our comparison predicate. This new comparison predicate will be exactly the same as the predicate returned by super, with the single change of having the constantValue of the rightExpression modified.

Finally…

- (void) setPredicate:(NSPredicate *)newPredicate {
    if ([newPredicate isKindOfClass:[NSComparisonPredicate class]]) {

        NSComparisonPredicate * comparison = (NSComparisonPredicate *)newPredicate;

        NSExpression * right = [comparison rightExpression];
        NSNumber * value = [right constantValue];

        NSTimeInterval base = 0;
        NSInteger unit = 0;
        [self getBase:&base unit:&unit fromTimeInterval:[value doubleValue]];

        value = [NSNumber numberWithDouble:base];
        right = [NSExpression expressionForConstantValue:value];

        [[self unitPopUpButton] selectItemAtIndex:unit];

        newPredicate = [NSComparisonPredicate predicateWithLeftExpression:[comparison leftExpression] 
                                                          rightExpression:right 
                                                                 modifier:[comparison comparisonPredicateModifier] 
                                                                     type:[comparison predicateOperatorType] 
                                                                  options:[comparison options]];
    }

    [super setPredicate:newPredicate];
}

@end

This is the final method that we need to implement. This method will take an NSPredicate and extract various values from it to reflect in the UI. Like its inverse -predicateWithSubpredicates:, we’re going to let super handle most of the work. However, we need to do a bit of manipulation first. We’re going to extract the constantValue of the comparison predicate’s rightExpression. This NSNumber is the absolute time interval, in seconds. We’re going to take that time interval and run it through our conversion method to turn it into a base and a unit. The unit is used to select the appropriate item in the unitPopUpButton, and the base is put back into the NSPredicate. When we pass this new predicate up to super, the base will be placed inside the text field (the third view returned from -templateViews).

Using this template

Now that we have this template, we have to actually use it. Fortunately, it’s pretty simple:

NSPredicateEditor * editor = ...; //some NSPredicateEditor. perhaps an IBOutlet?
NSMutableArray * rowTemplates = [[editor rowTemplates] mutableCopy];

NSArray * leftExpressions = [NSArray arrayWithObject:[NSExpression expressionForKeyPath:@"duration"]];
DDTimeIntervalRowTemplate * intervalTemplate = [[DDTimeIntervalRowTemplate alloc] initWithLeftExpressions:leftExpressions];
[rowTemplates addObject:intervalTemplate];
[intervalTemplate release];

[editor setRowTemplates:rowTemplates];
[rowTemplates release];

This should be fairly easy to understand. We’re simply adding an instance of DDTimeIntervalRowTemplate to the list of templates available to our NSPredicateEditor. This row template recognizes one left expression, the keyPath @"duration". This means that all predicates generated by this row template will be of the form duration <operator> <time interval>. Pretty easy.

Localizing this template

To localize this row template, we’ll need the following somewhere in a comment:

NSLocalizedStringFromTable(@"%[duration]@ %[is, is not, is greater than, is less than, is greater than or equal to, is less than or equal to]@ %@ %[seconds, minutes, hours, days, weeks]@", @"Predicate", @"the text of the duration template")

This will generate a rather lengthy strings file (30 different key-value pairs), with each value having four positional specifiers (as was mentioned above), one for each of the views returned from -templateViews. For a more complete explanation of how localizing a predicate editor, see my earlier post on that very topic.

Wrap-Up

So there you have it! This may be a rather simple and contrived example, but a useful one. Hopefully this will help you as you wrap your brain around NSPredicateEditor and its supporting classes. There are still a couple gotchas, but we’ll learn about those in another post.

Abusing NSPredicate

NSPredicate is a nifty class. It’s intended use is to be an objected-oriented way of expressing a truth statement. We see this being used with things like -[NSArray filteredArrayUsingPredicate:], -[NSFetchRequest setPredicate:], and more. NSPredicate, however, can offer us a lot more.

There are 2 things that make NSPredicate so insanely awesome:

  1. Its string parser
  2. The evaluation power of NSExpression (one of NSPredicate's support classes)

NSPredicate Parsing

One of the best things about NSPredicate is its parsing engine. Given a well-formatted NSString, NSPredicate will turn it into a tree of NSCompoundPredicate and NSComparisonPredicate objects. This can be used to our advantage. If we have a string that represent some boolean expression, we can tap NSPredicate to parse it for us and give us back an organized syntax tree.

For this example, let’s consider the following:

(a | b) & c | (b | d)

If we just try to run this through the predicate parser, we’ll get an exception, for 2 reasons:

  1. A predicate is a truth statement. This expression is simply a value. It is not being compared to anything, and as such does not represent a true or false statement, but simply a value.
  2. Each sub-expression of a predicate must itself be a predicate. Since “c" (for example) is not being compared to anything, it is not a valid statement.

Fixing the first problem is easy. We can just tack “= 0" onto the end of the statement:

(a | b) & c | (b | d) = 0

As for fixing the second, we can do that with a little bit of find-and-replace by regular expression. For simplicity, we’ll say that each one of these identifiers (a, b, c, and d) must be alphanumeric. We can then use something like NSRegularExpression (on iOS) or RegexKit (on Mac) to replace “([a-z0-9]+)" with "\1 = 0”, thereby giving us:

(a = 0 | b = 0) & c = 0 | (b = 0 | d = 0) = 0

We can now run this through +[NSPredicate predicateWithFormat:] to get our parsed tree. From there you can recursively walk the tree and manipulate things as much as you like.

NSExpression Solving

Let’s say you have a string that contains some sort of mathematical statement. Perhaps it’s user-entered, or perhaps it’s dynamically generated. For our purposes, we’re going to use:

22/7.0 + (13*42) - 1024

Our calculator shows us that we should be getting -474.857142857142857 as the result.

But how can we evaluate this dynamically? Well, you could use something like Graham Cox’s excellent GCMathParser to parse and evaluate it, but this has a limitation: it’s a bit difficult (read: nearly impossible) to extend to use functions not supported by the parser.

Fortunately, we can abuse NSPredicate to do the work for us.

NSPredicate has a wonderful parsing engine (which we’ve already abused), and it builds a tree built out of NSExpression objects, which have this wonderful method called -expressionValueWithObject:context: which will evaluate everything for us. Unfortunately, there’s no way to build an NSExpression tree out of an arbitrary NSString. However, NSPredicate can.

So we just need to make out statement into a predicate, run it through the parser, and extract the appropriate value! This is really simple. To make something into a predicate, we simply need to add an operator and a comparison value:

22/7.0 + (13*42) - 1024 = 0

Now, we can pump it through NSPredicate:

NSPredicate * parsed = [NSPredicate predicateWithFormat:@"22/7.0 + (13*42) - 1024 = 0"];

We know that this is an NSComparisonPredicate (since it’s of the form <expression> <operator> <expression>), so we can simply extract the leftExpression:

NSExpression * left = [(NSComparisonPredicate *)parsed leftExpression];

And finally, we can now evaluate it:

NSNumber * result = [left expressionValueWithObject:nil context:nil];
NSLog(@"result: %@", result);

Wonderfully, this logs -474.8571428571429, which is exactly what we were hoping for.