A Few Examples

an Avatar filter I made that responds to eye "openness"

a UI controlled pixelation filter I made

A month or so ago, one of my friends from out of town came to visit. I mentioned to him that I've been doing a lot of creative coding, and showed him some of the Processing scripts I've made.

During that conversation, he mentioned SparkAR, and I bookmarked it ("neat" I believe was the word I used). Later that week, I downloaded it during a bit of boredom. I was hooked immediately by the potential of the tool.

What is SparkAR?

SparkAR is best described as an AR Photoshop + Unity + scripting environment (kinda). It allows creators to make their own filters for Facebook or Instagram, without having too much in the way of technical knowledge. Creators can import 3D assets, textures, and utilize face tracking to make fun effects for IG users.

For myself though, I was interested in the scripting functionality. I wanted to make cool interactive filters, and use my programming background to make up for my lack of a design skillset. The API is written in Javascript, so I thought it'd be relatively easy to get into.

And I'll be the first to admit, the API is bad. Like... bad bad. It's not well updated, the documentation isn't clear, the index takes you to a broken Facebook link, and the community on StackOverflow or otherwise is quite small and/or non-existent.

get used to seeing this really annoying error

Even with all that, I'd like to impart a few lessons I've learned using SparkAR scripting this past few weeks.

1) The Native Slider
2) Monitoring Multiple Things
3) The Reactive Paradigm, Variables, and You

Let's jump into it.

The Native Slider

github example

SparkAR has a built in slider that allows users to control the "intensity" of your filter. I'm using the slider here to modify the opacity of the pixelation filter.

var lastSliderValue = 0.5;
const slider = NativeUI.slider;
const pixelatedMat = Materials.get('pixelatedMat');

slider.value.monitor({fireOnInitialValue: false}).subscribe(function(val) {
    lastSliderValue = val.newValue;
    // Choose your desired slider value
    pixelatedMat.opacity = lastSliderValue;

});

NativeUI.slider.value = lastSliderValue;
NativeUI.slider.visible = true;

The Reactive Paradigm, and You

github example

Spark Studio recommends that you use their reactive programming technique over the traditional imperative method. The problem is, they don't tell you where and where you can't use it!

Basically what will happen is that it will fire once when the script starts, but it won't "subscribe" to the changes of the eye (blinking for example).

I've found that reactive value setting only works in non conditional, left = right type statements. For example:

leftIris.hidden = leftEye.openness.lt(Reactive.val(0.5)); // works
leftIris.hidden = leftEye.openness < Reactive.val(0.5) ? true : false; // will NOT work

// multi line example
leftIris.hidden = leftEye.openness.lt(Reactive.val(0.5)); // works
rightIris.hidden = rightEye.openness.lt(Reactive.val(0.5)); // works
material.opacity =   leftIris.hidden && rightIris.hidden ? 1.0 : 0.0; // will NOT work

Monitoring Multiple Things

github example

In the multi line example above, I was looking for a way to subscribe to two or more elements concurrently. Fortunately, there's a method for that!

monitorMany() accepts an array of scalars (0.0 -> 1.0) and subscribes to their value. Inside the subscription I've captured and converted the values to my desire:

const leftOpen = leftEye.openness;
const rightOpen = rightEye.openness;

Reactive.monitorMany([leftOpen, rightOpen]).subscribe(function(event) {
    const isLeftOpen = event.newValues["0"] > 0.5;
    const isRightOpen = event.newValues["1"] > 0.5;

    if (isLeftOpen && isRightOpen) arrow.opacity = Reactive.val(1.0);
    else arrow.opacity = Reactive.val(0.0);
}) 

Conclusion

SparkAR is a really cool technology with a ton of potential. I hope these tips will save you a bit of hair pulling as you script your filters. Have fun out there!