Understanding Shaders in SpriteKit
If you’re new to game development, you’ve probably heard of shaders but don’t quite understand them. If you’re new to SpriteKit, you’ve probably hit a few speed bumps working with shaders. Since I’m still learning more about the two, I figured it would be nice to put up a concrete example that covers cropping and effects using shaders. Specifically, we’ll cover two concepts:
- Using alpha, black, and white pixel values to determine which pixels to display.
- Porting shaders found on Shadertoy into SpriteKit.
Here is what it’ll look like when we’re done.
If you have never worked with a shader in SpriteKit before, you should probably check out this quick introductory tutorial. And if you’re interested in shaders, but not SpriteKit, you may still find a thing or two that grabs you’re attention.
Getting Started
The first thing to do is find an image that you would like to mask or modify in some way. For this example, I’ll be using a circular health bar that wraps around a shield graphic. Note the image is a .png, and that in between the black lines along with the outside is fully transparent. We’ll try to satisfy three requirements.
- Only show health in between the two black lines outlining the image.
- Allow for the health section to be partially full.
- Add movement to the health bar.
A Basic Shader
To apply a basic shader to our image, we’ll need to tell SpriteKit to attach a shader to the SKSpriteNode, and we’ll also need to create an empty text file ending in .fsh that contains our shader code. First, the shader code:
Next, attach the shader in SpriteKit:
As you can see to the right, the shader turns every pixel green. This is because main()
is the entry point, and by default gl_FragColor
is the return value. So, for each pixel of the image, this snippet of code runs. And when the code runs, it is telling each pixel that the color should be green (note the r,g,b,a value of the vec4()
call).
Detecting Alpha Values
Now that we can change the color of pixels, the next step is to only change the color of fully transparent pixels. We’ll change our shader code to the following:
Now, our image fully displays, and all transparent pixels have been replaced with green pixels. Specifically, we create val
which contains the pixel in the actual texture. The alpha value (val.a
) is transparent if it equals 0.0. So, we either replace the pixel with a green one, or set the return value to the pixel from the actual texture.
Masking & Gradients
The obvious problem with the previous step is that we have to get rid of everything outside of the circle. We only want to fill the inside. In addition, we want the inside to fill only a percentage equal to the health of the player. We’re going to add a separate image to act as both an image mask, and a gradient fill.
First, let’s talk about the mask. This circle is the exact same size as the shield above. So, if a pixel falls inside of both the circle and the shield, we know we want to draw it. If it falls outside, we can get rid of it. That’s how we’ll achieve the health bar look.
Second, let’s talk about the gradient. Notice that a radial gradient has been applied with pure black meeting pure white at the top. The general idea here is that we can check pixel colors. Imagine a player with health of 75%. We can check all pixels that are at least 75% black, and turn them on. Anything else gets turned off.
To make these two concepts, we’ll have to attach SKUniform’s to the shader. Think of this as passing variables through to the shader. In this case, we’ll need two variables. One for health, and one for the image above. Our Swift code should now look like:
We’re basically supplying a float and SKTexture to the shader. Then, we can modify the shader to produce our desired result.
Here is the result:
A few notes on the inline code comments:
[1] At the beginning of this tutorial, we checked val.a == 0.0
and now we’re checking val.a less than 0.1
. This is because there are partially transparent pixels around the edges of the circle that we’re being excluded. A limit of 0.1 set the best result.
[2] You can test the black value of a pixel against any of the r, g, b values of the vec4. I just used .r
because that is what I’ve seen the most.
**[3] **Similar to [1], I could have check the mask for pixels that are fully visible where grad.a == 1.0
, but a tolerance was needed to produce the best results.
Porting From Shadertoy
When browsing Shadertoy, you’ll notice certain variables that aren’t available to you, or that break your script. Here are the most common ones I’ve seen, and how to use them in SpriteKit.
iGlobalTime | u_time |
iResolution | u_sprite_size |
fragCoord.xy | gl_FragCoord.xy |
iChannelX | SKUniform with name of “iChannelX” containing SKTexture |
fragColor | gl_FragColor |
Using Existing Animation Examples
Now that we know how to port from Shadertoy, let’s try it with a few examples.
Example 1: Bullseye
Example 2: Wobble Spiral
Example 3:Glowing Thing
Further Reading & Next Steps
Resources:
- Making a Pixel Shader for iOS8 with SpriteKit
- Hub.ae’s shader intro and bloom/blur tutorial.
- SKShader documentation.
- WWDC14 PDF
- OpenGL ES 2.0 API Quick Reference Card
- Alternative to Shadertoy: GLSLSandbox
Beyond this, I’m still interested in the performance of shaders, and confidently knowing when to use them. For example, the 6th shader in the original gif above starts to lag a bit if you leave it running too long. Is it because of poor code, or is it too intensive? Similarly, would it be better to use 20 static images to represent health chunks of the circle, and just manage the health bar in code? I’ve often heard that using shaders is the “right way”, but I’d like to know with certainty when that is true. Perhaps another part to this tutorial is in order.
As far as the 6th shader is concerned, the common wisdom, as far as I know, is to avoid if conditionals (seems to be less of a problem now), trigonometric functions, since those take a lot of instructions to execute (quite often people use look up textures to skip those computations on the GPU and some say that trigonometric on the GPU aren't that bad any more) and doing too many calculations in one line, since the driver is probably generating a lot of temporary variables to perform those (if you're interested, check out these GDC slides http://www.gdcvault.com/play/1018182/Low-Level-Thinking-in-High). If I had to guess where the lag is coming from, I'd say it's either a precision issue with the time float or the trigonometric function start to get wonky with higher number ranges. To answer your larger question, in my opinion using shaders is great if you want to use less memory and want a smooth animation, I also think it's nice, that all that's needed, is setting just one uniform in a shader. If your shaders are getting too complex or you want to add special details by hand a sprite sheet may be better. I also prefer shaders since there has to be less redundant data in memory, given that a lot of the pixels in a sprite sheet probably aren't changing. If your shaders are performing badly look into calculating data higher up the chain, for instance maybe you can do calculations per vertex and have the results interpolated over the fragments or calculate data on the cpu and save the results in a look up texture, so that the gpu just has to do a texture lookup.
Thanks for the detailed reply. That's helpful to place some rules into the thought process. The link you provided is a bit above my head right now. Still some work to do before I can comfortably process that. I'll get there eventually.
Thanks a lot for that article! I'm quite new to game developement (i've been coding for years though), Swift and SpriteKit so we are pretty much in the same spot. I'm working on a space exploration game and I was wondering if shaders would be helpful for a few things. I'm not quite there yet, but when I start attacking the subject, maybe I can share some of my findings since I would be probably using shaders along with SKShapeNodes and SKEmitterNodes. Anyway, thanks a lot for the insight! Very helpful! One question though: in your case, do you create the textures and shader every time the player's health change? Can you simply change the uniform's values and the shader will adjust everything by itself every frame?
You should be able to just update the uniforms. I don't remember recreating the shader, but I don't have my sample code handy. Would definitely love to see what you come up with – there isn't much activity with advanced SpriteKit out there, so any content you come up with would be nice to read.
I might be completely wrong but it seems like the gradient or something does not work since iOS 10. Or something has changed. Or my implementation (a bit different) is incorrect. Can you confirm that this works in iOS 10 (iOS 10.2 etc) still?
Btw I am referring to a possible problem with shader3.fsh… val and/or grad does not read in properly, it seems.
I just tried your code again, using your exact same swift/shader code, and yes there is a problem starting from shader3.fsh, ie where you first use the gradient. It works on iOS 9ish, but not on a iOS 10.2.1 device. And I think I found a way to get away the problem on iOS 10.2: do NOT put the textures inside a sprite atlas in a .xcassets folder. If I put the sprite image files outside of a sprite atlas (still inside the .xcassets folder) things work OK/the same way even on iOS 10.2. Maybe the reason is related to app thinning, or the atlas, or something else.
Interesting, I haven't tried it yet in iOS10. If it is asset related, that wouldn't be the first time a sprite atlas vs .xcassets vs image in directory has broken things in SpriteKit. If I find a specific answer, I'll update it here. Thanks for the note.
By the way, it seems the u_sprite_size uniform has been removed as of iOS10, at least according to Apples docs: https://developer.apple.com/reference/spritekit/skshader "Since the removal of u_sprite_size, if your shader requires the size of the sprite in pixels, you are responsible for creating the necessary uniform or attribute." There is sample code for recreating this in the docs themselves, but thought it was worth pointing out.