"Nobody puts Baby in a corner."

As I was unhappy with the introductory Nuke courses provided by FXPHD, I decided to give their competitor Digital Tutors a try. They take a different tack, showing more of the fundamentals behind everything than any practical applications. So while I know a lot of what goes behind what I already know, I didn't learn any useful skills a Nuke professional will need to know. I should probably just blame this on it being an introductory course more than anything.

One section they do go over is the Corner Pin technique. If you've ever seen a computer screen on TV or in the Movies there's about a 99% chance that they were looking at a blank screen. Its much easier and more flexible to add the screen in post rather than to have it designed, created, and played back in sync for every take live on set where time is money.

The four tracks for each of the 4 corners

After reading in your footage you start by tracking each of the four corners you want to track. In the footage provided by Digital Tutors they show a wall mounted TV. You want to pick four high contrast areas in the corners to get the best track. You also want to place the trackers in a clockwise order, otherwise you'll have a issue lining up the corner pins. 

Image inserted inserted after Corner Pin is linked to trackers

You'll then need to read in your insert footage and attach the corner pin node remembering to link the trackers to the four corner pins by control clicking and dragging in the same clockwise order. Then Merge the insert over the original footage.

Corner Pin inserted footage after grade

Then comes the finessing. I changed the blend mode to "Screen" to preserve the reflections over the inserted footage. To help the reflections, a grade was put on the display before the inserted footage to make the black screen closer to true black and not a washed out grey. I then added a edge blur to blend the sharp edges into the black edges of the display. I added a blur node to the insert footage mainly because the inserted footage was 1080p and the Digital Tutors stock footage it is going into was 480p. Finally a grade was put on the inserted footage so to match the brightness and contrast of what a screen would look playing on a TV outside.

The final Corner Pin script as seen in Nuke

I'm not happy with the final results. Partly because of the poor footage resolution provided by the tutorial, but mainly because the workflow they suggest does not leave room for much flexibility. You do not have much control of the reflection or the size and placement of the inserted footage. If you place a transform node after the corner pin the center point is scales from is not relative to the center of the corner pin. You could approximate the corner pin center but it was just easier for me to adjust the Corner Pin node's "from" settings to offset the footage. Also the final grade of matching the inserted footage to the background footage is fixed and can't change to if the background gets brighter, darker, etc. Any change to one would require hand animating the grading of the other. I feel there must be a better workflow to give more flexibility and results with greater accuracy. Back to the tutorial drawing board. 

"I got no strings, to hold me down, to make me fret, or make me frown..."

This is my first "finished" shot in Nuke. It is a wire and mat removal project as part of the Nuke 102 course on FXPHD.  As far as VFX shots go, this is as basic as you get. No matter if its replacing a green screen with a digital set, removing a billboard that you couldn't get clearance for, or in this case painting out wires; Painting out objects is the most common VFX shot there is these days. Likewise the same process's used in all of those VFX shots are the same used in this VFX shot.

The first step is identifying exactly what needs to be done, in this case you can see its removing the guide wire and the fall mat between the actress and the door. There are some reflections of the wire in the window and it does appear the actress' zipper is undone; but I'll save those bits for when I'm more experienced in the finer details.

After identifying the parts we need to remove, we can then move on to replicating what would be there by creating a "Clean Plate". As suggested in the tutorial the easiest way is to look for a frame without the wire, or with the least amount of it as possible. I found towards the end of the footage at frame 180 the most amount of clean real estate to draw from. However the shot is rather high at this point and the space below the curb has very little real estate. I found a frame at 190 that went quite low, relatively clean. Between these two frames we should be able to cover enough of the section we need to replace for the entire video. 

The individual frames of video were then taken into Photoshop where I painted out the wire and mat. This was mainly done using the clone stamp tool. There is a clone tool within Nuke, which the instructor suggested against using, after using it later I can understand why, there is not much control and the results are underwhelming. Because at one point the camera tilts below the frame I extended it slightly. 

Then I had to track the footage. Since we're going to stick the clean footage over the bad, we need to have the clean plate follow wherever the bad goes. If the camera was locked off this wouldn't be much of a problem since the area's above and below the curb wouldn't move. This handheld shot pans and tilts but never zooms, so we can use the 2D tracker instead of Nuke's 3D tracker. This involves finding a clear, high contrasting, to point the tracker box at. If the point is good enough you can hit track and you're done. In this case the point either became obscured or blurred out due to the quick pan and tilt of the camera. To get around this you must find the frame where the track errors and create an "Off-set track" or another point for it to track for a few frames before the original point starts to track correctly. This wasn't a difficult track, but because of the footage provided was already low contrast and such low resolution it required a lot of off-set tracking. Luckily Nuke's tracker is very skilled and we got a good track. 

You then want to bring in your clean footage, roto the sections you want to use, and premultiply it thereby cutting out the parts you don't want to use. You then apply your tracking info to this and merge it over your original footage. I then added motion blur to the plates using that tracking data so the plate and background blur equally. 

The next bit is where the real grunt work happens in VFX: Rotoscoping. You have to trace everything you want to show up infront of the clean plates to essentially cut it out of the overlayed plate. For this shot it was the door, steering wheel, left leg, right leg, and her right hand. The steering wheel was quite easy as it doesn't change shape at all. I just made a shape, applied the same tracking data as the clean plate and done. Everything else wiggles, tilts, folds, and worse yet: blurs. Also, because the movement is so fast and subtle, you couldn't just keyframe the big movements and let the computer handle all the frames in between. All 200 frames had to be done individually for every shape. Another drawback to this shot was its low resolution and contrast. Many times you are looking at jumbled pixels or white on white trying to see where the correct line should be. Ideally everything would be a razor sharp clearly discernible line. Then because of the rapid movements the line begins to blur. I don't have much experience with the Nuke Roto tools so using their "Feathering" to account for the motion blured pixels was hit or miss. 

Obviously Rotosocoping is more art that science. Having come from After Effects with its "Rotobrush" this experience was far more laborious. I am unsure why Nuke doesn't have such a tool in their arsenal, maybe because the results are still better doing it by hand? Still hope they add it in the future. I know I have some tutorials ahead about Rotoscoping. I'm praying the have some tips to speed things up. 

One issue is that there is a section where two cars travel in front of our "Clean" background but still behind the wire. You could use the same technique of grabing the frame, painting out the wire in Photoshop and merging it back in to just that one frame of video. Yet because the car moves you would have to do this for every frame the car is visible. The gentleman doing the tutorial suggests using Nuke's Time Offset node. Essentially grabbing a frame from the car's past and painting it over its present. The only issue is the results are quite bad. The light hitting the car is darker than before so it looks like you're painting a completely different shade. Instead I opted for something else he suggested, using Nuke's own clone feature. This requires painting the 20 individual frames the two cars pass using a similar method to Photo's clone stamp tool. The results are not nearly as good as Photoshop, which can't be a surprise. Also Nuke started to bog down under the load of many brush strokes, as it records each one (909 in total).

Considering it was my first real Nuke composite I'm quite pleased with it. The roto could still use some tweaking, especially around that damn labyrinthine hand. Also the paint work over the cars is a bit dodgy. A good effects supervisor would likely give me a list of notes, which I wouldn't mind. It'll only make me better. For now though I'll consider it a fine first step. With that I leave you with the final results of this hard labor: Simulated Violence. 

Note: I'm sure what the effects people intended was to have the car roto'd out and insert the dummy after the real girl disappeared out of frame but I only took the shot this far. Also, I color matched it the best I could but they didn't give me a lot to work with. 

"You want to lead one day? Then learn how to follow."

I have made my way through the first set of NUKE tutorials from FXPHD: NUK101 and NUK102. They are quite old tutorials, using NUKE 4.7, I am currently on 6.3v4. They cover the standard composting fare such as keying, tracking, and roto.

 Where it gets interesting is where the tutorials briefly touch on the more advanced tools available within Nuke such as the 3D camera and projection mapping.I would have loved to delved deep into these features but the instructor giving the tutorials is horrible.

He's constantly messing up, stop/starting the video, and apologizing. It feels like someone showed him how to do it once and now he's trying to remember. They were too clumsy to follow, however the wire removal one was good enough to figure out what he was trying to do. I'll have a post on that later.

"I say we take off and NUKE the entire site from orbit. It's the only way to be sure."

Been using NUKE for about a month now. The list of popular compositing program starts with After Effects but ends in Flame and Nuke.

Nuke Interface

Originally built to simply render Flame files, it soon became to go-to compositor after the previous number one, Shake, was discontinued by Apple. Their are numerous difference between Nuke and everyone else (Python scripting, Floating point, Scanline rendering. 3D Camera support) but the most obvious difference to the user is the node based workflow. Unlike a layer based system such as After Effects; Nuke and Flame work of of discrete Nodes that feed into one another. Creative Cow gives a good analogy of Layers being a stack of photos and Nodes having everything spread out. This way you can see, change, and combine everything in anyway you want, with no issues of a layer hierarchy. Its this flexibility that has allowed it to become the gold standard in feature film and TV composting. 

With that flexibility has also come an almost infinite complexity. Where once a simple act of dragging an effect on a clip in the timeline and tweaking has now become and act of setting up 10 nodes and tweaking all 10 of those nodes. Someone less experienced, such as myself, can spend hours tweaking every setting of every node to find you have completely lost the pot and it works no better if not worse than when you started. Sure you can a similar result to After Effects with less nodes, but at that point, you might as well use After Effects. The complexity is something I will just have to get used to and one day will hopefully relish. 

I have been quite proficient at After Effects. Coming from Photoshop its layout and functionally made sense to me. Even if I wasn't sure where the function was I was looking for, I could find it because of my relations to the "Adobe UI Language". With Nuke I know where I want to go, but have to relearn how to walk to get there. Through a combination of tutorials and just messing around I'm rehabilitating my compositing legs.

What a wire removal Nuke scrip looks like

I don't meant for this to be a comprehensive review, just an initial impression. I'll update with more as I finalize test shots. 

"Renew, renew, renew..."

There I was, turning 30, as you do. Going through that early onset mid-life crisis associated with realizing your not a kid anymore, that your scale of thinking ahead is not the next month, but the next 10, 20, 30 years. It can be a bit overwhelming, touching the obelisk like that. And with that vantage point comes a harsh reality: I'm not where I want to be. Not even close. 

So where do want to be? Siting in a dark office hunched over a computer clicking on pixels, making an image from nothing, and seeing it up on the big screen. Its the job that has fascinated me most since my fontanel grew in. So I spent 30 years delaying the inescapable truth of what I want to do with my life, now's the relatively easy part: realizing it.

This blog will document my every step of going from inexperienced, to knowledgable, to employed. Along the way you're sure to see all my mistakes, my frustration, and inevitable failures. Yet from those learning opportunities will come success, joy, and pretty pictures. 

Its not if, its when.