Category: Video

Now in Living Color – The Process of Colorization

January 17, 2019

One of my colorized shots used in They Shall Not Grow Old

I haven’t posted a lot of work lately, but not because I’ve been lazy. I spent the better part of 2018 as a Color Compositor and Matte Painter on Peter Jacksons’s colorized documentary They Shall Not Grow Old, that examines the life of soldiers during World War I. My experience in the past year really helped me analyze my own work, and recognize issues that I might not have discovered myself until several revisions later. Each day at the studio, I submitted drafts of my shots and we analyzed them in the theater to work them down to the smallest detail. I think one of the biggest improvements I gained through the WWI project is attention to highlights and shadows. Previously, I had been adding color to the highlights as a way to bring color into the overall image, but for the WWI footage, they wanted clean, white highlights and clean, black shadows; no tinting, no compensating for the atmosphere of the shot. Since I now incorporate this into my own projects, I think it forces me to work more color into the areas where there would naturally be more or less saturation, instead of relying on an overall base tint to mimic the look of film stocks.

The most popular question I receive is “How do you do this? Can I do this myself?” When I first started colorizing footage, I could hardly find any information on how to do it. There still isn’t much information out there, and there certainly aren’t any walkthroughs or 101 tutorials. In a nutshell, it requires manual application of every individual color in the shot. There isn’t a special program that does it for me; it’s a totally manual process. (There are some impressive algorithms and automated colorization programs right now, but they are not at a point yet where they can colorize footage in fine detail.) Since there isn’t one way, or a right way to do it, it’s up to the artist to choose the best and most efficient method. The largest task at hand isn’t getting color into the image, but getting all of those colors to move naturally within the footage. The most straightforward method is to paint and animate every frame one at a time, and I used this method on the very first piece of footage I colorized (see The Munsters – Season 2 opening.) Painting frame by frame was exhausting and frustrating, but out of the grief, it gave me a better understanding of how these areas of color need to move within the image and it pushed me to find a better, more efficient way to do this.

Since then and currently, I rotoscope the footage using motion tracking. Rotoscoping involves outlining an area of footage, tracking its motion through the shot, and making sure it stays in the appropriate area. Rotoscoping is usually done on a smaller scale to remove wires from a shot, change the color of an object, or to place new objects/characters in a shot. Rarely are they rotoscoped entirely unless it’s for something like colorization or 3D conversion. Motion tracking still requires frame-by-frame analysis, but it only requires manual intervention every 3 or 4 frames instead of every single frame. It might not sound like a lot, but that cuts down the task by about 75{4bdb5961094c0eb4a6c257d9af4d1ff762679304c55b8a1a4d68ed7046a336af} and depending on the shot, some objects can be tracked with almost no intervention at all. This has greatly increased the efficiency of getting color into the shot, and with all the time saved, it gives an opportunity to work more detail into the image. Here is a quick step by step of the process I use:

The Process

      1. Select and edit the footage – Since I do everything myself, the effort it will take to rotoscope a shot is important to consider. I usually try to find a 30-60 second scene that has a good joke or punchline, and also has the type of movement I can feasibly rotoscope on my own. Backgrounds are important to consider, but they are a little easier since there isn’t usually a lot of detailed movement. I also get a chance to break these scenes down, add some music cues, a title card, adjust the laugh track, etc. which is a fun bonus.

        Editing My Fair Munster in Adobe Premiere

      2. Rotoscope – This is by far the most time-consuming part of the process. I start with a main character in the scene, usually with the face, and work out from there. There is no secret, just a lot of motion tracking, watching frame by frame to make sure the mask stays in line, and making adjustments as it changes. I’ve spent anywhere from 2 days to several weeks on some shots. Many times a shot will look simple enough like I could finish it in an hour or two, but then I’ll sit down and realize it’s going to take days for one reason or another.

        Grandpa Munster’s roto masks

      3. Assemble masks – This takes a long time and is also not the most fun part of the process, but importing the masks and arranging them in front of/behind each other is a crucial part of the process.
      4. Add color (finally!) – This is actually the fastest part of the process, and definitely the most fun. I can do color on a shot in one day, but it is always helpful to let it sit and come back later to pinpoint issues that stick out or require adjustments. Color is added in various ways depending on the brightness/contrast of the shot and how it absorbs the color. I use everything from solid colors, to levels to curves when adding the color. This is where an eye for color is essential. I sometimes use reference photos but if it’s for The Munsters, I Love Lucy or something else I know very well, I go without a reference and just make sure it looks good (Unless it’s something like a floral dress that can be matched to a real color reference.)

        Assembling the color masks in After Effects

      5. Detail it out – After the shot has color, I watch it over and over to look for masks that aren’t layered properly, or ones that need edge adjustments because they are bleeding over into other areas. At this point, the project file is getting pretty heavy and it takes longer to process changes so it’s important to do the rotoscoping correctly at the beginning.
      6. Render & Archive – When everything is finished, I render out a high quality, high resolution master file that I use to upload online. I save all the project files onto backup hard drives so I can always go back and make changes later.

The Results

Marineland Carnival behind the scenes with Fred Gwynne, Al Lewis and Yvonne De Carlo

The Munsters scare up some fun in this newly discovered holiday special

October 30, 2016
The Munsters at Marineland - Marineland Carnival, 1965

The Munsters roll up to Marineland in style

Reboots, updates and remakes be gone, the original Munster family is back! Though they were peacefully laid to rest after cancellation in 1966, the spooky family left behind one last fright for fans of the beloved series. Filmed in March of 1965 at the famous Marineland of the Pacific, The Munsters hosted a one-hour special that has finally been pulled from its deep, dark grave (which hasn’t been a bad resting place, as far as they’re concerned.)

It aired only once on Easter Sunday, April 18, 1965 and hasn’t seen the light of day in over 50 years. Some insisted that it was lost forever, but in true monster fashion, The Munsters’ famous appearance on CBS’s 1965 Marineland Carnival (subtitled The Munsters Visit Marined) has reemerged to join the world of the living. The special was written and produced independently from Universal’s writers and production team, and due to contractual agreements with Universal, CBS actually booked “The Munsters,” so the actors’ names do not even appear in the credits!

The premise of the special is that the Munster family has traveled to Marineland in hopes of finding a new pet for Eddie. “Something cuddly he can take to bed, like a shark or an eel,” says Lily. “But not the wind up kind; an electric eel,” Herman jokes. While shooting the first scene showing the family driving into Marineland, the famous Munster Koach was caught in rush-hour traffic, causing a complete standstill as motorists gawked at the famous car. In fact, the Munster Koach also had to be booked separately because like the cast, it too makes personal appearances, even to this day.

Yvonne De Carlo as Lily Munster behind the scenes at Marineland Carnival, 1965

Yvonne De Carlo looks ghoulish in the bright sun

Once inside, The Munsters join their tour guide, Sid Gould (remember him from The Lucy Show?) whose tour is frequently interrupted by Grandpa’s flying antics and sway pole routine. As the family makes their way around the park, we are treated to the California sea lions and a dolphin show, as well as comedy divers, a group of blues-singing walruses and even Surfin’ Annie, a surfboard-riding terrier.

Producer Charles Andrews describes the special as “a real, live, three-ring sea circus,” and if that’s not enough variety for you, musical acts by the New Christy Minstrels are also interspersed throughout the special. The popular group performs several numbers including a rendition of “This Land is Your Land” and one of their biggest hits, “Saturday Night” where they appear underwater in an aquarium. This was surely a fun special for kids in the 60s!

 

Al Lewis as Grandpa Munster behind the scenes at Marineland Carnival 1965

Al Lewis poses with a young fan during a break in filming

Interestingly, Marineland was gracious in hosting the production, but did not alter their shows or schedule by even a minute. Off camera, Fred Gwynne and Al Lewis spoke candidly about their love for animals, and the growing trend of families purchasing dolphins as pets. “I don’t know whether I like the idea of having a dolphin stable,” Fred says. “If your pet turns out to be smarter than you are, there could be trouble. But what really gets me down is the tossing of rubber balls to whales and dolphins. They did an autopsy on a whale recently and found a couple of balls in his stomach.” Being the gentle giant that he was, Gwynne kept careful count of the balls thrown during the two-day production on the Palos Verdes Peninsula in California.

A little later, while shooting a scene where Grandpa throws a fishing line, Al Lewis unknowingly gave the hand command for the four dolphins to do a high flying formation and he almost dropped his fish into the 640,000 gallon tank. “It was disconcerting, to put it mildly,” Lewis recalls. “As an animal lover, I felt I’d have a rapport with the dolphins and the whales. But obviously the dolphins didn’t penetrate my makeup. So I ask you, how much can they see?” he joked. “That’s something for those oceanographic scientists to ponder!”

Perhaps the high point of the special is when Herman Munster climbs out onto the Marineland deck to feed a fish to the famous pilot whale named Bubbles, right out of his hand. As the hour comes to a close and the family decides it’s time to go home, Eddie chooses a seal named Roo as his new pet. The Munster family says their thank yous and goodbyes to the Marineland staff, and as the Munster Koach lets out a loud roar, they drive away with the newest member of their family.

Marineland Carnival 1965 Regional Listing

The cast appears in a regional listing

So, why has this special been missing for so long and where has it been? It is known that the Marineland special was shot on video tape and for a lost program, that is not good because video tape both degrades faster than film and it became common practice for studios to erase tapes that they assumed would never be aired again, in favor of saving money and using them again. Fortunately, a kinescope protection print was made. A kinescope is a camera that films a video screen in order to preserve live programs or to air them again at a later date. Networks often used 16mm kinescopes for East/West coast transmissions due to different time zones, and luckily, at least one of these Marineland prints survived. The kinescope film remained in a private collection until the late 90s when it was donated to the Paley Center for Media in 1997, where it sat again for almost 20 years until being recently re-discovered in 2014. Since its airing in 1965, diehard fans have been on the hunt to find the legendary Marineland Carnival and the search has finally come to an end, as the lost special is now available for public viewing  in Los Angeles and New York at the Paley Center for Media.

 

Bringing The Munsters Back from the Grave

November 23, 2015
The Munsters sit down to dinner in this rare color shot from a View-Master reel.

The Munsters sit down to dinner in this rare color shot from a View-Master reel.

Most people who know me know that from the age of nine, I have been a Munsters fanatic. When I was younger, the show’s cartoonish approach to monsters and physical comedy drew me in, but as I grew up, it was the sly writing and comedic timing between Fred Gwynne and Al Lewis that made me appreciate the show even more. Unfortunately, audiences were only treated to the show in black and white, which veiled a shroud of mystery over everything from Grandpa’s lab to the shade of Herman’s green skin.

It has always been a dream of mine to see The Munsters in color and there are only a handful of instances where we see the family outside of their dreary, monochromatic world. The 1964 unaired pilot, as well as the 1966 feature film Munster, Go Home! were both shot in Technicolor. However, the pilot features a different set, different cast and is a far cry from the show it would eventually become. By 1966, audiences were dying to see The Munsters in their full, Technicolor glory and Munster, Go Home! finally delivered. Unfortunately, producers decided to change the plot and move the Munsters to England, leaving fans with only a small glimpse inside the creepy Munster mansion during the film’s first five minutes.

After stumbling upon Stuart Manning’s wonderful colorization of The Addams Family opening, I was so impressed that I decided to give it a try myself. I am familiar with colorizing still images, but moving images are a completely different animal and initially, the project seemed daunting, if not impossible. I started researching to find the easiest colorization method. I had attempted colorization years ago, creating a 2-second animated GIF of Herman walking down the stairs from the first season opening and it was bad. It was really, really bad and THAT took me hours. I initially attempted rotoscoping in Adobe After Effects. Motion tracking on individual objects was incredibly time consuming and required frame by frame tracing. In the end, my computer just couldn’t handle it and I knew there had to be a better solution. After diving into the world of YouTube and Google searching for any tutorials on colorization, I came up dry and it looked like I was going to have to create my own process.

A still from the unaired color pilot

A disheartened Marilyn retreats to her bedroom in this still from the unaired color pilot

Before I go any further, I want to address the purists. The ones who consider colorization to be blasphemy of the film and television world. I know. Colorization in the past was not good. In fact, it was pretty terrible and even today, colorizations are typically flat with little shading and color variation. I’m with you guys; they are bad. But I’m not here to do bad things! Especially not to a show so near and dear to my heart. There is also a fairly large crowd that argues black and white programs should not be colorized because it goes against the director’s vision. I completely agree with this argument for certain films, but in this case, I don’t believe the director chose black and white for his vision. In fact, I know he didn’t. In a 2002 interview, Munsters makeup creator Karl Silvera explained, “It was $10,000 more to do [the show] in color and neither CBS or Universal were willing to pick up that extra $10,000 per segment. That’s why it was shot in black and white.” Coupled with the fact the pilot was shot in color, as well as the follow-up movie, it is clear that the reason television audiences were never treated to seeing the Munster family in “dying” color was strictly budgetary.

I spent a day or two thinking of how I was going to approach colorizing hundreds upon hundreds of individual images and I finally streamlined a process. After importing all 1,317 frames of the 44-second opening, I broke them down into scenes. Each scene consists of an establishing shot of the character, followed by a quick zoom into a closeup. The establishing shots and close ups would be easy enough, but the zooms seemed like a challenge when coloring frame by frame. Using the same color palette for both parts of the shot did not work well, so I had to color each section (establishing and close-up) separately and find a way to transition the colors during the zoom.

Speaking of colors, let’s talk about that: my personal belief is that a colorized image should not look colorized. I have a personal vendetta against poor colorizations that take too many artistic liberties. Was that woman really wearing a bright purple skirt with a neon yellow shirt? Are you sure that guy’s hat was red? They are silly color choices that look neither realistic or aesthetically pleasing and just because you can color an object any color you like, it doesn’t mean you should. Whenever I colorize something, my thought process is not “how can I make this look cool?” rather “how can I make this look like it was shot in color?” Many times, people throw color in where it doesn’t belong. The sad truth is that sometimes objects are boring and in fact, most objects individually are pretty boring. Blonde hair is not yellow, it’s a shade of gray and brown. Teeth are not white, they are gray and yellow. Black? You can’t just not colorize it. It has a lot of blue, purple and green. Color takes a lot of work.

The dress in question.

The dress in question.

I did, however, take two artistic liberties in this project. As I learned from Stuart Manning’s blog post, early color television programs often chose yellow in place of white to add an extra pop of color, while still registering as a white object on black and white sets. The dress that we see Marilyn wearing in the Munsters opening was white, and this can be seen in a few color promo photos featuring the first actress to play Marilyn, Beverley Owen. Yellow seemed like the natural choice for several reasons. It adds necessary color to the shot and I have no doubt that the wardrobe department would have chosen something similar, had it been shot in color. Yellow also fits with the grayscale values of the dress, so it looks very natural. Had I tried to color the dress red, green or any other color, it would not “stick” to the gray values as easily and would not look very good. My final reasoning in choosing yellow was because recent Munsters memorabilia items feature the character of Marilyn wearing a yellow dress that looks similar to the one seen in the opening. Although it has nothing to do with the show’s original creative team, I feel that it supports my choice. The second artistic decision I made was the color of The Munsters’ drippy, melted wax logo and title cards. In the pilot, the text was a minty blue, almost the color of Herman’s skin. I felt that the color just did not look right when placed over the colorful house and characters. I decided to use a bright green color with a shade of yellow. This color was used in the main title card for Munster, Go Home! and I felt that it gives a nice contrast of color while keeping the palette organic to the show.

Choosing the colors for the other characters was fairly simple because I know these characters better than I know some people in real life. The only color reference I used was for Lily’s purple coat. I knew it was purple and gray, but to get the tones just right, I needed to reference Munster, Go Home! The rest of the colors like Herman’s skin, Eddie’s suit and Grandpa’s red cape were done without reference. I know I probably should have, but I did not feel it was necessary. This is also one of my favorite parts about colorization. The image slowly starts to develop and you get a feel for the tone of the image, then in a snowball effect, all of the other colors fall into place and it’s like magic. I colorized the beginning and ending frames of each character and worked through the frames in reverse to eventually match the colors.

Once I finalized my technique and fell into a groove, the process was fairly painless. I fully colorized several images, almost as key frames, throughout the entire segment, then adjusted for the small movements in between. I knew I would be working frame by frame, but I was not prepared for the sheer amount of time required for this adventure. In the end, I colorized every single frame by hand and even re-colorized the portions of Eddie and Marilyn when I decided on better color choices. In all, I colorized nearly 2,000 images over the course of 80 hours in one very, very full week. Final adjustments and clean-up took an additional two days and finally hitting the “export” button was shared with a stiff drink and a deep sense of pride. I sent myself through colorization boot camp, honed my Photoshop skills and fulfilled my childhood dream. I am very happy with the end result and, call me a masochist, I am looking forward to doing it again.