‘The Art and Science of Digital Compositing’ – Chapter 1:Introduction to Digital Compositing Notes
‘CGI’ (computer-generated imagery) is a vague term used for visual effects in films. Although technically, it can be used in any visual work created on a computer.
Two types of CGI, 3D and 2D. 3D is in the form of models, 3D animation and lighiting, and 2D consists of imagery such as graphics and photographs.
“Digital Compositing: The digitally manipulated combination of at least
two source images to produce an integrated result.”
Good digital compositing will not reveal each image’s source.
Digital compositing is not exclusive to film.
Good digital compositing requires good technical understanding of tools, and good artistic judgement.
Oscar G. Rejlander combined imagery from 32 negatives to produce ‘The Two Ways of Life’. The work was poorly received as compositing in photography was seen as dishonest. However, this technique evolved into optical compositing that was used in King Kong, 1933, and eventually evolved into digital compositing.
A ‘matte’ is to exclude an object from the rest of an image. Typically, an object against a screen.
Elements within a composite are reffered to as ‘layers’. Original footage shot with a camera is a ‘plate’.
History of Visual Effects
Special effects tend to be illusionary. This can be mechanical effects that exist in live-action. This can be props and scale models.
I find this live-action trickery really interesting, as they create such a simple scene in such an intricate method. The lord of the Rings used a very complicated set to create this illusion.
Illusions like this can be recreated with digital methods. I know in video games, more specifically old 3D games, distant mountains would only be slightly larger than the player.
Special effects can look more realistic than digital.
The earliest forms of special effects would involve splicing film together. In 1895, Alfred Clark filmed a reenactment of Mary Queen of Scots beheading. The executioner brought down the axe, the actors froze and Mary QoS was replaced by a headless dummy.
George Melies used a similar technique called the ‘stop trick’.
Melies also used the ‘matte shot’ in his film, “A Trip to the Moon”. Which uses painted background and cut out canvases in the foreground to create a fantasy environment.
Stop-frame animation allowed for moving characters and objects in films. A famous example of this technique is in King Kong.
2001: A Space Odyssey used realistic miniature models of spaceships flmed with a natural depth of field.
Digital visual effects became more common throughout the 90s. Virtual environments allowed for more freedom within a scene. Fantasy films often used backdrops, now whole 3D enironments could be created. These could also be used in realistic environments with camera shots that would be near impossible to film.
Digital VFX allowed for simulations in film. This could be things like crowd simulations. Massive Crowd Software- created for battle scenes in The Lord of The Rings allowed for tens of thousands of digital agents to play certain characters and perform certain behaviors.
2.5D (Parallax Effect)
As I don’t have a good video camera, but a DSLR, I thought I could perhaps produce an animation from the photographs. Gillian showed me a method of producing an animation from still images called “2.5D”. This involves creating masks from different images and setting them at different depths (between foreground and background) then panning a virtual camera to give the illusion of depth.
This can be done with one single image, but it may be far more interesting to collage many different images to get a more abstract effect.
This video describes the process well. However I really dislike the images he produces. I think it may be the strange warpy animation, or the uncanny valley feeling or just the content of the images.
I think the effect fails when it is used in attempt to create a realistic scene. It has such a cheap feel to it but I think it would be far more interesting in a collage of images from different origins.
We looked into one point and two point motion tracking. We selected an area of a video, After Effects tracks the point, then animates the path. The user can then parent another component to the track and it will animate to match a movement in the original video. It is important to choose an area of high contrast for tracking as the software will find the area easier to track.
This will only track one point’s position in the x and y axis. However, to track scale and rotation, you must select two points.
In this tutorial, we looked into 4 pin tracking. We attatched a graphic to a mobile phone screen. We used the perspective setting in the motion tracker, then applied an animation to the mask.
We then used the 3D camera to generate planes. I ran into an issue where After Effects would close with no error once the planes were generated. I discovered that this was due to the fact, After Effects was saving the cache on a drive that didn’t have enough space for it.
We learned how to create precompositions, so we could apply to 3D planes within another video. We also created a mask to prevent objects in the scene from appearing under the graphics. This involved drawing out a mask, setting it to track, then adjusting it every couple of keyframes.
We brought Cinema 4D objects into After Effects. I ran into issues where AE would get stuck finishing tracking the 3D camera. I may have to go and adjust the memory settings.
We would then create a ground plane, then apply a 3D object to it. This object came in without any shadow.
To fix this, we created a ground plane in cinema 4D with a directional light casting a shadow onto it.
We then isolated the shadow layer, changed the multi-track so it showed only the shadows. From here we used the blendmode to make the white pixels transparent so the shadow would appear but the plane was invisible.
My Pluralsight notes tend to take up alot of room with not so interesting content so I will write it in another blog here.
I moved into my new flat on Maryhill road in September. My room faces north towards the high rises on the opposite side of Maryhill. Although these high rises are often criticised for their ugliness, I’ve always felt an aesthetic appreciation for them. Despite the fact the highrises have been getting demolished in the past decade, Glasgow still seems to have dozens, whereas Dundee there are only a few remaining.
As the sun starts to set, I can see the in the highrise individually turn on. Then they begin to turn off again after 10pm, but never completely. There’s something about this I find comforting. Like each light is representitive as an individual or family, just getting on with life.
I had an idea a couple of months ago, where one of these high rises could be edited to appear like a fruit machine, with the windows lighting up to form the flashing patterns.
My plan was just to create an animation where I coloured in the windows to do this, but with this project I could bring in 3D objects, such as a handle on the side that gets pulled.
This video uses a combination of photography and illustration to produce environments. It also uses the parralax effect. This captures the sort of effect I’d like to achieve in this project.
This video has similarities with the idea of using the high rises in Glasgow- old architecture. It uses very simple animation. It’s not really that impressive aesthetically but it does the job.
This piece is alot closer to stopframe animation than the 2.5D scene I wish to achieve, but I like the obvious tearing of the objects in the scene. It doesn’t pretend to be anything other than a collage. I do think this piece is visually stunning.
This is another really simple animation. I could create a scene like this but just add some depth to it.
I plan to go to library to find any architecture or collage books for reference. I will need to play close attention to visuals as I feel like my work is often weak visually, without trying to sound pretentious, I tend to focus on the idea/concept behind my work more than anything else, sometimes at the expense of the visual quality.
The Age of Collage
“For many artists, the search for the ideal images and the process of cutting…is the true creative process, and it cannot be replaced by working at a computer”
Eugenia Loli- digital collage artist
Segei Sviatcho- intentionally makes imperfect cuts
Or “maximise the illusionism”.
Cubist collage- Perhaps I could explore going completely abstract?
Dada and surrealist collages seem to have more of the aesthetic I am going for. Dada collage often had political goals…
Pop art dadaism – Eduardo Paolozzi (I think he has an exhibition at the national gallery at the moment).
Describes the act of collage as violent. A mutilation. He will often leave copyright signs from the pictures he takes- “he raises questions that disturb the relationship between the consumer and the producer”.
Matthieu Bourel- “collage is like assembling a ‘fictional puzzle that doesn’t exist, but which might have a solution’.”
I find it easy to visualise this piece as a 2.5D animation. The couple in the foreground feel somewhat seperated to the background, perhaps by colour, or their more round shape. I also like how Johanna produced the background here, it appears as one solid photo, however I can count at least 4 images within it, all of natural water-like textures.
Brighton based artist, Kieren Copper A.K.A Cur3es creates collages from scanning in real images and manipulating them in photoshop. His work tends to take the form of landscapes although he makes some really horrible geometric collages.
Most of these compositions will be constructed from photographs, illustrations, paintings. Some artists source their images from online, others will scan in physical images and some will work completely physically. The more you rely on digital the more freedom you have, and perhaps the more difficult it becomes. Limitations tend to spur on creativity. Digital collage has a huge sample set, and has complete control over scale. As I am new to collage, it may be easier to work from physical cuttings to begin with to feel the limitation.
Playing with collage
Create a collage of scenes I’ve seen in Glasgow?
Boxing match under motorway
Man with traffic cone
Man dancing outside Munn’s bar
Student with electric scooter
When I was walking back from the studio last week, I wandered past two men having a boxing match under the motorway.
Parallax Effect (2.5D)
The 2.5D effect can be created in After Effects by making layers of a photoshop file, 3D layers, then giving them depth. A 3D camera is then created and can pan/ zoom/ orbit to create a sense of 3D.
Below I made a quick example…
I really like this effect, and the limited colours of the GIF make it appear even better. This technique would be recreated with more layers, and a more interesting scene.
I added movement in the y axis as well as some rotation in the y and x axis. This creates a more dynamic movement. The rotation increases the feeling of depth.
I like being able to see the edges of the image. I might start to push the collage deeper into the centre.
I also need to find a way to extend the run time of these animations. They may have to zoom in/out of the animation to make it appear as though there is more to the collage.
The left side seems too dense. Concept for attempt at 3D.
Maya Live Link
I tried to set up a live link with AE in order to bring the lever, camera and light into after effects so I can edit it within the video. I spent ages trying to install it, and then establish the link. Once I got this to work, the scene was constantly dark and objects had weird transform values so I decided to ditch it. Instead I will just render the lever to look like its on the side of a building and render the animation then import it to AE.
This method is slow, and I may need to create another render if the perspective isn’t correct.
Black background is creating issues with lever. I went back into maya and created a self illuminating green material that would be easy to key out. I didn’t want it to appear in the reflections of the lever so I fiddled around with the render settings until I got the right result.
I was then able to attatch this to my highrise.
I also didn’t like the density in the left side of the composition, so I rearanged it slightly until it felt right.
I like the space in the left side of the composition now.
I didn’t know how to conserve the composition when adding depth to it so a lot of trial and error was involved.
I removed any rotation animation from the 3D Camera as it often warped the images and led to unpredictable movements.
To make it feel more natural, I added a small amount of camera shake by adding a wiggle parameter to the camera orientation.
To finish things off, I added traffic sounds.
I would like to light up the multi-story flats in such a way they resembled a fruit machine. I would also like to add the fruit machine sound effects to this as I feel like they would cut through the ambient environment sounds.