Skip to content

White Paper

gadfly16 edited this page May 11, 2019 · 4 revisions

The Teflon Reference Configuration

A fairly general post-production pipeline and workflow for cinematic productions.

Abstract

In this document we present a folder structure and Teflon configuration for a generic post-production pipeline.

Teflon is a framework that allows its users to express project structure and logic freely, so NOTHING in this document is mandatory. This is just a reference that can help you to start using Teflon quickly. Later you can use it as the basis of your own designs. This configuration also helps us to implement Teflon, since the system has to be able to execute this particular one.

While Teflon is pretty flexible it still has some logic hard wired into it. This document may mention these Teflon specific properties, but it wont discuss them in detail. Please refer to the Design Proposal for more details of Teflon specific mechanics.

This document is under heavy development, with frequent incompatible updates. It's here only to promote discussion. If you have questions come and chat with us on Gitter. If you have remarks, suggestions or ideas, please feel free to create an issue.

The Workflow

In this section we describe the workflow that we want to implement with this configuration. This workflow is somewhat general, but it's main targets are live action movies with a considerable number of VFX shots in it. While it may be prove useful for other purposes, like computer generated animation films, but that field could clearly benefit from a dedicated configuration.

The basic workflow consists of the following blocks of processes:

  1. Preparation

  2. Assigning cinematic metadata to dailies based on the clapperboard.

  3. Renaming camera files to reflect the clapperboard data.

  4. Production

  5. Convert dailies to the review and edit formats.

  6. Transfer edit and review files to concerned parties.

  7. Edit the film.

  8. Transfer XML file containing the edit decisions.

  9. Create folder structures and clip conversions for VFX shots based on the edit decisions.

  10. Do VFX.

  11. Transfer and convert VFX shots for insertion to the timeline.

  12. Grade requested shots.

  13. Do sound works on requested shots.

  14. Delivery

  15. Mastering.

  16. Converting for delivery formats.

The main advantage of Teflon is that the production processes can happen concurrently. For example grading and even VFX work can be started pretty early after editing has been started. This may sound strange but this is the point of our approach. During the editing process the director may request some preliminary grading for some takes marking them on the timeline. When the system parses the XML it discovers these flags it can warn the colorist that some preliminary LUTs have been requested. When the colorist is ready with the LUTs the system applies them, so in the editing room the editor and director immediately sees the changes right after they are ready. Without doing anything.

This is possible because we disrupt the economy of the post production process: by conventions and automation we make operations traditionally seen as expensive feel cheap. The best example that illustrates this idea is the situation when changes on the editing timeline causes headaches on the VFX end. While an event like this will always have consequences, we can make it cheaper by automating things, and by setting up the process in a clever and coherent way. If we do a good job, what the VFX artist will perceive from all of this is that she need to process some additional frames on the clip she is working on.

All the technical conventions in this configuration serves this purpose. Conventions of this configuration can be grouped like the following:

  • Naming conventions
  • Time and frame numbering conventions
  • Process conventions

We discuss these conventions in following sections in detail.

The Project Structure

This section describes how a show is structured. For detailed descriptions about how objects behave plese refer to the description of their type.

The Show Root

The show root has the type show which is the boundary of self containment. A valid show always contains itself, that if we move or copy the show root we can be confident that the copy will behave exactly as the original, because it has no outside dependencies.

The show also expresses the highest level of administration. For example this is the highest level where the system look for prototypes during new object creation.

The output of tree -L 1 . in the show root of a newly created directory will look like the following:

.
├── asset
├── edit
├── grade
├── in
├── out
├── sound
└── vfx

In and Out Directories

This configuration is using synchronized directories for communication between departments and/or facilities. Directories named in and out are usually having some roles in this synchronization process.

In the case of the show root the in directory is for incoming files. If someone is getting a file, she may not know immediately where it exactly belongs, she can put it here. It's a shared responsibility of the show's contributors to not reference files from this directory, but move the files where they belong instead and reference them there.

On the other hand the out in the show root is for the outgoing files of the show. For example previews are collected here, and the final delivery versions of a finished film ends up here.

Directories with the same in and out names can also be found in the department directories. These directories are usually configured to do one-way synchronization between sites and devices.

The Asset Directory

The asset directory holds together all the digital assets of a show, like the dailies, the script, application specific tools, etc.

.
├── app
├── dailies
├── journal
├── script
└── still

Department Directories

The edit, grade, sound and vfx directories are for different departments of the show, that require their own file-system structure.

Naming Conventions

This configuration uses a naming convention for Teflon objects and files that is both human and machine readable. This means that file and directory names must be easy on the eye and the brain, and they must be concise at the same time. We can make life much more easier if our programs could also deduce some information from file and directory names. For example when we create a new object the program can use the target file name to deduce the type of the object, and we can use teflon new Sc_14 instead of teflon new -t scene Sc_14.

Naming conventions are recommendations, they are not mandatory in the sense that the system is designed to handle situations breaking these conventions and doesn't use these properties for automation logic. In other words it's not the name of the object that triggers the logic usually but metadata assigned to the object.

Spaces in Names

Spaces in file names are allowed, so it's the user's decision to use them or not. Spaces look better in graphical user interfaces, but can cause minor escaping headaches on the command line. Teflon's approach toward spaces is that they should not cause any problems and by using ZSH's auto-completion the user can mitigate the escaping problems.

Directory Names

Directory names that express function start with lowercase, and directory names that express identity start with uppercase letters. For example in the case of the name tif the name expresses the functionality of the folder, which is to hold versions of something in tif format. On the other hand in the case of Sc_14 the name expresses that this is scene number 14 which is the identity of a cinematic object.

File Names

The first letter of files doesn't have an implied meaning like directory names, the capitalization of the first letter of files is meaningless. In other words file names always express identity.

Sequence Numbers

Sequence numbers are preceded with . and are not padded with zeroes. For example example.1.tif complies to the convention while example_0001.tif does not for several reasons. First the frame number is not preceded with a . and the string representation of the number is polluted with meaningless zeros.

We decided to stick to the simple idea of eliminating frame padding for several reasons. Frame padding is a solution to a display problem that is solvable by other and better means. The industry is using frame padding only to support some ancient sorting algorithms. By using more sophisticated sort algorithms we can eliminate the necessity of the meaningless zeros and the benefits are far greater than the losses.

All that said this is still only a convention that makes everybody's life easier. Teflon can handle sequences with frame padding, we just don't use them in this configuration.

Extensions

File extensions are lowercase and conventionally three characters long. Please use the canonical extensions, since the configuration is using them as a hint of the file type.

Versions

Versions signatures are always at the end of the base filename before the sequence number and/or the file extension. Version numbers are integer numbers preceded with a lowercase v. Version signatures can have a note appended after the version number. The following is a list of valid versions of objects:

Sc_4_Sh_3_Vfx_1_v1_scetch.mp4
Sc_4_Sh_3_Vfx_1_v2_still_not_there.mp4
Sc_4_Sh_3_Vfx_1_v3_almost.1234.exr
Sc_4_Sh_3_Vfx_1_v4.mp4

For further discussion of versioning mechanics please read the Versioning section.

Time and Numbering Conventions

Managing timecodes properly throughout the project is unavoidable, and while Teflon absolutely do this, it doesn't solve all problems of all departments. When we want to integrate VFX into our pipeline with the least friction possible we need to take their specific needs into consideration. It's common knowledge that working with frame numbers up to the millions is really uncomfortable. If we want to avoid this situation we have to introduce a comfortable and consistent renumbering scheme. At first it may seem a bit complicated but in reality it is not that bad.

The basic idea is that the VFX clips are positioned relative to their base take.

The Type System

Cinematic Types

  • film: a sequence of scenes, abbreviated as Film in file names
  • series: a sequence of films, abbreviated as Season in file names
  • scene: a sequence of shots, abbreviated as Sc in file names
  • shot: a camera setting covering a part of a scene, abbreviated as Sh in file names
  • take: a recorded attempt of a shot, abbreviated as Tk in file names
  • clip: a continuous segment of a take, abbreviated as Cl in file names
  • vfx clip: a continuous segment of a take including VFX elements or edits, abbreviated as Vfx in file names
  • base: a take that a VFX shot is based on primarily
  • plate: a visual component of a composition that makes up a shot

Cinematic structures are usually represented by directory structures in the file-system. For example a scene is represented by a directory named Sc_<id>, while shots belonging to the scene are directories in it with a name like Sc_<id>. This is not mandatory, it's only a convention, but it makes life much easier.

Transforming Cinematic Structures to File Names

The configuration uses a naming convention that allows the user to express cinematic identities both as file-system structures and as names of single files. For example a shot in the show hierarchy may look like $SHOW/Sc_4/Sh_2, while in a file name belonging to the shot it will read Sc_4_Sh_2_.... Basically we simply replace the directory separators to underscores.

Asset Types

  • app: application specific assets
  • dailies: camera footage from the principal shooting
  • script: scripts and journals of the show and the shooting
  • still: still images

Shot Component Types

  • comp: composition files, like Nuke or Fusion scripts
  • cgi: 3d scene files, like Houdini or Maya scenes
  • plate: plates of the cinematic entity referenced by the compositions
  • camera: camera data

Versioning

Teflon currently does not provide a native versioning facility, instead it uses a set of conventions to let the user save states of objects.