The Path of the Righteous

Oct 31, 2014 by     4 Comments    Posted under: 3dsMax, Maxscript, pipeline

This month’s tutorial is designed for beginner scripters. Anybody who has worked in an environment where the folder structure isn’t locked down will tell you that a Wild West approach always ends badly. What happens if somebody leaves? or is sick and has to pass their working shot over to somebody else? It’s something we have to be aware of and build systems to handle. Conceiving a default set of folders for the type of work you do is a useful starting point to this process, and relevant even if there is only one of you. Most people will want to customise their folder structure, so if you start to design a base folder there’s a couple of approaches.

  • You can make an empty structure and manually copy this when you start a new project
  • You can make a tool that builds the structure automatically

Either approach is fine, but a tool is much COOLER.

 

Take The Right Path

wizard of oz

What might be important is how you construct the paths to your file structure in projects. Here’s the string concatenation way:

"\\\\server\\projectname\\sequence" + "\\shot" + "\\animation" + "\\maxfiles"

Remember that you have the option of adding a forward slash instead of using “\\” but the result for me is the same, it’s a file path that you have to be careful to format properly in order for it to parse correctly. Any portion of the code that misses the path separator could mean an unusable path. You could use stringstream and format, but it’s still a similar sort of concatenation operation.

Pathconfig Structure

pcap

The Pathconfig struct has a few useful methods. It’s likely that (like me) you looked at the first few methods and assumed it was something to do with the built-in max project structure. However, keep going, it has some very useful functions lower down. The single most useful function which I tend to use all the time is appendPath. This is great because you don’t have to use string concatenation, it automatically adds the correct separators for you:

pathconfig.appendPath "\\\\server\\projectname\\bad" "dates"

So this is fine when you’re adding one folder name onto another. You could reason that this is an unnecessary step to something simple. You might be right, but for me it is to do with code clarity. I deal with 1000s of lines of code that has references to a tightly bound folder structure. So when I see appendPath highlighted in a code block, I know exactly what is happening at that point when overviewing code. It’s a subtle thing but allows me to quickly revisit code and see exactly what is happening.

While this might be all fine with a single concatenation, what if you want to build a path from a number of variables? Here is were we can learn from other programming languages. Both dotnet and python have path modules that allow us to construct paths easilly.

--Dotnet Join Path
(dotnetclass "system.io.path").combine #("C:/the", "of", "monarch", "the", "I_am")

# Python Join Path 
from os import path
os.path.join ( "C:/the", "of", "monarch", "the", "I_am" )

so to create something like this in MXS, you can easily use something like:

-- Like the join function in dotnet or python
fn joinPath paths =
(
local fullPath = ""
for i in paths do fullPath = pathconfig.appendPath fullPath (i as string)
fullPath
)

joinPath #("C:/Snakes/Why","did", "it", "have", "to", "be", "snakes")

Some other useful functions in pathconfig

pathConfig.convertPathToRelativeTo <path1> <path2>

I use this to truncate paths if it is too long to display on something like a messagebox. Often, the user knows the root of the project so only really needs to see the important information. Remember, the longer path should be the first argument

fullPath = "\\Server\\Share\\Project\\Scene01\\Shot06\\Animation\\XAF"
rootPath = "\\Server\\Share\\Project"

trPath = pathConfig.convertPathToRelativeTo rootPath fullPath
messagebox ("The file has been written to\n\n" + trPath) title:"PathConfig" beep:false

This is useful to extract the last folder in a filepath

trPath2 = pathConfig.stripPathToLeaf fullPath
messagebox ("The file has been written to the " + trPath2 + " directory") title:"PathConfig" beep:false

The main reason is I know exactly what is happening when I see the stripPathToLeaf function. You can do the same with filterstring, but I think pathconfig is a clearer approach than this:

pathEle = filterstring "\\Server\\Share\\Project\\Scene01\\Shot06\\Animation\\XAF" "\\"
if pathEle.count > 0 then
trPath3 = pathEle[pathEle.Count]
pathConfig.isPathRootedAtBackslash fullPath

This is useful to decide if you are working locally or not, sometimes if an artist drags a file into the viewport they can end up working outside pipeline and I need a way of detecting if this happens.

Building a simple project configuration tool

You can discuss for more hours in a day the nuances of folder structure, but ultimately, any structure is better than no structure. However, you don’t have to create a complex script to do something like this. 3dsMax has a built in project configuration that you can use if you want to setup a basic project structure, but can also create your own from the existing configuration if you need.

pathConfig.doProjectSetupStepsUsingDirectory <ProjectPath>

The useful thing about this call is that it will make an additional file with an .mxp extension. This looks like this:

[Directories]
Animations=.\sceneassets\animations
Archives=.\archives
AutoBackup=.\autoback
BitmapProxies=.\proxies
Downloads=.\downloads
Export=.\export
Expressions=.\express
Images=.\sceneassets\images
Import=.\import
Materials=.\materiallibraries
MaxStart=.\scenes
Photometric=.\sceneassets\photometric
Previews=.\previews
ProjectFolder=C:\Users\LoneRobot\Desktop\meh
RenderAssets=.\sceneassets\renderassets
RenderOutput=.\renderoutput
RenderPresets=.\renderpresets
Sounds=.\sceneassets\sounds
VideoPost=.\vpost
[XReferenceDirs]
Dir1=.\scenes
[BitmapDirs]
Dir1=C:\Program Files\Autodesk\3ds Max 2014\Maps
Dir2=C:\Program Files\Autodesk\3ds Max 2014\Maps\glare
Dir3=C:\Program Files\Autodesk\3ds Max 2014\Maps\adskMtl
Dir4=C:\Program Files\Autodesk\3ds Max 2014\Maps\Noise
Dir5=C:\Program Files\Autodesk\3ds Max 2014\Maps\Substance\noises
Dir6=C:\Program Files\Autodesk\3ds Max 2014\Maps\Substance\textures
Dir7=C:\Program Files\Autodesk\3ds Max 2014\Maps\mental_mill
Dir8=C:\Program Files\Autodesk\3ds Max 2014\Maps\fx
Dir9=.\downloads

You realise that this is just an ini file, which means we can use this, edit it to add new folders and structures to suit our production, or completely remove the default max project structure.

-- add some custom paths
Scenes=.\scenes
Scenes1=.\scenes\shots\animation
Scenes2=.\scenes\shots\maxfiles

-- add some asset folders
Assets1=.\assets\characters
Assets2=.\assets\sets
Assets3=.\assets\props

If we want to parse this mxp file and create our own pipeline structure building function, we can write a simple function like this:

fn createProjectfromMXP root mxpFile =
(
if doesfileexist root and doesfileexist mxpFile then
(
--grab the directory keys from the mxp file as an array
folderKeys = getINISetting projectPreset "directories"

-- loop over the keys and get the value
for key in folderKeys do
(
-- construct the new folder path
newFolder = (pathconfig.appendPath newProjectRoot (getINISetting projectPreset "directories" key))
makeDir newFolder
format "Created new folder % in Project %\n" newFolder root
)
)
)

createProjectfromMXP "D:\TheLastCrusade" "C:\Users\LoneRobot\Desktop\ProjectDefault.mxp"

It is project creation at it’s simplest. I’ve extended this idea to having XML folder presets for each discipline, having a default preset but also allowing each project to customise the folder structure to their needs. It becomes useful with the different requirements of jobs and means the same tool can be used each time to generate the structure.

I hope this has made you think about how you handle files and pathing in all of your scripts. Please note, I don’t think concatenation is wrong, I am simply proposing a consistent approach in the way other programming languages are able to. When you start to build a larger codebase for your pipelines, it is something that I find much easier to visualise when revisiting scripts.

Appendix

One consideration when writing a tool to abstract folder structures into XML format was the nature of what to store. It’s a great example of something that would appear to non-programmers as a pointless endeavour but a perfect challenge to a programmer. The question I asked myself was:

How do you make sure that the list of folders you are about to write is as economical as possible?

Lets take something like a folder such as the Python install on my computer. Imagine this was the structure I wanted to write to file. Id grab the folders using something like this

pyDirs = (dotnetclass "system.IO.Directory").GetDirectories @"C:/Python27" \
 "*" (dotnetclass "System.IO.SearchOption").AllDirectories
pyDirs.count 
--632

So out of that list, there are quite a few nested subfolders.

C:\Python27
C:\Python27\Tools
C:\Python27\Tools\Scripts

Logically, you don’t need to store the first two entries here – they are made automatically in creating the last one. So you need to check the array to see what the longest elements are and cross reference them with the shortest.  Whilst I’m certain my approach can be optimised massively, it takes a few seconds to analyse the most efficient folder list to write to file.

fn ProcessUniqueDirectoryArray root returnSortedbyName:true debug:false = 
( 
 if doesfileexist root then
 (
 -- compare path length 
 -- Used by the qsort method to sort an array according to longest>shortest path length
 fn comparePathLength v1 v2 =
 (
 local pathLen = v1.count<v2.count
 case pathLen of
 (
 false: -1
 true: 1
 default:0
 )
 ) 
 clearlistener()
 start = timeStamp()
 --Use dotnet to get All Folders in a directory (EnumerateFiles is faster but only in later frameworks)
 DirStructure = (dotnetclass "system.IO.Directory").GetDirectories root "*" (dotnetclass "System.IO.SearchOption").AllDirectories
 -- Strip the pathroot to provide a neutral path root location
 -- This is so we can combine the neutral path with a new root location later
relPaths = (for dir in DirStructure collect substituteString dir (root + "\\") "")
-- sort the path array by longest to shortest path order
 qsort relPaths comparePathLength
 if Debug then format "Path length Counts %\n" (for i in relpaths collect i.count)
 -- Grab the folder count before we do anything 
 PreProcessCount = relPaths.count
-- Loop through each folder from the start (the longest path) 
 for i in relpaths do
 ( 
 if debug then format "path loop %\n" i
 -- For each path in the array, loop through the paths backwards and check if the text is contained within the longer path
 -- This is the core to calculating the most optimized path list
 -- i.e if you have a directory "C:\Python27\Lib\logging", you dont need to store "C:\Python27\Lib\" or "C:\Python27"
 -- as they will be created by storing the longest unique path
 for p = relpaths.count to 1 by -1 do
 (
 -- Check that the path we are checking in our p loop is not the i loop path (otherwise it will be removed)
 if not relpaths[p] == i then
 (
 --if debug then format "checking % is contained in %\n" relpaths[p] i
 if matchpattern i pattern:(relpaths[p]+"*") then 
 (
 -- If the shorter path is found in the longer path, we get the location of the shorter path 
 --if debug then format "Path similarity found % % \n" relpaths[p] i
 --dont need to use find item as the p loop is the same
 -- as we are going backwards we can safely delete items
 --it = finditem relpaths relpaths[p]
 if debug then format "p-loop Integer % | FindItem Integer % \n" p it 
 if debug then format "************************************* Deleting path from array : % *************************************\n" relpaths[p]
 deleteitem relpaths p 
 
 )
 --else if debug then format "% is a unique path\n" relpaths[p]
 )
 ) 
 )
if debug then format "Post Path Analysis Count : %\n" relPaths.count
 qsort relPaths comparePathLength
 if debug then format "re-sorted by length\n" 
 if debug then for i in relpaths do print i
 if returnSortedbyName then sort relpaths
 if debug then format "re-sorted by name\n" 
 if debug then for i in relpaths do print i
 
 if debug then format "PreProcess Count : % | PostProcess Count %\n" PreProcessCount relPaths.count
 if debug then format "Processed in % seconds\n" ((timeStamp() - start) / 1000.0)
 relPaths 
 )
)
ProcessUniqueDirectoryArray @"C:\Python27" debug:true

Running this yields the following:

PreProcess Count : 632 | PostProcess Count 391

So a little bit of thought can never hurt…

404