The Path of the Righteous
This month’s tutorial is designed for beginner scripters. Anybody who has worked in an environment where the folder structure isn’t locked down will tell you that a Wild West approach always ends badly. What happens if somebody leaves? or is sick and has to pass their working shot over to somebody else? It’s something we have to be aware of and build systems to handle. Conceiving a default set of folders for the type of work you do is a useful starting point to this process, and relevant even if there is only one of you. Most people will want to customise their folder structure, so if you start to design a base folder there’s a couple of approaches.
- You can make an empty structure and manually copy this when you start a new project
- You can make a tool that builds the structure automatically
Either approach is fine, but a tool is much COOLER.
Take The Right Path
What might be important is how you construct the paths to your file structure in projects. Here’s the string concatenation way:
"\\\\server\\projectname\\sequence" + "\\shot" + "\\animation" + "\\maxfiles"
Remember that you have the option of adding a forward slash instead of using “\\” but the result for me is the same, it’s a file path that you have to be careful to format properly in order for it to parse correctly. Any portion of the code that misses the path separator could mean an unusable path. You could use stringstream and format, but it’s still a similar sort of concatenation operation.
The Pathconfig struct has a few useful methods. It’s likely that (like me) you looked at the first few methods and assumed it was something to do with the built-in max project structure. However, keep going, it has some very useful functions lower down. The single most useful function which I tend to use all the time is appendPath. This is great because you don’t have to use string concatenation, it automatically adds the correct separators for you:
pathconfig.appendPath "\\\\server\\projectname\\bad" "dates"
So this is fine when you’re adding one folder name onto another. You could reason that this is an unnecessary step to something simple. You might be right, but for me it is to do with code clarity. I deal with 1000s of lines of code that has references to a tightly bound folder structure. So when I see appendPath highlighted in a code block, I know exactly what is happening at that point when overviewing code. It’s a subtle thing but allows me to quickly revisit code and see exactly what is happening.
While this might be all fine with a single concatenation, what if you want to build a path from a number of variables? Here is were we can learn from other programming languages. Both dotnet and python have path modules that allow us to construct paths easilly.
--Dotnet Join Path (dotnetclass "system.io.path").combine #("C:/the", "of", "monarch", "the", "I_am")
# Python Join Path from os import path os.path.join ( "C:/the", "of", "monarch", "the", "I_am" )
so to create something like this in MXS, you can easily use something like:
-- Like the join function in dotnet or python fn joinPath paths = ( local fullPath = "" for i in paths do fullPath = pathconfig.appendPath fullPath (i as string) fullPath ) joinPath #("C:/Snakes/Why","did", "it", "have", "to", "be", "snakes")
Some other useful functions in pathconfig
pathConfig.convertPathToRelativeTo <path1> <path2>
I use this to truncate paths if it is too long to display on something like a messagebox. Often, the user knows the root of the project so only really needs to see the important information. Remember, the longer path should be the first argument
fullPath = "\\Server\\Share\\Project\\Scene01\\Shot06\\Animation\\XAF" rootPath = "\\Server\\Share\\Project" trPath = pathConfig.convertPathToRelativeTo rootPath fullPath messagebox ("The file has been written to\n\n" + trPath) title:"PathConfig" beep:false
This is useful to extract the last folder in a filepath
trPath2 = pathConfig.stripPathToLeaf fullPath messagebox ("The file has been written to the " + trPath2 + " directory") title:"PathConfig" beep:false
The main reason is I know exactly what is happening when I see the stripPathToLeaf function. You can do the same with filterstring, but I think pathconfig is a clearer approach than this:
pathEle = filterstring "\\Server\\Share\\Project\\Scene01\\Shot06\\Animation\\XAF" "\\" if pathEle.count > 0 then trPath3 = pathEle[pathEle.Count]
This is useful to decide if you are working locally or not, sometimes if an artist drags a file into the viewport they can end up working outside pipeline and I need a way of detecting if this happens.
Building a simple project configuration tool
You can discuss for more hours in a day the nuances of folder structure, but ultimately, any structure is better than no structure. However, you don’t have to create a complex script to do something like this. 3dsMax has a built in project configuration that you can use if you want to setup a basic project structure, but can also create your own from the existing configuration if you need.
The useful thing about this call is that it will make an additional file with an .mxp extension. This looks like this:
[Directories] Animations=.\sceneassets\animations Archives=.\archives AutoBackup=.\autoback BitmapProxies=.\proxies Downloads=.\downloads Export=.\export Expressions=.\express Images=.\sceneassets\images Import=.\import Materials=.\materiallibraries MaxStart=.\scenes Photometric=.\sceneassets\photometric Previews=.\previews ProjectFolder=C:\Users\LoneRobot\Desktop\meh RenderAssets=.\sceneassets\renderassets RenderOutput=.\renderoutput RenderPresets=.\renderpresets Sounds=.\sceneassets\sounds VideoPost=.\vpost [XReferenceDirs] Dir1=.\scenes [BitmapDirs] Dir1=C:\Program Files\Autodesk\3ds Max 2014\Maps Dir2=C:\Program Files\Autodesk\3ds Max 2014\Maps\glare Dir3=C:\Program Files\Autodesk\3ds Max 2014\Maps\adskMtl Dir4=C:\Program Files\Autodesk\3ds Max 2014\Maps\Noise Dir5=C:\Program Files\Autodesk\3ds Max 2014\Maps\Substance\noises Dir6=C:\Program Files\Autodesk\3ds Max 2014\Maps\Substance\textures Dir7=C:\Program Files\Autodesk\3ds Max 2014\Maps\mental_mill Dir8=C:\Program Files\Autodesk\3ds Max 2014\Maps\fx Dir9=.\downloads
You realise that this is just an ini file, which means we can use this, edit it to add new folders and structures to suit our production, or completely remove the default max project structure.
-- add some custom paths Scenes=.\scenes Scenes1=.\scenes\shots\animation Scenes2=.\scenes\shots\maxfiles -- add some asset folders Assets1=.\assets\characters Assets2=.\assets\sets Assets3=.\assets\props
If we want to parse this mxp file and create our own pipeline structure building function, we can write a simple function like this:
fn createProjectfromMXP root mxpFile = ( if doesfileexist root and doesfileexist mxpFile then ( --grab the directory keys from the mxp file as an array folderKeys = getINISetting projectPreset "directories" -- loop over the keys and get the value for key in folderKeys do ( -- construct the new folder path newFolder = (pathconfig.appendPath newProjectRoot (getINISetting projectPreset "directories" key)) makeDir newFolder format "Created new folder % in Project %\n" newFolder root ) ) ) createProjectfromMXP "D:\TheLastCrusade" "C:\Users\LoneRobot\Desktop\ProjectDefault.mxp"
It is project creation at it’s simplest. I’ve extended this idea to having XML folder presets for each discipline, having a default preset but also allowing each project to customise the folder structure to their needs. It becomes useful with the different requirements of jobs and means the same tool can be used each time to generate the structure.
I hope this has made you think about how you handle files and pathing in all of your scripts. Please note, I don’t think concatenation is wrong, I am simply proposing a consistent approach in the way other programming languages are able to. When you start to build a larger codebase for your pipelines, it is something that I find much easier to visualise when revisiting scripts.
One consideration when writing a tool to abstract folder structures into XML format was the nature of what to store. It’s a great example of something that would appear to non-programmers as a pointless endeavour but a perfect challenge to a programmer. The question I asked myself was:
How do you make sure that the list of folders you are about to write is as economical as possible?
Lets take something like a folder such as the Python install on my computer. Imagine this was the structure I wanted to write to file. Id grab the folders using something like this
pyDirs = (dotnetclass "system.IO.Directory").GetDirectories @"C:/Python27" \ "*" (dotnetclass "System.IO.SearchOption").AllDirectories pyDirs.count --632
So out of that list, there are quite a few nested subfolders.
C:\Python27 C:\Python27\Tools C:\Python27\Tools\Scripts
Logically, you don’t need to store the first two entries here – they are made automatically in creating the last one. So you need to check the array to see what the longest elements are and cross reference them with the shortest. Whilst I’m certain my approach can be optimised massively, it takes a few seconds to analyse the most efficient folder list to write to file.
fn ProcessUniqueDirectoryArray root returnSortedbyName:true debug:false = ( if doesfileexist root then ( -- compare path length -- Used by the qsort method to sort an array according to longest>shortest path length fn comparePathLength v1 v2 = ( local pathLen = v1.count<v2.count case pathLen of ( false: -1 true: 1 default:0 ) ) clearlistener() start = timeStamp() --Use dotnet to get All Folders in a directory (EnumerateFiles is faster but only in later frameworks) DirStructure = (dotnetclass "system.IO.Directory").GetDirectories root "*" (dotnetclass "System.IO.SearchOption").AllDirectories -- Strip the pathroot to provide a neutral path root location -- This is so we can combine the neutral path with a new root location later relPaths = (for dir in DirStructure collect substituteString dir (root + "\\") "") -- sort the path array by longest to shortest path order qsort relPaths comparePathLength if Debug then format "Path length Counts %\n" (for i in relpaths collect i.count) -- Grab the folder count before we do anything PreProcessCount = relPaths.count -- Loop through each folder from the start (the longest path) for i in relpaths do ( if debug then format "path loop %\n" i -- For each path in the array, loop through the paths backwards and check if the text is contained within the longer path -- This is the core to calculating the most optimized path list -- i.e if you have a directory "C:\Python27\Lib\logging", you dont need to store "C:\Python27\Lib\" or "C:\Python27" -- as they will be created by storing the longest unique path for p = relpaths.count to 1 by -1 do ( -- Check that the path we are checking in our p loop is not the i loop path (otherwise it will be removed) if not relpaths[p] == i then ( --if debug then format "checking % is contained in %\n" relpaths[p] i if matchpattern i pattern:(relpaths[p]+"*") then ( -- If the shorter path is found in the longer path, we get the location of the shorter path --if debug then format "Path similarity found % % \n" relpaths[p] i --dont need to use find item as the p loop is the same -- as we are going backwards we can safely delete items --it = finditem relpaths relpaths[p] if debug then format "p-loop Integer % | FindItem Integer % \n" p it if debug then format "************************************* Deleting path from array : % *************************************\n" relpaths[p] deleteitem relpaths p ) --else if debug then format "% is a unique path\n" relpaths[p] ) ) ) if debug then format "Post Path Analysis Count : %\n" relPaths.count qsort relPaths comparePathLength if debug then format "re-sorted by length\n" if debug then for i in relpaths do print i if returnSortedbyName then sort relpaths if debug then format "re-sorted by name\n" if debug then for i in relpaths do print i if debug then format "PreProcess Count : % | PostProcess Count %\n" PreProcessCount relPaths.count if debug then format "Processed in % seconds\n" ((timeStamp() - start) / 1000.0) relPaths ) ) ProcessUniqueDirectoryArray @"C:\Python27" debug:true
Running this yields the following:
PreProcess Count : 632 | PostProcess Count 391
So a little bit of thought can never hurt…
4 Comments + Add Comment
Got anything to say? Go ahead and leave a comment!
Want to know about updates the moment they happen? Subscribe to LoneRobot.net
- SliderMan – Does everything that a slider can
- Grabbing The Material Preview
- The Path of the Righteous
- I Don’t Need Regular Expressions
- Kinetic Energy
- What we are today comes from our thoughts of yesterday
- Autodesk Webinar
- Two Clicks From Amsterdam
- WindowBox Replanted
- SpeechBot – A handy script to load and save morpher keys