Articles Archive
Articles Search
Director Wiki
  

Beyond Desktop Video

October 19, 1999
by Brennan Young

About QuickTime

The Quicktime format is one of Apple's greatest successes. Some would say it is their only really successful technology beyond the Macintosh. Ironically, that success has been hard earned. When Apple's 'postage stamp sized' desktop video appeared, it was not taken very seriously by professional video editors. This was hardly surprising, given that hard disk space and RAM were at a premium.

Slowly, and not least because of its wholesale adoption by the multimedia community where tools like Director played a central role, Quicktime has become the standard for digital video editing, and even looks set to eclipse the ugly, expensive and intrusive Real Networks products in the streaming market too. There are now other competitors in the 'desktop video' scene, too. It has been an uphill struggle all the way, and continues to be so, but Quicktime seems to be a rare case of carefully assembled good tech winning over slickly-marketed, but substandard products.

The buzz about Quicktime over the last year or so seems to be focused mainly on streaming. Apple stands to make some good (and honest) money if they can crack Real's dominance. Unfortunately, this high profile marketing battle conceals some of Quicktime's more fascinating dimensions.

Quicktime is often described as a 'multimedia' technology. There was a time when I found this label a little exaggerated, but since version 3, I became curious about just how 'multi' the supported media were. It turns out that Quicktime is maturing into an elegant and powerful multimedia format which already matches (and even overtakes) Shockwave on many levels.

Just as Macromind usurped Apple's flagship authoring product via the back door by adding interactivity to the Videoworks editing tool, originally designed as a utility to support Hypercard, Quicktime could easily do the same to Director. If you still think Quicktime is only for desktop video, audio and VR, it's probably worth paying close attention here. Even though Quicktime movies need no longer behave quite like what we have come to expect from Quicktime movies, they can usually run just fine inside Director no matter how many weird and wonderful things they include.

Quicktime is now a standard technology for digital video and audio. Many other media types are also supported, but perhaps not encouraged as much as they could be. We can perhaps speak of 'desktop video inertia', where it is commonly assumed that Quicktime can do nothing else.

Tracks

In Quicktime, each media type occupies a track, and a Quicktime movie can contain a large number of tracks. Visual or 'spatial' tracks are somewhat similar to sprite channels in Director, as they can have a drawing layer, a graphics mode (ink), width, height, a location within the window, rotation and so on. In the case of sprite tracks (to be covered in more detail later) the tracks themselves can contain an arbitrary number of sprites with their own drawing layer, graphics mode, width, height, location etc.

Most users of Quicktime are also familiar with audio tracks, which can have volume and balance. A lesser know type, MIDI tracks can have specific instruments applied and those instruments can be shaped and customised with Quicktime's excellent built-in software synthesiser. Before you rush out and buy a license for Beatnik, take a close look at the MIDI and 'soundtracker' capabilities of Quicktime.

Most users of Quicktime are also familiar with audio tracks, which can have their own volume and balance. A lesser know type, MIDI, music or instrument tracks can have specific instruments applied using your own samples and those instruments can be shaped and customised with Quicktime's excellent built-in software synthesiser. Before you rush out and buy a license for the Beatnik Xtra, take a close look at the MIDI and 'soundtracker' capabilities of Quicktime.

The Quicktime softsynth also has a 'secret' GUI. When you see it and start to use it, you will realise why Apple left it undocumented, but it does actually work. Import a MIDI file into Quicktime Player, 'Get Info' on the Music Track. Hold down the Option (Alt) key and double-click on an instrument, then click on the 'Edit...' button, which is normally greyed-out. In the centre of the weird 'card' interface which you should now see is a black square, or perhaps a padlock icon, click on it and you can access some of the finer features of the Quicktime softsynth. The slider controls are very rough so you may have to use the scroller arrows, but you can shape your own audio samples using Filters, Envelopes and LFO's.

Here is a short and incomplete list of some other Quicktime track types:

A large number of media types are supported by Quicktime itself. There are more, but I think this should indicate that desktop video is just the tip of the iceberg.

Combining this hidden power with Director as a container will lead to some interesting new territory. Indeed, if you are using Director mostly as a container for Quicktime, you might even find that you don't need Director at all for some jobs. After all, why waste processor power and RAM on two multimedia engines when one will do the job just fine. The interactivity is there in Quicktime, and more importantly, the tools are here to make it now.

Getting away from the desktop video mindset when working with Quicktime does not happen easily. Some track types are so new that they are barely supported, some (like teletext) are obscure enough that they would probably raise eyebrows even at Apple if somebody started using them. Alas Director's Quicktime support exposes a tiny fragment of what is actually available. Another problem is that until recently, many track types could only be created or edited by small freeware or shareware applications, most of which are Mac only.

Some interactive non-video track types have received a fair amount of attention. QTVR has become an industry in itself, and is fairly well covered by various Director Xtras and documentation, although the latest version at time of writing (shipping with Director 7.02) has a known issue where sounds in VR panoaramas do not play on the Mac. This makes it impossible for Director to take advantage of sounds located in stereo within a VR panorama, which adds extra dimensions to the experience.

Sprite Tracks

The most interesting track type, as far as I am concerned, is the sprite track. I imagine most Director users will feel the same way.

Sprite tracks with motion paths first appeared with Quicktime 2. They have the advantage of using a fraction of the bandwidth of an equivalent digital video movie, such as may be exported with Director's own built-in Quicktime exporter, because seperate media are stored once only and then moved around in layers, much like in Director itself. The bandwidth advantages are obvious to anybody who develops media for the web.

Editors

Quicktime 3 brought along a large number of enhancements to the Quicktime format, the most important perhaps being one-to-one feature matching between Macintosh and Windows versions. More interesting though, was that sprites could be provided with interactivity, something which Apple call 'wired action atoms'. The complexity of the wired sprite API is only now beginning to be realised, and several tools now exist which make these features available to the multimedia community.

If you are a user of MacOS 8.5 or later, you can play with the freeware tool Spritz as a first step into the possibilities of wired actions. Custom events (beyond the standard set of mouseEvents and idle) are supported, so sprites can send messages to each other, and a fair assortment of actions are available, but as the ReadMe puts it, "The downside [is] that since it is free I don't have unlimited time to work on it", and indeed, the current version is too unstable and has too many rough edges for serious use. Nonetheless, it is good enough for simple interactive projects such as slideshows. Give the guy some feedback and maybe it will improve more more quickly. Here's a simple example of a movie made with Spritz:

It should be mentioned at this point that there is a standard, cross platform Quicktime authoring tool, Apple's own QuickTime Player (previously MoviePlayer). If you are at all serious about multimedia authoring, you really should get a license to access the pro features of this tool. It's inexpensive, especially when bundled with other products such as MacOS, and allows you to assemble quite sophisticated multimedia documents with the full range of different track types, almost from scratch.

Quicktime Player (Pro) is particularly useful for making rough edits in digital audio and video using the clipboard. Tracks can be layered, stretched, rotated and scaled (both in space and time), transparency and effects can be applied. MIDI instruments can be chosen and shaped, text tracks can be made into hyperlinks, chapters, or karaoke lyrics, sprite media can be exchanged, and much more. Many hours of rendering time can be saved by using Quicktime Player instead of Adobe Premiere for simple (or even advanced) editing and compositing. Even given that the current GUI design of Quicktime Player leaves a lot to be desired, and hardly qualifies for the 'pro' nametag a pro license for Quicktime is a must-have.

Elsewhere, the 'next generation' Quicktime editors have started to appear. GoLive was one of the first HTML editors to offer this feature, and Adobe's latest version of same is quite good, although geared strongly to web output. As mentioned earlier, Macromedia's Flash 4 also supports some Quicktime features.

One of the first dedicated Quicktime editors to go beyond scratching the surface was Electrifier Pro, which makes authoring features easily available without resorting to the 'keyhole surgery' of Quicktime Player. Additionally, it affords simple interactive sprite editing. Unfortunately, the wired atoms supported by Electrifier Pro are so limited as to be likely to disappoint hardened Lingo coders. Particularly limiting is that only a few events are supported, with no ability to create custom messages, variables, logical control structures and so on.

Nonetheless, the Electrifier Pro user interface is very well designed (borrowing many ideas from the 'undead' mTropolis) and it is a very good tool for assembling simpler, low bandwidth interactive Quicktime movies from pre-existing content. Multimedia authors who are not particularly comfortable with writing scripts in ASCII will find many useful avenues for this tool. It is also likely that the next version of Electrifier will address many of its present shortcomings in a flexible and elegant way.

LiveStage

A more recent tool on the scene is totallyHip's totallyHip's LiveStage. A Windows version of LiveStage 1 was recently released, so non-Mac users can also get in on the action. I have been so consistently impressed with it that I have chosen to devote most of the rest of this article to this remarkeable product.

LiveStage 1.0 has actually been quietly smouldering for a while now, but I expect that after LiveStage Pro appears, this hot item is going to start a conflagration when people realise what it is capable of. LiveStage Pro should reach 2.0 around the time this article gets published at DOUG, I am using the first release candidate at time of writing and it's very stable. Well, to put it bluntly, LiveStage Pro kicks butt. It is only fairly recently that totallyHip themselves realised what a monster they had on their hands and put the price of LS1.0 up from a throwaway $199 to a more respectable $349. (I can be quite cavalier about this of course because I bought a license when it was cheap). This pricing is likely to differ by the time LiveStage Pro is commercially available so check the totallyHip website for details. It should also be noted that the quirky (and somewhat unique) web graphics editor WebPainter, also from totallyHip, and a pro license for Quicktime are bundled with LiveStage, making it a pretty good deal all round.

A preview edition of LiveStage Pro recently won 'Best of Show' at MacWorld. Notably Matthew Peterson's remarkeable LiveStage-authored Quicktime movies helped to show off the product's depth and power. (Most significant were his implementation of the Logo programming language and drawing program with unlimited undo and pdf export! Check them out and be amazed.)

You can download demos of both versions from the totallyHip website. These will allow you to export finished movies, but saving of projects is disabled. Fortunately I have been participating the LiveStage Pro Alpha and Beta phases so I can whet appetites with some of the goodies which should be available by the time this article appears.

Using LiveStage

LiveStage is a serious authoring tool with a complete scripting language, Qscript, taking a central role. More so, in some ways, than Lingo does in Director. I wonder whether 'scriptless authors' will be able to get the most out of LiveStage, but if you are at all competent with Lingo, you will be impressed with what Qscript has to offer. The recent introduction of 'behaviors' might also broaden the audience somewhat. For good measure, LiveStage Pro also includes a simple multistate button editor.

LiveStage 1 was almost wholly designed for authoring wired sprite tracks. It also allows some access to the Quicktime musical instrument architecture, which means that a full range of musical sounds and sound effects are available at negligible bandwidth. Wired actions can also refer to and manipulate other tracks which can not in themselves be imported into or edited in LiveStage 1, i.e. it includes none of the 'media assembly' features found in most other Quicktime editors.

The result of this, unless you are working with sprites alone, is that when you export a Quicktime movie from LiveStage 1, you usually still have some work to do in Electrifier Pro or Quicktime Player so that the wired atoms have something to 'chew on'. This means that it is not really a tool which you would use for 'putting everything together'. LiveStage Pro is a more elaborate affair, supporting many more track types, so it would normally be possible to polish your work without having to import it into other tools afterwards.

It is very important to note that LiveStage exports 'ordinary' Quicktime movies. You can play them almost anywhere you'd play a Quicktime movie, as long as the system has Quicktime installed. LiveStage itself has a preview window, (which is seperate from 'the stage', the window used to lay out content), so you can 'compile' and view your movie while you are working on it.

However, unlike Director, where Macromedia has full control over the design of the player and the editor, totallyHip's engineers have the curious job of wrapping Apple's wired Quicktime atoms with a higher level interface. This means that there is a limit to what can be made available to the end user because such tools ultimately have to conform to Apple's Quicktime API, which is apparently not as well documented as it could be. As an example, on the (excellent) Livestage support mailing lists, occasional questions about obscure Qscript actions or properties are enthusiastically answered with "try such and such, if you find out how it works, let us all know".

Nevertheless, totallyHip's engineers have done a fantastic job of exposing a pretty complete range of wired atoms to the Qscript environment. In a few cases, atoms have been included without the engineers being fully clear about their functionality. This is quite opposite to the approach of most other authoring tools, where 'strange' or 'risky' functionality goes undocumented to 'protect' the poor user. (To many professionals, this is an insulting conceit.)

LiveStage treats you like an adult, so you'd better be prepared for some mysterious dark corners. This is a delectable prospect for some people, more unsettling for others. Jaded lingo coders whose next barrier is the XDK, and those who are looking for alternatives after the horrors of the Shockwave Remote will find very much to enjoy in Livestage. Anybody who wants to get stuck in closer with Quicktime without relying so much on Lingo will also be amply rewarded. Instead of always using Director for even the simplest projects, you can perhaps get by with a rich wired movie running in a simple custom container authored with a free application such as iShell.

LiveStage is probably not an ideal tool for absolute beginners, but it is a multimedia pro's fantasy. I would even go so far as to say that it is an essential part of a modern multimedia web designer's arsenal. You simply can't do Quicktime justice without it. The ability to breathe the kind of fluid interactivity we have come to expect from Shockwave into Quicktime movies is surely too good an opportunity to pass over. It is also fully AppleScriptable, so running on a Mac web server and with appropriate cgi, you can generate Quicktime movies on the fly for visitors to your website for a tiny fraction of the cost of Macromedia's Generator. This has been tried already during the Beta phase and it works quite well.

Scripting with Qscript

The LiveStage 1 working process was somewhat hampered by a quirky user interface with a list of 'Actions' (something like keyframes) instead of a timeline. LiveStage Pro bows to convention and has a timeline window, (something like Director's score) and it is here that the samples of the different tracks can be manipulated. In LS1, simple animations can be made along paths, but this has been dropped in the Pro version in favor of more flexible 'tween' and 'modifier' tracks, which do the same and more.

The heart of both versions of LiveStage is the scripting window. Typographical errors are avoided by presenting all the Qscript symbols as drag and drop icons which provide parameter templates, similar to GoLive's Javascript palette. This works rather well. A standard set of handlers are represented as seperate text fields arranged in a list. Adding your own custom handlers is no problem.

So what can you do with Qscript? Well, quite a lot. Qscript is syntactically closer to the C family than to Lingo. 'Dot syntax' (recently introduced to Director) is spoken exclusively here. Syntax will not be a problem to most Lingo users, especially those who have embraced the changes in D7, or anyone that has worked with javascript.

Here is a small, but useful example script which, when pasted into the 'idle' handler of a sprite, causes that sprite to behave like a compass if its parent track is part of a QTVR movie:

GlobalVars angle
IF (angle != PanAngle)
    Rotate(PanAngle-angle)
ENDIF
angle = PanAngle

Qscript symbols (keywords) are divided into three basic categories, 'Actions', 'Properties' and 'Other'. Actions and Properties always have a target object, although if you are lazy they all have default targets. In the example above, the default target of a sprite action such as Rotate is the sprite which 'contains' the wired action, where the property panAngle is associated exclusively with VR panoramas, so that is the default target.

Getting and setting properties is achieved by a Qscript accessor designed for that job. Most properties additionally have equivalent 'SetTo' and 'SetBy' actions. This is nice and clean and will doubtless please people like Paul Hemmer, but it might seem odd to some Lingo users that, for example movieRate, setRateTo and setRateBy all refer to the same property. In this case, you must use the first of these to get that property, and one of the other two to set it.

When typing Qscript, you put the target first, followed by a dot, then the action or property which is being called:

SpriteNamed("Frisbee").MoveBy(10,0)

Under 'Other' is found an assortment of useful and more specialised scripting items. Local and Global variables are distinguished from one another, LiveStage Pro introduces sprite variables and movie variables, giving four basic levels of 'visibility' or scope. Unlike Lingo, all local variables must be declared inside handlers, which took me some getting used to. This will be transparently obvious to 'real' programmers, although with the exception of arrays, the datatypes of variables need not be specfied.

There's a fairly complete set of boolean and arithmetic operators, a random function, but no thanks to Apple, no trig functions. Control structures such as IF, ELSE and WHILE are available, a FOR...NEXT loop and a SWITCH...CASE structure have been provided also.

Two basic data types are supported, integer and float, with booleans making an appearance here and there. One dimensional arrays of fixed size can be used. Some actions and properties (such as those that set the graphics modes) require special predefined constants as parameters. LiveStage Pro provides basic support for string handling, but this is still very primitive by comparison to Lingo, again limited by Quicktime itself.

Significantly, custom handlers can also be constructed, but parameter passing and return values are not supported. This is another shortcoming in the QTAPI, and is likely to be added if and when Apple get their act together. In LS1, custom handlers are referred to by an ID number;

SpriteNamed("Frisbee").ExecuteEvent(9463)

...although LiveStage Pro adds a preprocessor so that (amongst other things) constants may be used for custom handler names, image names, constants and so on. A call to a custom handler using a constant would therefore look something like this:

SpriteNamed("Frisbee").ExecuteEvent($"dissolve")

It's rather difficult to return to 'flat' programming after getting used to object oriented approaches in Lingo. Well, Qscript is not really flat. The objects, properties and wired actions available are properly encapsulated. LiveStage Pro offers two kinds of inheritance; MakeNewSprite (which allows sprites to be duplicated), and 'behaviors' which are very similar to their namesakes in Director, but there can be serious performance hits when more than a small number of sprites have long 'idle' handlers. The best practical solution is to make 'manager' and 'controller' objects, using the other sprites as a site to receive events but little else. I'm hoping that Apple will pay close attention to this area so that we can usefully achieve the kind of performance that is possible with Director, with behaviors being shared by large numbers of sprites.

Shortcomings

There are certainly some design improvements that need to be made to LiveStage, behaviors must be authored with an external text editor, and the 'stage' layout is a little inconsistent. Although with this much unique power, plenty can be forgiven. As is common in the software industry, TotallyHip were obliged to 'freeze' the feature set at some point, or we wouldn't be seeing a product at all. Even more killer features are to be expected in future versions, but first they must sell some units.

Other major shortcomings in the Quicktime API are text and math. Quicktime has not yet dealt adequately with the cross-platform issues of fonts and font sizes, and there is no way to display dynamically edited strings. Additionally, the lack of trig functions, or even a square root, puts some limits on the kind of content that is possible right now. There are workarounds. Some folks have made their own text engine in pure Qscript, and an abbreviated lookup table for trigonometry. Another interesting solution is to get Flash to handle text and math. Flash can then send Qscript messages to wired tracks with any relevant data. (Messaging the other way is more restricted.)

On the subject of messaging, Quicktime 4 provides 'inter-movie communication', something like 'tell window' in Director but which is handled transparently by the browser plug-in without any need for LiveConnect or other flaky technologies. Quicktime wired sprites can also send string messages to their container applications, but the latest version of the Quicktime Xtra shipping with Director supports none of these great facilities yet. I have, with some difficulty, managed to get Director to communitcate with wired sprites. The technique is too slow for serious use, but it is perhaps worth investigating until Macromedia improve Director's Quicktime support.

LiveStage is a little Quicktime editor which can perhaps best be seen as complimentary to tools like Flash and Director, although it also competes with both of those tools on a surprising number of levels. Anybody from the Videoworks days will tell you that a 'little Quicktime editor' can go an awfully long way in just a few years.

Conclusion

All of the technologies I have been focusing on in this article are ideal for low bandwidth, internet use, and with Quicktime's excellent capacity for handling broadcast quality media, such technologies will scale well as broadband becomes more common, offering something much more than just 'TV on the net'. Apple would be foolish to underprioritise wired Quicktime movies, simply because they can add things to the streamed media which the other streaming technologies can't touch.

Similarly, Macromedia would be foolish to ignore these powerful new dimensions which are revealing themselves within Quicktime. Macromedia might not see Quicktime as a threat to its flagship technologies -- may even laugh it off as small fry, pointing at third party statistics of the relative sizes of the installed user base of Quicktime and Shockwave -- but a closer look reveals that Quicktime is indeed evolving into what Apple call 'an operating system for media'. Significantly Quicktime represents a more portable, open, flexible and more standards-based technology for multimedia than Director can be. Just consider its adoption by the Motion Picture Experts Group for example.

Macromedia therefore needs to integrate these new dimensions much better within its exisitng, sucessful product range. Best for us (as Director users) would be if the set of properties and actions offered by a future Quicktime Xtra were extended to support a fuller range of track types and wired actions reliably, explicitly and by design, rather than unreliably, underdocumented and by accident.

The multimedia landscape will be different within eighteen months or so, (as it always is) and I am sure that many multimedia designers' conception of Quicktime will have been as radically transformed as mine has been since I realised there was more to it than video. The Quicktime iceberg will not sink the Titanic this time round, but it's passing very close to the keel...

Brennan Young is an English freelance multimedia designer / programmer living and working in Copenhagen, Denmark. He started making interactive presentations on Commodore and Acorn computers as a teenager in the early 1980s, but went on to study Fine Art and Art history at Goldsmith's College, London where he discovered Director. He teaches and lectures on multimedia at various institutions around Denmark. When he is not fiddling with interesting authoring software, he composes and performs music, writes theoretical and practical articles, and 'makes art' for exhibitions or private consumption.

Copyright 1997-2024, Director Online. Article content copyright by respective authors.