Distributed Processing Basics

Distributed processing accelerates processing by distributing the work to multiple computers that have been chosen to provide more processing power. You can submit batches of processing jobs to the Apple Qmaster distributed processing system, which allocates those jobs to other computers in the most efficient way (described in more detail in How the Apple Qmaster System Distributes Batches).

Computers that submit batches to the Apple Qmaster distributed processing system are called clients. A job is a processing task such as a Compressor preset-source pair, or a Shake file, or other files or commands that use UNIX commands to specify settings such as rendering instructions and file locations and destinations.

Figure. Diagram showing the batch, the Apple Qmaster cluster, and the processed files.

A batch is one or more jobs submitted for processing at one time. The procedure is analogous to printing multipage documents from a word processing program; the files are spooled and processed in the background. Although a batch can include just one job, you will typically want to submit several jobs at once for processing. Similarly, several people can use the same Apple Qmaster system at the same time, with several client computers sending batches in the same time frame. Batches are managed and distributed by the computer that is designated as the Apple Qmaster cluster controller, which is described in the next section.

There are three approaches you can take toward setting up a distributed processing system:

Important: When using distributed processing (This Computer Plus, QuickClusters, and managed clusters), you may be asked to authenticate yourself. This is because Apple Qmaster must share a folder, using NFS sharing, with links to the media files so that the processing computers can access the media files. This makes this folder available to any computer that knows its IP address. When using distributed processing with confidential media, make sure the computers are protected by a firewall.

For more information about the various components that make up a distributed processing system, see Basic Components of the Apple Qmaster Distributed Processing System.