Module Farm

Module Farm introduces the concept of a farm object (an object with a hidden implementation). A farm consists of a number of workers that process jobs. The Perform method of a farm object allows the programmer to insert a job into the job queue of the farm. Idle workers (those that are not doing something) periodically pick a job from the queue to perform. When the job queue is empty and no workers are working, we say that the farm is idle.

It is important to note that the workers are simple-minded in the sense that should an exception occur while performing a job, the job is terminated without any indication to the programmer, and the worker becomes idle again. It is thus advisable to include exception handling code in the job itself.

The Perform method of a farm uses a special calling convention. The argument of this method must be an expression denoting a function application. For example, say we would like to turn a function invocation of F with two arguments into a job, we must write:

var frm = Farm_NewFarm(10);

frm.Perform(F(a, b))

It is important to know that the arguments to the function application are evaluated before the job is started. A typical application of a farm object is the following stupid web crawler program with 10 parallel workers:

import Farm;

var F = Farm_NewFarm(10);

var ProcessPage = fun(url)

var page = GetURL(url);

every a in Elem(page, "a") do

F.Perform(ProcessPage(a.href))

end

end;

F.Perform(ProcessPage("http://www.nowhere.com"));

while !F.Idle() do

Sleep(10000)

end