The WebL language described in this document is version 3.0. The WebL home page is located at http://www.compaq.com/WebL on the World-Wide Web.

 

Acknowledgements: WebL was initially designed and implemented by Thomas Kistler and Hannes Marais. Service combinators were contributed by Luca Cardelli and Rowan Davies. Tom Rodeheffer suggested many improvements to the language and implementation. Monika Henzinger, Jeff Dean, Brian Eberman and Jin Yu contributed many suggestions, bug fixes, and improvements. Cynthia Hibbard, Dominique Marais, and Krishna Bharat corrected several mistakes in the user manual. Since the release of the software in July 1998, many corrections and improvements have been made by WebL users themselves. The list of contributors and their contributions are contained in the file BugList.java, which is part of the WebL source distribution.

 

(c) Copyright Compaq Computer Corporation, 1998-1999.

All rights reserved.

 

The WebL software contains regular expression software developed by

Daniel F. Savarese. Copyright (c) 1997-1999 by Daniel F. Savarese.

All rights reserved.

 

Table of Contents

CHAPTER 1 Introduction 11

CHAPTER 2 The Language Core 17

Basic Terminology 17

Expressions 17

Value Types 18

Constants 19

Operators 19

Statements 20

Variables and Scoping 20

Constructors 22

Dynamic Types 23

Type Nil 23

Type Bool 23

Type Char 24

Type String 25

Type Int 25

Type Real 26

Type List 26

Type Set 27

Type Fun 27

Type Object 29

Type Meth 30

Types Page, Piece, PieceSet and Tag 31

Value Equality 31

Operators 32

Statements 35

Statement Sequences 35

If Statement 35

While Statement 36

Repeat Statement 36

Try Statement 36

Every Statement 38

Lock Statement 38

Begin Statement 39

Return Statement 39

Built-in Functions 40

Modules 46

Service Combinators 47

Services 47

Sequential execution S ? T 48

Concurrent execution S | T 48

Time-out timeout(t, S) 49

Repetition Retry(S) 49

Non-termination Stall() 49

CHAPTER 3 Pages 51

Basic Protocol Terminology 51

Markup 54

Retrieving Page Objects 59

CHAPTER 4 The Markup Algebra 67

Pages, Tags, Pieces, and Piece Sets 67

Tags 68

Pieces 68

Piece Sets 69

Searching Functions 70

Element search 70

Pattern search 71

PCData search 73

Sequence search 74

Paragraph search 75

Filtering Pieces 78

Miscellaneous Functions 80

Piece Comparison 83

Piece Set Operators and Functions 87

I. Basic Operators 88

II. Positional Operators 89

III. Hierarchical Operators 93

IV. Regional Operators 95

V. Miscellaneous Functions 98

Page Modification 106

Creating Pieces 106

Inserting Pieces 108

Deleting Pieces 109

Replacing Pieces 111

CHAPTER 5 Modules 113

Module Base64 115

Module Browser 116

Module Cookies 117

Module Farm 119

Module Files 121

Module Java 124

Module Servlet 131

Module Str 136

Module Url 138

Module WebCrawler 141

Module WebServer 143

CHAPTER 6 Examples 149

Reading Grades 149

WebCrawler 153

Highlight Proxy 156

CHAPTER 7 WebL Quick Reference 159

Running WebL Programs 159

WebL EBNF 162

Operator Precedence 166

Operators 168

Functions 173

Exceptions 182

Regular Expressions 195

List of Tables

TABLE 1. Constant examples 19

TABLE 2. Operator examples 19

TABLE 3. Constructor expressions 22

TABLE 4. Boolean Expressions 23

TABLE 5. Character Expressions 24

TABLE 6. Escape Sequences 24

TABLE 7. String Expressions 25

TABLE 8. Integer Expressions 25

TABLE 9. Real Expressions 26

TABLE 10. List Expressions 26

TABLE 11. Set Expressions 27

TABLE 12. WebL Core Operators 33

TABLE 13. Core Built-in Functions 41

TABLE 14. Supported MIME Types 63

TABLE 15. Functions to Retrieve Web Pages 64

TABLE 16. Fields of the option object 65

TABLE 17. Piece Set Searching Functions 79

TABLE 18. Miscellaneous Functions 82

TABLE 19. Comparing Pieces x and y 86

TABLE 20. Piece and Piece Set Operators 101

TABLE 21. Piece and Piece Set Functions 104

TABLE 22. Formal Definitions of Piece Set Operators 105

TABLE 23. Page Modification Functions 112

TABLE 24. Standard WebL Modules 113

TABLE 25. Module Base64 115

TABLE 26. Module Browser 116

TABLE 27. Module Cookies 118

TABLE 28. Module Farm 120

TABLE 29. Methods of Farm Objects 120

TABLE 30. Module Files 121

TABLE 31. Module Java 128

TABLE 32. Conversion of Java types into WebL types 129

TABLE 33. Conversion of WebL types into Java types 130

TABLE 34. Format of the Servlet request parameter object 134

TABLE 35. Format of the Servlet response parameter object 135

TABLE 36. Module Str 136

TABLE 37. Module Url 139

TABLE 38. URL constituents 140

TABLE 39. Module WebServer 145

TABLE 40. Fields of the Request Object 146

TABLE 41. Fields of the Response Object 147

TABLE 42. WebL Command Line Options 160

TABLE 43. String and Character Escape Sequences 165

TABLE 44. Operator Precedence Table 166

TABLE 45. WebL Operators 168

TABLE 46. Built-in Functions 173

TABLE 47. Exceptions thrown by the built-in functions 183

TABLE 48. Quantified Atoms 195

TABLE 49. Quantified Atoms with Minimal Matching 196

TABLE 50. Atoms 197

TABLE 51. Perl5 Extended Regular Expressions 198

List of Figures

FIGURE 1. Converting Markup into Tag and PCData Sequences 68

FIGURE 2. Piece Notation 69

FIGURE 3. Results of Searching for "WebL" 71

FIGURE 4. Nested Unnamed Pieces 84

FIGURE 5. Example of Position Numbering 85

FIGURE 6. Operation of P without Q 96

FIGURE 7. Operation of P intersect Q 97

FIGURE 8. Flattening a Piece Set 99

FIGURE 9. Application of the Content Function 100

FIGURE 10. Application of the NewPiece function 107

FIGURE 11. Copying Pieces during Inserts 109

FIGURE 12. Deleting Pieces 110

Introduction

The Language Core

Pages

The Markup Algebra

Modules

Examples

WebL Quick Reference