Monday, January 29, 2007

An Introduction to Web Development with PLT Scheme - The View

Introduction

First of all, thanks for the positive response to part 1 of this introduction. Comments are very welcome.

The goal of this introduction is to write a "Mini Reddit" called "List It!". The first part focused on the model component i.e. the underlying data and how to implement it using an SQLite database. This second part will concentrate on the view component, which presents the data to the user. The third part will be on the controller.

Our user interface consists of two pages: the frontpage shows the list of links with the highest scores and the submission page where a user can enter the name and url of a new entry. To give an impression of where we are headed, here is screenshots of the final result:



In the Model-View-Controller organization, the controller receives a request from the client (user), retrieves data from the model, lets the view construct a web page and the sends a response back to the client. The view thus consists of a series of functions turning data
into web-pages.

Representing XHTML as X-expressions

PLT Scheme supports multiple ways of representing web-pages. One option is to use strings:

"<html><head><title>A Title</title>
<body>A body</body></html>".
Since the structure of the document is lost with this choice, all but the most simple manipulations of the document, becomes harder to program than they should.

Instead we will use X-expressions (S-expressions representing XML) to represent our web-pages. The above example becomes:
  `(html (head (title "A title"))
(body "A body"))
Besides X-expressions another popular choice in the Scheme world is SXML (if you are into serious XML-manipulations look into SXML and SSAX), which are similar to X-expressions in terms of use.

The beauty of X-expressions is that, since they are normal lists, we can use the builtin functions to manipulate them. Another major convenience is we can use unquote (written as a comma) to insert the result of a Scheme expressions into the X-expression:
   `(body "The result of 1+2 is: " ,(number->string (+ 1 2)))
which, when evaluated, results in
    (body "The result of 1+2 is: " "3").
Attributes are placed right after the tag:
    `(body ((color "white") (bgcolor "black"))
"Hello world")
The WEB library

To avoid writting boilerplate stuff (charset, stylesheet, content type etc.) each and every time a web page is constructed, we will use the general web framework from PLaneT (the PLT Scheme "CPAN" - note that packages are downloaded automatically if needed).

> (require (planet "html.scm" ("soegaard" "web.plt" 1 0)))
> (html-page #:body `(p "Hello world"))
(html
(head (title "A title")
(link ((rel "stylesheet") (type "text/css") (href "")))
(meta ((http-equiv "Content-Type") (content "text/html;charset=UTF-8"))))
(body (h1 "A Header")
(p "Hello world")))
In the example above, the html-page only received the body, so it used the site wide default arguments for the title, the stylesheet and the "header". Obviously we need to override the defaults for our List It! site:

(override-default current-page-title"List it!")
(override-default current-page-header'(h1 ((class "page_header")) "List it!"))
(override-default current-page-style-sheet "http://localhost/stylesheet.css")

After overriding the defaults we get:
    (html
(head (title "List it!")
(link ((rel "stylesheet") (type "text/css")
(href "http://localhost/stylesheet.css")))
(meta ((http-equiv "Content-Type")
(content "text/html;charset=UTF-8"))))
(body (h1 ((class "page_header")) "List it!")
(p "Hello world")))


Use the keywords #:title, #:header, #:body, #:style-sheet, and #:style-sheet with html-page, in order not to use the site wide defaults.


The Submit-new-entry page

The funtion html-submit-new-page builds an X-expression representing the submission page. It takes no arguments, since it doesn't rely on data from the model:

(define (html-submit-new-page)
(html-page
#:title "List it! - submit"
#:header '(h1 "List it!")
#:body
`(div (h2 "Submit a new entry")
,(html-form
"submitnewform" "control.scm"
(html-input "action" #:value "submit" #:type 'hidden)
`(table (tr (td "url") (td ,(html-input "url" #:type 'text #:value "http://")))
(tr (td "title") (td ,(html-input "title" #:type 'text #:value "A title"))))
(html-input "submit" #:value "submit")))))


The submission page consists of a form with two input fields. One for the url and one for the title. The function html-form rececives a name of the form, the action. The default method for submitting is POST, but one can use the keyword argument #:method to choose something else. The function html-input builds an input field. Other types of input fields are: 'image and 'hidden.

Clicking the submit button will post the parameters action, url and title to control.scm.


The Frontpage

The submitted entries are ranked according to their score. The entry with the highest score has rank 1, the next rank 2 and so on. Only a limited number of entries can be showed at a time, so we imagine the entries divided into pages. The function html-front-page below receives three arguments: a page number, the rank of the first entry of the page in question and the list of entries
(define (html-front-page page-number rank-of-first-entry entries)
(html-page
#:body `(div
,(html-menu)
,(html-list-of-entries page-number rank-of-first-entry entries))))
The function html-menu generates an X-expression for the "menu" on top of the screen The function html-list-of-entries generates an X-expression representing the list of entries.


The menu

The "menu" consists so far of a single menu item "submit-new-link".
(define (html-menu)
`(a ((href "control.scm?action=submitnew")) "submit-new-link"))
The list of entries

The list of entries is received by html-list-of-entries as a list of vectors.

(#4("entry_id" "title" "url" "score")
#4("1" "Everything Scheme" "http://www.scheme.dk/blog/" "42")
#4("3" "PLT Scheme" "http://www.plt-scheme.org" "9")
...)

The first vector holds the column names in the database, and can thus safely be ignored.
(define (html-list-of-entries page-number rank-of-first-entry entries)
`(div ((class "entries"))
,@(list-ec
(if (not (null? entries)))
(:list entry (index i) (cdr entries))
(:match #(id header url score) entry)
`(table ...))))

A DIV of class "entries" is used around the list of entries, so we single them out in the stylesheet. Next we use an eager comprehension list-ec to generate a list of X-expressions, one for each entry. Since we want the result to be of the form (div (class "entries") (table ...) (table ...) ...) and not (div (class "entries") ((table ...) (table ...) ...)) we use unquote-splicing ,@ instead of just unquote.

The eager comprehension first checks to see, whether entries is the empty list - and if so, an empty list of X-expressions is generated.

The clause (list entry (index i) (cdr entries)) iterates over the elements of the list entries (minus the first element), binding entry to each element in turn, also the index variable i counts from 0 and up.

The clause (:match #(id header url score) entry) uses pattern matching to bind the variables id, header, url, and score to the elements of the entry vector.

The final expression `(table ...) can now use the variables to generate the X-expression for the entry in question.

The table is used to position the rank, the up- and down-arrow and the link. I am able to do the same in CSS - but not in all browsers at the same time.
`(table ((class "entry"))
(tr (td ((class "rank")) ,(number->string (+ i rank-of-first-entry)))
(td ,(let ((form (format "arrowform~a" id)))
(html-form form "control.scm"
#:atts '((class "arrows"))
(html-input "arrowitem" #:type 'hidden)
(html-input "entry_id" #:type 'hidden #:value id)
(html-input "action" #:type 'hidden #:value "updown")
`(div ,(html-a-submit form "arrowitem" "up"
(html-icon 'go-up #:class "arrow")))
,(html-a-submit form "arrowitem" "down"
(html-icon 'go-down #:class "arrow")))))
(td (div (a ((href ,url)) ,header))
(span ((class "score")) "score: " ,score)))))))
The alert reader will probably wonder, why the form is introduced. Why aren't the arrows two simple links to, say,

control.scm?action=updown&arrowitem=down&entry_id=1 ?

Well, a click at the above url would result in the entry getting a vote. The problem lies in what happens next. If the user now reloads the page, the entry will get an extra vote. Bookingmarking the page will also result in extra votes, when the user returns.

The form-solution sends the parameters "behind the scenes", so that the url becomes control.scm. This obviously solves the bookmarking problem. The reload problem is solved via the Post/Redirect/Get pattern - which I'll discuss in detail in the post on the controller.

The function html-a-submit builds a link that submits the form (it's in the web-framework), it involves a little JavaScript.
(define (html-a-submit formname formitem id text)
`(a ((href ,(string-append
(format "javascript:document.~a.~a.value='~a';" formname formitem id)
(format "document.~a.submit();" formname))))
,text))
The function html-icon returns an X-expression `(img ...) representing various images. In this tutorial only the up- and down-arrow is supported, but if anyone is interested, I have a version supporting all of Tango (an excellent source of icons).

The Program

;;; view.scm  --  Jens Axel Soegaard

(module view mzscheme
(provide (all-defined))

(require (lib "kw.ss")
(planet "42.ss" ("soegaard" "srfi.plt"))
(planet "html.scm" ("soegaard" "web.plt" 1 0))
(planet "web.scm" ("soegaard" "web.plt" 1 0)))

;;;
;;; SITE WIDE DEFAULTS
;;;

(override-default current-page-title "List it!")
(override-default current-page-header '(h1 ((class "page_header")) "List it!"))
(override-default current-page-style-sheet "http://localhost/stylesheet.css")

;;;
;;; FRONT PAGE(S)
;;;

(define (html-front-page page-number rank-of-first-entry entries)
(html-page
#:body `(div
,(html-menu)
,(html-list-of-entries page-number rank-of-first-entry entries))))

(define (html-menu)
`(a ((href "control.scm?action=submitnew")) "submit-new-link"))

(define (html-submit-new-page)
(html-page
#:title "List it! - submit"
#:header '(h1 "List it!")
#:body
`(div (h2 "Submit a new entry")
,(html-form
"submitnewform" "control.scm"
(html-input "action" #:value "submit" #:type 'hidden)
`(table (tr (td "url") (td ,(html-input "url" #:type 'text #:value "http://")))
(tr (td "title") (td ,(html-input "title" #:type 'text #:value "A title"))))
(html-input "submit" #:value "submit")))))

(define/kw (html-icon name #:key (class #f))
(define (icon-absolute-url name)
(format "/~a.png" name))
(if class
`(img ((class ,class) (src ,(icon-absolute-url name))))
`(img ( (src ,(icon-absolute-url name))))))

(define (html-list-of-entries page-number rank-of-first-entry entries)
`(div ((class "entries"))
,@(list-ec
(if (not (null? entries)))
(:list entry (index i) (cdr entries))
(:match #(id header url score) entry)
`(table ((class "entry"))
(tr (td ((class "rank")) ,(number->string (+ i rank-of-first-entry)))
(td ,(let ((form (format "arrowform~a" id)))
(html-form form "control.scm"
#:atts '((class "arrows"))
(html-input "arrowitem" #:type 'hidden)
(html-input "entry_id" #:type 'hidden #:value id)
(html-input "action" #:type 'hidden #:value "updown")
`(div ,(html-a-submit form "arrowitem" "up" (html-icon 'go-up #:class "arrow")))
(html-a-submit form "arrowitem" "down" (html-icon 'go-down #:class "arrow")))))
(td (div (a ((href ,url)) ,header))
(span ((class "score")) "score: " ,score)))))))

; html-redirect-page
; a standard text to show, when redirecting
(define (html-redirect-page body)
(html-page #:title "Redirecting"
#:body body))

)

Saturday, January 20, 2007

An Introduction to Web Development with PLT Scheme

Introduction

From time to time people ask how to develop for the web with PLT Scheme on the PLT mailing list. The quick answer is "Just as in any other language", but that's not how to get people hooked on Scheme. To write a decent web-application require knowledge of a range of subjects such as HTML, databases, servlets, and web-servers. For some reason there is a lack of tutorials on these subjects, so I have decided to make an attempt at writing, if not a complete tutorial, then an elaborate get-started example.

The example application will be a mini version of Reddit called ListIt. The front page consists of a list of links to interesting articles, users can vote the articles up and down, and submit new articles. The hope is that the example is small enough to be easily understood, but on the other hand large enough to illustrate as many aspects as possible. Please leave comments on the blog: Did the example hit home? Is a paragraph in need of a rewrite? Did I skip something?

Model-View-controller

First things first. How should the program be organized? There is no need to reinvent the wheel, so I have chosen to use the Model-View-Controller architecture, which works just as well for web applications as it does for graphical user interfaces.

In a nutshell the Model-View-Controller architecture works like this: The model holds the data, the view displays data. User interactions goes through the controller in order to keep a separation between the model and the view. (See section 22.3 of HTDP or Wikipedia for more on MVC ).

In our case we will represent the model, the view and the controllers as three separate Scheme modules. The model will use a database to hold the links, the view will consists of functions generating HTML and the controller will the web-servlet that reacts on the user actions.

The Model

Today we will concentrate on the model. Each entry in our database consists of an entry-id , a title to display, an url to the article and a score representing the votes. Since we expect many entries in our database, we will think of them as divided into page. The number of entries in each page is given by the parameter PAGE-LIMIT.

The interface to our model consists of the following functions:

insert-entry: title url score -> entry-id
Insert a new entry into the database.

increase-score : entry-id ->
Increase the score of an existing entry

decrease-score : entry-id ->
Decrease the score of an existing entry

top : natural -> (list (list entry-id title url score))
Return the given number of entries with the highest scores

page : natural -> (list (list entry-id title url score))
Return the list of entries in the given page.

url-in-db? : url -> boolean
Is the url already listed?

These functions are the only ones to be exposed to the controller.

Implementation of the model

To implement these functions we will use an SQLite database. It wouldn't be unreasonable to argue that it would be easier to use a hash-table, but I want to illustrate how to use SQLite.

SQLite is small database engine, which comes in the form of a single self-contained, zero-configuration DLL-file on Windows or a a so-file on other platforms. We will use Jay McCarthy and Noel Welsh's PLT Scheme bindings sqlite.plt . On top of these binding we'll use a S-expression to SQL-string library written by me (it will appear on PLaneT soon - it has been submitted). On Windows you download SQLite by pasting the following into the DrScheme interaction window (the REPL):

(require (planet "download-sqlite.scm"
("soegaard" "sqlite.plt" 1 0))


To use the two SQLite packages, we start our module with

(require

(planet "sqlite.ss" ("jaymccarthy" "sqlite.plt"))
(planet "sqlite.ss" ("soegaard" "sqlite.plt" 1 0)))

Opening the database is simple:

(define db (open (string->path "c:/listit.sb")))
.

At least it will be, after it is created. The following function creates an empty database with a single table "entries":

(define (create-table-entries)
(exec/ignore
db
#<<SQL
CREATE TABLE entries (
entry_id INTEGER PRIMARY KEY,
title TEXT,
url TEXT,
score INTEGER )
SQL
))


The #<< starts a so called here-string. The function exec/ignore executes an SQL-statement and ignores the result (there is no S-expression syntax for CREATE TABLE yet).

Once we have created the our table, we can begin writing the functions in our interface.
The first is:

(define (insert-entry title url score)
(insert db (sql (INSERT INTO entries (title url score)
VALUES (,title ,url ,score)))))


The macro sql converts an S-expression representation of an SQL-statement into a string, which is then handed to SQLite by insert. The string produced by the sql macro from
(insert-entry "Everything Scheme" "http://www.scheme.dk/blog/" 42)
becomes
"INSERT INTO entries (title, url, score) VALUES ('Everything Scheme', 'http://www.scheme.dk/blog/', '42')".

The remaining functions from the interface are all simple SQL-statements, which can be studied in the full program below.

A loose end: In the source below the parameter current-database is used to hold the database. As a convenience I have with the help of syntax-id-rules defined the identifier db to expand to (current-database). But in order make everything work also as when the database isn't created yet, the actual definition below is a little more involved.

Testing

To test the model, open the "model.scm" in DrScheme. In the "Language" menu use "Choose Language" to choose the "Module" language. Click and "Run" and you are ready to test it:

Welcome to DrScheme, version 369.3-svn10jan2007.
Language: (module ...).
> (insert-entry "Everything Scheme" "http://www.scheme.dk/blog/" 42)
1
> (insert-entry "Reddit" "http://www.reddit.com" 7)
2
> (insert-entry "PLT Scheme" "http://www.plt-scheme.org" 5)
3
> (top 2)
(#4("entry_id" "title" "url" "score")
#4("1" "Everything Scheme" "http://www.scheme.dk/blog/" "42")
#4("2" "Reddit" "http://www.reddit.com" "7"))
> (increase-score 3)
> (increase-score 3)
> (increase-score 3)
> (increase-score 3)
> (top 2)
(#4("entry_id" "title" "url" "score")
#4("1" "Everything Scheme" "http://www.scheme.dk/blog/" "42")
#4("3" "PLT Scheme" "http://www.plt-scheme.org" "9"))
>


The Program


;;; model.scm  -- Jens Axel Søgaard

(module model mzscheme
(provide (all-defined))

(require
(planet "sqlite.ss" ("jaymccarthy" "sqlite.plt"))
(planet "sqlite.ss" ("soegaard" "sqlite.plt" 1 0)))

;;; CONFIGURATION

(define PAGE-LIMIT (make-parameter 50)) ; number of entries on each page
(define DATABASE-PATH (string->path "listit.db"))
; initialization of the db happens at the first run, see bottom of this file
(define current-database (make-parameter #f))

;;; CONVENIENCE

; define db to be short for (current-database)
(define-syntax db
(syntax-id-rules () [db (or (current-database)
(let ([d (open DATABASE-PATH)])
(current-database d)
d))]))

;;; DATABASE CREATION

(define (create-table-entries)
(exec/ignore
db
#<<SQL
CREATE TABLE entries (
entry_id INTEGER PRIMARY KEY,
title TEXT,
url TEXT,
score INTEGER )
SQL
))

(define (drop-table-entries)
(exec/ignore db "DROP TABLE entries"))


;; DATABASE INSERTION AND UPDATES

(define (insert-entry title url score)
(insert db (sql (INSERT INTO entries (title url score)
VALUES (,title ,url ,score)))))

(define (increase-score entry-id)
(update db (sql (UPDATE entries
SET (score = (+ score 1))
WHERE (= entry_id ,entry-id)))))

(define (decrease-score entry-id)
(update db (sql (UPDATE entries
SET (score = (- score 1))
WHERE (= entry_id ,entry-id)))))


;;; DATABASE RETRIEVAL

(define (top n)
(select db (sql (SELECT (entry_id title url score)
FROM entries
ORDER-BY (score DESC)
LIMIT ,n))))

(define (page n)
(select db (sql (SELECT (entry_id title url score)
FROM entries
ORDER-BY (score DESC)
LIMIT ,(PAGE-LIMIT) OFFSET ,(* (PAGE-LIMIT) n)))))

(define (entries-with-url url-str)
(select db (sql (SELECT (entry_id title url score)
FROM entries
WHERE ,(format "url='~a'" url-str)))))

(define (url-in-db? url-str)
(let ([result (entries-with-url db url-str)])
(if (null? (entries-with-url db url-str))
#f
result)))


;;; INITIALIZATION

; on first run, create tables
(unless (and (file-exists? DATABASE-PATH)
(table-exists? db "'entries'"))
(create-table-entries)
(current-database (make-parameter (open DATABASE-PATH)))))

Labels: , , , , ,