Saturday, January 20, 2007

An Introduction to Web Development with PLT Scheme

Introduction

From time to time people ask how to develop for the web with PLT Scheme on the PLT mailing list. The quick answer is "Just as in any other language", but that's not how to get people hooked on Scheme. To write a decent web-application require knowledge of a range of subjects such as HTML, databases, servlets, and web-servers. For some reason there is a lack of tutorials on these subjects, so I have decided to make an attempt at writing, if not a complete tutorial, then an elaborate get-started example.

The example application will be a mini version of Reddit called ListIt. The front page consists of a list of links to interesting articles, users can vote the articles up and down, and submit new articles. The hope is that the example is small enough to be easily understood, but on the other hand large enough to illustrate as many aspects as possible. Please leave comments on the blog: Did the example hit home? Is a paragraph in need of a rewrite? Did I skip something?

Model-View-controller

First things first. How should the program be organized? There is no need to reinvent the wheel, so I have chosen to use the Model-View-Controller architecture, which works just as well for web applications as it does for graphical user interfaces.

In a nutshell the Model-View-Controller architecture works like this: The model holds the data, the view displays data. User interactions goes through the controller in order to keep a separation between the model and the view. (See section 22.3 of HTDP or Wikipedia for more on MVC ).

In our case we will represent the model, the view and the controllers as three separate Scheme modules. The model will use a database to hold the links, the view will consists of functions generating HTML and the controller will the web-servlet that reacts on the user actions.

The Model

Today we will concentrate on the model. Each entry in our database consists of an entry-id , a title to display, an url to the article and a score representing the votes. Since we expect many entries in our database, we will think of them as divided into page. The number of entries in each page is given by the parameter PAGE-LIMIT.

The interface to our model consists of the following functions:

insert-entry: title url score -> entry-id
Insert a new entry into the database.

increase-score : entry-id ->
Increase the score of an existing entry

decrease-score : entry-id ->
Decrease the score of an existing entry

top : natural -> (list (list entry-id title url score))
Return the given number of entries with the highest scores

page : natural -> (list (list entry-id title url score))
Return the list of entries in the given page.

url-in-db? : url -> boolean
Is the url already listed?

These functions are the only ones to be exposed to the controller.

Implementation of the model

To implement these functions we will use an SQLite database. It wouldn't be unreasonable to argue that it would be easier to use a hash-table, but I want to illustrate how to use SQLite.

SQLite is small database engine, which comes in the form of a single self-contained, zero-configuration DLL-file on Windows or a a so-file on other platforms. We will use Jay McCarthy and Noel Welsh's PLT Scheme bindings sqlite.plt . On top of these binding we'll use a S-expression to SQL-string library written by me (it will appear on PLaneT soon - it has been submitted). On Windows you download SQLite by pasting the following into the DrScheme interaction window (the REPL):

(require (planet "download-sqlite.scm"
("soegaard" "sqlite.plt" 1 0))


To use the two SQLite packages, we start our module with

(require

(planet "sqlite.ss" ("jaymccarthy" "sqlite.plt"))
(planet "sqlite.ss" ("soegaard" "sqlite.plt" 1 0)))

Opening the database is simple:

(define db (open (string->path "c:/listit.sb")))
.

At least it will be, after it is created. The following function creates an empty database with a single table "entries":

(define (create-table-entries)
(exec/ignore
db
#<<SQL
CREATE TABLE entries (
entry_id INTEGER PRIMARY KEY,
title TEXT,
url TEXT,
score INTEGER )
SQL
))


The #<< starts a so called here-string. The function exec/ignore executes an SQL-statement and ignores the result (there is no S-expression syntax for CREATE TABLE yet).

Once we have created the our table, we can begin writing the functions in our interface.
The first is:

(define (insert-entry title url score)
(insert db (sql (INSERT INTO entries (title url score)
VALUES (,title ,url ,score)))))


The macro sql converts an S-expression representation of an SQL-statement into a string, which is then handed to SQLite by insert. The string produced by the sql macro from
(insert-entry "Everything Scheme" "http://www.scheme.dk/blog/" 42)
becomes
"INSERT INTO entries (title, url, score) VALUES ('Everything Scheme', 'http://www.scheme.dk/blog/', '42')".

The remaining functions from the interface are all simple SQL-statements, which can be studied in the full program below.

A loose end: In the source below the parameter current-database is used to hold the database. As a convenience I have with the help of syntax-id-rules defined the identifier db to expand to (current-database). But in order make everything work also as when the database isn't created yet, the actual definition below is a little more involved.

Testing

To test the model, open the "model.scm" in DrScheme. In the "Language" menu use "Choose Language" to choose the "Module" language. Click and "Run" and you are ready to test it:

Welcome to DrScheme, version 369.3-svn10jan2007.
Language: (module ...).
> (insert-entry "Everything Scheme" "http://www.scheme.dk/blog/" 42)
1
> (insert-entry "Reddit" "http://www.reddit.com" 7)
2
> (insert-entry "PLT Scheme" "http://www.plt-scheme.org" 5)
3
> (top 2)
(#4("entry_id" "title" "url" "score")
#4("1" "Everything Scheme" "http://www.scheme.dk/blog/" "42")
#4("2" "Reddit" "http://www.reddit.com" "7"))
> (increase-score 3)
> (increase-score 3)
> (increase-score 3)
> (increase-score 3)
> (top 2)
(#4("entry_id" "title" "url" "score")
#4("1" "Everything Scheme" "http://www.scheme.dk/blog/" "42")
#4("3" "PLT Scheme" "http://www.plt-scheme.org" "9"))
>


The Program


;;; model.scm  -- Jens Axel Søgaard

(module model mzscheme
(provide (all-defined))

(require
(planet "sqlite.ss" ("jaymccarthy" "sqlite.plt"))
(planet "sqlite.ss" ("soegaard" "sqlite.plt" 1 0)))

;;; CONFIGURATION

(define PAGE-LIMIT (make-parameter 50)) ; number of entries on each page
(define DATABASE-PATH (string->path "listit.db"))
; initialization of the db happens at the first run, see bottom of this file
(define current-database (make-parameter #f))

;;; CONVENIENCE

; define db to be short for (current-database)
(define-syntax db
(syntax-id-rules () [db (or (current-database)
(let ([d (open DATABASE-PATH)])
(current-database d)
d))]))

;;; DATABASE CREATION

(define (create-table-entries)
(exec/ignore
db
#<<SQL
CREATE TABLE entries (
entry_id INTEGER PRIMARY KEY,
title TEXT,
url TEXT,
score INTEGER )
SQL
))

(define (drop-table-entries)
(exec/ignore db "DROP TABLE entries"))


;; DATABASE INSERTION AND UPDATES

(define (insert-entry title url score)
(insert db (sql (INSERT INTO entries (title url score)
VALUES (,title ,url ,score)))))

(define (increase-score entry-id)
(update db (sql (UPDATE entries
SET (score = (+ score 1))
WHERE (= entry_id ,entry-id)))))

(define (decrease-score entry-id)
(update db (sql (UPDATE entries
SET (score = (- score 1))
WHERE (= entry_id ,entry-id)))))


;;; DATABASE RETRIEVAL

(define (top n)
(select db (sql (SELECT (entry_id title url score)
FROM entries
ORDER-BY (score DESC)
LIMIT ,n))))

(define (page n)
(select db (sql (SELECT (entry_id title url score)
FROM entries
ORDER-BY (score DESC)
LIMIT ,(PAGE-LIMIT) OFFSET ,(* (PAGE-LIMIT) n)))))

(define (entries-with-url url-str)
(select db (sql (SELECT (entry_id title url score)
FROM entries
WHERE ,(format "url='~a'" url-str)))))

(define (url-in-db? url-str)
(let ([result (entries-with-url db url-str)])
(if (null? (entries-with-url db url-str))
#f
result)))


;;; INITIALIZATION

; on first run, create tables
(unless (and (file-exists? DATABASE-PATH)
(table-exists? db "'entries'"))
(create-table-entries)
(current-database (make-parameter (open DATABASE-PATH)))))

Labels: , , , , ,

13 Comments:

Blogger Jay McCarthy said...

Very cool! I hope to see the other entries.

16:05  
Blogger Jens Axel Søgaard said...

Thanks. As you probably guessed the next two posts will be about the implementation of the view and the controller.

22:36  
Blogger joorce said...

It's nice to see "real" code in scheme.

Looking forward the other entries too.

23:52  
Blogger Geoff Knauth said...

Oh so readable! You make it look easy. If I could use your code for my PostgreSQL and MySQL databases, I think I could switch my personal DB webapps from Java/Python to Scheme in no time. I especially appreciate your sql macro for constructing SQL strings. Does your macro handle embedded single quotes and double quotes? What if I have a string that looks like an sexp (is surrounded by parens)? Are there problems in the encoding or decoding of those?

Thanks for the tip about #<< here-strings!

20:27  
Blogger James said...

Great article! I am looking forward to the future installments. Any possibility you can illustrate file upload and security/session handling. Maybe fastcgi config too too ;^).

14:19  
Blogger Jens Axel Søgaard said...

Hi Geoff,

Thanks. The intention is that the sql macro handles converting strings automatically. However you can use prepare to "compile" your SQL-statement with parameters marked with "?" and then later apply the statement to actual parameters. In this way, there is no risk of SQL injections.

I'll post an example later on.

17:50  
Blogger Jens Axel Søgaard said...

Hi James,

Thanks for the suggestion. I might write about file upload and sessions. Since I'm using the PLT web-server, I have no need for FastCGI. Do I guess correctly that you want to use Apace? If so note, that you can use Apache and the PLT Scheme web-server at the same time.
See for example this discussion on using the PLT Web Server and Apache at the same time in the mailing list archive.

18:00  
Blogger Ramon said...

Love the article, there's not enough examples of Scheme web apps out there. Looking forward to further installments.

05:59  
Blogger pragmatic said...

Thanks for the example. Can't wait to see more of them!

07:05  
Blogger rohan said...

Thanks jens,

I have just stumbled on this while looking for info on web dev with
plt scheme.

Some things I noticed while overcoming bumps in getting it to run:
1. here-strings: It is great to see these in the plt language as they
are so useful. But as with here strings in other languages making
sure that the terminator is not indented, nor has trailing spaces
is crucial. I was bitten by this when getting the example up and
running. Emacs indenting is a blessing most of the time. ;-)

2. I am guessing, but it would seem that you must use absolute paths
for the path to the sqlite file. I tried something like this:
(string->path "~/devel/scheme/database.sb") and it failed, but
hardcoding an absolute path seemed to do the trick.

But, now it is running fabulously, and I love the schemey sql
notation; very nice.

I have been really enjoying your blog, so many useful bits and pieces,
and thanks for all the work you put into it.

Looking forward to reading the next installment of the plt web
dev. tutorial.

Rohan

P.S. I am very glad you included using a relational database backend,
makes this example much more relevant for all those people who need to
use such things for getting work done, or who are just very used to
using them. It also nicely demonstrated how easily you can have plt up and running
with a SQL system. :-)

10:55  
Blogger Pierre said...

I love your programming style, I intend to try to learn from it. I've been coding scheme for a couple of years, but I still haven't quite got the hang of picking exactly the right mix of macros and runtime. I think emulating this style of yours will help me.

20:26  
Blogger Jens Axel Søgaard said...

Pierre,

Thanks for the nice words.

I intend to update the tutorial to work with PLT v400 soonish, so stay tuned.

/Jens Axel

20:44  
Blogger Jay Godse said...

You can see a pile of SQLite related videos at YouTube which show how to use and program against SQLite.

16:10  

Post a Comment

Links to this post:

Create a Link

<< Home