Web development with Python
|
|
Introduction
Primarily two kinds of tools: Those that are just called through CGI and
its derivatives, and those that launch their own server which may include an HTTP
server or can be called from a web server. The former are easier to set up and
don't require anything else besides the web server (possibly with extra modules
like mod_fastcgi to improve performance), but they seem to offer less features
than application servers.
Why dump PHP and write web apps as long-running processes?
"PHP is an embarrassment, a blight upon my craft. It’s so broken, but
so lauded by every empowered amateur who’s yet to learn anything else, as to
be maddening. It has paltry few redeeming qualities and I would prefer to forget
it exists at all. But I’ve got to get this out of my system. So here goes, one
last try." http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/
From comp.lang.python: "A long running process has lots of benefits
that makes design and development easier and makes your app faster.
- Code is loaded/parsed/compiled only once and kept in memory (but check
for memory leaks)
- Don't underestimate the costs of process creation. It doesn't scale
unless you fork() childs which is more complex and still doesn't scale as
well as a long running process.
- you can do all costly and time consuming operations on startup like
loading all configuration files and databases like i18n
stuff.
- performance tweaks like database connection pool works only with a long
running process.
- in-process caching is usually easier to implement and faster. In some
cases only in-process works.
- you can have background tasks without the requirement of an outside
service like cron + wget.
- async server based on a select()-like reactor are only possible when
a single process handles the incoming connections.
- A long running process is more powerful and the 'enterprise' way of
doing it right. All important and large scale Python and Java solutions
are using long running processes instead of a CGI-like model.
- PHP's way is a poor man's solution to compensate for bad design and
memory holes. PHP's ancestor PHTML still bleeds through and makes you suffer
for mistakes of the past."
"Plus, if you manage it right, you have a guarantee that you can never
be in a half-updated state - that's somewhat tricky when you have inter-dependencies
in PHP code and the file system itself manages things. What happens if a request
comes in while you're half-way through uploading new code to the server? By
default, you could use half old code and half new code. Keeping everything in
memory makes it easier to prevent that."
Why use a web framework?
Some of the benefits of using a web framework (from http://cherrypy.org/):
- A fast, HTTP/1.1-compliant, WSGI thread-pooled webserver.
- Easy to run multiple HTTP servers (e.g. on multiple ports) at once.
- A powerful configuration system for developers and deployers alike.
- A flexible plugin system.
- Built-in tools for caching, encoding, sessions, authorization, static
content, and many more.
- Swappable and customizable...everything.
- Built-in profiling, coverage, and testing support.
- Runs on Python 2.5+, 3.1+, Jython and Android.
Another take:
- CRSF protection
- keeping things DRY
- templating (or you could use many others)
- user management
- administrative interface
- database creation/introspection
- i18n
- an ecosystem of pluggable add-on apps
- URL routing
- view decorators
- easily swappable back-ends
- active development across multiple lines of business
- GIS support
- abstracted ORM (or you could use SQLObject or its kin) to allow you
mobility between DB back-ends should you want to
"I would recommend Django as it seems to scale up nicely and I know
that it has an active community which can be a godsend for getting help as a
"newbie". That being said it does have a bit of a learning curve compared
to some of the lighter frameworks, but not so much of one as to counteract the
advantage of scalability. Again, this is all based on opinions that I have read
(and remember) and not on any facts.
Maybe this
article will help you. The comments
on /. should round up anything missing from the article (I hope).
CGI, FastCGI/SCGI, WSGI, mod_python?
In a nutshell:
- mod_python is a module for Apache that mimicked mod_php, but is pretty much
deprecated
- CGI is the first, language-neutral way to have a web server run an application.
Because it launches a process for each request, it's only recommended in small
applications
- FastCGI and SCGI (similar to FastCGI but easier to implement)
are enhanced alternatives to CGI, namely long-running processes
so the host doesn't waste time relaunching the web application (and interpreter,
in the case of Python) with each request
- WSGI
is a Python-specific interface
specification that documents the way the application (your Python app)
and its container (Flup, gunicorn, uWSGI, etc.) speak to each other, and
is available for the major players (Apache, Nginx,
Lighttpd, Cherokee, Varnish, etc.). WSGI doesn't cover how the container speaks
to the outside world (HTTP, FastCGI, etc.) This is the prefered option to run Python applications.
CGI Sample1
Once Python is installed, here's how to configure .htaccess to let Apache
run Python scripts:
- Options -Indexes +ExecCGI
- AddHandler cgi-script .py
Next, create the following hello.py in your web
server's cgi-bin/ directory (Note: If editing on a Windows host before uploading
the file on a Linux host, watch out for CRLF end characters: Run "dos2unix"
to take care of this):
- #!/usr/bin/env python
-
- # enable debugging
- import cgitb
- cgitb.enable()
-
- print "Content-Type: text/plain"
- print
- print "Hello, World!"
Run "chmod 755" on it, and aim at http://localhost/cgi-bin/hello.py
CGI Sample2
This shows how to handle missing items in a form, and the importance of using
a framework to separate code and HTML output:
- #!/usr/bin/env python
-
- # Import modules for CGI handling
- import cgi, cgitb, traceback, sys
-
- cgitb.enable()
-
- # Create instance of FieldStorage
- form = cgi.FieldStorage()
-
- print "Content-Type: text/html;charset=iso-8859-1\n\n"
-
- try:
- #OK name = form.getvalue('name')
- name = form['name'].value
- mail = form['mail'].value
- tel = form['tel'].value
- except Exception:
- #pass
- #traceback.print_exc(file=sys.stdout)
- print "Error in
script."
- sys.exit(0)
-
- header = """
- <html>
- <head>
- <meta http-equiv="content-type"
content="text/html; charset=iso-8859-1" />
- <title>My site</title>
- <link href="/style.css" rel="stylesheet"
type="text/css" />
- </head>
-
- <body>"""
-
- footer = """
- </body>
- </html>"""
-
- #print "Name is %s" % (cgi.escape(name), )
-
- print header
-
- print "Hello"
-
- print footer
WSGI
For simple, test applications, you can use Python's (2.5 and later) included
web server and WSGI server through a package named wsgiref. If you want to access the script from a remote host,
make sure the
script listens on the host's IP address instead of "localhost".
As explained, simply run "python environment.py" and aim your browser
at http://localhost:8051.
For real world applications, the recommended solution is running Apache and its mod_wsgi module in daemon
mode : "The preferred mode
of running a WSGI application under mod_wsgi if you have no idea of how to setup
and tune Apache to match your specific web application, is daemon mode. Even
if you reckon you do know how to setup Apache it is still safer to use daemon
mode. Even if your WSGI application is running in daemon mode, if the only thing
you are using Apache for is to host the WSGI application and serve static files,
then it is also recommended that you use worker MPM rather than prefork MPM
as worker MPM will cut down on memory use by the Apache child processes."
("Why
are you using embedded mode of mod_wsgi?")
Here's a list of
WSGI servers which looks pretty thorough.
Note that two mod_wsgi modules are available: The one for Apache is by Graham
Dumpleton and is still active and recommended, while the one for Nginx by
Manlio Perillo isn't
as stable, is no longer developed and is not recommended ("Blocking
requests and nginx version of mod_wsgi").
As of 0.8.40, Nginx supports the uwsgi protocol by default. As
for Lighttpd...
WSGI is Python only,
and is a specification (API) for web servers and application servers to communicate
with web applications. Rather than a language agnostic protocol, a standard
function signature is defined:
- def simple_app(environ, start_response):
- """Simplest possible application
object"""
- status = '200 OK'
- response_headers = [('Content-type','text/plain')]
- start_response(status, response_headers)
- return ['Hello world!\n']
That is a complete (if limited) WSGI application. A web server with WSGI
support (such as Apache with mod_wsgi) can invoke this function whenever a request
arrives. Note that the uwsgi server implemented by the uWSGI project is no longer
Python-specific (it can also handle Perl, Ruby, Lua, etc.)
The reason this is so great is that we can avoid the messy step of converting
from a HTTP GET/POST to CGI to Python, and back again on the way out. It's a
much more direct, clean and efficient linkage." (source)
"WSGI is a Python calling convention. Flask and other frameworks provide
a callable object that takes 'environ' and 'start_headers' parameters and return
an iterable of bytes. Various WSGI servers can call these objects to make requests.
http://www.python.org/dev/peps/pep-3333/
Some servers like gunicorn speak HTTP themselves on one side, and directly
WSGI on the other side. Others like flup, rather than HTTP, speak FastCGI or
SCGI (which are network protocols) with a front-end server like Apache, lighttpd
or nginx.
Apache’s mod_wsgi is a bit special: both the Apache integration and the worker
processes (assuming daemon mode) are managed by mod_wsgi. They speak an internal/private
protocol together." (source)
"Note that although mod_wsgi has features similar to FASTCGI/SCGI solutions,
it isn't intended to be a replacement for those hosting mechanisms in all situations
for Python web hosting. Specifically, mod_wsgi is not designed for nor intended
for use in over allocated shared mass virtual hosting setups for different users
on a single Apache instance" (http://code.google.com/p/modwsgi/)
Here's how to install and configure Nginx and uWSGI (which is faster than gunicorn and better
integrated with Nginx) on a Debian host:
- apt-get update
- apt-get install nginx
Check if the uWSGI package is available, and its version:
- vi /etc/apt/sources.list:
deb http://ftp.fr.debian.org/debian
testing main
deb http://ftp.debian.org/debian/ wheezy-updates main
- apt-get install uwsgi uwsgi-plugin-python
If the packages are a bit old, and you need more recent features,
you'll have to compile the latest source, either manually or through pip:
- apt-get install python python-dev
- pip install http://projects.unbit.it/downloads/uwsgi-lts.tar.gz
Alternatively, some users recommend running Python through virtualenv and
installing uWSGI therein:
- apt-get install python-virtualenv
- virtualenv env
- env/bin/pip install http://projects.unbit.it/downloads/uwsgi-lts.tar.gz
"Nginx natively includes support for upstream servers speaking the uwsgi
protocol since version 0.8.40. If you are unfortunate enough to use an older
version (that nevertheless is 0.7.63 or newer), you can find a module in the
nginx directory of the uWSGI distribution." http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html
- Upgrade Nginx from 0.7.63 to a more recent release (at least 0.8.40)
- Figure out how to install uWSGI
- Figure out how to configure Nginx to speak to uWSGI
- Figure out how to write a Python application an run it through Nginx
+ uWSGI
FCGI and some light framework
Since mod_python is Apache-specific and hasn't been updated in a while, and
CGI is slow for bigger sites, FastCGI
was invented: In this case, the application is compiled and runs as a stand-alone
application which is called by the web server through a Unix/TCP port. On Apache,
FastCGI is available through either mod_fastcgi or mod_fcgid ("mod_fcgi
is NOT a replacement for mod_fastcgi").
Note that FastCGI will keep a process in RAM either until a timeout is reached
or the script has answered so many requests. Then, FastCGI will kill the process
and start a new process. So if you're using a shared host, it's a better idea
to test new scripts on a local server where you're free to restart scripts at
will, and only upload scripts once they're good to go.
Here's an example of a basic script running on a shared host that uses mod_fcgid
+ Flup to run a Python script:
- #!/usr/bin/env python2.6
-
- import cgitb
-
- # enable debugging
- cgitb.enable()
-
- def myapp(environ, start_response):
- start_response('200
OK', [('Content-Type', 'text/plain')])
- return ['Done!\n']
-
- if __name__ == '__main__':
- from flup.server.fcgi
import WSGIServer
- WSGIServer(myapp).run()
Tips
- To find which version of Apache is running on a shared server (might
not work on a shared server): httpd -v.
- If uploading a Python web app on a shared host, if you need to install extra
Python software, you might need to use sys.path/$PYTHONPATH so Python can locate
them. Also take a look at virtualenv
to help with organizing the local python modules.
- In case you use a shared server that relies on FastCGI, check that Flup
isn't already installed. If it is, all you have to do is use fcgi.py and
your Python script. Check whether to use "#!/usr/bin/env python2.6"
or "#!/usr/bin/env python"
- When called from mod_fcgid through eg. Flup, a Python web script
must be made executable (chmod +x myscript.py). This is not necessary when
called from mod_wsgi.
- If editing a Python script on a Windows host and uploading it to run
on a Linux host, watch out for the line endings (CRFL vs. LF).
- If more than one version of Python is installed, you can either use
"#!/usr/bin/env python" to use the first one available, or use
"#!/usr/bin/env python2.6" to specifically use release 2.6.
- To debug a Python web application, log on to the host, and run the script
by hand, eg. "python ./application.fcgi". This doesn't catch all errors,
so check the server logs for more information.
- The web server error log is usually located in /var/log... which might
be off-limit on a shared host.
- Note that a FastCGI script isn't being stopped and replaced right away
by a new version that you uploaded, which is another reason to test a script
on a host where you have admin control before uploading it to a shared host
- For debugging, use the logging module + logging handler instead of relying
on stderr/stdout
- Nginx/Cherokee + uWSGI
(confusingly, uWSGI is the server while uwsgi
is a binary protocol a bit faster than http) is supposed to be an easier
way to run Python web scripts than Apache + mod_wsgi (and much better than
Apache + mod_fcgid + Flup).
Nginx + mod_wsgi is not recommended (Blocking
requests and nginx version of mod_wsgi compares Graham Dumpleton's mod_wsgi
for Apache with Manlio Perillo's mod_wsgi
for nginx)
- "The Gunicorn
"Green Unicorn" is a Python Web Server Gateway Interface HTTP
Server for Unix. It is a pre-fork worker model, ported from Ruby's Unicorn
project. The Gunicorn server is broadly compatible with a number of web
application frameworks, simply implemented, light on server resources and
fairly fast."
Further Reading
WSGI Frameworks
http://bottlepy.org/docs/dev/
To read
DONE python
& fcgi verses mod_python
DONE Plain
old Python with FastGCI
DONE HOWTO Use Python in
the web
DONE (uses mod_fastcgi) Apache,
FastCGI and Python by Ferry Boender, version 1.0, Feb 12, 2009
DONE (doesn't deal with how to use multiple DB servers in write mode) Scaling Python for High-Load
Web Sites
DONE The
Web Server Benchmarking We Need
OLD Cameron
Laird's personal notes on Python and the Web
DONE How to
serve a WSGI application via CGI
DONE Getting
Python to work (CGI)
DONE Python
- CGI Programming
DONE Five
minutes to a Python CGI
DONE 4.4.
CGI - Dynamic Web Pages
DONE Programming
CGI With Python
DONE http://stackoverflow.com/questions/219110/how-python-web-frameworks-wsgi-and-cgi-fit-together
DONE http://stackoverflow.com/questions/257481/whats-the-difference-between-scgi-and-wsgi
DONE http://stackoverflow.com/questions/1747266/is-there-a-speed-difference-between-wsgi-and-fcgi
DONE http://www.vmunix.com/mark/blog/archives/2006/01/02/fastcgi-scgi-and-apache-background-and-future/
DONE http://helpful.knobs-dials.com/index.php/CGI,_FastCGI,_SCGI,_WSGI,_servlets_and_such
DONE http://www.subspacefield.org/~travis/python_web_frameworks.html
DONE http://geekfault.org/2010/03/01/nginx-et-python-le-perfect-setup/
Which solution to run Python web apps?
Ideally, write the web application with a framework that supports WSGI, and
configure the web server to support WSGI directly instead of using SCGI/FastCGI and
some wrapper to WSGI.
Apache
mod_wsgi
mod_fcgid + wrapper (Flup, uWSGI,
etc.)
Here's how to connect mod_fcgid to a WSGI application:
Cherokee
Includes support for uWSGI
Lighttpd
Nginx
FastCGI = slow
mod_wsgi for Nginx = deadware, slow
uWSGI = alternative to mod_wsgi and requires recompiling Nginx
gunicorn = basic HTTP server that supports WSGI; less features than uWSGI,
but doesn't require recompiling Nginx
TurboGears
Django
- http://snakelets.sourceforge.net/
- "Snakelets is a Python web application server. This project provides a
threaded web server, Ypages (HTML+Python language, similar to Java's JSPs) and
Snakelets: code-centric page request handlers (similar to Java's Servlets). It
is able to run off a CD (read-only mode).
Snakelet's focus is on understanding the way dynamic web sites are created
(the code is very compact and easy to understand), and make this process as easy
as possible, while still providing a rich and powerful server environment."
- http://www.4suite.org/
- "an open-source platform for XML and RDF processing"
- Not sure if it can be used as a stand-alone tool to build web applications,
or can requires additional stuff
- "Unlike most other Python application servers, like Zope and Webware,
SkunkWeb forks processes. This ensures reliability (a child process can't
take down the whole server) and scalability (SkunkWeb can use all the processors
it can get its hands on)."
- Templating language is STML, which looks about as sophisticated as Zope's
ZPT... Too sophisticated for a templating language maybe
- SkunkWeb components come in several flavors: includes, regular components,
and data components. Components are just pages containing Python or STML
code. SkunkWeb servers can talk to each other, and send components to one
another - you can have for example, a SkunkWeb server just do database connections,
from which other SkunkWeb servers talk to this one server here.
- Databases : SkunkWEB supports many databases. It can support any
database for which there is existing Python support, but for optimum performance
and connection caching the following three are best supported: MySQL,
PostgreSQL and Oracle (and soon FirebirdSQL.) SkunkWEB also has an
included (though separately available) object-relational interface called
PyDO. PyDO is lightweight, somewhat transparent and easy to use.
In addition to the above databases, it also supports SapDB and sqlite.
- Can run either its own HTTP server or be called from Apache through
mod_skunkweb (Apache talks to the daemon either by mod_skunkweb or by a
CGI)
- Some items are URL-accessible (html, py), while others will return a
404 if accessed (inc, pyinc, etc.)
- "A distinction is drawn between url-accessible, or "top-level"
documents, and the various types of subsidiary components that can be referenced
from them. The main difference between them is that top-level executable
documents are executed in a Python namespace with an object in it called
CONNECTION, which serves the same roles typically played by Request and
Response objects in frameworks like Java Servlets, Zope, ASP, etc; subsidiary
documents may not have that object in their namespace, for reasons which
will become clear."
- All these components are compiled to Python bytecode, and their compiled
representations are stored in a configurably located directory called the
compileCacheRoot (and, by default but depending on SkunkWeb's configuration,
in memory).
- http://www.twistedmatrix.com
- "an event-driven networking framework written in Python".
Pure-Python framework for programming network services and applications
- Twisted's "official" templating system is Woven. However,
Twisted also works with Quixote and it is quite possible to adapt other
frameworks to it.
- Network programming with the Twisted framework (Part 1,
2,
3,
and 4)
- Twisted Python by Stephen
Figgins
- An Introduction to the Twisted
Networking Framework by Itamar Shtull-Trauring
- http://www.movatis.com/crusader/
- Got this e-mails when asking for tutorials: "The version we're
using is stable and reliable. But as there was very little response from
the open source community the Sourceforge CVS and RPMs are out of date.
Probably there are too many open source application servers around and we
didn't market the project well enough (we're not a venture capital backed
company like Zope). Actually we're planning to close the project on Sourceforge
and come up with a brand new version later. So currently I must recommend
against using Crusader."
- Didn't find any tutorial
- "Crusader is a powerful application server for Python based upon
a scaleable, extensible and easy-to-use general purpose server framework.
- Also offers SpellCaster to write XML-RPC-callable components, and SnakeCharmer
which is an advanced PSP implementation
- "There are currently only Linux-RPMs available but Crusader also
runs on Windows and integrates nicely with NTs service model. We will supply
a windows setup soon."
- "Snake Charmer is a powerful Python Server Pages implementation
based on the Crusader Server Framework. Snake Charmer allows you to mix
HTML and Python in a ASP/JSP/PHP way, send HTTP redirects and status codes,
access GET/POST data, supports distributed session management, application
scope instances, file inclusion, precompiling, caching and more."
- http://draco.boskant.nl/
- Looks very slick, but very little traffic in the mailing
list
- Seems similar to CherryPy, but only requires mod_python and an SQL server,
ie. no server. Seems to only run under Apache
- "All handler code is grouped per directory under the document root
and put in a file named ‘ handler .py’1. This file is a Python source file
which we’ll call the directory handler. In this directory handler we can
define handler functions for templates in that directory. To define a handler
function for a template, you need to subclass the class Handler (defined
in draco.handler), and put it in a file named ‘ handler .py’. Methods defined
in this handler class correspond to templates with the same name plus the
default extension."
- http://www.object-craft.com.au
(jump to here
to get a glimpse of what an appplication in Albatross looks like)
- "The ability to place Python code for each page in a dynamically
loaded module, or to place each page in its own class in a single mainline"
- "Applications that can be deployed as either CGI programs or as
mod_python module with minor changes to program mainline"
- (From happy user Igor Sidorenkov) "It is not ModularSessionApp,
but RandomModularSessionApp in their application model that fits closely
to standard website organization approach. Just go through their Popview
example and rewrite Popview4 to use RandomModularSessionApp."
- Untar and run "python setup.py install"
- Create cgi-bin/simple.html :
<html>
<head>
<title>My
CGI application</title>
</head>
<body>
Hello
from my second simple CGI application!
</body>
</html>
- Create cgi-bin/simple.py :
#!/usr/bin/python
from albatross
import SimpleContext
ctx = SimpleContext('.')
templ = ctx.load_template('simple.html')
templ.to_html(ctx)
print
'Content-Type: text/html'
print
ctx.flush_content()
Resources
- Karrigell is a simple web
programming solution, under active development
- Embedded web and relational database server (Gadfly)
- Code can be either stand-alone Python or "Python inside HTML"
- Little but steady traffic in the mailing
list
Quixote puts all the code and HTML for a Web-based application into a Python
package containing .py and .ptl files, and provides a simple framework for publishing
code and objects on the Web.
There are three components to a Quixote app:
- A driver script, usually a CGI or FastCGI script, acting as the interface
between the web server and your Quixote application. Here's a basic example:
from
quixote import enable_ptl, Publisher
enable_ptl()
app = Publisher("quixote.demo")
app.setup_logs()
app.publish_cgi()
- A configuration file
- Your application in a Python package or module
An application's code lives in a Python package that contains both .py and
.ptl files. Complicated logic should be in .py files, while .ptl files,
ideally, should contain only the logic needed to render your Web interface and
basic objects as HTML. As long as your driver script calls enable_ptl(),
you can import PTL modules (.ptl files) just as if they were Python modules.
Quixote's publisher will start at the root of this package, and will treat
the rest of the URL as a path into the package's contents. Here are some
examples, assuming that the URL_PREFIX is "/q", your web server is
setup to rewrite /q requests as calls to (eg.) /www/cgi-bin/books.cgi, and the
root package for your application is 'books'. For instance, a call to http://localhost/g
calls books._q_index().
PTL, Python Template Language, is used to mix HTML with Python code. An import
hook is defined so that PTL files can be imported just like Python modules.
The basic syntax of PTL is the same as Python's, with a few small changes, as
seen in this example:
- template barebones_header(title=None,
- description=None):
- """
- <html><head>
- <title>%s</title>
- """ % html_quote(str(title))
- if description:
- '<meta name="description"
content="%s">' % html_quote(description)
- '</head><body bgcolor="#ffffff">'
Templates in PTL are analogous to Python's functions. Templates are
defined using the template keyword, obviously, and they can't have docstrings.
The difference between a template and a regular Python function is that inside
a template the results of expressions are accumulated, and the return value
of the template is simply all those results concatenated together
Will work with any Web server that supports CGI, FastCGI, SCGI protocol
(note that although the mod_python module for Apache 2 should work on Windows,
the Python SCGI package will not due to Unix-specific APIs like fork(), etc.),
along with Apache's mod_python and AOLserver's PyWX. Provides its own templating
language called PTL (Python Template Language). Quixote can be restarted automatically
by Apache. Thus, Quixote has the advantage of providing a OO-based model to
develop web applications without requiring an application server if you don't
want to run any (in which case, you'll use CGI or mod_python) or run one if
you want to increase performance (FastCGI or SCGI). In case you
wish to use ZODB to ensure persistance, take a look at Dulcinea,
a collection of modules useful for developing applications with
Quixote and the ZODB.
Setup
- Install Python. If you already have Python installed and it is older
than 2.2, run cd /tmp/Python-2.1.2/Tools/compiler ; python setup.py install
- Download and untar the Quixote package, and run python setup.py install.
FYI, stuff is installed under d:\Python22\Lib\site-packages\quixote\
- Open a terminal window, and run the following commands to check that
Quixote was installed correctly:
$ python
>>> import
quixote
>>> quixote.enable_ptl()
>>> import quixote.demo
- From the original package, copy demo/demo.cgi and demo/demo.conf into
your web server's cgi-bin directory. I prefer to create a sub-directory
cgi-bin/quixote
- Open demo.cgi, and edit the shebbang line to point to where the python
interpreter is located. For instance:
#!d:\Python22\python.exe
- Open demo.conf, and change the paths to fit your setup, eg.
DEBUG_LOG
= "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-debug.log"
ERROR_LOG
= "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-error.log"
- Aim at http://localhost/cgi-bin/quixote/demo.cgi
The page is actually displayed through the "app = Publisher('quixote.demo')" line
in demo.cgi, which calls the _q_index(request) method of the pages.ptl template
located in the Quixote demo package that was installed previously (eg. d:\Python22\Lib\site-packages\quixote\demo\).
Programming
Every namespace (package or module) that Quixote uses must supply two special
identifiers: _q_index() and _q_exports. _q_index is supplied in the _init_.py
module in a package, eg. splat/web/__init__.py includes this line:
- from splat.web.index import _q_index
_q_index is equivalent to index.html, and is a callable, ie. a function,
a method, or a PTL template, which returns the default page for a namespace.
_q_exports is used to explicitely tell Quixote which routines are callable,
ie. can be accessed through a URL. This is a way to make sure users don't start
calling routines that should remain off-limit. Here's an example:
- #splat/web/__init__.py:
- _q_exports = ['about']
_q_getname() can be used as a fallback before returning a 404. This is also
convenient when you wish to map a URL to a virtual object, ie. object publishing.
For instance, you could use _q_getname() to map the URL http://localhost/bug/0124
to a bug entry in the SQL server:
- from quixote.errors import TraversalError
- from splat.web.util import get_bug_database
- [...]
- def _q_getname (request, name):
- try:
- bug_id = int(name)
- except ValueError:
- raise TraversalError("invalid
bug ID: %r (not an integer)" % name)
- bug_db = get_bug_database()
- bug = bug_db.get_bug(bug_id)
- if bug is None:
- raise TraversalError("no
such bug: %r" % bug_id)
- return header("Bug %s" % bug) + format_bug_page(bug)
+ footer()
A "callable" is a function that can be called through a URL, eg.
http://localhost/widgets/ . Quixote
requires you to be explicit; a _q_exports variable lists all the methods that
are public. Methods not listed in _q_exports are private. Quixote puts all the
code and HTML for a Web-based application into a Python package containing .py
and .ptl files, and provides a simple framework for publishing code and objects
on the Web. For example, the MEMS Exchange interface lives in a package nmed
mems.ui. Once our Web server has been configured to direct all requests to Quixote,
accessing http://fab/ will run the function mems.ui._q_index. Accessing http://fab/run/
runs mems.ui.run._q_index .
PTL, Python Template Language, is used to mix HTML with Python code. An import
hook is defined so that PTL files can be imported just like Python modules.
Inside a PTL template, you can use all of Python's control-flow statements.
There are three components to a Quixote application:
- A driver script, usually a CGI or FastCGI script. This is the
interface between your web server (eg., Apache) and the bulk of your application
code, eg. demo.cgi. The driver script is responsible for creating a Quixote
publisher customized for your application and invoking its publishing loop.
- A configuration file. This file specifies various features of
the Publisher class, such as how errors are handled, the paths of various
log files, and various other things. Read through quixote/config.py for
the full list of configuration settings. The most important configuration
parameters are:
ERROR_EMAIL e-mail address to which errors will be mailed
ERROR_LOG
file to which errors will be logged
For development/debugging, you
should also set DISPLAY_EXCEPTIONS true and SECURE_ERRORS false; the defaults
are the reverse, to favour security over convenience.
- Finally, the bulk of the code will be in a Python package or module,
called the root namespace. The Quixote publisher will be set up to
start traversing at the root namespace.
cgi-bin/quixote/demo.cgi
- #!d:\Python22\python.exe
- # Example driver script for the Quixote demo: publishes the contents
of the quixote.demo package.
- from quixote import enable_ptl, Publisher
- # Install the import hook that enables PTL modules.
- enable_ptl()
- # Create a Publisher instance
- app = Publisher('quixote.demo')
- # (Optional step) Read a configuration file
- app.read_config("demo.conf")
- # Open the configured log files
- app.setup_logs()
- # Enter the publishing main loop
- app.publish_cgi()
d:\Python22\Lib\site-packages\quixote\demo\__init__.py
- _q_exports = ["simple", "error", "widgets",
"form_demo"]
-
- import sys
- from quixote.demo.pages import _q_index
- from quixote.demo.widgets import widgets
- from quixote.demo.forms import form_demo
- from quixote.demo.integer_ui import IntegerUI
-
- def simple (request):
- # This function returns a plain text document,
not HTML.
- request.response.set_content_type("text/plain")
- return "This is the Python function 'quixote.demo.simple'.\n"
-
- def error (request):
- raise ValueError, "this is a Python exception"
-
- def _q_getname(request, component):
- return IntegerUI(request, component)
- a URL maps to either a namespace (package, module, class instance) or
a callable (function, method, PTL template)
- if a URL maps to a namespace, Quixote looks for a callable _q_index()
in that namespace and calls it
- _q_index() doesn't have to be explicitly exported by your namespace;
if it exists, it will be used
- anything your application prints to standard output goes to Quixote's
debug log. (If you didn't specify a debug log in your config file, debug
messages are discarded.)
_q_exports = ["simple", "error"]
This means that only these two names are explicitly exported by the Quixote
demo. (The empty string is implicitly exported from a namespace if a _q_index()
callable exists there -- thus "/qdemo/" is handled by _q_index() in
the "quixote.demo" namespace. Arbitrary names may be implicitly exported
using a _q_getname() function; see Lesson 4 below.)
Q&A
What does python setup.py install actually do?
How can I improve performance?
Set up your web server to handle Python scripts : CGI, FastCGI, SCGI,
Apache's mod_python, AOLServer's PyWX
Does it run on Windows?
We are mainly familiar with Unix, and develop and deploy Quixote under
Linux. However, we've had several reports of people using Quixote under Windows,
more-or-less successfully. There are still a few Unix-isms in the code, but
they are being rooted out in favour of portability.
Resources
Moved to its own page.
A framework available from http://webware.sourceforge.net/.
The main component is WebKit,
which is an application server that sits between the web server and your
WebWare-based Python application. The reason for an independ server along
with a web server is the same as eg. CherryPy: A smarter web server is required
to build OO web applications, ie. a web server that just delivers files from
a filesystem is not adapter to OO programming.
Click on the picture below to see where WebWare stands in the architecture
(diagram taken from Jason Hildebrand's Introduction
to Webware: Developing Web Applications in Python)
WebKit
To install WebKit on a Windows host...
- Check that the version of Python you installed supports thread:
- python
- import thread
- If you get an error message, you'll have to recompile Python
with support for threads
- Download and unpack WebWare
- cd \path\to\Webware
- python install.py
- (enter password; As indicated, the password is saved in clear text
in Webware 0.8/WebKit/Configs/Application.config)
- Install the CGI adapter
- If using Apache: The mod_webkit is the fastest way to
connect to Webware, and is only supported for Apache. CGI is also
available to Apache users through wkcgi (see below)
- If using another web server: From the WebWare package, copy
the wkcgi (WebKit CGI adapter) wkcgi.exe which sits between the
web server and the WebKit application server: copy WebKit\Adapters\wkcgi\wkcgi.exe
\path\to\cgi-bin
- Go to the WebKit sub-directory in the WebWare package, and run appserver.bat.
The AppServer is a program that must be running anytime you want to
access your servlets or PSP files. By default, WebKit will try to talk
to the AppServer on host localhost on port 8086 (you can run "netstat
-an" to check that an an application is listening on TCP 8086)
- http://localhost/cgi-bin/wkcgi.exe/
Testing Webware
(Not sure at this point what I'm doing... I got the following instructions
here)
To build your own Webware server:
- Cd to the Webware package, and run "python bin/MakeAppWorkDir.py c:/tmp/webware",
where c:/tmp/webware is where Webware will install its directory tree
- cd c:/tmp/webware
- Start the server with "./AppServer"
- Copy the CGI adapter: cp [webware package]\WebKit\Adapters\wkcgi\wkcgi.exe
C:\www\cgi-bin\
- Aim your browser to http://localhost/cgi-bin/wkcgi.exe
You should see "Welcome to Webware!", which is actually c:\tmp\webware\MyContext\Main.py
Hello, world!
- from WebKit.Servlet import Servlet
- class Hello(Servlet):
- def respond(self, trans):
- trans.response().write('Content-type:
text/html\n\nHello, world!\n')
Resources
Q&A
What does "python install.py" actually do?
How does running "http://localhost/cgi-bin/wkcgi.exe/" ends up
displaying stuff in the Examples/ directory?
What is CGIWrapper?
So far, CGIWrapper has received little attention among the Webware community.
Most people gravitate to its alternative, WebKit, which is described later.
http://webware.sourceforge.net/Papers/IntroToWebware.html
Where do I go after running "python install" in the WebWare package?
I followed the directions, and ran "python bin/MakeAppWorkDir.py c:/tmp/webware",
followed by "cd c:/tmp/webware", "./AppServer" to launch
the server, copied C:\tmp\webware\WebKit.cgi into my web server's cgi-bin/ directory,
and aimed at http://localhost/cgi-bin/WebKit.cgi/:
- ERROR Traceback (most recent call last):
- File ".\WebKit\Adapters\CGIAdapter.py", line 48,
in run
- IOError: [Errno 9] Bad file descriptor
Using http://localhost/cgi-bin/WebKit.cgi/MyContext/Main returns the same
thing.
WebKit.cgi doesn't
work under Windows. Use wkcgi.exe instead, ie. once the Webware server is running,
aim at http://localhost/cgi-bin/wkcgi.exe
- http://jonpy.sf.net/
- Mailing
list filled with spam
- Unzip the package, and run "python setup.py install", which compiles
and copies stuff in python\Lib\site-packages\jon
- The jon package includes the following modules which can be imported
via eg. "import jon.cgi as cgi":
- cgi (abstraction layer for CGI-style applications), modpy (a connector to use the abstraction layer with mod_python),
and fcgi (a connector to use the abstraction layer with the FastCGI
protocol)
- mime - a simple MIME parser for reading multipart/form-data HTTP
requests
- wt - a simple yet extremely powerful web templating system
- session - session management for HTTP requests
- dbpool - a database connection pool
Hello, world!
After installing the Jonpy package, just create this hello.py file in the
cgi-bin/ directory of your web server and aim at http://localhost/cgi-bin/hello.py:
- #!C:\Python22\python.exe
-
- import jon.cgi as cgi
-
- class Handler(cgi.Handler):
- def process(self, req):
- req.set_header("Content-Type",
"text/plain")
- req.write("<html><body>Hello,
%s!\n</body></html>" % req.params.get("greet",
"world"))
-
- cgi.CGIRequest(Handler).process()
Important: If copy/pasting this sample, it might not run due to invisible
characters chocking the Python interpreter. Just remove and re-add leading tabs
on each line, and give it another go.
CGI Module
Code to handle a request must subclass the abstract class cgi.Handler .
- import jon.cgi as cgi
-
- class Handler(cgi.Handler):
- def process(self, req):
- req.set_header("Content-Type", "text/plain")
- req.write("Hello, %s!\n" % req.params.get("greet",
"world"))
-
- cgi.CGIRequest(Handler).process()
This class uses a single method process which receives a single parameter
of type cgi.Request, which is used to retrieve information about the request
and to send the response. A subclass of cgi.Request is used to call the handler.
Which subclass is used depends on the protocol used to communicate with the
web server. This module provides cgi.CGIRequest which implements the standard
CGI protocol.
Globals
Functions
- html_encode(raw)
- url_encode(raw)
- url_decode(enc)
- traceback(req, html=0)
Classes
Error(Exception)
SequencingError(Error)
Request
Public Instance Variables
- params
- stdin
- cookies
- environ
- aborted
Protected Instance Variables
Public Methods
- __init__(self, handler_type)
- output_headers(self)
- clear_headers(self)
- add_header(self, hdr, val)
- set_header(self, hdr, val)
- del_header(self, hdr)
- set_buffering(self, f)
- flush(self)
- clear_output(self)
- error(self, s)
- write(self, s)
- traceback(self)
Protected Methods
- _init(sself)
- _write(self, s)
- _flush(self)
- _mergevars(self, encoded)
- _mergemime(self, contenttype, encoded)
- _read_cgi_data(self, environ, inf)
CGIRequest(Request)
Handler
Public Methods
- process(self, req)
- traceback(self, req)
DebugHandlerMixIn
DebugHandler(DebugHandlerMixIn, Handler)
modpy module
Classes
Error(Exception)
Request(cgi.Request)
fcgi module
Classes
Server
Public Methods
- __init__(self, handler_types, max_requests=0, params=None, request_type=Request,
threading_level=1)
- exit(self)
- run(self)
Request(cgi.Request)
mime module
Classes
Error(Exception)
Entity(rfc822.Message)
wt module
Classes
TemplateCode
GlobalTemplate(TemplateCode)
Public Class Variables
Public Methods
Handler(cgi.Handler)
DebugHandler(cgi.DebugHandlerMixIn, Handler)
session module
Classes
Error(Exception)
Session(dict)
Public Instance Variables
- relocated
- created
- new
- surl
Public Methods
- __init__(self, req, secret, cookie="jonsid", url=0, root="",
referer=None, sid=None, shash=None)
- save(self)
Public Static Methods
Protected Instance Variables
Protected Methods
- _make_hash(self, sid, secret)
- _create(self, secret)
- _load(self)
FileSession(Session)
Public Methods
- __init__(self, req, secret, basedir=None, **kwargs)
- save(self)
Public Static Methods
- tidy(max_idle=0, max_age=0, basedir=None)
Protected Methods
- _create(self, secret)
- _load(self)
SQLSession(Session)
Public Methods
- __init__(self, req, secret, dbc, table="sessions", **kwargs)
- save(self)
Public Static Methods
- tidy(dbc, table="sessions", max_idle=0, max_age=0)
Protected Methods
- _create(self, secret)
- _load(self)
dbpool module
Globals
Variables
- apilevel
- threadsafety
- paramstyle
- Warning
- Error
- InterfaceError
- DatabaseError
- DataError
- OperationalError
- IntegrityError
- InternalError
- ProgrammingError
- NotSupportedError
Functions
- set_database(dbmod, minconns)
- connect(...)
Q&A
ISAPI
Nufox
"Nufox is a
python XUL toolkit written ontop of Twisted and Nevow"
Resources