Author: Wesley J. Chun
Pub Date: 2012
Size: 10 Mb
Already know Python but want to learn more? A lot more? Dive into a variety of topics used in practice for real-world applications. Covers regular expressions, Internet/network programming, GUIs, SQL/databases/ORMs, threading, and Web development. Learn about contemporary development trends such as Google+, Twitter, MongoDB, OAuth, Python 3 migration, and Java/Jython. Presents brand new material on Django, Google App Engine, CSV/JSON/XML, and Microsoft Office. Includes Python 2 and 3 code samples to get you started right away! Provides code snippets, interactive examples, and practical exercises to help build your Python skills.
Middleware and Wrapping WSGI Applications
There might be situations in which you want to let the application run asis, but you want to inject pre or post-processing before (the request) or after the application executes (the response). This is commonly known as middleware, which is additional functionality that sits between the Web server and the Web application. You’re either massaging the data coming from the user before passing it to the application, or you need to do some final tweaks to the results from the application before returning the payload back to the user. This is commonly referred to as a middleware onion, indicating the application is at the heart, with additional layers in between.
Preprocessing can include activities, such as intercepting the request parameters; modifying them; adding or removing them; altering the environment (including any user-submitted form [CGI] variables); using the URL path to dispatch application functionality; forwarding or redirecting requests; load-balancing based on network traffic via the inbound client IP address; delegating to altered functionality (e.g., using the User-Agent header to send mobile users to a simplified UI/app); etc.
Examples of post-processing primarily involves manipulating the output from the application. The following script is an example, similar to the timestamp server that we created in Chapter 2, “Network Programming”: for each line from the application’s results, we’re going to prepend it with a timestamp. In practice of course, this is much more complicated, but this is an example similar to others you can find online that capitalize or lowercase application output. Here, we’ll wrap our call to simple_wsgi_app() with ts_simple_wsgi_app()and install the latter as the application that the server registers:
Updates to WSGI in Python 3
PEP 333 defined the WSGI standard for Python 2. PEP 3333 offers enhances to PEP 333 to bring the standard to Python 3. Specifically, it calls out that the network traffic is all done in bytes. While such strings are native to Python 2, native Python 3 strings are Unicode to emphasize that they represent text data while the original ASCII strings were renamed to the bytestype.
Specifically, PEP 3333 clarifies that “native” strings—the data type named str, regardless of whether you’re using Python 2 or 3—are those
used for all HTTP headers and corresponding metadata. It also states that “byte” strings are those which are used for the HTTP payloads (requests/responses, GET/POST/PUT input data, HTML output, etc.). For more information on PEP 333, take a look at its definition, which you can find at www.python.org/dev/peps/pep-3333/.
Independent of PEP 3333, there are other related proposals that will make for good reading. One is PEP 444, which is a first attempt to define a “WSGI 2,” if such a thing takes on that name. The community generally regards PEP 3333 as a “WSGI 1.0.1,” an enhancement to the original
PEP333 specification, whereas PEP 444 is a consideration for WSGI’s next generation.
Real-World Web Development
CGI was the way things used to work, and the concepts it brought still apply in Web programming today; hence, the reason why we spent so
much time looking at it. The introduction to WSGI brought you one step closer to reality.
Today, new Python Web programmers have a wealth of choices, and while the big names in the Web framework space are still Django, Pyramid, and Google App Engine, there are plenty more options for users to choose from—perhaps a mind-numbing selection, actually. Frameworks aren’t even necessary: you could go straight down to a WSGI-compliant Web server without any of the extra “fluff ” or framework features. However, the chances are more likely that you will go with a framework because of the convenience of having the rest of the Web stack available to you.
A modern Web execution environment will likely consist of either a multithreaded or multiprocess server model, signed/secure cookies, basic
user authentication, and session management. Many of these things regular application developers already know; authentication represents user registration with a login name and password, and cookies are ways of maintaining user information, sometimes session information, as well. We also know that in order to scale, Web servers need to be able to handle requests from multiple users; hence, the use of threads or processes. However, one thing that hasn’t been covered is the need for sessions. If you look at all the application code in this entire chapter that runs on Web servers, it might take a while for you to know that aside from the obvious differences from scripts that run from beginning to end or server loops which just run forever, Web applications (or servletsin Java parlance) are executed for every request. There’s no state saved within the code, and we already mentioned that HTTP is stateless, as well. In other words, don’t expect data to be saved in variables, global or otherwise. Think of a request like a single transaction. It comes in, does its business, and finishes, leaving nothing behind in the codebase.
This is why session management—saving of a user’s state across one or more requests within a well-defined duration of time—is needed. Generally, this is accomplished by using some sort of persistent storage, such as memcache, flat (or not-so-flat) files, and even databases. Developers can certainly roll their own, especially when writing lower-level code, as we’ve seen in this chapter. But without question this wheel has already been (re)invented several times, which is why many of the larger, more well-known Web frameworks, including Django, come with their own session management software. (This leads directly into our next chapter.)