Flea documentation contents¶

Flea¶

Flea is a library that helps you write functional tests for WSGI applications.

Flea uses CSS selectors and XPath to give you a simple but powerful tool to navigate and test the HTML responses generated by your WSGI web applications. Here’s an example of how easy it is to test a WSGI application:

>>> r = TestAgent(my_wsgi_app).get('/')
>>> print r.body
<html>
        <body>
                <a href="/sign-in">sign in</a>
        </body>
</html>
>>> r = r.click('sign in')
>>> r = r("form#login-form").fill(
...     username = 'root',
...     password = 'secret',
... ).submit()
>>> assert 'login successful' in r.body

Overview¶

The TestAgent class provides a user agent that drives a WSGI application:

>>> from flea import TestAgent
>>> agent = TestAgent(my_wsgi_app)

You can now use this agent to navigate your WSGI application by...

...making GET requests:

>>> r = r.get('/my-page')

...making POST requests:

>>> r = r.post('/contact', data={'message': 'your father smells of elderberries'})

...clicking links:

>>> # Click on a link with content 'foo'
>>> r = r.click("foo")

>>> # Click a link matching a regular expression
>>> import re
>>> r = r.click(re.compile('f.*o'))

>>> # Find a link using a CSS selector
>>> r = r("a#mylink").click()

>>> # Or an XPath expression
>>> r = r("//a[@id='mylink']").click()

...and submitting forms:

>>> r = r("form[name=login-form]").fill(username='me', password='123').submit()
>>> r = r("form[name=contact] button[name=send]").submit()

Finding elements¶

There are several methods for traversing the DOM. The simplest is usually to use CSS selectors:

>>> r.css("a.highlighted")
[<a class="highlighted"...>]

For more complex requirements you can also use XPath, with find() or with dictionary-style access:

>>> r.find("//a[@class='highlighted']")
[<a class="highlighted"...>]
>>> r["//a[@class='highlighted']"]
[<a class="highlighted"...>]

You can also call the TestAgent directly, passing either an XPath expression or a CSS selector. Flea will autosense the expression type:

>>> r("a.highlighted")
[<a class="highlighted"...>]
>>> r("//a[@class='highlighted']")
[<a class="highlighted"...>]

If an expression could be interpreted as both a valid XPath and CSS selector, flea defaults to ‘css’. You can force an expression to be interpreted as one or the other by passing a flavor argument:

>>> r("a.highlighted", 'css')
[<a class="highlighted"...>]
>>> r("//a[@class='highlighted']", 'xpath')
[<a class="highlighted"...>]

Filling and submitting forms¶

Although you can fill fields by altering the necessary DOM properties: checked (checkboxes, radio buttons), selected (select options), text (textareas) and value for other input types, it’s usually more convenient to use the fill() method, which presents a common interface to all control types.

When you fill in form fields, the underlying DOM is updated. This makes it really easy to check your form is correctly filled while developing tests:

>>> app = Response([
...     '<html>'
...     '<form>'
...         '<input type="text" name="subject" />'
...         '<textarea name="message"/>'
...     '</form>'
...     '</html>'
... ])
>>> r = TestAgent(app).get('/')
>>> r('form').fill(subject='hello', message='how are you?')
<...>

>>> # Display the updated HTML
>>> # You could also use r.serve() to interact with the completed form in a web browser
>>> r('form').html()
'<form><input type="text" name="subject" value="hello"><textarea name="message">how are you?</textarea></form>'

Text inputs and textareas:

>>> app = Response([
...     '<html>'
...     '<form>'
...         '<input type="text" name="subject" />'
...         '<textarea name="message"/>'
...     '</form>'
...     '</html>'
... ])
>>> r = TestAgent(app).get('/')
>>> r('input[name=subject]').fill('hello')
<...>
>>> r('textarea[name=message]').fill('world')
<...>
>>> r('form').submit_data()
[('subject', 'hello'), ('message', 'world')]

Checkboxes:

>>> app = Response([
...     '<html>'
...     '<form>'
...         '<input type="checkbox" name="opt-in" value="yes" />'
...         '<input type="checkbox" name="items" value="one" />'
...         '<input type="checkbox" name="items" value="two" />'
...         '<input type="checkbox" name="items" value="three" />'
...     '</form>'
...     '</html>'
... ])
>>> r = TestAgent(app).get('/')
>>> r('input[name=opt-in]').fill(True)
<...>
>>> r('input[name=items]').fill(['two', 'three'])
<...>
>>> r('form').submit_data()
[('opt-in', 'yes'), ('items', 'two'), ('items', 'three')]

Radio buttons:

>>> app = Response([
...     '<html>'
...     '<form>'
...         '<input type="radio" name="item" value="one" />'
...         '<input type="radio" name="item" value="two" />'
...         '<input type="radio" name="item" value="three" />'
...     '</form>'
...     '</html>'
... ])
>>> r = TestAgent(app).get('/')
>>> r('input[name=item]').fill('two')
<...>
>>> r('form').submit_data()
[('item', 'two')]

Select boxes

>>> app = Response([
...     '<html>'
...     '<form>'
...         '<select name="icecream">'
...             '<option value="strawberry">strawberry</option>'
...             '<option value="vanilla">vanilla</option>'
...         '</select>'
...         '<select name="cake" multiple="">'
...             '<option value="chocolate">chocolate</option>'
...             '<option value="ginger">ginger</option>'
...             '<option value="coffee">coffee</option>'
...         '</select>'
...     '</form>'
...     '</html>'
... ])
>>> r = TestAgent(app).get('/')
>>> r('select[name="icecream"]').fill('strawberry')
<...>
>>> r('select[name="cake"]').fill(['chocolate', 'coffee'])
<...>
>>> r('form').submit_data()
[('icecream', 'strawberry'), ('cake', 'chocolate'), ('cake', 'coffee')]

File uploads¶

To test file upload fields, you must pass a tuple of (filename, content-type, data) to fill(). The data part can either be a string:

>>> r = TestAgent(Response([
...         '<html>'
...         '<form name="upload" action="/" enctype="multipart/form-data">'
...                 '<input type="file" name="image"/>'
...         '</form>'
...         '</html>'
... ])).get('/')
>>> r("input[name=image]").fill(('icon.png', 'image/jpeg', 'testdata'))
<...>

Or a file-like object:

from StringIO import StringIO
r("input[name=image]").fill(('icon.png', 'image/jpeg', StringIO('aaabbbccc')))

Filling forms in a single call¶

The fill() method, when called on a form element, is a useful shortcut to filling in an entire form with a single call. Keyword arguments are used to populate input controls by id or name:

r = TestAgent(Response([
    '<html>'
            '<form name="login-form">'
                    '<input type="text" name="username"/>'
                    '<input type="text" name="password"/>'
            '</form>'
    '</html>'
])).get('/')
r = r["//form[@name='login-form']"].fill(
    username='fred',
    password='secret'
).submit()

XPath or CSS selector expressions may be used for fields whose names can’t be represented as python identifiers or when you need more control over exactly which fields are selected:

r = r("form[name=login-form]").fill(
        ('.//input[1]', 'fred'),
        ('.//input[2]', 'secret'),
).submit()

HTTP redirects¶

HTTP redirect responses (301 or 302) are followed by default. If you want to explicitly check for a redirect, you’ll need to specify follow=False when making the request. All methods associated with making a request - click, submit, get, post etc - take this parameter.

To follow a redirect manually:

>>> r = TestAgent(testapp).get('/')
>>> r = r("form[name=register]").submit(follow=False)
>>> r.request.request_path
'/register'
>>> r.response.status_code
302
>>> r = r.follow()
>>> r.request.request_path
'/'
>>> r.response.status_code
200

Querying WSGI application responses¶

Checking the content of the request¶

>>> assert r.request.path_info == '/index.html'
>>> print r.request.environ
{...}

r.request is a pesto.request.Request object, and all attributes of that class are available to examine.

Checking the content of the response¶

>>> assert r.response.content_type == 'text/html'
>>> assert r.response.status == '200 OK'

Note that r.response is a pesto.testing.TestResponse object, and all attributes of that class are available.

By default, responses are checked for a successful status code (2xx or 3xx), and an exception is raised for any other status code. If you want to bypass this checking, use the check_status argument:

>>> def myapp(environ, start_response):
...     start_response('500 Error', [('Content-Type', 'text/plain')])
...     return ['Sorry, an error occurred']
...
>>> TestAgent(myapp).get('/')
Traceback (most recent call last):
...
BadStatusError: GET '/' returned HTTP status '500 Error'
>>> TestAgent(myapp).get('/', check_status=False)
<flea.TestAgent ...>

Checking returned content.¶

The .body property contains the raw response from the server:

.. doctest::

>>> r = TestAgent(Response(["<html><p><strong>How now</strong> brown cow</p></html>"])).get('/')
>>> assert 'cow' in r.body

Any element selected via an xpath query has various helper methods useful for inspecting the document.

Checking if strings are present in an HTML element¶

>>> assert 'cow' in r('p')

Accessing the html of selected elements¶

>>> r('//p[1]').html()
'<p><strong>How now</strong> brown cow</p>'

Note that this is the html parsed and reconstructed by lxml, so is unlikely to be the literal HTML emitted by your application - use body for that.

Accessing textual content of selected elements¶

The striptags() method removes all HTML tags and normalizes whitespace to make string comparisons easier:

>>> r = TestAgent(Response([
... """
...     <html>
...         <p>
...             <strong>How now</strong>
...             brown
...             cow
...          </p>
...    </html>
... """])).get('/')
>>> r('//p[1]').striptags()
' How now brown cow '

Inspecting and interacting with a web browser¶

Flea gives you two methods for viewing the application under test in a web browser.

The showbrowser method opens a web browser and displays the content of the most recently loaded request:

>>> r.get('/').showbrowser()

The serve method starts a HTTP server running your WSGI application and opens a web browser at the location corresponding to the most recent request. For example, the following code causes a web browser to open at http://localhost:8080/foobar:

>>> r.get('/foobar').serve()

If you want to change the default hostname and port for the webserver you must specify these when first initializing the TestAgent object:

>>> r = TestAgent(my_wsgi_app, host='192.168.1.1', port='8888')
>>> r.get('/foobar').serve()

Now the web browser would be opened at http://192.168.1.1:8888/foobar.

One final note: the first request to the application is handled by relaying the most recent response received to the web browser, including any cookies previously set by the application. Also, if any methods have been called that access the lxml representation of an HTML response – eg finding elements by an XPath query or filling form fields – then the lxml document in its current state will be converted to a string and served to the browser, meaning that while the document should be logically equivalent, it will no longer be a byte-for-byte copy of the response content received from the WSGI application.

This only applies to the first request, and ensures that the web browser receives a copy of the page as currently in memory, with any form fields filled in and with any cookies set so that you can pick up in your web browser exactly where the TestAgent object left off.

API documention¶

exception flea.BadStatusError¶: Raised when a non-success HTTP response is found (eg 404 or 500)

class flea.ElementWrapper(agent, element)¶

Wrapper for an lxml.etree element, providing additional methods useful for driving/testing WSGI applications. ElementWrapper objects are normally created through the find/css methods of TestAgent instance:

>>> from pesto.response import Response
>>> myapp = Response(['<html><body><a href="/">link 1</a><a href="/">link 2</a></body></html>'])
>>> agent = TestAgent(myapp).get('/')
>>> elementwrapper = agent.find('//a')[0]

ElementWrapper objects have many methods and properties implemented as XPathMultiMethod objects, meaning their behaviour varies depending on the type of element being wrapped. For example, form elements have a submit method, a elements have a click method, and input elements have a value property.

checked¶: Return True if the element has the checked attribute

click(*args, **kwargs)¶

For elements matching a[@href]:

Follow a link and return a new instance of TestAgent

For elements matching input[@type='submit' or @type='image']|button[@type='submit' or not(@type)]:

Alias for submit

css(selector)¶: Return elements matching the given CSS Selector (see lxml.cssselect for documentation on the CSSSelector class.

fill(*args, **kwargs)¶

For elements matching form:

Fill the current form with data.

param *args: Pairs of (selector, value)

param **kwargs: mappings of fieldname to value

See the documentation for _set_value() implementations for individual form control types to see how values are processed as this varies between text inputs, selects, radio buttons, checkboxes etc

For elements matching input[@type='radio']:

Set the value of the radio button, by searching for the radio button in the group with the given value and checking it.

For elements matching textarea:

Set the value of a textarea control

For elements matching input[@type='file']:

Set the value of a file input box

For elements matching input:

Set the value of a (text, password, ...) input box

For elements matching select[@multiple]:

Set the values of a multiple select box

param values: list of values to be selected

For elements matching select[not(@multiple)]:

Set the values of a multiple select box

param values: list of values to be selected

find(path, namespaces=None, **kwargs)¶

Return elements matching the given xpath expression.

If the xpath selects a list of elements a ResultWrapper object is returned.

If the xpath selects any other type (eg a string attribute value), the result of the query is returned directly.

For convenience that the EXSLT regular expression namespace (http://exslt.org/regular-expressions) is prebound to the prefix re.

form¶: For elements matching input|textarea|button|select|form:

Return the form associated with the wrapped element.

html()¶: Return an HTML representation of the element

input_group(*args, **kwargs)¶: For elements matching textarea|input|select:

Return the group of inputs sharing the same name attribute

pretty()¶: Return an pretty-printed string representation of the element

selected¶: For elements matching option:

Return True if the given select option is selected

striptags()¶

Strip tags out of the element and its children to leave only the textual content. Normalize all sequences of whitespace to a single space.

Use this for simple text comparisons when testing for document content

Example:

>>> from pesto.response import Response
>>> myapp = Response(['<p>the <span>foo</span> is completely <strong>b</strong>azzed</p>'])
>>> agent = TestAgent(myapp).get('/')
>>> agent['//p'].striptags()
'the foo is completely bazzed'

submit(*args, **kwargs)¶

For elements matching input[@type='submit' or @type='image']|button[@type='submit' or not(@type)]:

Submit the form, returning a new TestAgent object, by clicking on the selected submit element (input of type submit or image, or button with type submit)

For elements matching form:

Submit the form, returning a new TestAgent object

submit_data(*args, **kwargs)¶

For elements matching input[@type='submit' or @type='image']|button[@type='submit' or not(@type)]:

Return a list of the data that would be submitted to the server in the format [(key, value), ...], without actually submitting the form.

For elements matching form:

Return a list of the data that would be submitted to the server in the format [(key, value), ...], without actually submitting the form.

submit_value¶

For elements matching input[@type='checkbox']:

Return the value of the selected checkbox element as the user agent would return it to the server in a form submission.

For elements matching input[@type='radio']:

Return the value of the selected radio element as the user agent would return it to the server in a form submission.

For elements matching select[@multiple]:

Return the value of the selected radio/checkbox element as the user agent would return it to the server in a form submission.

For elements matching select[not(@multiple)]:

Return the value of the selected radio/checkbox element as the user agent would return it to the server in a form submission.

For elements matching input[not(@type) or @type != 'submit' and @type != 'image' and @type != 'reset']:

Return the value of any other input element as the user agent would return it to the server in a form submission.

For elements matching input[@type != 'submit' or @type != 'image' or @type != 'reset']:

Return the value of any submit/reset input element

For elements matching textarea:

Return the value of any submit/reset input element

value¶

For elements matching input[@type='file']:

Return the value of the file upload, which will be a tuple of (filename, content-type, data)

For elements matching input|button:

Return the value of the input or button element

exception flea.NotARedirectError¶: Raised when an attempt is made to call follow() on a non-redirected response

class flea.PassStateWSGIApp(testagent)¶

WSGI application that replays the TestAgent’s cookies and currently loaded response to the downstream UA on the first request, thereafter proxies requests to the agent’s associated wsgi application.

Used by TestAgent.serve.

class flea.ResultWrapper(elements, expr=None)¶

Wrap a list of elements (ElementWrapper objects) returned from an xpath query, providing reasonable default behaviour for testing.

ResultWrapper objects usually wrap ElementWrapper objects, which in turn wrap an lxml element and are normally created through the find/findcss methods of TestAgent:

>>> from pesto.response import Response
>>> myapp = Response(['<html><body><p>item 1</p><p>item 2</p></body></html>'])
>>> agent = TestAgent(myapp).get('/')
>>> resultwrapper = agent.find('//p')

ResultWrapper objects have list like behaviour:

>>> len(resultwrapper)
2
>>> resultwrapper[0] 
<ElementWrapper <Element p at ...>>

Attributes that are not part of the list interface are proxied to the first item in the result list for convenience. These two uses are equivalent:

>>> resultwrapper[0].text
'item 1'
>>> resultwrapper.text
'item 1'

Items in the ResultWrapper are ElementWrapper instances, which provide methods in addition to the normal lxml.element methods (eg click(), setting/getting form field values etc).

filter(matcher)¶: Return a new ResultWrapper of the elements in elements where applying the function matcher to the element results in a truth value.

filter_on_text(matcher)¶: Return a new ResultWrapper of the elements in elements where applying the function matcher to the text contained in the element results in a truth value.

class flea.TestAgent(app, request=None, response=None, cookies=None, history=None, validate_wsgi=True, host='127.0.0.1', port='8080')¶

A TestAgent object provides a user agent for the WSGI application under test.

Key methods and properties:

get(path), post(path), post_multipart - create get/post requests for the WSGI application and return a new TestAgent object

request, response - the Pesto request and response objects associated with the last WSGI request.

body - the body response as a string

lxml - the lxml representation of the response body (only

applicable for HTML responses)

reset() - reset the TestAgent object to its initial state, discarding any form

field values

find() (or dictionary-style attribute access) - evalute the given

xpath expression against the current response body and return a ResultWrapper object.

click(linkspec, flavor='auto', ignorecase=True, index=0, follow=True, check_status=True, **kwargs)¶

Click the link matched by linkspec. See findlinks() for a description of the link finding parameters

Parameters:

linkspec – specification of the link to be clicked
flavor – if css, linkspec must be a CSS selector, which must returning one or more links; if xpath, linkspec must be an XPath expression returning one or more links; any other value will be passed to findlinks().
ignorecase – (see findlinks())
index – index of the link to click in the case of multiple matching links

css(selector)¶: Return elements matching the given CSS Selector (see lxml.cssselect for documentation on the CSSSelector class.

find(path, namespaces=None, **kwargs)¶

Return elements matching the given xpath expression.

If the xpath selects a list of elements a ResultWrapper object is returned.

If the xpath selects any other type (eg a string attribute value), the result of the query is returned directly.

For convenience that the EXSLT regular expression namespace (http://exslt.org/regular-expressions) is prebound to the prefix re.

findlinks(linkspec, flavor='auto', ignorecase=True, **kwargs)¶

Return a ResultWrapper of links matched by linkspec.

Parameters:	linkspec – specification of the link to be clicked ignorecase – if `True`, the link search will be case insensitive flavor – one of `auto`, `text`, `contains`, `startswith`, `re`

The flavor parameter is interpreted according to the following rules:

if auto: detect links based on the following criteria:

if linkspec is a regular expression or otherwise has a search method, this will be used to match links.

if linkspec is callable, each link will be tested against it in turn, and the first link that returns True will be selected.

otherwise contains matching will be used

if text: for links where the text of the link is linkspec

if contains: for links where the link text contains linkspec

if startswith: for links where the link text contains linkspec

if re: for links where the text of the link matches linkspec

follow()¶: If response has a 30x status code, fetch (GET) the redirect target. No entry is recorded in the agent’s history list.

follow_all()¶: If response has a 30x status code, fetch (GET) the redirect target, until a non-redirect code is received. No entries are recorded in the agent’s history list.

get(PATH_INFO='/', data=None, charset='UTF-8', follow=True, history=True, check_status=True, **kwargs)¶

Make a GET request to the application and return the response.

PATH_INFO: The path to request from the application. This must be URL encoded. As a convenience, query string data can be appended to this. Example value: agent.get('/my%20script?param=1')
data: Query string data to be appended to the request, must be either a dict or list of (name, value) tuples.
charset: Encoding used for any unicode values encountered in data
follow: If false, redirect responses will not be followed
history: If true, a new entry will be added to the history attribute for the resulting TestAgent object
check_status: If true, any non success HTTP status code will result in an AssertionError being raised

make_environ(REQUEST_METHOD='GET', PATH_INFO='', wsgi_input='', **kwargs)¶

Return a dictionary suitable for use as the WSGI environ.

PATH_INFO must be URL encoded. As a convenience it may also contain a query string portion which will be used as the QUERY_STRING WSGI variable.

new_session()¶: Return a new TestAgent with all cookies deleted. This gives an easy way to test session expiry.

post(PATH_INFO='/', data=None, charset='UTF-8', follow=True, history=True, check_status=True, **kwargs)¶

Make a POST request to the application and return the response.

PATH_INFO: The path to request from the application. This must be URL encoded.
data: POST data to be sent to the application, must be either a dict or list of (name, value) tuples.
charset: Encoding used for any unicode values encountered in data
follow: If false, redirect responses will not be followed
history: If true, a new entry will be added to the history attribute for the resulting TestAgent object
check_status: If true, any non success HTTP status code will result in an AssertionError being raised

post_multipart(PATH_INFO='/', data=None, files=None, charset='UTF-8', follow=True, history=True, check_status=True, **kwargs)¶

POST a request to the given URI using multipart/form-data encoding.

PATH_INFO: The path to request from the application. This must be URL encoded.
data: POST data to be sent to the application, must be either a dict or list of (name, value) tuples.
files: list of (name, filename, content_type, data) tuples. data may be either a byte string, iterator or file-like object.
history: If true, a new entry will be added to the history attribute for the resulting TestAgent object
check_status: If true, any non success HTTP status code will result in an AssertionError being raised

reload(follow=True, check_status=True)¶

Reload the current page, if necessary re-posting any data.

Form fields that have been filled in on the loaded page, they will be refilled on the reloaded page, provided that the reloaded page has exactly the same fields present in the same order.

reset()¶: Reset the lxml document, abandoning any changes made

response_class¶: alias of MockResponse

serve(open_in_browser=True)¶

Start a HTTP server for the application under test.

The host/port used for the HTTP server is determined by the host and port arguments to the TestAgent constructor.

The initial page rendered to the browser will the currently loaded document (in its current state - so if changes have been made, eg form fields filled these will be present in the HTML served to the browser). Any cookies the TestAgent has stored are also forwarded to the browser.

Subsequent requests from the browser are then proxied directly to the WSGI application under test.

showbrowser()¶: Open the current page in a web browser

start_response(status, headers, exc_info=None)¶: No-op implementation.

class flea.XPathMultiMethod¶: A callable object that has different implementations selected by XPath expressions.

flea.guess_expression_flavor(expr)¶

Try to guess whether expr is a CSS selector or XPath expression.

css is the default value returned for expressions valid in both syntaxes.

flea.is_html(response)¶: Return True if the response content-type header indicates an (X)HTML content part.

flea.parse_cookies(response)¶: Return a Cookie.BaseCookie object populated from cookies parsed from the response object

flea.uri_join_same_server(baseuri, uri)¶

Join two uris which are on the same server. The resulting URI will have the protocol and netloc portions removed. If the resulting URI has a different protocol/netloc then a ValueError will be raised.

>>> from flea import uri_join_same_server
>>> uri_join_same_server('http://localhost/foo', 'bar')
'/bar'
>>> uri_join_same_server('http://localhost/foo', 'http://localhost/bar')
'/bar'
>>> uri_join_same_server('http://localhost/rhubarb/custard/', '../')
'/rhubarb/'
>>> uri_join_same_server('http://localhost/foo', 'http://example.org/bar')
Traceback (most recent call last):
  ...
ValueError: URI links to another server: http://example.org/bar

flea.when(xpath_expr)¶: Decorator for methods having different implementations selected by XPath expressions.

Flea documentation contents¶

Flea¶

Overview¶

Finding elements¶

Filling and submitting forms¶

File uploads¶

Filling forms in a single call¶

HTTP redirects¶

Querying WSGI application responses¶

Checking the content of the request¶

Checking the content of the response¶

Checking returned content.¶

Checking if strings are present in an HTML element¶

Accessing the html of selected elements¶

Accessing textual content of selected elements¶

Inspecting and interacting with a web browser¶

API documention¶

Table Of Contents

This Page

param *args:	Pairs of `(selector, value)`
param **kwargs:	mappings of fieldname to value

Navigation

Flea documentation contents¶

Flea¶

Overview¶

Finding elements¶

Filling and submitting forms¶

File uploads¶

Filling forms in a single call¶

HTTP redirects¶

Querying WSGI application responses¶

Checking the content of the request¶

Checking the content of the response¶

Checking returned content.¶

Checking if strings are present in an HTML element¶

Accessing the html of selected elements¶

Accessing textual content of selected elements¶

Inspecting and interacting with a web browser¶

API documention¶

Table Of Contents

This Page

Quick search

Navigation