pesto.utils module API documention

General utility functions used within pesto.

These functions are reserved for internal usage and it is not recommended that users of the API call these functions directly

class pesto.utils.DelimitedInput(io, delimiter, consume_delimiter=True)

Wrap a PutbackInput to read as far as a delimiter (after which subsequent reads will return empty strings, as if EOF was reached)

Examples:

>>> from StringIO import StringIO
>>> s = StringIO('one--two--three')
>>> s.seek(0)
>>> p = PutbackInput(s)
>>> DelimitedInput(p, '--').read()
'one'
>>> DelimitedInput(p, '--').read()
'two'
>>> DelimitedInput(p, '--').read()
'three'
>>> DelimitedInput(p, '--').read()
''

read(size=-1)

Return up to size bytes of data from the stream until EOF or delimiter is reached.

readline(size=-1)

Read a single line of up to size bytes from the input stream, or up to delimiter if this is encountered before a complete line is read.

class pesto.utils.ExpandableOutput(bufsize=16384)

Write-only output object.

Will store data in a StringIO, until more than bufsize bytes are written, at which point it will switch to storing data in a real file object.

switch_to_file_storage()

Switch the storage backend to an instance of TemporaryFile.

write_file(data)

write, optimized for the TemporaryFile backend

write_stringio(data)

write, optimized for the StringIO backend.

class pesto.utils.MultiDict(data=None, **kwargs)

Dictionary-like object that supports multiple values per key. Insertion order is preserved

Synopsis:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d['a']
1
>>> d['b']
3
>>> d.getlist('a')
[1, 2]
>>> d.getlist('b')
[3]
>>> list(d.iterallitems())
[('a', 1), ('a', 2), ('b', 3)]

allitems()

Return a list of (key, value) pairs for each item in the MultiDict. Items with multiple keys will have multiple key-value pairs returned. Items are returned in insertion order:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.allitems()
[('a', 1), ('a', 2), ('b', 3)]

copy()

Return a shallow copy of the dictionary:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> copy = d.copy()
>>> copy
MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> copy is d
False

classmethod fromkeys(seq, value=None)

Create a new MultiDict with keys from seq and values set to value.

Example:

>>> MultiDict.fromkeys(['a', 'b'])
MultiDict([('a', None), ('b', None)])

Keys can be repeated:

>>> d = MultiDict.fromkeys(['a', 'b', 'a'])
>>> d.getlist('a')
[None, None]
>>> d.getlist('b')
[None]

get(key, default=None)

Return the first available value for key key, or default if no such value exists:

>>> d = MultiDict([('a', 1), ('a', 2)])
>>> d.get('a')
1
>>> print d.get('b')
None

getlist(key)

Return a list of all values for key key.

items()

Return a list of (key, value) tuples. Only the first (key, value) is returned where keys have multiple values:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.items()
[('a', 1), ('b', 3)]

iterallitems()

Like allitems, but return an iterator rather than a list.

iteritems()

Like items, but return an iterator rather than a list.

iterkeys()

Iterator for dictionary keys. Each key is returned only once, even if multiple values are present.

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.keys()
['a', 'b']

iterlistitems()

Like listitems, but return an iterator rather than a list.

iterlistvalues()

Like listvalues, but returns an iterator over the results instead of a list.

itervalues()

Like values, but return an iterator rather than a list.

keys()

Return dictionary keys. Each key is returned only once, even if multiple values are present.

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.keys()
['a', 'b']

listitems()

Like items, but returns lists of values:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.listitems()
[('a', [1, 2]), ('b', [3])]

listvalues()

Return a list of values. Each item returned is a list of values associated with a single key.

Example usage:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.listvalues()
[[1, 2], [3]]

pop(key, default=None)

Dictionary pop method. Return and remove the value associated with key. If more than one value is associated with key, only the first is returned.

Example usage:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.pop('a')
1
>>> d
MultiDict([('a', 2), ('b', 3)])
>>> d.pop('a')
2
>>> d
MultiDict([('b', 3)])
>>> d.pop('a')
Traceback (most recent call last):
    ...
KeyError: 'a'

popitem()

Return and remove the last (key, value) pair.

Example usage:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.popitem()
('b', 3)
>>> d.popitem()
('a', 2)
>>> d.popitem()
('a', 1)
>>> d.popitem() 
Traceback (most recent call last):
    ...
KeyError: 'popitem(): dictionary is empty'

update(other=None, **kwargs)

Update the MultiDict from another MultiDict:

>>> d = MultiDict()
>>> d.update(MultiDict([('a', 1), ('a', 2)]))
>>> d
MultiDict([('a', 1), ('a', 2)])

dictionary:

>>> d = MultiDict()
>>> d.update({'a': 1, 'b': 2})
>>> d
MultiDict([('a', 1), ('b', 2)])

iterable of key, value pairs:

>>> d = MultiDict()
>>> d.update([('a', 1), ('b', 2)])
>>> d
MultiDict([('a', 1), ('b', 2)])

or keyword arguments:

>>> d = MultiDict()
>>> d.update(a=1, b=2)
>>> d
MultiDict([('a', 1), ('b', 2)])

values()

Return values from the dictionary. Where keys have multiple values, only the first is returned:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.values()
[1, 3]

class pesto.utils.PutbackInput(io)

Wrap a file-like object to allow data read to be returned to the buffer.

Only supports serial read-access, ie no seek or write methods.

Example:

>>> from StringIO import StringIO
>>> s = StringIO("the rain in spain\nfalls mainly\non the plain\n")
>>> p = PutbackInput(s)
>>> line = p.readline()
>>> line
'the rain in spain\n'
>>> p.putback(line)
>>> p.readline()
'the rain in spain\n'

peek(size)

Peek ahead size bytes from the stream without consuming any data

putback(data)

Put data back into the stream

read(size=-1)

Return up to size bytes from the input stream.

readline(size=-1)

Read a single line of up to size bytes from the input stream.

class pesto.utils.ReadlinesMixin

Mixin that defines readlines and the iterator interface in terms of underlying readline method.

class pesto.utils.SizeLimitedInput(io, length)

Wrap an IO object to prevent reading beyond length bytes.

Example:

>>> from StringIO import StringIO
>>> s = StringIO("the rain in spain\nfalls mainly\non the plain\n")
>>> s = SizeLimitedInput(s, 24)
>>> len(s.read())
24
>>> s.seek(0)
>>> s.read()
'the rain in spain\nfalls '
>>> s.seek(0)
>>> s.readline()
'the rain in spain\n'
>>> s.readline()
'falls '

check_available(requested)

Return the minimum of requested and the number of bytes available in the stream. If requested is negative, return the number of bytes available in the stream.

read(size=-1)

Return up to size bytes from the input stream.

readline(size=-1)

Read a single line of up to size bytes from the input stream.

seek(pos, whence=0)

Seek to position pos. This is a wrapper for the underlying IO’s seek method.

tell()

Return the position of the file pointer in the stream.