pesto.utils API documention

General utility functions used within pesto.

These functions are reserved for internal usage and it is not recommended that users of the API call these functions directly

class pesto.utils.DelimitedInput(io, delimiter, consume_delimiter=True)

Wrap a PutbackInput to read as far as a delimiter (after which subsequent reads will return empty strings, as if EOF was reached)

Examples:

>>> from StringIO import StringIO
>>> s = StringIO('one--two--three')
>>> s.seek(0)
>>> p = PutbackInput(s)
>>> DelimitedInput(p, '--').read()
'one'
>>> DelimitedInput(p, '--').read()
'two'
>>> DelimitedInput(p, '--').read()
'three'
>>> DelimitedInput(p, '--').read()
''

Initialize an instance of DelimitedInput.

read(size=-1)

Return up to size bytes of data from the stream until EOF or delimiter is reached.

readline(size=-1)

Read a single line of up to size bytes from the input stream, or up to delimiter if this is encountered before a complete line is read.

class pesto.utils.ExpandableOutput(bufsize=16384)

Write-only output object.

Will store data in a StringIO, until more than bufsize bytes are written, at which point it will switch to storing data in a real file object.

Initialize an ExpandableOutput instance.

switch_to_file_storage()

Switch the storage backend to an instance of TemporaryFile.

write_file(data)

write, optimized for the TemporaryFile backend

write_stringio(data)

write, optimized for the StringIO backend.

class pesto.utils.MultiDict(*args, **kwargs)

Dictionary-like object that supports multiple values per key. Insertion order is preserved.

Synopsis:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d['a']
1
>>> d['b']
3
>>> d.getlist('a')
[1, 2]
>>> d.getlist('b')
[3]
>>> list(d.iterallitems())
[('a', 1), ('a', 2), ('b', 3)]

MultiDicts can be constructed in the following ways:

1. From a sequence of (key, value) pairs:

   >>> MultiDict([('a', 1), ('a', 2)])
   MultiDict([('a', 1), ('a', 2)])
   

2. Initialized from another MultiDict:

   >>> d = MultiDict([('a', 1), ('a', 2)])
   >>> MultiDict(d)
   MultiDict([('a', 1), ('a', 2)])
   

3. Initialized from a regular dict:

   >>> MultiDict({'a': 1})
   MultiDict([('a', 1)])
   

4. From keyword arguments:

   >>> MultiDict(a=1)
   MultiDict([('a', 1)])
   

allitems()

Return a list of (key, value) pairs for each item in the MultiDict. Items with multiple keys will have multiple key-value pairs returned:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.allitems()
[('a', 1), ('a', 2), ('b', 3)]

copy()

Return a shallow copy of the dictionary:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> copy = d.copy()
>>> copy
MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> copy is d
False

extend(*args, **kwargs)

Extend the MultiDict with another MultiDict, regular dictionary or a iterable of (key, value) pairs. This is similar to update() except that new keys are added to old keys.

classmethod fromkeys(seq, value=None)

Create a new MultiDict with keys from seq and values set to value.

Example:

>>> MultiDict.fromkeys(['a', 'b'])
MultiDict([('a', None), ('b', None)])

Keys can be repeated:

>>> d = MultiDict.fromkeys(['a', 'b', 'a'])
>>> d.getlist('a')
[None, None]
>>> d.getlist('b')
[None]

get(key, default=None)

Return the first available value for key key, or default if no such value exists:

>>> d = MultiDict([('a', 1), ('a', 2)])
>>> d.get('a')
1
>>> print d.get('b')
None

getlist(key)

Return a list of all values for key key.

items()

Return a list of (key, value) tuples. Only the first (key, value) is returned where keys have multiple values:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.items()
[('a', 1), ('b', 3)]

iterallitems()

Like allitems, but return an iterator rather than a list.

iteritems()

Like items, but return an iterator rather than a list.

iterkeys()

Iterator for dictionary keys. Each key is returned only once, even if multiple values are present.

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.keys()
['a', 'b']

iterlistitems()

Like listitems, but return an iterator rather than a list.

iterlistvalues()

Like listvalues, but returns an iterator over the results instead of a list.

itervalues()

Like values, but return an iterator rather than a list.

keys()

Return dictionary keys. Each key is returned only once, even if multiple values are present.

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.keys()
['a', 'b']

listitems()

Like items, but returns lists of values:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.listitems()
[('a', [1, 2]), ('b', [3])]

listvalues()

Return a list of values. Each item returned is a list of values associated with a single key.

Example usage:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.listvalues()
[[1, 2], [3]]

pop(key, *args)

Dictionary pop method. Return and remove the value associated with key. If more than one value is associated with key, only the first is returned.

Example usage:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.pop('a')
1
>>> d
MultiDict([('a', 2), ('b', 3)])
>>> d.pop('a')
2
>>> d
MultiDict([('b', 3)])
>>> d.pop('a')
Traceback (most recent call last):
    ...
KeyError: 'a'

popitem()

Return and remove a (key, value) pair from the dictionary.

The item popped is always the most recently added key and the first value associated with it:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.popitem()
('b', 3)
>>> d.popitem()
('a', 1)
>>> d.popitem()
('a', 2)
>>> d.popitem() 
Traceback (most recent call last):
    ...
KeyError: 'popitem(): dictionary is empty'

update(*args, **kwargs)

Update the MultiDict from another MultiDict, regular dictionary or a iterable of (key, value) pairs. New keys overwrite old keys - use extend() if you want new keys to be added to old keys.

Updating from another MultiDict:

>>> d = MultiDict([('name', 'eric'), ('occupation', 'lumberjack')])
>>> d.update(MultiDict([('mood', 'okay')]))
>>> d
MultiDict([('name', 'eric'), ('occupation', 'lumberjack'), ('mood', 'okay')])

from a dictionary:

>>> d = MultiDict([('name', 'eric'), ('hobby', 'shopping')])
>>> d.update({'hobby': 'pressing wild flowers'})
>>> d
MultiDict([('name', 'eric'), ('hobby', 'pressing wild flowers')])

an iterable of (key, value) pairs:

>>> d = MultiDict([('name', 'eric'), ('occupation', 'lumberjack')])
>>> d.update([('hobby', 'shopping'), ('hobby', 'pressing wild flowers')])
>>> d
MultiDict([('name', 'eric'), ('occupation', 'lumberjack'), ('hobby', 'shopping'), ('hobby', 'pressing wild flowers')])

or keyword arguments:

>>> d = MultiDict([('name', 'eric'), ('occupation', 'lumberjack')])
>>> d.update(mood='okay')

values()

Return values from the dictionary. Where keys have multiple values, only the first is returned:

>>> d = MultiDict([('a', 1), ('a', 2), ('b', 3)])
>>> d.values()
[1, 3]

class pesto.utils.PutbackInput(io)

Wrap a file-like object to allow data read to be returned to the buffer.

Only supports serial read-access, ie no seek or write methods.

Example:

>>> from StringIO import StringIO
>>> s = StringIO("the rain in spain\nfalls mainly\non the plain\n")
>>> p = PutbackInput(s)
>>> line = p.readline()
>>> line
'the rain in spain\n'
>>> p.putback(line)
>>> p.readline()
'the rain in spain\n'

Initialize a PutbackInput object from input stream io.

peek(size)

Peek ahead size bytes from the stream without consuming any data

putback(data)

Put data back into the stream

read(size=-1)

Return up to size bytes from the input stream.

readline(size=-1)

Read a single line of up to size bytes from the input stream.

class pesto.utils.ReadlinesMixin

Mixin that defines readlines and the iterator interface in terms of underlying readline method.

class pesto.utils.SizeLimitedInput(io, length)

Wrap an IO object to prevent reading beyond length bytes.

Example:

>>> from StringIO import StringIO
>>> s = StringIO("the rain in spain\nfalls mainly\non the plain\n")
>>> s = SizeLimitedInput(s, 24)
>>> len(s.read())
24
>>> s.seek(0)
>>> s.read()
'the rain in spain\nfalls '
>>> s.seek(0)
>>> s.readline()
'the rain in spain\n'
>>> s.readline()
'falls '

check_available(requested)

Return the minimum of requested and the number of bytes available in the stream. If requested is negative, return the number of bytes available in the stream.

read(size=-1)

Return up to size bytes from the input stream.

readline(size=-1)

Read a single line of up to size bytes from the input stream.

seek(pos, whence=0)

Seek to position pos. This is a wrapper for the underlying IO’s seek method.

tell()

Return the position of the file pointer in the stream.