Welcome to Thin Redis Object Layer’s documentation!

A library for light and predictable python object mappings to Redis

Indices and tables

• Index
• Module Index
• Search Page

Getting started

pip install trol and start defining your schema:

>>> import trol
>>> import redis
...
>>> class MyDatabase(trol.Database):
...   redis = redis.Redis()
...
...   favorite_breweries = trol.SortedSet('favbreweries', typ=trol.Model)
...
...   class Brewery(trol.Model):
...     def __init__(self, short_name):
...       self.id = short_name
...
...     location = trol.Property()
...     name = trol.Property(typ=str)
...     beers = trol.Set(typ=trol.Model)
...
...   class Beer(trol.Model):
...     def __init__(self, name, batch_number):
...       self.name = name
...       self.batch_number = batch_number
...
...     @property
...     def id(self):
...       return self.name + '@' + str(self.batch_number)
...
...     style = trol.Property()
...     rating = trol.Property(typ=int)
...
>>> brewery = MyDatabase.Brewery('frmt')
>>> brewery.location = (47.6490476, -122.3467747)
>>> brewery.name = "Fremont Brewing Company"
>>> lush = MyDatabase.Beer('Lush IPA', 120)
>>> lush.style = "Indian Pale Ale"
>>> lush.rating = 5
>>> universale = MyDatabase.Beer('Universale', 245)
>>> universale.style = "American Pale Ale"
>>> universale.rating = 5
>>> brewery.beers.add(lush, universale)
2
>>> MyDatabase.favorite_breweries.add(brewery, 10)
1
>>> set(MyDatabase.redis.keys()) == {
...   b'favbreweries',
...   b'Brewery:frmt:name',
...   b'Brewery:frmt:location',
...   b'Brewery:frmt:beers',
...   b'Beer:Lush IPA@120:style',
...   b'Beer:Lush IPA@120:rating',
...   b'Beer:Universale@245:style',
...   b'Beer:Universale@245:rating'
... }
True

Development

For local development, install the dependencies listed in requirements.txt and additionally the dev-dependencies in requirements-dev.txt.

As an additional development dependency, you should have Redis server installed locally. You can follow the instructions at https://redis.io/docs/getting-started/installation/ to get started. Note that on Ubuntu, you should install redis via apt rather than snap as the snap package does not include the redis-server binary.

Testing

Tests are written using nose2. Running tests can be accomplished by running python -m nose2.

Note that tests will run a redis-server process, so this command must be installed. as mentioned above, in addition to the requirements-dev.txt dependencies.

Model

class trol.model.Model

Model is a class that can hold and operate on trol properties and collections to form a trol object.

Attributes:

autocommit (bool): If True, properties will be immediatly commited to Redis when modified. Default is True

This attribute can be overriden for a single property by setting that properties autocommit attribute

alwaysfetch (bool): If True, property values will be fetched from Redis on every access. Deafault is False

This attribute can be overriden for a single property by setting that properties alwaysfetch attribute

commit(*propnames)

Saves properties in this model to Redis

Args:

*propnames (list[str]): The attribute nanes of properties which should be committed.

If none are provided, the default is to commits all propertoes in the model.

delete(*propnames)

Deletes properties in this model from Redis

Args:

*propnames (list[str]): The attribute nanes of properties which should be deleted.

If none are provided, the default is to deleted all properties in the model.

exists(*propnames)

Checks the properties in this model for existance in Redis

Args:

*propnames (list[str]): The attribute nanes of properties which should be checked for existance.

If at least one property is specified, True will be returnedif all proerties exist If none are provided, the default is to return true if any property exists.

expire(ttl=None, **kwargs)

Sets the expiration TTL on specified keys or a common TTL on all keys.

If both common TTL and property TTL are specified, the property TTL takes precidence. If neither are specified the call is a no-op.

Args:

ttl (float): TTL to apply to all keys. **kwargs (dict[str, float]): Key value pairs where the key is the property name and

value is the desired TTL in seconds. Expiration times will be rounded to the nearest millisecond.

invalidate(*propnames)

Mark properties in this model as invalid and requiring a fetch

Args:

*propnames (list[str]): The attribute nanes of properties which should be invalidated.

If none are provided, the default is to invalidate all propertoes in the model.

key

str: Redis key which prefixes any properties stored under this Model instance

By default this is {model_name}:{id}

Changing the key of an instance may cause data to be lost. It’s best to think of these models as a pointer and changing the keys is changing the value of the pointer

Example:

TODO: Write a new example

model_name

str: A name for this model which, in addition to its id which identify it in Redis

By default this is the class name

Example:

TODO: Write a new example

redis

Redis: The active Redis connection for this model

This model can have it’s own Redis connection of use connection of the Database that holds it

update(**kwargs)

Updates the local values of multiple properties and commits them if autocommit is set

Args:

**kwargs (dict[str, object]): Key value pairs where the key is the property name and value is what it should be set to

exception trol.model.ModelDeserializationError

class trol.model.ModelType(name, bases, _dict)

ModelType is a metaclass providing awareness of member trol objects to Model.

This type embeds a dict of Property which tracks any properties assigned at class load. It assigns the names of properties who are unamed This type embeds a dict of Collection which tracks any collections assigned at class load It will assign the names to any collections which did not have their names assigned

trol.model.deserialize_model(byts)

Deserialize a key reference into a model instance

The model class, id, model_name, and key will be set on deserialization Any custom attributes of the instance will not, and __init__ will not be called

Args:

bytes: The key reference

Returns:

model (Model): The deserialized model

trol.model.serialize_model(model)

Serialize a model instance into a key reference

The model class, id, model_name, and key will be preserved on serialization Any custom attributes of the instance will not

Args:

model (Model): The model to serialize

Returns:

bytes: The key reference

Property

class trol.property.Nil

Class of indicator value for unset properties

class trol.property.Property(name=None, typ=None, autocommit=None, alwaysfetch=None, serializer=None, deserializer=None)

Property provides a field to a Model, backed in Redis, as if it were a local property.

The expectation is that this Property object is embedded as a class level attribute The class it is embedded in should define redis and key attrbutes to define where this property is stored

Attributes:

autocommit (bool): Commit the local value to Redis on every assignment

If set to None, assignments will check the holder object for an autocommit value, and default to True if not present Useful if you don’t want to worry about forgetting to commit

alwaysfetch (bool): Fetch the latest value from the database

If set to None, gets will check the holder object for an alwaysfetch value, and default to True if not present Useful if you have multiple threads or processes frequently modifying the database

serializer (Callable[[object], bytes]): A function or callable which will be used for serializing the value before storage in Redis

Although bytes is the most general output type, this function may also output str, int, or any other type redis=py will accept Default is pickle.dumps, which is not human readable, but will work with most python object

deserializer (Callable[[bytes], object]): A function or callable which will be used for deserializing the value after reciving it from redis

Default is pickle.loads, the counterpart for the deafault serializer

commit(obj)

Commits this properties value to Redis

Does nothing if the value is nil, which means there is nothing to write

Args:

obj (object): This property’s holder

Returns:

bool: True if the set transaction was successful. False otherwise

delete(obj)

Deletes the key of this property from Redis

Args:

obj (object): This property’s holder

Returns:

bool: True if a the key was deleted. False if it didn’t exist.

exists(obj)

Checks whether or not a value is set for this property in Redis

Args:

obj (object): This property’s holder

Returns:

bool: True if the key have a value in Redis. False otherwise

expire(obj, ttl)

Sets expiration on the ket of this property in Redis

NOTE:

Expiration is not handled internally to trol, so if a key has expired and alwaysfetch is not set to True, a non-nil value may be returned after expiration.

Args:

obj (object): This property’s holder ttl (float): Time to live in seconds. Precisions beyond 1 millisecond will be rounded to

the nearest millisecond, which is the minimum resolution of a Redis timeout.

Returns:

bool: True if the expire was set successfully. False otherwise

fetch(obj)

Retrieves and sets the value of this property

Args:

obj (object): This property’s holder

Returns:

bytes: The data retrieved or None in the case of a key not found

invalidate(obj)

Invalidates the local value to indicate a fetch must be done

Args:

obj (object): This property’s holder

key(obj)

Gets the key where this property’s data exists in Redis

The key for this property is the key of it’s holder, with :<name> appended

Args:

obj (object): Model instance holding this property.

Returns:

str: The key which can be used to get this property’s data from Redis

static mangle(name)

Creates a mangled version of the inputted name

Mangling produces a name unlikely to colide with other attribute names.

Args:

name (str): The name which should be mangled

Returns:

str: Mangled name

name

str: The name for this property, which will be used to determine the key when bound to a Model

set(obj, value)

Sets the value stored in the holder obj

Args:

obj (object): This propery’s holder value (object): The value to set

value(obj)

Gets the value stored in the holder obj

Sets the property value attribute in the holder if it does not already exist

Args:

obj (object): This propery’s holder

Returns:

object: The local value of this property

trol.property.nil = <trol.property.Nil object>

An indicator value for unset properties

Database

Collection

Collections which proxy access to Redis storage

This file is Copyright (c) 2010 Tim Medina Licenced under the MIT License availible in the root of the trol source repo It has been modifed to hit the overal model of trol, but is largely the same externally Thanks to the guys at Redisco for the great work! https://github.com/kiddouk/redisco

Here are some of the modifications from the origonal Redisco conatainers:

• Values are serialized and deserialized allowing more arbitrary objects to be stored
• Comparison operations do not trigger network transfer of container members
• Set copy does not tigger network transfer of set members
• Functions which require multiple explicitly use the pipeline to avoid multiple network calls
• Some functions now use a “scratch-pad” in redis to avoid transfering full collections
• The db attribute has been renamed to redis to match other places in trol
• There is no default expire time
• A Redis connection must be specified on construction, unless accessed through a Model
• There is no support for TypedList or NonPersitentList

Unlike properties, collections call out to Redis on every transaction by default Caching for collections is a trickier prospect, and therefore is not attempted

>>> import trol
>>> from redis import Redis
>>> redis = Redis()

class trol.collection.Collection(name=None, redis=None, typ=None, key=None, serializer=None, deserializer=None)

Base class for all collections. This class should not used directctly This class provides the redis attribute :members:

clear()

Remove the collection from the redis storage

Returns:None

>>> s = trol.Set('test_clear', redis)
>>> s.add('1')
1
>>> s.clear()
>>> s.members
set()

key

str: The key in Redis which contains the data this object references

Setting this attribute directly to non-None will override name in determining key

>>> s = trol.Set(name='foo')
>>> s.key
'foo'
>>> s.key = 'bar'
>>> s.key
'bar'

name

str: The name for this collection, which will be used to determine the key if bound to a Model

>>> class Alpha(trol.Model):
...     def __init__(self, ident):
...         self.id = ident
...
...     storage = trol.Set(name="store")
...
>>> a = Alpha('xyz')
>>> a.key
'Alpha:xyz'
>>> a.storage.key
'Alpha:xyz:store'

pipeline

redis.Pipeline: A pipeline object to execute buffered commands

The collection waits until you call this function for the first time to intialize the pipeline A unique pipeline will be created for each thread that accesses this property, so it is thread safe

redis

redis.Redis: A connection object for interacting with Redis

set_expire(time)

Allow the key to expire after time seconds.

Parameters:time – time expressed in seconds. Returns:None

>>> s = trol.Set('test_set_expire', redis)
>>> s.add("1")
1
>>> s.set_expire(1)
>>> from time import sleep
>>> sleep(1.5)
>>> s.members
set()

class trol.collection.Hash(*args, field_typ=<class 'str'>, field_serializer=None, field_deserializer=None, **kwargs)

This class represent a hash (i.e. dict) object as seen in Redis.

dict

Returns all the fields and values in the Hash.

Returns:dict with all the hash fields and values

>>> h = trol.Hash('test_hgetall', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.dict == {"a": 1, "b": 2, "c": 3}
True

get(field, default=None, raise_error=False)

Returns the value stored in the field, or the default value unless raise_error is True.

Parameters:

• field – the bytes or string field key to look up.
• default – the value to return if the field is not found.
• raise_error – whether to raise a KeyError if the key is not found.

>>> h = trol.Hash('test_hget', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hget("b")
2
>>> h.hget("d") is None
True
>>> h.hget("d", default=0)
0
>>> h.hget("d", raise_error=True)
Traceback (most recent call last):
...
KeyError: 'd'

hdel(*fields)

Delete one or more hash fields by key.

Parameters:fields – on or more fields to remove. Returns:the number of fields that were removed

>>> h = trol.Hash('test_hdel', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hdel("a", "b")
2
>>> h.dict
{'c': 3}

hexists(field)

Returns True if the field exists, False otherwise.

>>> h = trol.Hash('test_hexists', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hexists("a")
True
>>> h.hexists("d")
False

hget(field, default=None, raise_error=False)

Returns the value stored in the field, or the default value unless raise_error is True.

Parameters:

• field – the bytes or string field key to look up.
• default – the value to return if the field is not found.
• raise_error – whether to raise a KeyError if the key is not found.

>>> h = trol.Hash('test_hget', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hget("b")
2
>>> h.hget("d") is None
True
>>> h.hget("d", default=0)
0
>>> h.hget("d", raise_error=True)
Traceback (most recent call last):
...
KeyError: 'd'

hgetall()

Returns all the fields and values in the Hash.

Returns:dict with all the hash fields and values

>>> h = trol.Hash('test_hgetall', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.dict == {"a": 1, "b": 2, "c": 3}
True

hincrby(field, increment=1)

Increment the value of the field. :returns: the value of the field after incrementation

>>> h = trol.Hash('test_hincrby', redis)
>>> h.hincrby("key", 10)
10
>>> h.hincrby("key", 2)
12

hkeys()

Returns all fields name in the Hash

>>> h = trol.Hash('test_hkeys', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hkeys()
['a', 'b', 'c']

hlen()

Returns the number of elements in the Hash.

Returns:the number of elements in the hash.

>>> h = trol.Hash('test_hlen', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hlen()
3

hmget(fields, default=None, raise_error=False)

Returns the values stored in the fields, or the default value unless raise_error is True.

Parameters:

• fields – an iterable of byte or string fields to retrieve.
• default – the value to return if the field is not found.
• raise_error – whether to raise a KeyError if the key is not found.

>>> h = trol.Hash('test_hmget', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hmget(["a", "b"])
[1, 2]
>>> h.hmget(["c", "d"])
[3, None]
>>> h.hmget(["c", "d"], default=0)
[3, 0]
>>> h.hmget(["c", "d"], raise_error=True)
Traceback (most recent call last):
...
KeyError: 'd'

hmset(mapping)

Sets or updates the fields with their corresponding values.

Parameters:mapping – a dict with keys and values Returns:True if the operation succeeded

>>> h = trol.Hash('test_hmset', redis)
>>> h.hmset({"a": 1, "b": 2, "c": 3})
True
>>> h.dict == {"a": 1, "b": 2, "c": 3}
True
>>> h.hmset({})
True
>>> h.dict == {"a": 1, "b": 2, "c": 3}
True

hset(field, value)

Set field in the Hash to value.

Returns:1 if field is a new value and 0 if it was updated.

>>> h = trol.Hash('test_hset', redis)
>>> h.hset("bar", "foo")
1
>>> h.hset("bar", "baz")
0
>>> h.dict
{'bar': 'baz'}

hvals()

Returns all the values in the Hash

Returns:list with all the hash values

>>> h = trol.Hash('test_hvals', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hvals()
[1, 2, 3]

items() → a set-like object providing a view on D's items

keys()

Returns all fields name in the Hash

>>> h = trol.Hash('test_hkeys', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hkeys()
['a', 'b', 'c']

update(*args, **kwargs)

Sets or updates the fields with their corresponding values, accepting args like the native python dict.update

Returns:None

>>> h = trol.Hash('test_set_update', redis)
>>> h.update({"a": 1, "b": 2, "c": 3})
>>> h.dict == {"a": 1, "b": 2, "c": 3}
True
>>> h.update(d=4)
>>> h["d"]
4
>>> h.update([("e", 5)])
>>> h["e"]
5

values()

Returns all the values in the Hash

Returns:list with all the hash values

>>> h = trol.Hash('test_hvals', redis)
>>> h.update(a=1, b=2, c=3)
>>> h.hvals()
[1, 2, 3]

class trol.collection.List(name=None, redis=None, typ=None, key=None, serializer=None, deserializer=None)

This class represent a list object as seen in redis.

all()

Returns all items in the list.

append(*values)

Push the value into the list from the right side

Parameters:values – a list of values or single value to push Returns:long representing the size of the list.

>>> l = trol.List('test_rpush', redis)
>>> l.rpush(['a', 'b'])
2
>>> l.rpush(['c', 'd'])
4
>>> l.members
['a', 'b', 'c', 'd']
>>> l.rpush()
4

copy(key)

Copy the list to a new list.

Warning

If destination key already contains a value, it clears it before copying.

Returns:a list object pointing to the copy

>>> l = trol.List('test_list_copy', redis)
>>> l.extend(['a', 'b', 'c'])
>>> copy = l.copy('copy')
>>> copy.members
['a', 'b', 'c']

count(value)

Return number of occurrences of value.

Parameters:value – a value tha may be contained in the list

>>> l = trol.List('test_count', redis)
>>> l.extend(['duck', 'duck', 'duck', 'goose'])
>>> l.count("duck")
3
>>> l.count("goose")
1
>>> l.count("possum")
0

extend(iterable)

Extend list by appending elements from the iterable.

Parameters:iterable – an iterable objects.

>>> l = trol.List('test_extend', redis)
>>> l.extend(['a', 'b'])
>>> l.members
['a', 'b']
>>> l.extend(['c', 'd'])
>>> l.members
['a', 'b', 'c', 'd']

lindex(idx)

Return the value at the index idx

Parameters:idx – the index to fetch the value. Returns:the value or None if out of range.

>>> l = trol.List('test_lindex', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.lindex(1)
'b'

llen()

Returns the length of the list.

lpop()

Pop the first object from the left.

Returns:the popped value.

>>> l = trol.List('test_lpop', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.lpop()
'a'
>>> l.lpop()
'b'
>>> l.members
['c']
>>> l.lpop()
'c'
>>> l.lpop() is None
True

lpush(*values)

Push the value into the list from the left side

Parameters:values – a list of values or single value to push Returns:long representing the number of values pushed.

>>> l = trol.List('test_lpush', redis)
>>> l.lpush(['a', 'b'])
2
>>> l.lpush(['c', 'd'])
4
>>> l.members
['d', 'c', 'b', 'a']
>>> l.lpush()
4

lrange(start, stop)

Returns a range of items.

Parameters:

• start – integer representing the start index of the range
• stop – integer representing the size of the list.

>>> l = trol.List('test_lrange', redis)
>>> l.push(['a', 'b', 'c', 'd'])
4
>>> l.lrange(1, 2)
['b', 'c']

lrem(value, num=1)

Remove first occurrence of value.

Returns:the number of removed elements

>>> l = trol.List('test_lrem', redis)
>>> l.extend(['duck', 'duck', 'duck', 'goose'])
>>> l.lrem("duck")
1
>>> l.lrem("duck", 3)
2
>>> l.members
['goose']

lset(idx, value)

Set the value in the list at index idx

Returns:True is the operation succeed.

>>> l = trol.List('test_lset', redis)
>>> l.push(['a', 'b', 'c'])
3
>>> l.lset(0, 'e')
True
>>> l.members
['e', 'b', 'c']

ltrim(start, end)

Trim the list from such that it only includes elements from start to end inclusive.

Returns:True if the operation succeeded

>>> l = trol.List('test_ltrim', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.ltrim(0, 1)
True
>>> l.members
['a', 'b']

members

Return all items in the list.

pop()

Pop the first object from the right.

Returns:the popped value.

>>> l = trol.List('test_rpop', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.rpop()
'c'
>>> l.rpop()
'b'
>>> l.members
['a']
>>> l.rpop()
'a'
>>> l.rpop() is None
True

pop_onto(key)

Remove an element from the list, atomically add it to the head of the list indicated by key

Parameters:key – the key of the list receiving the popped value. Returns:the popped (and pushed) value

>>> l = trol.List('list_rpoplpush1', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.rpoplpush('list_rpoplpush2')
'c'
>>> l2 = trol.List('list_rpoplpush2', redis)
>>> l2.members
['c']

push(*values)

Push the value into the list from the right side

Parameters:values – a list of values or single value to push Returns:long representing the size of the list.

>>> l = trol.List('test_rpush', redis)
>>> l.rpush(['a', 'b'])
2
>>> l.rpush(['c', 'd'])
4
>>> l.members
['a', 'b', 'c', 'd']
>>> l.rpush()
4

remove(value, num=1)

Remove first occurrence of value.

Returns:the number of removed elements

>>> l = trol.List('test_lrem', redis)
>>> l.extend(['duck', 'duck', 'duck', 'goose'])
>>> l.lrem("duck")
1
>>> l.lrem("duck", 3)
2
>>> l.members
['goose']

reverse()

Reverse the list in place.

Note

This command must make two network calls, transferring the list each way

Returns:None

>>> l = trol.List('test_reverse', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.members
['a', 'b', 'c']
>>> l.reverse()
>>> l.members
['c', 'b', 'a']

rpop()

Pop the first object from the right.

Returns:the popped value.

>>> l = trol.List('test_rpop', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.rpop()
'c'
>>> l.rpop()
'b'
>>> l.members
['a']
>>> l.rpop()
'a'
>>> l.rpop() is None
True

rpoplpush(key)

Remove an element from the list, atomically add it to the head of the list indicated by key

Parameters:key – the key of the list receiving the popped value. Returns:the popped (and pushed) value

>>> l = trol.List('list_rpoplpush1', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.rpoplpush('list_rpoplpush2')
'c'
>>> l2 = trol.List('list_rpoplpush2', redis)
>>> l2.members
['c']

rpush(*values)

Push the value into the list from the right side

Parameters:values – a list of values or single value to push Returns:long representing the size of the list.

>>> l = trol.List('test_rpush', redis)
>>> l.rpush(['a', 'b'])
2
>>> l.rpush(['c', 'd'])
4
>>> l.members
['a', 'b', 'c', 'd']
>>> l.rpush()
4

shift()

Pop the first object from the left.

Returns:the popped value.

>>> l = trol.List('test_lpop', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.lpop()
'a'
>>> l.lpop()
'b'
>>> l.members
['c']
>>> l.lpop()
'c'
>>> l.lpop() is None
True

trim(start, end)

Trim the list from such that it only includes elements from start to end inclusive.

Returns:True if the operation succeeded

>>> l = trol.List('test_ltrim', redis)
>>> l.extend(['a', 'b', 'c'])
>>> l.ltrim(0, 1)
True
>>> l.members
['a', 'b']

unshift(*values)

Push the value into the list from the left side

Parameters:values – a list of values or single value to push Returns:long representing the number of values pushed.

>>> l = trol.List('test_lpush', redis)
>>> l.lpush(['a', 'b'])
2
>>> l.lpush(['c', 'd'])
4
>>> l.members
['d', 'c', 'b', 'a']
>>> l.lpush()
4

class trol.collection.Set(name=None, redis=None, typ=None, key=None, serializer=None, deserializer=None)

This class represents a Set in redis.

add(*values)

Add the specified members to the Set.

Parameters:values – a list of values or a simple value. Returns:integer representing the number of value added to the set.

>>> s = trol.Set('test_sadd', redis)
>>> s.sadd(1, 2, 3)
3
>>> s.sadd(4)
1
>>> s.sadd(4)
0
>>> s.sadd()
0
>>> s.members == {1, 2, 3, 4}
True

copy(key)

Copy the set to another key and return the new Set.

Warning

If the new key already contains a value, it will be overwritten.

>>> s1 = trol.Set('test_set_copy1', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2 = s1.copy('test_set_copy2')
>>> s2.members == {"a", "b", "c"}
True

difference(key, *other_sets)

Return a new Set representing the difference of n sets.

Parameters:

• key – String representing the key where to store the result.
• other_sets – list of other Set.

Returns:

a new Set representing the difference of this set and the other sets.

>>> s1 = trol.Set('test_difference1', redis)
>>> s2 = trol.Set('test_difference2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'e'])
2
>>> s3 = s1.difference('test_difference3', s2)
>>> s3.members == {'a', 'b'}
True

difference_update(*other_sets)

Update the set, removing elements found in others.

Parameters:other_sets – list of Set Returns:None

>>> s1 = trol.Set('test_difference_update1', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2 = trol.Set('test_difference_update2', redis)
>>> s2.add(['b', 'c', 'd'])
3
>>> s1.difference_update(s2)
>>> s1.members
{'a'}

intersection(key, *other_sets)

Return a new Set representing the intersection of the other sets.

Parameters:

• key – String representing the key where to store the result.
• other_sets – list of other Set.

Returns:

a new Set representing the intersection of the other sets.

>>> s1 = trol.Set('test_intersection1', redis)
>>> s2 = trol.Set('test_intersection2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'e'])
2
>>> s3 = s1.intersection('test_intersection3', s2)
>>> s3.members
{'c'}

intersection_update(*other_sets)

Update the set, keeping only elements found in it and all other_sets.

Parameters:other_sets – list of Set Returns:None

>>> s1 = trol.Set('test_intersection_update1', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2 = trol.Set('test_intersection_update2', redis)
>>> s2.add(['b', 'c', 'd'])
3
>>> s1.intersection_update(s2)
>>> s1.members == {'b', 'c'}
True

isdisjoint(other)

Return True if the set has no elements in common with other.

Parameters:other – another Set Returns:boolean

>>> s1 = trol.Set('test_isdisjoint1', redis)
>>> s2 = trol.Set('test_isdisjoint2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'd', 'e'])
3
>>> s1.isdisjoint(s2)
False
>>> s2.remove('c')
1
>>> s1.isdisjoint(s2)
True

issubset(other_set)

Test whether every element in the set is in other.

Parameters:other_set – another Set to compare to.

>>> s1 = trol.Set('test_issubset1', redis)
>>> s2 = trol.Set('test_issubset2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add('b')
1
>>> s2.issubset(s1)
True
>>> s2.add('d')
1
>>> s2.issubset(s1)
False

issuperset(other_set)

Test whether every element in other is in the set.

Parameters:other_set – another Set to compare to.

>>> s1 = trol.Set('test_issuperset1', redis)
>>> s2 = trol.Set('test_issuperset2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add('b')
1
>>> s1.issuperset(s2)
True

members

Return the stored content of the Set.

pop()

Remove and return (pop) a random element from the Set.

Returns:String representing the value poped.

>>> s = trol.Set('test_spop', redis)
>>> s.add("a")
1
>>> s.spop()
'a'
>>> s.members
set()

remove(*values)

Remove the values from the Set if they are present.

Parameters:values – a list of values or a simple value. Returns:boolean indicating if the values have been removed.

>>> s = trol.Set('test_srem', redis)
>>> s.add([1, 2, 3])
3
>>> s.srem([1, 3])
2

sadd(*values)

Add the specified members to the Set.

Parameters:values – a list of values or a simple value. Returns:integer representing the number of value added to the set.

>>> s = trol.Set('test_sadd', redis)
>>> s.sadd(1, 2, 3)
3
>>> s.sadd(4)
1
>>> s.sadd(4)
0
>>> s.sadd()
0
>>> s.members == {1, 2, 3, 4}
True

scard()

Returns the cardinality of the Set.

Returns:integer cardinality of the Set.

>>> s = trol.Set('test_scard', redis)
>>> s.add(['a', 'b', 'c'])
3
>>> s.scard()
3

sdiff(*other_sets)

Performs a difference between Sets and return the result without storing.

Note

This function return a Python set object, not a Set. See func:difference.

>>> s1 = trol.Set('test_sdiff1', redis)
>>> s2 = trol.Set('test_sdiff2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'e'])
2
>>> s1.sdiff(s2) == {'a', 'b'}
True

sinter(*other_sets)

Performs an intersection between Sets and return the result without storing.

Note

This function return a Python set object, not a Set. See func:intersection.

>>> s1 = trol.Set('test_sinter1', redis)
>>> s2 = trol.Set('test_sinter2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'e'])
2
>>> s1.sinter(s2)
{'c'}

sismember(value)

Return True if the provided value is in the Set.

>>> s = trol.Set('test_sismember', redis)
>>> s.add(['a', 'b', 'c'])
3
>>> s.sismember('d')
False
>>> s.clear()

spop()

Remove and return (pop) a random element from the Set.

Returns:String representing the value poped.

>>> s = trol.Set('test_spop', redis)
>>> s.add("a")
1
>>> s.spop()
'a'
>>> s.members
set()

srandmember()

Return a random member of the set.

>>> s = trol.Set('test_srandmember', redis)
>>> s.add(['a', 'b', 'c'])
3
>>> s.srandmember() in { 'a', 'b', 'c' }
True

srem(*values)

Remove the values from the Set if they are present.

Parameters:values – a list of values or a simple value. Returns:boolean indicating if the values have been removed.

>>> s = trol.Set('test_srem', redis)
>>> s.add([1, 2, 3])
3
>>> s.srem([1, 3])
2

sunion(*other_sets)

Performs a union between Sets and return the result without storing.

Note

This function return a Python set object, not a Set. See func:union.

>>> s1 = trol.Set('test_sunion1', redis)
>>> s2 = trol.Set('test_sunion2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['c', 'e'])
2
>>> s1.sunion(s2) == {'a', 'b', 'c', 'e'}
True

union(key, *other_sets)

Return a new Set representing the union of the other sets.

Parameters:

• key – String representing the key where to store the result.
• other_sets – list of other Set.

Returns:

a new Set representing the union of the other sets.

>>> s1 = trol.Set('test_union1', redis)
>>> s2 = trol.Set('test_union2', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2.add(['d', 'e'])
2
>>> s3 = s1.union('test_union3', s2)
>>> s3.members == {'a', 'c', 'b', 'e', 'd'}
True

update(*other_sets)

Update the set, adding elements from all other_sets.

Parameters:other_sets – list of Set Returns:None

>>> s1 = trol.Set('test_set_update1', redis)
>>> s1.add(['a', 'b', 'c'])
3
>>> s2 = trol.Set('test_set_update2', redis)
>>> s2.add(['b', 'c', 'd'])
3
>>> s1.update(s2)
>>> s1.members == {'a', 'b', 'c', 'd'}
True

class trol.collection.SortedSet(name=None, redis=None, typ=None, key=None, serializer=None, deserializer=None)

This class represents a SortedSet in redis. Use it if you want to arrange your set in any order.

add(members, score=1)

Add members in the set and assign them the score.

Parameters:

• members – a list of item or a single item
• score – the score the assign to the item(s)

>>> s = trol.SortedSet('test_zadd', redis)
>>> s.add('a')
1
>>> s.zscore('a')
1.0
>>> s.add('b', 20)
1
>>> s.zscore('b')
20.0
>>> s.add({'c':5, 'd':6})
2
>>> s.zscore('d')
6.0

between(min, max, limit=None, offset=None)

Returns the list of the members of the set that have scores between min and max.

Note

The min and max are inclusive when comparing the values.

Parameters:

• min – the minimum score to compare to.
• max – the maximum score to compare to.
• limit – limit the result to n elements
• offset – Skip the first n elements

>>> s = trol.SortedSet('test_between', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.add('c', 30)
1
>>> s.between(20, 30)
['b', 'c']

eq(value)

Returns the elements that have value for score.

ge(v, limit=None, offset=None, withscores=False)

Returns the list of the members of the set that have scores greater than or equal to v.

Parameters:

• v – the score to compare to.
• limit – limit the result to n elements
• offset – Skip the first n elements

gt(v, limit=None, offset=None, withscores=False)

Returns the list of the members of the set that have scores greater than v.

incr_by(value, att)

Increment the score of the item by value

Parameters:

• value – the value to add to the current score
• att – the member to increment

Returns:

the new score of the member

>>> s = trol.SortedSet('test_zincrby', redis)
>>> s.add('a', 10)
1
>>> s.zincrby(10, 'a')
20.0

le(v, limit=None, offset=None)

Returns the list of the members of the set that have scores less than or equal to v.

Parameters:

• v – the score to compare to.
• limit – limit the result to n elements
• offset – Skip the first n elements

lt(v, limit=None, offset=None)

Returns the list of the members of the set that have scores less than v.

Parameters:

• v – the score to compare to.
• limit – limit the result to n elements
• offset – Skip the first n elements

members

Returns the members of the set.

rank(elem)

Returns the rank of the element.

>>> s = trol.SortedSet('test_zrank', redis)
>>> s.add({'a': 30, 'b':20, 'c':10})
3
>>> s.zrank('b')
1

remove(*values)

Remove the values from the SortedSet

Returns:True if at least one value is successfully removed, False otherwise

>>> s = trol.SortedSet('test_zrem', redis)
>>> s.add('a', 10)
1
>>> s.zrem('a')
1
>>> s.members
[]

revmembers

Returns the members of the set in reverse.

revrank(member)

Returns the ranking in reverse order for the member

>>> s = trol.SortedSet('test_zrevrank', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.revrank('a')
1

score(elem)

Return the score of an element

>>> s = trol.SortedSet('test_zscore', redis)
>>> s.add("a", 10)
1
>>> s.score("a")
10.0

zadd(members, score=1)

Add members in the set and assign them the score.

Parameters:

• members – a list of item or a single item
• score – the score the assign to the item(s)

>>> s = trol.SortedSet('test_zadd', redis)
>>> s.add('a')
1
>>> s.zscore('a')
1.0
>>> s.add('b', 20)
1
>>> s.zscore('b')
20.0
>>> s.add({'c':5, 'd':6})
2
>>> s.zscore('d')
6.0

zcard()

Returns the cardinality of the SortedSet.

>>> s = trol.SortedSet('test_zcard', redis)
>>> s.add("a", 1)
1
>>> s.add("b", 2)
1
>>> s.add("c", 3)
1
>>> s.zcard()
3

zincrby(value, att)

Increment the score of the item by value

Parameters:

• value – the value to add to the current score
• att – the member to increment

Returns:

the new score of the member

>>> s = trol.SortedSet('test_zincrby', redis)
>>> s.add('a', 10)
1
>>> s.zincrby(10, 'a')
20.0

zrange(start, stop, withscores=False)

Returns all the elements including between start (non included) and stop (included).

Parameters:withscore – True if the score of the elements should also be returned

>>> s = trol.SortedSet('test_range', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.add('c', 30)
1
>>> s.zrange(1, 3)
['b', 'c']
>>> s.zrange(1, 3, withscores=True)
[('b', 20.0), ('c', 30.0)]

zrangebyscore(min, max, **kwargs)

Returns the range of elements included between the scores (min and max)

>>> s = trol.SortedSet('test_zrangebyscore', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.add('c', 30)
1
>>> s.zrangebyscore(20, 30)
['b', 'c']

zrank(elem)

Returns the rank of the element.

>>> s = trol.SortedSet('test_zrank', redis)
>>> s.add({'a': 30, 'b':20, 'c':10})
3
>>> s.zrank('b')
1

zrem(*values)

Remove the values from the SortedSet

Returns:True if at least one value is successfully removed, False otherwise

>>> s = trol.SortedSet('test_zrem', redis)
>>> s.add('a', 10)
1
>>> s.zrem('a')
1
>>> s.members
[]

zremrangebyrank(start, stop)

Remove a range of element between the rank start and stop both included.

Returns:the number of item deleted

>>> s = trol.SortedSet('test_zremrangebyrank', redis)
>>> s.add("a", 10)
1
>>> s.add("b", 20)
1
>>> s.add("c", 30)
1
>>> s.zremrangebyrank(1, 2)
2
>>> s.members
['a']

zremrangebyscore(min_value, max_value)

Remove a range of element by between score min_value and max_value both included.

Returns:the number of items deleted.

>>> s = trol.SortedSet('test_zremrangebyscore', redis)
>>> s.add("a", 10)
1
>>> s.add("b", 20)
1
>>> s.add("c", 30)
1
>>> s.zremrangebyscore(10, 20)
2
>>> s.members
['c']

zrevrange(start, end, **kwargs)

Returns the range of items included between start and stop in reverse order (from high to low)

>>> s = trol.SortedSet('test_zrevrange', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.add('c', 30)
1
>>> s.zrevrange(1, 2)
['b', 'a']
>>> s.clear()

zrevrangebyscore(max, min, **kwargs)

Returns the range of elements included between the scores (min and max)

>>> s = trol.SortedSet('test_zrevrangebyscore', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.add('c', 30)
1
>>> s.zrevrangebyscore(30, 20)
['c', 'b']

zrevrank(member)

Returns the ranking in reverse order for the member

>>> s = trol.SortedSet('test_zrevrank', redis)
>>> s.add('a', 10)
1
>>> s.add('b', 20)
1
>>> s.revrank('a')
1

zscore(elem)

Return the score of an element

>>> s = trol.SortedSet('test_zscore', redis)
>>> s.add("a", 10)
1
>>> s.score("a")
10.0

Lock

class trol.lock.Lock(name=None, timeout=None, sleep=0.1, blocking_timeout=None, lock_class=None, thread_local=True)

Lock provides a Redis-backed distributed lock on a Model or Database instance.

Lock should be a class member of a Model class, and provides a unique lock to each instance of that model. It is essentially a factory class for redis-py’s Lock objects

Attributes:

timeout (float or None): Maximum time the lock can be held before it expires, releasing it automatically. When set to None, the lock never expired. sleep (float): Time, in seconds, to sleep between each attempt to acquire the lock. blocking_timeout (float or None): Maximum time to spend acquiring the lock. lock_class (type or None): Class used to construct the lock. See redis.Lock for the canonical version. thread_local (bool): Whether to use threadlocal storage when to store the reservation token.

Todo

Lock currently only works when bound to an object, and not directly from the Database class. Lock should be refactored as a subclass of redis.lock.Lock to allow direct operations (instead of being built during the __get__ access) and/or Database should be refactored to no longer rely on janky “class-binding”.

>>> import trol
>>> import time
>>> from threading import Thread
>>> from redis import Redis
...
>>> class Sleepy(trol.Model):
...     redis = Redis()
...     sleepy_lock = trol.Lock()
...     sleep = time.sleep
...
...     def __init__(self, id):
...         self.id = id
...     
...     def print(self, t, msg):
...         with self.sleepy_lock:
...             self.sleep(t)
...             print(msg)
...
>>> sleepy = Sleepy('foo')
>>> Thread(target=sleepy.print, args=(3, "ok, go ahead")).start()
>>> Thread(target=sleepy.print, args=(1, "my turn!")).start()
>>> time.sleep(5)
ok, go ahead
my turn!

Just like with Property, the name of the lock is used to form it’s Redis key and can either by specified explicitly in the constructor or inferred from the attribute name in a Model.

>>> with sleepy.sleepy_lock:
...     print(sleepy.redis.keys())
[b'Sleepy:foo:sleepy_lock']
>>> print(sleepy.redis.keys())
[]

build(obj)

Builds a lock instance and assigns it to a field in obj for later retrieval.

Args:

obj (object): Model instance for which locking is provided.

Returns:

object: Lock of type specified as lock_class. Probably redis.Lock.

key(obj)

Gets the key where this lock exists in Redis

The key for this lock is the key of it’s holder, with :<name> appended

Args:

obj (object): Model instance for which locking is provided.

Returns:

str: The key which can be used to synchronize this lock data in Redis

static mangle(name)

Creates a mangled version of the given name.

Mangling produces a name unlikely to collide with other attribute names.

Args:

name (str): The name which should be mangled

Returns:

str: Mangled name

name

str: The name for this property, which will be used to determine the key when bound to a Model

Utilities

A set of utility functions for use in multiple parts of trol

This module currently contains the serialize and deserialize methods for the Property and Collection classes I implement

>>> import redis
>>> import trol
...
>>> class HotNewClass:
...     def __init__(self, howhot):
...         self.howhot = howhot
...
>>> @trol.serializer(HotNewClass)
... def hotnew_serializer(hnc):
...     print("HNC IS BEING SERIALIZED!")
...     return '<HOT>{}'.format(hnc.howhot)
...
>>> @trol.deserializer(HotNewClass)
... def hotnew_deserializer(byts):
...     print("RETURN OF THE HNC!")
...     howhot = int(byts.decode('utf-8').strip('<HOT>'))
...     return HotNewClass(howhot)
...
>>> class SweetModel(trol.Model):
...     def __init__(self, ident, redis):
...         self.id = ident
...         self.redis = redis
...
...     bar = trol.Property(typ=HotNewClass)
...
>>> r = redis.Redis('localhost')
>>> sm = SweetModel('xyz', r)
>>> sm.bar = HotNewClass(10)
HNC IS BEING SERIALIZED!
>>> r.get(sm.key + ':bar')
b'<HOT>10'
>>> sm.invalidate()
>>> sm.bar.howhot
RETURN OF THE HNC!
10
>>> r.flushall()
True

class trol.util.Deserializer(typ)

A class containing the provided deserialize functions for selected type

The deserializers here are aim for human-readability, which means they are not their most optimized If you want better performance through avoidance to and from str, implement a new deserializer

Attributes:

func (Callable[[bytes], object]: The deserialization function this class is constructed to use

The return type is determined by the type used to construct this object

Args:

typ (type): The type of objects which will be deserialized by this Deserializer

Currently supported types are: str, int, float, and bytes

class trol.util.Serializer(typ)

A class containing the provided serialize functions for selected type

The serializers here are aim for human-readability, which means they are not their most optimized If you want better performance through avoidance to and from str, implement a new serializer

Attributes:

func (Callable[[object], object]: The serialization function this class is constructed to use

The input and return types are determined by the type used to construct this object The output type will be a serial type accepted by redis-py

Args:

typ (type): The type of objects which will be serialized by this Serializer

Currently supported types are: str, int, float, and bytes

trol.util.deserializer(cls)

A convinience decorator to register a deserializer

trol.util.deserializers = {<class 'str'>: <function deserialize_str>, <class 'int'>: <function deserialize_int>, <class 'float'>: <function deserialize_float>, <class 'bytes'>: <function deserialize_bytes>, <class 'bool'>: <function deserialize_bool>, <class 'trol.model.Model'>: <function deserialize_model>}

dict[type, Callable[[bytes], object]]: A dictionary of deserializers known trol classes

Additonal entries can be added to support new deserializable types There should be an entry here for each one in serializers

trol.util.serializer(cls)

A convinience decorator to register a serializer

trol.util.serializers = {<class 'str'>: <function serialize_str>, <class 'int'>: <function serialize_int>, <class 'float'>: <function serialize_float>, <class 'bytes'>: <function serialize_bytes>, <class 'bool'>: <function serialize_bool>, <class 'trol.model.Model'>: <function serialize_model>}

dict[type, Callable[[object], bytes]]: A dictionary of serializers known trol classes

Additonal entries can be added to support new serializable types