Collections (Data Structures + helpers)¶

Table of Contents

Collections (Data Structures + helpers)

Functions, classes and/or types which either are, or are related to Python variable storage types (dict, tuple, list, set etc.)

Object-like Dictionaries (dict’s)¶

Have you ever wanted a dictionary that works like an object, where you can get/set dictionary keys using attributes (x.something) as easily as you can with items (x['something'])?

We did. So we invented DictObject, a sub-class of the built-in dict, making it compatible with most functions/methods which expect a dict (e.g. json.dumps()).

You can create a new DictObject and use it just like a dict, or you can convert an existing dict into a DictObject much like you’d cast any other builtin type.

It can also easily be cast back into a standard dict when needed, without losing any data.

Creating a new DictObject and using it ¶

Since DictObject is a subclass of the builtin dict, you can instantiate a new DictObject in the same way you would use the standard dict class:

>>> d = DictObject(hello='world')
>>> d
{'hello': 'world'}
>>> d['hello']
'world'
>>> d.hello
'world'
>>> d.lorem = 'ipsum'
>>> d['orange'] = 'banana'
>>> d
{'hello': 'world', 'lorem': 'ipsum', 'orange': 'banana'}

Converting an existing dictionary (dict) into a DictObject ¶

You can convert an existing dict into a DictObject in the same way you’d convert any other object into a dict:

>>> y = {"hello": "world", "example": 123}
>>> x = DictObject(y)
>>> x.example
123
>>> x['hello']
'world'
>>> x.hello = 'replaced'
>>> x
{'hello': 'replaced', 'example': 123}

It also works vice versa, you can convert a DictObject instance back into a dict just as easily as you converted the dict into a DictObject.

>>> z = dict(x)
>>> z
{'hello': 'replaced', 'example': 123}

Dict-able NamedTuple’s ¶

While collections.namedtuple()’s can be useful, they have some quirks, such as not being able to access fields by item/key (x['something']). They also expose a method ._asdict(), but cannot be directly casted into a dict using dict(x).

Our dictable_namedtuple() collection is designed to fix these quirks.

What is a dictable_namedtuple and why use it?¶

Unlike the normal namedtuple() types, ``dictable_namedtuple``s add extra convenience functionality:

Can access fields via item/key: john['first_name']

Can convert instance into a dict simply by casting: dict(john)

Can set new items/attributes on an instance, even if they weren’t previously defined.

NOTE: You cannot edit an original namedtuple field defined on the type, those remain read only

There are three functions available for working with dictable_namedtuple classes/instances, each for different purposes.

dictable_namedtuple() - Create a new dictable_namedtuple type for instantiation.

convert_dictable_namedtuple() - Convert an existing namedtuple instance (not a type/class) into a dictable_namedtuple instance.

subclass_dictable_namedtuple() - Convert an existing namedtuple type/class (not an instance) into a dictable_namedtuple type for instantiation.

Importing dictable_namedtuple functions ¶

from collections import namedtuple
from privex.helpers import dictable_namedtuple, convert_dictable_namedtuple, subclass_dictable_namedtuple

Creating a NEW dictable_namedtuple type and instance ¶

If you’re creating a new Named Tuple, and you want it to support dictionary-like access, and have it able to be converted into a dict simply through dict(my_namedtuple), then you want dictable_namedtuple()

Person = dictable_namedtuple('Person', 'first_name last_name')
john = Person('John', 'Doe')
dave = Person(first_name='Dave', last_name='Smith')
print(dave['first_name'])       # Prints:  Dave
print(dave.first_name)          # Prints:  Dave
print(john[1])                  # Prints:  Doe
print(dict(john))               # Prints:  {'first_name': 'John', 'last_name': 'Doe'}

Converting an existing namedtuple instance into a dictable_namedtuple instance ¶

If you have existing Named Tuple instances, e.g. returned from a python library, then you can use convert_dictable_namedtuple() to convert them into dictable_namedtuple’s and gain all the functionality mentioned at the start of this section.

Person = namedtuple('Person', 'first_name last_name')  # This is an existing namedtuple "type" or "class"
john = Person('John', 'Doe')  # This is an existing namedtuple instance
john.first_name               # This works on a standard namedtuple. Returns: John
john[1]                       # This works on a standard namedtuple. Returns: Doe
john['first_name']            # However, this would throw a TypeError.
dict(john)                    # And this would throw a ValueError.

# We can now convert 'john' into a dictable_namedtuple, which will retain the functionality of a
# namedtuple, but add to the functionality by allowing dict-like key access, updating/creating new
# fields, as well as painlessly casting to a dictionary.

d_john = convert_dictable_namedtuple(john)
d_john.first_name               # Returns: John
d_john[1]                       # Returns: Doe
d_john['first_name']            # Returns: 'John'
dict(d_john)                    # Returns: {'first_name': 'John', 'last_name': 'Doe'}

Converting an existing namedtuple type/class into a dictable_namedtuple type/class ¶

If you have existing Named Tuple type/class then you can use subclass_dictable_namedtuple() to convert the type/class into a dictable_namedtuple type/class and gain all the functionality mentioned at the start of this section. (NOTE: it’s usually easier to just replace your namedtuple calls with dictable_namedtuple)

Person = namedtuple('Person', 'first_name last_name')  # This is an existing namedtuple "type" or "class"
# We can now convert the 'Person' type into a dictable_namedtuple type.
d_Person = subclass_dictable_namedtuple(Person)
# Then we can use this converted type to create instances of Person with dictable_namedtuple functionality.
john = d_Person('John', 'Doe')
john.first_name               # Returns: John
john[1]                       # Returns: Doe
john['first_name']            # Returns: 'John'
dict(john)                    # Returns: {'first_name': 'John', 'last_name': 'Doe'}

Copyright:

    +===================================================+
    |                 © 2019 Privex Inc.                |
    |               https://www.privex.io               |
    +===================================================+
    |                                                   |
    |        Originally Developed by Privex Inc.        |
    |        License: X11 / MIT                         |
    |                                                   |
    |        Core Developer(s):                         |
    |                                                   |
    |          (+)  Chris (@someguy123) [Privex]        |
    |          (+)  Kale (@kryogenic) [Privex]          |
    |                                                   |
    +===================================================+

Copyright 2019     Privex Inc.   ( https://www.privex.io )

class privex.helpers.collections.DictDataClass[source]¶

This is a base class for use with Python 3.7+ dataclass ‘s, designed to make dataclasses more interoperable with existing dictionaries, and allows them to be used like dictionaries, similar to Dictable, but more powerful / flexible.

The most notable difference between this and Dictable - is that DictDataClass uses the attribute raw_data on your dataclass to store any excess attributes when your dataclass is initialised from a dictionary with from_dict() or from_list(), allowing you to retrieve any dictionary keys which couldn’t be stored on your dataclass instance.

Basic Example:

>>> from dataclasses import dataclass, field
>>> from privex.helpers import DictDataClass, DictObject
>>> from typing import Union
>>>
>>> @dataclass
>>> class ExampleDataclass(DictDataClass):
...     example: str = 'hello world'
...     lorem: int = 999
...     raw_data: Union[dict, DictObject] = field(default_factory=DictObject, repr=False)
...     ### ^ The raw, unmodified data that was passed as kwargs, as a dictionary
...     ### For DictDataClass to work properly, you must include the raw_data dataclass field in your dataclass.
...
>>> edc = ExampleDataclass.from_dict(dict(example='test', hello='this is an example'))
>>> edc.example
'test'
>>> dict(edc)
{'example': 'test', 'lorem': 999}

Thanks to raw_data - you can access any extraneous items contained within the dictionary used in from_dict() as if they were part of your dataclass. You can also access and set any attribute using standard dictionary item syntax with square brackets like so:

>>> edc.hello               # This is not in the original dataclass attributes, but is proxied from raw_data
'this is an example'
>>> edc['hello']            # Also works with item / dict syntax
'this is an example'
>>> edc['hello'] = 'world'  # You can set attributes using "key" / dict-like syntax
>>> edc.hello
'world'
>>> edc.hello = 'test'      # You can also set raw_data keys using standard attribute dot-notation syntax.
>>> edc['hello']
'test'

Dictionary casting modes / ``__iter__`` configuration modes

There are a total of four (4) dict` conversion modes that you may use for a given class.

These modes control whether raw_data is used when an instance is being casted via dict(obj), the order in which it’s merged with the instance attributes, along with the option to include just the dataclass attributes, or just the raw_data.

Available DictConfig.dict_convert_mode options:

* **Fallback / Dataclass / Instance Attributes Only** - When ``dict_convert_mode`` is empty (``None`` or ``""``), or
  it can't be matched against a pre-defined conversion mode, when an instance is converted into a :class:`.dict` - only the
  attributes of the instance are used - :attr:`.raw_data` keys are ignored.

  ``dict_convert_mode`` settings: ``None``, ``""``, ``"none"`` or any other invalid option.

* Raw Data Only ( ``raw`` / ``raw_data`` ) - When this mode is used, converting the instance to a ``dict`` will
  effectively just return ``raw_data``, while still enforcing the ``dict_exclude`` and ``dict_listify`` settings.

  ``dict_convert_mode`` settings: ``"raw"``, ``"raw_data"``, ``"rawdata"``, or any other value beginning with ``raw``


* Merge with Dataclass Priority - In this mode, both :attr:`.raw_data` and the instance's attributes are used when converting
  into a :class:`.dict` - first the :attr:`.raw_data` dictionary is taken, and we merge the instance attributes on top of it.

  This means the instance/dataclass attributes take priority over the ``raw_data`` attributes, which will generally result
  in only ``raw_data`` keys which don't exist on the instance having their values used in the final ``dict``.

  ``dict_convert_mode`` settings: ``"merge_dc"`` / ``"merge_ins"`` / ``"merge_dataclass"``

* Merge with :attr:`.raw_data` Priority - In this mode, both :attr:`.raw_data` and the instance's attributes are used when
  converting into a :class:`.dict` - first the instance attributes are converted into a dict, then we merge the
  :attr:`.raw_data` dictionary on top of it.

  This means the :attr:`.raw_data` keys take priority over the instance/dataclass attributes, which will generally result in
  only instance attributes which don't exist in ``raw_data`` having their values used in the final ``dict``.

  ``dict_convert_mode`` settings: ``"merge_rd"`` / ``"merge_raw"`` / ``"merge_raw_data"``

By default, the conversion mode is set to merge_dc, which means when an instance is converted into a dict - the dataclass instance attributes are merged on top of the raw_data dictionary, meaning raw_data is included, but dataclass attributes take priority over raw_data keys.

Changing the conversion mode

>>> from dataclasses import dataclass, field
>>>
>>> @dataclass
>>> class MyDataclass(DictDataClass):
...     class DictConfig:
...         dict_convert_mode = 'merge_dc'
...
...     hello: str
...     lorem: str = 'ipsum'
...     raw_data: Union[dict, DictObject] = field(default_factory=DictObject, repr=False)
>>> dc = MyDataclass.from_dict(dict(hello='test', example=555, test='testing'))
>>> dc.hello
'test'
>>> dc.hello = 'replaced'
>>> dict(dc)
{'hello': 'replaced', 'example': 555, 'test': 'testing', 'lorem': 'ipsum'}
>>> dc.DictConfig.dict_convert_mode = 'merge_raw'
{'hello': 'test', 'lorem': 'ipsum', 'example': 555, 'test': 'testing'}
>>> dc.DictConfig.dict_convert_mode = None
{'hello': 'replaced', 'lorem': 'ipsum'}

privex.helpers.collections.DictDataclass¶: alias of privex.helpers.collections.DictDataClass

class privex.helpers.collections.DictObject[source]¶

A very simple dict wrapper, which allows you to read and write dictionary keys using attributes (dot notation) PLUS standard item (key / square bracket notation) access.

Example Usage (creating and using a new DictObject):

>>> d = DictObject(hello='world')
>>> d
{'hello': 'world'}
>>> d['hello']
'world'
>>> d.hello
'world'
>>> d.lorem = 'ipsum'
>>> d['orange'] = 'banana'
>>> d
{'hello': 'world', 'lorem': 'ipsum', 'orange': 'banana'}

Example Usage (converting an existing dict):

>>> y = {"hello": "world", "example": 123}
>>> x = DictObject(y)
>>> x.example
123
>>> x['hello']
'world'
>>> x.hello = 'replaced'
>>> x
{'hello': 'replaced', 'example': 123}

class privex.helpers.collections.Dictable[source]¶

A small abstract class for use with Python 3.7 dataclasses.

Allows dataclasses to be converted into a dict using the standard dict() function:

>>> @dataclass
>>> class SomeData(Dictable):
...     a: str
...     b: int
...
>>> mydata = SomeData(a='test', b=2)
>>> dict(mydata)
{'a': 'test', 'b': 2}

Also allows creating dataclasses from arbitrary dictionaries, while ignoring any extraneous dict keys.

If you create a dataclass using a dict and you have keys in your dict that don’t exist in the dataclass, it’ll generally throw an error due to non-existent kwargs:

>>> mydict = dict(a='test', b=2, c='hello')
>>> sd = SomeData(**mydict)
TypeError: __init__() got an unexpected keyword argument 'c'

Using from_dict you can simply trim off any extraneous dict keys:

>>> sd = SomeData.from_dict(**mydict)
>>> sd.a, sd.b
('test', 2)
>>> sd.c
AttributeError: 'SomeData' object has no attribute 'c'

class privex.helpers.collections.Mocker(modules: dict = None, attributes: dict = None, *args, **kwargs)[source]¶

This mock class is designed to be used either to act as a stand-in “noop” (no operation) object, which could be used either as a drop-in replacement for a failed module / class import, or for certain unit tests.

If you need additional functionality such as methods having actual behaviour, you can set attributes on a Mocker instance to either a lambda, or point them at a real function/method:

>>> m = Mocker()
>>> m.some_func = lambda a: a+1
>>> m.some_func(5)
6

Example use case - fallback for unimportant module imports

Below is a real world example of using Mocker and privex.helpers.decorators.mock_decorator() to simulate pytest - allowing your tests to run under the standard unittest framework if a user doesn’t have pytest (as long as your tests aren’t critically dependent on PyTest).

Try importing pytest then fallback to a mock pytest:

>>> try:
...     import pytest
... except ImportError:
...     from privex.helpers import Mocker, mock_decorator
...     print('Failed to import pytest. Using privex.helpers.Mocker to fake pytest.')
...     # Make pytest pretend to be the class 'module' (the class actually used for modules)
...     pytest = Mocker.make_mock_class('module')
...     # To make pytest.mark.skip work, we add the fake module 'mark', then set skip to `mock_decorator`
...     pytest.add_mock_module('mark')
...     pytest.mark.skip = mock_decorator
...

Since we added the mock module mark, and set the attribute skip to point at mock_decorator, the test function test_something won’t cause a syntax error. mock_decorator will just call test_something() which doesn’t do anything anyway:

>>> @pytest.mark.skip(reason="this test doesn't actually do anything...")
... def test_something():
...     pass
>>>
>>> def test_other_thing():
...     if True:
...         return pytest.skip('cannot test test_other_thing because of an error')
...
>>>

Generating “disguised” mock classes

If you need the mock class to appear to have a certain class name and/or module path, you can generate “disguised” mock classes using make_mock_class() like so:

>>> redis = Mocker.make_mock_class('Redis', module='redis')
>>> redis
<redis.Redis object at 0x7fd7402ea4a8>

A :class:`.Mocker` instance has the following behaviour

Attributes that don’t exist result in a function being returned, which accepts any arguments / keyword args, and simply returns None

Example:

>>> m = Mocker()
>>> repr(m.randomattr('hello', world=123))
'None'

Arbitrary attributes x.something and items x['something'] can be set on an instance, and they will be similarly returned when they’re accessed. Attributes and items share the same key/value’s, so the following examples are all accessing the same data:

Example:

>>> m = Mocker()
>>> m.example = 'hello'
>>> m['example'] = 'world'
>>> print(m.example)
world
>>> print(m['example'])
world

You can add arbitrary “modules” to a Mocker instance. With only the name argument, add_mock_module() will add a “module” under the instance, which is really just another Mocker instance.

Example:

>>> m = Mocker()
>>> m.add_mock_module('my_module')
>>> m.my_module.example = 'hello'
>>> print(m.my_module['example'], m.my_module.example)
hello hello

add_mock_module(name: str, value=None, mock_attrs: dict = None, mock_modules: dict = None)[source]¶

Add a fake sub-module to this Mocker instance.

Example:

>>> m = Mocker()
>>> m.add_mock_module('my_module')
>>> m.my_module.example = 'hello'
>>> print(m.my_module['example'], m.my_module.example)
hello hello

Parameters

name (str) – The name of the module to add.
value – Set the “module” to this object, instead of an instance of Mocker
mock_attrs (dict) – If value is None, then this can optionally contain a dictionary of attributes/items to pre-set on the Mocker instance.
mock_modules (dict) – If value is None, then this can optionally contain a dictionary of “modules” to pre-set on the Mocker instance.

add_mock_modules(*module_list, _dict_to_attrs=True, _parse_dict=True, **module_map)[source]¶

>>> hello = Mocker.make_mock_class('Hello')
>>> hello.add_mock_modules(
...     world={
...         'lorem': 'ipsum',
...         'dolor': 123,
...     }
... )

Parameters

module_list –
_parse_dict –
_dict_to_attrs –
module_map –

Returns

classmethod make_mock_class(name='Mocker', instance=True, simple=False, attributes: dict = None, modules: dict = None, **kwargs) → Union[Any, privex.helpers.collections.Mocker, Type[privex.helpers.collections.Mocker]][source]¶

Return a customized mock class or create an instance which appears to be named name

Allows code which might check x.__class__.__name__ to believe it’s the correct object.

Using the kwarg module you can change the module that the class / instance appears to have been imported from, allowing for quite deceiving fake classes and instances.

Example usage:

>>> redis = Mocker.make_mock_class('Redis', module='redis')
>>> # As seen below, the class appears to be called Redis, and even claims to be from the module `redis`
>>> redis
<redis.Redis object at 0x7fd7402ea4a8>
>>> print(f'Module: {redis.__module__} - Class Name: {redis.__class__.__name__}')
Module: redis - Class Name: Redis

Creating methods/attributes dynamically

You can set arbitrary attributes to point at a function, or just set them to a lambda:

>>> redis.exists = lambda key: 1
>>> redis.exists('hello')
1
>>> redis.hello()  # Non-existent attributes just act as a function that eats any args and returns None
None

Parameters

name – The name to write onto the mock class’s __name__ (and __qualname__ if not specified)
instance (bool) – If True then the disguised mock class will be returned as an instance. Otherwise the raw class itself will be returned for you to instantiate yourself.
simple (bool) – When True, generates a very basic, new class - not based on Mocker, which contains the attributes/methods defined in the param attributes.
kwargs – All kwargs (other than qualname) are forwarded to __init__ of the disguised class if instance is True.
attributes (dict) – If simple is True, then this dictionary of attributes is used to generate the class’s attributes, methods, and/or constructor. If simple is False, and instance is True, these attributes are passed to the constructor of the Mocker clone that was generated.
modules (dict) – If simple is False, and instance is True, this dict of modules are passed to the constructor of the Mocker clone that was generated.

Key str qualname

Optionally specify the “qualified name” to insert into __qualname__. If this isn’t specified, then name is used for qualname, which is fine for most cases anyway.

Key str module

Optionally override the module namespace that the class is supposedly from. If not specified, then the class will just inherit this module (privex.helpers.common)

Returns

class privex.helpers.collections.OrderedDictObject[source]¶: Ordered version of DictObject - dictionary with attribute access. See DictObject

privex.helpers.collections.convert_dictable_namedtuple(nt_instance, typename=None, module=None, **kwargs) → Union[NamedTuple, Dict][source]¶

Convert an existing collections.namedtuple() instance into a dictable_namedtuple instance.

Example

First we create a namedtuple type Person

>>> from collections import namedtuple
>>> Person = namedtuple('Person', 'first_name last_name')

Next we create an instance of Person called John Doe, and we can confirm it’s a normal namedtuple, as we can’t access first_name by item/key.

>>> john = Person('John', 'Doe')
>>> john['first_name']
TypeError: tuple indices must be integers or slices, not str

Using convert_dictable_namedtuple(), we can convert john from a normal namedtuple, into a dictable_namedtuple.

This enables many convenience features (see dictable_namedtuple() for more info) such as easy casting to a dict, and accessing fields by item/key (square brackets):

>>> from privex.helpers import convert_dictable_namedtuple
>>> d_john = convert_dictable_namedtuple(john)
>>> d_john
Person(first_name='John', last_name='Doe')
>>> d_john['first_name']
'John'
>>> dict(d_john)
{'first_name': 'John', 'last_name': 'Doe'}

Parameters

nt_instance – An instantiated namedtuple object (using a type returned from collections.namedtuple())
typename (str) – Optionally, you can change the name of your instance’s class, e.g. if you provide a Person instance, but you set this to Man, then this will return a Man instance, like so: Man(first_name='John', last_name='Doe')
module (str) – Optionally, you can change the module that the type class belongs to. Otherwise it will inherit the module path from the class of your instance.

Key bool read_only

(Default: False) If set to True, the outputted dictable_namedtuple instance will not allow new fields to be created via attribute / item setting.

Return dictable_namedtuple

The instance you passed nt_instance, converted into a dictable_namedtuple

privex.helpers.collections.copy_class(obj: Type[T], name=None, deep_copy=True, deep_private=False, **kwargs) → Union[Type[T], type][source]¶

Attempts to create a full copy of a type or class, severing most object pointers such as attributes containing a dict / list, along with classes or instances of classes.

Example:

>>> class SomeClass:
>>>     example = 'lorem ipsum'
>>>     data = ['hello', 'world']
>>>     testing = 123
>>>
>>> from privex.helpers import copy_class
>>> OtherClass = copy_class(SomeClass, name='OtherClass')

If you then append to the list attribute data on both SomeClass and OtherClass - with a different item appended to each class, you’ll see that the added item was only added to data for that class, and not to the other class, proving the original and the copy are independent from each other:

>>> SomeClass.data.append('lorem')
>>> OtherClass.data.append('ipsum')
>>> SomeClass.data
['hello', 'world', 'lorem']
>>> OtherClass.data
['hello', 'world', 'ipsum']

Parameters

obj (Type[T]) – A type / class to attempt to duplicate, deep copying each individual object in the class, to avoid any object pointers shared between the original and the copy.
name (str|None) – The class name to use for the copy of obj. If not specified, defaults to the original class name from obj
deep_copy (bool) – (Default: True) If True, uses copy.deepcopy() to deep copy each attribute in obj to the copy. If False, then standard references will be used, which may result in object pointers being copied.
deep_private (bool) – (Default: False) If True, copy.deepcopy() will be used on “private” class attributes, i.e. ones that start with __. If False, attributes starting with __ will not be deep copied, only a standard assignment/reference will be used.
kwargs – Additional advanced settings (see keyword pydoc entries for this function)
use_bases (bool) – (Default: True) If True, copy the inheritance (bases) from obj into the class copy.
quiet (bool) – (Default False) If True, log deep copy errors as debug level (usually silent in production apps) instead of the louder warning.
bases (tuple) – A tuple of classes to use as “bases” (inheritance) for the class copy. If not specified, copies __bases__ from the original class.
module (str) – If specified, overrides the module __module__ in the class copy with this string, instead of copying from the original class.

Return Type[T] obj_copy

A deep copy of the original obj

privex.helpers.collections.copy_class_simple(obj: Type[T], name=None, qualname=None, module=<class 'privex.helpers.types.AutoDetected'>, allow_attrs: list = None, ban_attrs: list = None, **kwargs)[source]¶

This is an alternative to copy_class() which simply creates a blank class, then iterates over obj.__dict__, using setattr() to copy each attribute over to the cloned class.

It uses _q_copy() to safely deep copy any attributes which are object references, and thus need their reference pointers severed, to avoid edits to the copy affecting the original (and vice versa).

Parameters

obj – The class to duplicate
name (str) – The class name to set on the duplicate. If left as None, the duplicate will retain the original obj name.
qualname (str) – The qualified class name to set on the duplicate.
module (Optional[str]) – The module path to set on the duplicate, e.g. privex.helpers.common
allow_attrs (list) – Optionally, you may specify additional private attributes (ones which start with __) that are allowed to be copied from the original class to the duplicated class.
ban_attrs (list) –
Optionally, you may blacklist certain attributes from being copied from the original class to the duplicate.

Blacklisted attributes take priority over whitelisted attributes, so you may use this to cancel out any attributes in the default attribute whitelist DEFAULT_ALLOWED_DUPE which you don’t want to be copied to the duplicated class.
kwargs –
bases (tuple|list) – If specified, overrides the default inherited classes (obj.__bases__) which would be set on the duplicated class’s __bases__.

Returns

privex.helpers.collections.copy_func(f: Callable, rewrap_classmethod=True, name=None, qualname=None, module=<class 'privex.helpers.types.AutoDetected'>, **kwargs) → Union[Callable, classmethod][source]¶: Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)

privex.helpers.collections.dictable_namedtuple(typename, field_names, *args, **kwargs) → Union[Type[collections.namedtuple], dict][source]¶

Creates a dictable_namedtuple type for instantiation (same usage as collections.namedtuple()) - unlike namedtuple, dictable_namedtuple instances allow item (dict-like) field access, support writing and can be painlessly converted into dictionaries via dict(my_namedtuple).

Named tuple instances created from dictable_namedtuple types are generally backwards compatible with any code that expects a standard collections.namedtuple() type instance.

Quickstart

>>> from privex.helpers import dictable_namedtuple
>>> # Define a dictable_namedtuple type of 'Person', which has two fields - first_name and last_name
>>> p = dictable_namedtuple('Person', 'first_name last_name')
>>> john = p('John', 'Doe')    # Alternatively you can do p(first_name='John', last_name='Doe')
>>> john.first_name            # You can retrieve keys either via attributes (dot notation)
'John'
>>> john['last_name']          # Via named keys (square brackets)
'Doe'
>>> john[1]                    # Or, via indexed keys (square brackets, with integer keys)
'Doe'
>>> john.middle_name = 'Davis' # You can also update / set new keys via attribute/key/index
>>> dict(john)                 # Newly created keys will show up as normal in dict(your_object)
{'first_name': 'John', 'last_name': 'Doe', 'middle_name': 'Davis'}
>>> john                       # As well as in the representation in the REPL or when str() is called.
Person(first_name='John', last_name='Doe', middle_name='Davis')

This function adds / overrides the following methods on the generated namedtuple type:

_asdict
__iter__
__getitem__
__getattribute__
__setitem__
__setattr__
__repr__

Extra functionality compared to the standard namedtuple() generated classes:

Can access fields via item/key: john['first_name']

Can convert instance into a dict simply by casting: dict(john)

Can set new items/attributes on an instance, even if they weren’t previously defined. john['middle_name'] = 'Davis' or john.middle_name = 'Davis'

Example Usage

First we’ll create a named tuple typle called Person, which takes two arguments, first_name and last_name.

>>> from privex.helpers import dictable_namedtuple
>>> Person = dictable_namedtuple('Person', 'first_name last_name')

Now we’ll create an instance of Person called john. These instances look like normal namedtuple’s, and should be generally compatible with any functions/methods which deal with named tuple’s.

>>> john = Person('John', 'Doe')   # Alternatively you can do Person(first_name='John', last_name='Doe')
>>> john
Person(first_name='John', last_name='Doe')

Unlike a normal namedtuple type instance, we can access fields by attribute (.first_name), index ([0]), AND by item/key name (['last_name']).

>>> john.first_name
'John'
>>> john[0]
'John'
>>> john['last_name']
'Doe'

Another potentially useful feature, is that you can also update / create new fields, via your preferred method of field notation (other than numbered indexes, since those don’t include a field name):

>>> john['middle_name'] = 'Davis'
>>> john.middle_name = 'Davis'

We can also convert john into a standard dictionary, with a simple dict(john) cast. You can see that the new field we added (middle_name) is present in the dictionary serialized format.

>>> dict(john)
{'first_name': 'John', 'last_name': 'Doe', 'middle_name': 'Davis'}

Parameters

typename (str) – The name used for the namedtuple type/class
field_names (str) – One or more field names separated by spaces, e.g. 'id first_name last_name address'

Key bool read_only

(Default: False) If set to True, the outputted dictable_namedtuple instance will not allow new fields to be created via attribute / item setting.

Return Type[namedtuple] dict_namedtuple

A dict_namedtuple type/class which can be instantiated with the given field_names via positional or keyword args.

privex.helpers.collections.generate_class(name: str, qualname: str = None, module: str = None, bases: Union[tuple, list] = None, attributes: Dict[str, Any] = None, **kwargs) → Any[source]¶

A small helper function for dynamically generating classes / types.

Basic usage

Generating a simple class, with an instance constructor, a basic instance method, and an instance factory classmethod:

>>> import random
>>> from privex.helpers.collections import generate_class
>>> def hello_init(self, example: int):
...     self.example = example
...
>>> Hello = generate_class(
...     'Hello', module='hello',
...     attributes=dict(
...         __init__=hello_init, lorem=lambda self: self.example * 10,
...         make_hello=classmethod(lambda cls: cls(random.randint(1, 100)))
...     )
... )
...
>>> h = Hello(123)
>>> h.lorem()
1230
>>> j = Hello.make_hello()
>>> j.example
77
>>> j.lorem()
770

Generating a child class which inherits from an existing class (the parent(s) can also be a generated classes):

>>> World = generate_class(
...     'World', module='hello', bases=(Hello,), attributes=dict(ipsum=lambda self: float(self.example) / 3)
... )
>>> w = World(130)
>>> w.lorem()
1300
>>> w.ipsum()
43.333333333333336

Parameters

name (str) – The name of the class, e.g. Hello
qualname (str) – (Optional) The qualified name of the class, e.g. for nested classes A -> B -> C, class C would have the __name__: C and __qualname__: A.B.C
module (str) – (Optional) The module the class should appear to belong to (sets __module__)
bases (tuple|list) – (Optional) A tuple or list of “base” / “parent” classes for inheritance.
attributes (dict) – (Optional) A dictionary of attributes to add to the class. (can include constructor + methods)
kwargs –

Returns

privex.helpers.collections.generate_class_kw(name: str, qualname: str = None, module: str = None, bases: Union[tuple, list] = None, **kwargs) → Type[source]¶

Same as generate_class(), but instead of a dict attributes parameter - all additional keyword arguments will be used for attributes

Example:

>>> def lorem_init(self, ipsum=None):
...     self._ipsum = ipsum
...
>>> Lorem = generate_class_kw('Lorem',
...     __init__=lorem_init, hello=staticmethod(lambda: 'world'),
...     ipsum=property(lambda self: 0 if self._ipsum is None else self._ipsum)
... )
>>> l = Lorem()
>>> l.ipsum()
0
>>> l.hello()
'world'

privex.helpers.collections.is_namedtuple(*objs) → bool [source]¶

Takes one or more objects as positional arguments, and returns True if ALL passed objects are namedtuple instances

Example usage

First, create or obtain one or more NamedTuple objects:

>>> from collections import namedtuple

>>> Point, Person = namedtuple('Point', 'x y'), namedtuple('Person', 'first_name last_name')

>>> pt1, pt2 = Point(1.0, 5.0), Point(2.5, 1.5)
>>> john = Person('John', 'Doe')

We’ll also create a tuple, dict, and str to show they’re detected as invalid:

>>> normal_tuple, tst_dict, tst_str = (1, 2, 3,), dict(hello='world'), "hello world"

First we’ll call is_namedtuple() with our Person NamedTuple object john:

>>> is_namedtuple(john)
True

As expected, the function shows john is in-fact a named tuple.

Now let’s try it with our two Point named tuple’s pt1 and pt2, plus our Person named tuple john.

>>> is_namedtuple(pt1, john, pt2)
True

Since all three arguments were named tuples (even though pt1/pt2 and john are different types), the function returns True.

Now we’ll test with a few objects that clearly aren’t named tuple’s:

>>> is_namedtuple(tst_str)   # Strings aren't named tuples.
False
>>> is_namedtuple(normal_tuple)    # A plain bracket tuple is not a named tuple.
False
>>> is_namedtuple(john, tst_dict)  # ``john`` is a named tuple, but a dict isn't, thus False is returned.
False

Original source: https://stackoverflow.com/a/2166841

Parameters: objs (Any) – The objects (as positional args) to check whether they are a NamedTuple
Return bool is_namedtuple: True if all passed objs are named tuples.

privex.helpers.collections.make_dict_tuple(typename, field_names, *args, **kwargs)[source]¶

Generates a collections.namedtuple() type, with added / modified methods injected to make it into a dictable_namedtuple.

Note: You probably want to be using dictable_namedtuple() instead of calling this directly.

privex.helpers.collections.subclass_dictable_namedtuple(named_type: type, typename=None, module=None, **kwargs) → type [source]¶

Convert an existing collections.namedtuple() type into a dictable_namedtuple.

If you have an INSTANCE of a type (e.g. it has data attached), use convert_dictable_namedtuple()

Example:

>>> from collections import namedtuple
>>> from privex.helpers import subclass_dictable_namedtuple
>>> # Create a namedtuple type called 'Person'
>>> orig_Person = namedtuple('Person', 'first_name last_name')
>>> # Convert the 'Person' type into a dictable_namedtuple
>>> Person = subclass_dictable_namedtuple(orig_Person)
>>> john = Person('John', 'Doe')   # Create an instance of this dictable_namedtuple Person
>>> john['middle_name'] = 'Davis'

Parameters

named_type (type) – A NamedTuple type returned from collections.namedtuple()
typename (str) – Optionally, you can change the name of your type, e.g. if you provide a Person class type, but you set this to Man, then this will return a Man class type.
module (str) – Optionally, you can change the module that the type class belongs to. Otherwise it will inherit the module path from named_type.

Key bool read_only

(Default: False) If set to True, the outputted dictable_namedtuple type will not allow new fields to be created via attribute / item setting.

Return type dictable_namedtuple

Your named_type converted into a dictable_namedtuple type class.

Module Contents ¶

Functions

copy_class(obj[, name, deep_copy, deep_private])

Attempts to create a full copy of a type or class, severing most object pointers such as attributes containing a dict / list, along with classes or instances of classes.

convert_dictable_namedtuple(nt_instance[, …])

Convert an existing collections.namedtuple() instance into a dictable_namedtuple instance.

dictable_namedtuple(typename, field_names, …)

Creates a dictable_namedtuple type for instantiation (same usage as collections.namedtuple()) - unlike namedtuple, dictable_namedtuple instances allow item (dict-like) field access, support writing and can be painlessly converted into dictionaries via dict(my_namedtuple).

is_namedtuple(*objs)

Takes one or more objects as positional arguments, and returns True if ALL passed objects are namedtuple instances

make_dict_tuple(typename, field_names, …)

Generates a collections.namedtuple() type, with added / modified methods injected to make it into a dictable_namedtuple.

subclass_dictable_namedtuple(named_type[, …])

Convert an existing collections.namedtuple() type into a dictable_namedtuple.

Classes

Dictable()

A small abstract class for use with Python 3.7 dataclasses.

DictDataClass()

This is a base class for use with Python 3.7+ dataclass ‘s, designed to make dataclasses more interoperable with existing dictionaries, and allows them to be used like dictionaries, similar to Dictable, but more powerful / flexible.

DictObject

A very simple dict wrapper, which allows you to read and write dictionary keys using attributes (dot notation) PLUS standard item (key / square bracket notation) access.

OrderedDictObject

Ordered version of DictObject - dictionary with attribute access.

MockDictObj

alias of builtins.dict

Dictable()

A small abstract class for use with Python 3.7 dataclasses.

Mocker(modules, attributes, *args, **kwargs)

This mock class is designed to be used either to act as a stand-in “noop” (no operation) object, which could be used either as a drop-in replacement for a failed module / class import, or for certain unit tests.

OrderedDictObject

Ordered version of DictObject - dictionary with attribute access.