database¶

Classes¶

`Base`	Base class used for declarative class definitions.
`NumpyType`	Allows the creation of types which add additional functionality
`DataBaseEntry`	Base class used for declarative class definitions.

Functions¶

`_dataframe`(db, table)
`_to_numpy`(buffer, shape, dtype)
`process_responsibility_column`(df[, dtype])
`process_mask_column`(df, column)
`db_to_pandas`(db[, dtype, table, process])	for interactive use
`update_database`(db, explanation, time_taken[, multi, ...])
`add_to_database`(db, args, target, confidence, ...[, ...])
`initialise_rex_db`(name[, echo])

Module Contents¶

database._dataframe(db, table)¶

database._to_numpy(buffer, shape, dtype)¶

database.process_responsibility_column(df, dtype=np.float32)¶

database.process_mask_column(df, column)¶

database.db_to_pandas(db, dtype=np.float32, table='rex', process=True)¶: for interactive use

database.update_database(db, explanation, time_taken, multi=False, clauses=None, analysis_results=None)¶

Parameters:

explanation (rex_xai.explanation.explanation.Explanation | rex_xai.explanation.multi_explanation.MultiExplanation)
time_taken (float)

database.add_to_database(db, args, target, confidence, responsibility, sufficiency_mask, sufficiency_confidence, contrastive_mask, contrastive_confidence, inverse_classification, inverse_confidence, complete_mask, complete_classification, complete_confidence, area, entropy, insertion_curve, deletion_curve, time_taken, passing, failing, depth_reached, avg_box_size, multi=False, multi_no=None)¶

Parameters:

args (rex_xai.input.config.CausalArgs)
target (int)
confidence (float | None)
sufficiency_confidence (float | None)
contrastive_confidence (float | None)
inverse_classification (int | None)
inverse_confidence (float | None)
complete_classification (int | None)
area (float | None)
entropy (float | None)
insertion_curve (float | None)
deletion_curve (float | None)
time_taken (float)
passing (int)
failing (int)
depth_reached (int)
avg_box_size (float)

class database.Base¶

Bases: sqlalchemy.orm.DeclarativeBase

Base class used for declarative class definitions.

The _orm.DeclarativeBase allows for the creation of new declarative bases in such a way that is compatible with type checkers:

from sqlalchemy.orm import DeclarativeBase


class Base(DeclarativeBase):
    pass

The above Base class is now usable as the base for new declarative mappings. The superclass makes use of the __init_subclass__() method to set up new classes and metaclasses aren’t used.

When first used, the _orm.DeclarativeBase class instantiates a new _orm.registry to be used with the base, assuming one was not provided explicitly. The _orm.DeclarativeBase class supports class-level attributes which act as parameters for the construction of this registry; such as to indicate a specific _schema.MetaData collection as well as a specific value for :paramref:`_orm.registry.type_annotation_map`:

from typing_extensions import Annotated

from sqlalchemy import BigInteger
from sqlalchemy import MetaData
from sqlalchemy import String
from sqlalchemy.orm import DeclarativeBase

bigint = Annotated[int, "bigint"]
my_metadata = MetaData()


class Base(DeclarativeBase):
    metadata = my_metadata
    type_annotation_map = {
        str: String().with_variant(String(255), "mysql", "mariadb"),
        bigint: BigInteger(),
    }

Class-level attributes which may be specified include:

Parameters:

metadata – optional _schema.MetaData collection. If a _orm.registry is constructed automatically, this _schema.MetaData collection will be used to construct it. Otherwise, the local _schema.MetaData collection will supercede that used by an existing _orm.registry passed using the :paramref:`_orm.DeclarativeBase.registry` parameter.
type_annotation_map – optional type annotation map that will be passed to the _orm.registry as :paramref:`_orm.registry.type_annotation_map`.
registry – supply a pre-existing _orm.registry directly.

Added in version 2.0: Added DeclarativeBase, so that declarative base classes may be constructed in such a way that is also recognized by PEP 484 type checkers. As a result, DeclarativeBase and other subclassing-oriented APIs should be seen as superseding previous “class returned by a function” APIs, namely _orm.declarative_base() and _orm.registry.generate_base(), where the base class returned cannot be recognized by type checkers without using plugins.

__init__ behavior

In a plain Python class, the base-most __init__() method in the class hierarchy is object.__init__(), which accepts no arguments. However, when the _orm.DeclarativeBase subclass is first declared, the class is given an __init__() method that links to the :paramref:`_orm.registry.constructor` constructor function, if no __init__() method is already present; this is the usual declarative constructor that will assign keyword arguments as attributes on the instance, assuming those attributes are established at the class level (i.e. are mapped, or are linked to a descriptor). This constructor is never accessed by a mapped class without being called explicitly via super(), as mapped classes are themselves given an __init__() method directly which calls :paramref:`_orm.registry.constructor`, so in the default case works independently of what the base-most __init__() method does.

Changed in version 2.0.1: _orm.DeclarativeBase has a default constructor that links to :paramref:`_orm.registry.constructor` by default, so that calls to super().__init__() can access this constructor. Previously, due to an implementation mistake, this default constructor was missing, and calling super().__init__() would invoke object.__init__().

The _orm.DeclarativeBase subclass may also declare an explicit __init__() method which will replace the use of the :paramref:`_orm.registry.constructor` function at this level:

class Base(DeclarativeBase):
    def __init__(self, id=None):
        self.id = id

Mapped classes still will not invoke this constructor implicitly; it remains only accessible by calling super().__init__():

class MyClass(Base):
    def __init__(self, id=None, name=None):
        self.name = name
        super().__init__(id=id)

Note that this is a different behavior from what functions like the legacy _orm.declarative_base() would do; the base created by those functions would always install :paramref:`_orm.registry.constructor` for __init__().

class database.NumpyType(*args, **kwargs)¶

Bases: sqlalchemy.types.TypeDecorator

Allows the creation of types which add additional functionality to an existing type.

This method is preferred to direct subclassing of SQLAlchemy’s built-in types as it ensures that all required functionality of the underlying type is kept in place.

Typical usage:

import sqlalchemy.types as types


class MyType(types.TypeDecorator):
    """Prefixes Unicode values with "PREFIX:" on the way in and
    strips it off on the way out.
    """

    impl = types.Unicode

    cache_ok = True

    def process_bind_param(self, value, dialect):
        return "PREFIX:" + value

    def process_result_value(self, value, dialect):
        return value[7:]

    def copy(self, **kw):
        return MyType(self.impl.length)

The class-level impl attribute is required, and can reference any TypeEngine class. Alternatively, the load_dialect_impl() method can be used to provide different type classes based on the dialect given; in this case, the impl variable can reference TypeEngine as a placeholder.

The TypeDecorator.cache_ok class-level flag indicates if this custom TypeDecorator is safe to be used as part of a cache key. This flag defaults to None which will initially generate a warning when the SQL compiler attempts to generate a cache key for a statement that uses this type. If the TypeDecorator is not guaranteed to produce the same bind/result behavior and SQL generation every time, this flag should be set to False; otherwise if the class produces the same behavior each time, it may be set to True. See TypeDecorator.cache_ok for further notes on how this works.

Types that receive a Python type that isn’t similar to the ultimate type used may want to define the TypeDecorator.coerce_compared_value() method. This is used to give the expression system a hint when coercing Python objects into bind parameters within expressions. Consider this expression:

mytable.c.somecol + datetime.date(2009, 5, 15)

Above, if “somecol” is an Integer variant, it makes sense that we’re doing date arithmetic, where above is usually interpreted by databases as adding a number of days to the given date. The expression system does the right thing by not attempting to coerce the “date()” value into an integer-oriented bind parameter.

However, in the case of TypeDecorator, we are usually changing an incoming Python type to something new - TypeDecorator by default will “coerce” the non-typed side to be the same type as itself. Such as below, we define an “epoch” type that stores a date value as an integer:

class MyEpochType(types.TypeDecorator):
    impl = types.Integer

    cache_ok = True

    epoch = datetime.date(1970, 1, 1)

    def process_bind_param(self, value, dialect):
        return (value - self.epoch).days

    def process_result_value(self, value, dialect):
        return self.epoch + timedelta(days=value)

Our expression of somecol + date with the above type will coerce the “date” on the right side to also be treated as MyEpochType.

This behavior can be overridden via the coerce_compared_value() method, which returns a type that should be used for the value of the expression. Below we set it such that an integer value will be treated as an Integer, and any other value is assumed to be a date and will be treated as a MyEpochType:

def coerce_compared_value(self, op, value):
    if isinstance(value, int):
        return Integer()
    else:
        return self

Warning

Note that the behavior of coerce_compared_value is not inherited by default from that of the base type. If the TypeDecorator is augmenting a type that requires special logic for certain types of operators, this method must be overridden. A key example is when decorating the _postgresql.JSON and _postgresql.JSONB types; the default rules of TypeEngine.coerce_compared_value() should be used in order to deal with operators like index operations:

from sqlalchemy import JSON
from sqlalchemy import TypeDecorator


class MyJsonType(TypeDecorator):
    impl = JSON

    cache_ok = True

    def coerce_compared_value(self, op, value):
        return self.impl.coerce_compared_value(op, value)

Without the above step, index operations such as mycol['foo'] will cause the index value 'foo' to be JSON encoded.

Similarly, when working with the ARRAY datatype, the type coercion for index operations (e.g. mycol[5]) is also handled by TypeDecorator.coerce_compared_value(), where again a simple override is sufficient unless special rules are needed for particular operators:

from sqlalchemy import ARRAY
from sqlalchemy import TypeDecorator


class MyArrayType(TypeDecorator):
    impl = ARRAY

    cache_ok = True

    def coerce_compared_value(self, op, value):
        return self.impl.coerce_compared_value(op, value)

Parameters:

args (Any)
kwargs (Any)

impl¶

cache_ok = True¶

Indicate if statements using this ExternalType are “safe to cache”.

The default value None will emit a warning and then not allow caching of a statement which includes this type. Set to False to disable statements using this type from being cached at all without a warning. When set to True, the object’s class and selected elements from its state will be used as part of the cache key. For example, using a TypeDecorator:

class MyType(TypeDecorator):
    impl = String

    cache_ok = True

    def __init__(self, choices):
        self.choices = tuple(choices)
        self.internal_only = True

The cache key for the above type would be equivalent to:

>>> MyType(["a", "b", "c"])._static_cache_key
(<class '__main__.MyType'>, ('choices', ('a', 'b', 'c')))

The caching scheme will extract attributes from the type that correspond to the names of parameters in the __init__() method. Above, the “choices” attribute becomes part of the cache key but “internal_only” does not, because there is no parameter named “internal_only”.

The requirements for cacheable elements is that they are hashable and also that they indicate the same SQL rendered for expressions using this type every time for a given cache value.

To accommodate for datatypes that refer to unhashable structures such as dictionaries, sets and lists, these objects can be made “cacheable” by assigning hashable structures to the attributes whose names correspond with the names of the arguments. For example, a datatype which accepts a dictionary of lookup values may publish this as a sorted series of tuples. Given a previously un-cacheable type as:

class LookupType(UserDefinedType):
    """a custom type that accepts a dictionary as a parameter.

    this is the non-cacheable version, as "self.lookup" is not
    hashable.

    """

    def __init__(self, lookup):
        self.lookup = lookup

    def get_col_spec(self, **kw):
        return "VARCHAR(255)"

    def bind_processor(self, dialect): ...  # works with "self.lookup" ...

Where “lookup” is a dictionary. The type will not be able to generate a cache key:

>>> type_ = LookupType({"a": 10, "b": 20})
>>> type_._static_cache_key
<stdin>:1: SAWarning: UserDefinedType LookupType({'a': 10, 'b': 20}) will not
produce a cache key because the ``cache_ok`` flag is not set to True.
Set this flag to True if this type object's state is safe to use
in a cache key, or False to disable this warning.
symbol('no_cache')

If we did set up such a cache key, it wouldn’t be usable. We would get a tuple structure that contains a dictionary inside of it, which cannot itself be used as a key in a “cache dictionary” such as SQLAlchemy’s statement cache, since Python dictionaries aren’t hashable:

>>> # set cache_ok = True
>>> type_.cache_ok = True

>>> # this is the cache key it would generate
>>> key = type_._static_cache_key
>>> key
(<class '__main__.LookupType'>, ('lookup', {'a': 10, 'b': 20}))

>>> # however this key is not hashable, will fail when used with
>>> # SQLAlchemy statement cache
>>> some_cache = {key: "some sql value"}
Traceback (most recent call last): File "<stdin>", line 1,
in <module> TypeError: unhashable type: 'dict'

The type may be made cacheable by assigning a sorted tuple of tuples to the “.lookup” attribute:

class LookupType(UserDefinedType):
    """a custom type that accepts a dictionary as a parameter.

    The dictionary is stored both as itself in a private variable,
    and published in a public variable as a sorted tuple of tuples,
    which is hashable and will also return the same value for any
    two equivalent dictionaries.  Note it assumes the keys and
    values of the dictionary are themselves hashable.

    """

    cache_ok = True

    def __init__(self, lookup):
        self._lookup = lookup

        # assume keys/values of "lookup" are hashable; otherwise
        # they would also need to be converted in some way here
        self.lookup = tuple((key, lookup[key]) for key in sorted(lookup))

    def get_col_spec(self, **kw):
        return "VARCHAR(255)"

    def bind_processor(self, dialect): ...  # works with "self._lookup" ...

Where above, the cache key for LookupType({"a": 10, "b": 20}) will be:

>>> LookupType({"a": 10, "b": 20})._static_cache_key
(<class '__main__.LookupType'>, ('lookup', (('a', 10), ('b', 20))))

Added in version 1.4.14: - added the cache_ok flag to allow some configurability of caching for TypeDecorator classes.

Added in version 1.4.28: - added the ExternalType mixin which generalizes the cache_ok flag to both the TypeDecorator and UserDefinedType classes.

See also

Augmenting Existing Types

_types.TypeDecorator.process_bind_param()

class database.DataBaseEntry(id, path, target, confidence, responsibility, responsibility_shape, sufficiency_mask, mask_shape, sufficiency_confidence, time_taken, contrastive_mask=None, contrastive_confidence=None, inverse_classification=None, inverse_confidence=None, complete_mask=None, complete_classification=None, complete_confidence=None, passing=None, failing=None, area=None, entropy=None, insertion_curve=None, deletion_curve=None, total_work=None, multi=False, multi_no=None, depth_reached=None, avg_box_size=None, tree_depth=None, search_limit=None, iters=None, min_size=None, distribution=None, distribution_args=None, initial_radius=None, radius_eta=None, method=None, spotlights=0, spotlight_size=0, spotlight_eta=0.0, obj_function=None)¶

Bases: Base

Base class used for declarative class definitions.

The _orm.DeclarativeBase allows for the creation of new declarative bases in such a way that is compatible with type checkers:

from sqlalchemy.orm import DeclarativeBase


class Base(DeclarativeBase):
    pass

The above Base class is now usable as the base for new declarative mappings. The superclass makes use of the __init_subclass__() method to set up new classes and metaclasses aren’t used.

When first used, the _orm.DeclarativeBase class instantiates a new _orm.registry to be used with the base, assuming one was not provided explicitly. The _orm.DeclarativeBase class supports class-level attributes which act as parameters for the construction of this registry; such as to indicate a specific _schema.MetaData collection as well as a specific value for :paramref:`_orm.registry.type_annotation_map`:

from typing_extensions import Annotated

from sqlalchemy import BigInteger
from sqlalchemy import MetaData
from sqlalchemy import String
from sqlalchemy.orm import DeclarativeBase

bigint = Annotated[int, "bigint"]
my_metadata = MetaData()


class Base(DeclarativeBase):
    metadata = my_metadata
    type_annotation_map = {
        str: String().with_variant(String(255), "mysql", "mariadb"),
        bigint: BigInteger(),
    }

Class-level attributes which may be specified include:

Parameters:

metadata – optional _schema.MetaData collection. If a _orm.registry is constructed automatically, this _schema.MetaData collection will be used to construct it. Otherwise, the local _schema.MetaData collection will supercede that used by an existing _orm.registry passed using the :paramref:`_orm.DeclarativeBase.registry` parameter.
type_annotation_map – optional type annotation map that will be passed to the _orm.registry as :paramref:`_orm.registry.type_annotation_map`.
registry – supply a pre-existing _orm.registry directly.

Added in version 2.0: Added DeclarativeBase, so that declarative base classes may be constructed in such a way that is also recognized by PEP 484 type checkers. As a result, DeclarativeBase and other subclassing-oriented APIs should be seen as superseding previous “class returned by a function” APIs, namely _orm.declarative_base() and _orm.registry.generate_base(), where the base class returned cannot be recognized by type checkers without using plugins.

__init__ behavior

In a plain Python class, the base-most __init__() method in the class hierarchy is object.__init__(), which accepts no arguments. However, when the _orm.DeclarativeBase subclass is first declared, the class is given an __init__() method that links to the :paramref:`_orm.registry.constructor` constructor function, if no __init__() method is already present; this is the usual declarative constructor that will assign keyword arguments as attributes on the instance, assuming those attributes are established at the class level (i.e. are mapped, or are linked to a descriptor). This constructor is never accessed by a mapped class without being called explicitly via super(), as mapped classes are themselves given an __init__() method directly which calls :paramref:`_orm.registry.constructor`, so in the default case works independently of what the base-most __init__() method does.

Changed in version 2.0.1: _orm.DeclarativeBase has a default constructor that links to :paramref:`_orm.registry.constructor` by default, so that calls to super().__init__() can access this constructor. Previously, due to an implementation mistake, this default constructor was missing, and calling super().__init__() would invoke object.__init__().

The _orm.DeclarativeBase subclass may also declare an explicit __init__() method which will replace the use of the :paramref:`_orm.registry.constructor` function at this level:

class Base(DeclarativeBase):
    def __init__(self, id=None):
        self.id = id

Mapped classes still will not invoke this constructor implicitly; it remains only accessible by calling super().__init__():

class MyClass(Base):
    def __init__(self, id=None, name=None):
        self.name = name
        super().__init__(id=id)

Note that this is a different behavior from what functions like the legacy _orm.declarative_base() would do; the base created by those functions would always install :paramref:`_orm.registry.constructor` for __init__().

__tablename__ = 'rex'¶

id¶

path¶

target¶

confidence¶

time¶

responsibility¶

responsibility_shape¶

total_work¶

passing¶

failing¶

sufficiency_mask¶

mask_shape¶

sufficiency_confidence¶

contrastive_mask¶

contrastive_confidence¶

inverse_classification¶

inverse_confidence¶

complete_mask¶

complete_classification¶

complete_confidence¶

area¶

entropy¶

insertion_curve¶

deletion_curve¶

multi¶

multi_no¶

depth_reached¶

avg_box_size¶

tree_depth¶

search_limit_per_iter¶

iters¶

min_size¶

distribution¶

distribution_args¶

spatial_radius¶

spatial_eta¶

method¶

spotlights¶

spotlight_size¶

spotlight_eta¶

obj_function¶

search_limit = None¶

database.initialise_rex_db(name, echo=False)¶

database¶

Classes¶

Functions¶

Module Contents¶

Navigation

Related Topics