README.md

@@ -5,8 +5,8 @@ etc. Objects inheriting from the `CO3` base class can then define data transform
 that connect to database components, and can be automatically collected for coordinated
 database insertion.
 
-`co3` attempts to provide a general interface for interacting with storage media (e.g.,
+`co3` attempts to provide a general interface for interacting with a storage media (e.g.,
-databases, pickled objects, VSS framework, in-memory key-value stores, etc). The following
+database, pickled objects, VSS framework, in-memory key-value stores, etc). The following
 top-level classes capture the bulk of the operational model:
 
 - **Database**: reference to a storage medium, with an `Accessor` for accessing data,

co3/co3.py

@@ -1,9 +1,8 @@
 '''
-CO3 is an abstract base class for scaffolding object hierarchies and managing
+CO3 is an abstract base class for scaffolding object hierarchies and managing operations
-operations with associated database schemas. It facilitates something like a
+with associated database schemas. It facilitates something like a "lightweight ORM" for
-"lightweight ORM" for classes/tables/states with fixed transformations of
+classes/tables/states with fixed transformations of interest. The canonical use case is
-interest. The canonical use case is managing hierarchical document relations,
+managing hierarchical document relations, format conversions, and syntactical components.
-format conversions, and syntactical components.
 
 Generic collation syntax:
 
@@ -23,13 +22,12 @@ Generic collation syntax:
 
 .. admonition:: On multi-key attachment
 
-    One possible quirk of the current collation registry scheme is the rather
+    One possible quirk of the current collation registry scheme is the rather black and
-    black and white nature of key attachment. You either specify a single key,
+    white nature of key attachment. You either specify a single key, possibly to several
-    possibly to several groups, or allow any key via passthrough under an
+    groups, or allow any key via passthrough under an implicit group. There's no explicit
-    implicit group. There's no explicit "multi-key" pattern to make use of
+    "multi-key" pattern to make use of here, be it through "restricted passthrough"
-    here, be it through "restricted passthrough" (method still parameterized by
+    (method still parameterized by the key, but only allows keys from a provided list) or
-    the key, but only allows keys from a provided list) or just simple
+    just simple duplicated attachment. To demonstrate via the above example:
-    duplicated attachment. To demonstrate via the above example:
 
     .. code-block:: python
 
@@ -56,8 +54,8 @@ Generic collation syntax:
 
                 ...
 
-    or with a central handler and separate collation points (at least when the
+    or with a central handler and separate collation points (at least when the key list is
-    key list is small):
+    small):
 
     .. code-block:: python
 
@@ -73,41 +71,38 @@ Generic collation syntax:
             def key2(self):
                 self._handle_supported_keys('key2')
 
-    The former scales better and allows general key rejection patterns if
+    The former scales better and allows general key rejection patterns if needed, while
-    needed, while the latter integrates a bit better with the formal collation
+    the latter integrates a bit better with the formal collation process, e.g., will
-    process, e.g., will throw ``ValueErrors`` based on key mismatches
+    throw ``ValueErrors`` based on key mismatches automatically.
-    automatically.
 '''
 import inspect
 import logging
 from collections import defaultdict
+from functools import wraps, partial
 
 
 logger = logging.getLogger(__name__)
 
-
 def collate(key, groups=None):
     '''
     Collation decorator for CO3 subtype action registry.
 
-    Dynamic decorator; can be used as ``collate`` without any arguments, or
+    Dynamic decorator; can be used as ``collate`` without any arguments, or with all. In
-    with all. In the former case, ``key`` will be a function, so we check for
+    the former case, ``key`` will be a function, so we check for this.
-    this.
 
     .. admonition:: Usage
 
-        Collation registration is the process of exposing various actions for
+        Collation registration is the process of exposing various actions for use in
-        use in **hierarchical collection** (see ``Mapper.collect``). Collation
+        **hierarchical collection** (see ``Mapper.collect``). Collation *keys* are unique
-        *keys* are unique identifiers of a particular action that emits data.
+        identifiers of a particular action that emits data. Keys can belong to an arbitrary
-        Keys can belong to an arbitrary number of *groups*, which serve as
+        number of *groups*, which serve as semantically meaningful collections of similar
-        semantically meaningful collections of similar actions. Group
+        actions. Group assignment also determines the associated *collation component*
-        assignment also determines the associated *collation component* to be
+        to be used as a storage target; the results of actions $K_G$ belonging to group
-        used as a storage target; the results of actions $K_G$ belonging to
+        $G$ will all be stored in the attached $G$-component. Specification of key-group
-        group $G$ will all be stored in the attached $G$-component.
+        relations can be done in a few ways:
-        Specification of key-group relations can be done in a few ways:
 
-        - Explicit key-group specification: a specific key and associated
+        - Explicit key-group specification: a specific key and associated groups can be
-          groups can be provided as arguments to the decorator:
+          provided as arguments to the decorator:
 
           .. code-block:: python
 
@@ -132,14 +127,14 @@ def collate(key, groups=None):
                   ...
               }
 
-          If ``groups`` is left unspecified, the key will be attached to the
+          If ``groups`` is left unspecified, the key will be attached to the default
-          default ``None`` group.
+          ``None`` group.
 
-        - Implicit key-group association: in some cases, you may want to
+        - Implicit key-group association: in some cases, you may want to support an entire
-          support an entire "action class," and associate any operations under
+          "action class," and associate any operations under the class to the same storage
-          the class to the same storage component. Here we still use the notion
+          component. Here we still use the notion of connecting groups to components, but
-          of connecting groups to components, but allow the key to be
+          allow the key to be dynamically specified and passed through to the collation
-          dynamically specified and passed through to the collation method:
+          method:
 
           .. code-block:: python
 
@@ -165,31 +160,27 @@ def collate(key, groups=None):
 
           A few important notes:
 
-          - Implicit key-group specifications attach the *group* to a single
+          - Implicit key-group specifications attach the *group* to a single method,
-            method, whereas in the explicit case, groups can be affiliated with
+            whereas in the explicit case, groups can be affiliated with many keys. When
-            many keys. When explicitly provided, only those exact key values
+            explicitly provided, only those exact key values are supported. But in the
-            are supported. But in the implicit case, *any* key is allowed; the
+            implicit case, *any* key is allowed; the group still remains a proxy for the
-            group still remains a proxy for the entire action class, but
+            entire action class, but without needing to map from specifically stored key
-            without needing to map from specifically stored key values. That
+            values. That is, the utility of the group remains consistent across implicit
-            is, the utility of the group remains consistent across implicit
             and explicit cases, but stores the associations differently.
-          - The ``None`` key, rather than point to a ``(<method>,
+          - The ``None`` key, rather than point to a ``(<method>, <group-list>)`` tuple,
-            <group-list>)`` tuple, instead points to a dictionary of
+            instead points to a dictionary of ``group``-``method`` pairs. When attempting
-            ``group``-``method`` pairs. When attempting execute a key under a
+            execute a key under a particular group, the group registry indicates
-            particular group, the group registry indicates whether the key is
+            whether the key is explicitly supported. If ``None`` is present for the group,
-            explicitly supported. If ``None`` is present for the group, then
+            then ``key_registry[None][<group-name>]`` can be used to recover the method
-            ``key_registry[None][<group-name>]`` can be used to recover the
+            implicitly affiliated with the key (along with any other key under the group).
-            method implicitly affiliated with the key (along with any other key
+          - When any method has been implicitly registered, *any* key (even when
-            under the group).
+            attempting to specify an explicit key) will match that group. This can
-          - When any method has been implicitly registered, *any* key (even
+            effectively mean keys are not unique when an implicit group has been
-            when attempting to specify an explicit key) will match that group.
+            registered. There is a protection in place here, however; in methods like
-            This can effectively mean keys are not unique when an implicit
+            ``CO3.collate`` and ``Mapper.collect``, an implicit group must be directly
-            group has been registered. There is a protection in place here,
+            named in order for a given key to be considered. That is, when attempting
-            however; in methods like ``CO3.collate`` and ``Mapper.collect``, an
+            collation outside specific group context, provided keys will only be
-            implicit group must be directly named in order for a given key to
+            considered against explicitly registered keys.
-            be considered. That is, when attempting collation outside specific
-            group context, provided keys will only be considered against
-            explicitly registered keys.
     '''
     func = None
     if inspect.isfunction(key):
@@ -209,7 +200,6 @@ def collate(key, groups=None):
 
     return decorator
 
-
 class FormatRegistryMeta(type):
     '''
     Metaclass handling collation registry at the class level.
@@ -235,8 +225,8 @@ class FormatRegistryMeta(type):
                 for _, method in methods:
                     register_action(method)
 
-        # add final registered formats for the current class, overwriting any
+        # add final registered formats for the current class, overwriting any found in
-        # found in superclass chain
+        # superclass chain
         for attr_name, attr_value in attrs.items():
             register_action(attr_value)
 
@@ -245,49 +235,41 @@ class FormatRegistryMeta(type):
 
         return super().__new__(cls, name, bases, attrs)
 
-
 class CO3(metaclass=FormatRegistryMeta):
     '''
     Base class supporting the central "COllate, COllect, COmpose" paradigm.
 
-    - Collate: organize and transform conversion outputs, possibly across class
+    - Collate: organize and transform conversion outputs, possibly across class components
-      components
+    - Collect: gather core attributes, conversion data, and subcomponents for DB insertion
-    - Collect: gather core attributes, conversion data, and subcomponents for
+    - Compose: construct object-associated DB table references through the class hierarchy
-      DB insertion
-    - Compose: construct object-associated DB table references through the
-      class hierarchy
 
     .. admonition:: on action groups
 
-        Group keys are simply named collections to make it easy for storage
+        Group keys are simply named collections to make it easy for storage components to
-        components to be attached to action subsets. They do _not_ augment the
+        be attached to action subsets. They do _not_ augment the action registration
-        action registration namespace, meaning the action key should still be
+        namespace, meaning the action key should still be unique; the group key is purely
-        unique; the group key is purely auxiliary.
+        auxiliary.
 
         Action methods can also be attached to several groups, in case there is
-        overlapping utility within or across schemas or storage media. In this
+        overlapping utility within or across schemas or storage media. In this case, it
-        case, it becomes particularly critical to ensure registered ``collate``
+        becomes particularly critical to ensure registered ``collate`` methods really are
-        methods really are just "gathering results" from possibly heavy-duty
+        just "gathering results" from possibly heavy-duty operations, rather than
-        operations, rather than performing them when called, so as to reduce
+        performing them when called, so as to reduce wasted computation.
-        wasted computation.
 
     .. admonition:: New: collation caching
 
         To help facilitate the common pattern of storing collation results, a
-        ``collate_cache`` parameter has been added to store key-group indexed
+        ``collate_cache`` parameter has been added to store key-group indexed collation
-        collation results. (Note: now requires explicit superclass
+        results. (Note: now requires explicit superclass instantiation.)
-        instantiation.)
     '''
-
     def __init__(self):
         self._collate_cache = {}
 
     @property
     def attributes(self):
         '''
-        Method to define how a subtype's inserts should be handled under
+        Method to define how a subtype's inserts should be handled under ``collect`` for
-        ``collect`` for canonical attributes, i.e., inserts to the type's
+        canonical attributes, i.e., inserts to the type's table.
-        table.
         '''
         return vars(self)
 
@@ -302,15 +284,14 @@ class CO3(metaclass=FormatRegistryMeta):
     def collation_attributes(self, key, group):
         '''
         Return "connective" collation component data, possibly dependent on
-        instance-specific attributes and the action arguments. This is
+        instance-specific attributes and the action arguments. This is typically the
-        typically the auxiliary structure that may be needed to attach to
+        auxiliary structure that may be needed to attach to responses from registered
-        responses from registered ``collate`` calls to complete inserts.
+        ``collate`` calls to complete inserts.
 
-        Note: this method is primarily used by ``Mapper.collect()``, and is
+        Note: this method is primarily used by ``Mapper.collect()``, and is called just
-        called just prior to collector send-off for collation inserts and
+        prior to collector send-off for collation inserts and injected alongside collation
-        injected alongside collation data. Common structure in collation
+        data. Common structure in collation components can make this function easy to
-        components can make this function easy to define, independent of action
+        define, independent of action group for instance.
-        group for instance.
         '''
         return {}
 
@@ -369,3 +350,5 @@ class CO3(metaclass=FormatRegistryMeta):
             self._collate_cache[(key, group)] = result
 
         return result
+
+

co3/component.py

@@ -1,11 +1,12 @@
-class Component[T]:
+'''
-    '''
+Component
-    Component
 
-    General wrapper for storage components to be used in various database contexts. Relations
+General wrapper for storage components to be used in various database contexts. Relations
-    can be thought of generally as named data containers/entities serving as a fundamental
+can be thought of generally as named data containers/entities serving as a fundamental
-    abstractions within particular storage protocols.
+abstractions within particular storage protocols.
-    '''
+'''
+
+class Component[T]:
     def __init__(self, name, obj: T):
         self.name = name
         self.obj  = obj
@@ -18,3 +19,4 @@ class Component[T]:
 
     def get_attributes(self):
         raise NotImplementedError
+

co3/domain.py

@@ -1,24 +0,0 @@
-from contextlib import contextmanager
-
-from co3 import Resource
-
-
-class Domain[R: Resource]:
-    '''
-    General Domain class
-
-
-    '''
-    def __init__(self, content):
-        pass
-    
-    def get_resource(self, url: URL) -> Resource:
-        pass
-
-    @contextmanager
-    def connect(self, timeout=None):
-        raise NotImplementedError
-
-class SelectableDomain(Domain):
-    def select(self, component, *args, **kwargs):
-        raise NotImplementedError

co3/engine.py

@@ -13,7 +13,7 @@ class Engine:
     derivative Engines, like SQLEngine, mostly just wrap another engine-like object, this
     is not the rule. That is, inheriting Engine subtypes shouldn't necessarily expect to
     rely on another object per se, and if such an object is required, _this_ is the class
-    meant to be the skeleton that supports its creation (and not merely a wrapper for some
+    is meant to be skeleton to supports its creation (and not merely a wrapper for some
     other type, although it may appear that way when such a type is in fact readily
     available).
 

co3/indexer.py

@@ -11,10 +11,12 @@ logger = logging.getLogger(__name__)
 
 class Indexer:
     '''
-    Indexer base class
+    Indexer class
 
     Provides restricted access to an underlying Accessor to enable more efficient, superficial
-    caching. Note that cache clearing is to be handled by a wrapper class, like the Database.
+    caching.
+
+    Cache clearing is to be handled by a wrapper class, like the Database.
 
     Caching occurs at the class level, with indexes prefixed by table's origin Composer.
     This means that cached selects/group-bys will be available regardless of the provided
@@ -259,12 +261,12 @@ class Indexer:
             agg_on = agg_on_names
             index_on = index_on_names
 
-        # print(f'rows_are_mappings: {rows_are_mappings}')
+        #print(f'rows_are_mappings: {rows_are_mappings}')
-        # print(f'group_by: {group_by}')
+        #print(f'group_by: {group_by}')
-        # print(f'agg_on: {agg_on}')
+        #print(f'agg_on: {agg_on}')
-        # print(f'agg_on_names: {agg_on_names}')
+        #print(f'agg_on_names: {agg_on_names}')
-        # print(f'index_on: {index_on}')
+        #print(f'index_on: {index_on}')
-        # print(f'index_on_names: {index_on_names}')
+        #print(f'index_on_names: {index_on_names}')
 
         # "group by" block ID and wrangle the links into a list
         group_by_idx = {}
@@ -315,7 +317,6 @@ class Indexer:
 
         return list(group_by_idx.values())
 
-
 class CacheBlock:
     '''
     Wraps up a set of query parameters for a specific entity, and provides cached access

co3/managers/sql.py

@@ -75,7 +75,7 @@ class SQLManager(RelationalManager[SQLTable]):
     '''
     def __init__(self, *args, **kwargs):
         '''
-        The insert lock is a *re-entrant lock*, meaning the same thread can acquire the
+        The insert lock is a *reentrant lock*, meaning the same thread can acquire the
         lock again without deadlocking (simplifying across methods of this class that
         need it).
         '''
@@ -112,13 +112,11 @@ class SQLManager(RelationalManager[SQLTable]):
     def insert(
         self,
         connection,
-        component: SQLTable,
+        component,
         inserts: list[dict],
         commit=True
     ):
         '''
-        Insert a group of 
-
         Parameters:
         '''
         with self._insert_lock:
@@ -137,13 +135,6 @@ class SQLManager(RelationalManager[SQLTable]):
         Perform provided table inserts, aligning the insert format of
         ``Collector.collect_inserts()``.
 
-        Note that the regular ``insert`` actually supports the usual notion of a "bulk
-        insert," inserting many entries under a single table. This method simply supports
-        the same but across multiple tables. It does so by just making calls to
-        ``insert()`` after grouping entries for each component in the provided ``inserts``
-        dict, only committing the transaction after all components inserts have been
-        staged.
-
         Parameters:
             inserts: component-indexed dictionary of insert lists
         '''

co3/medium.py

@@ -1,101 +0,0 @@
-import logging
-from contextlib import contextmanager
-
-from co3.engine import Engine, Connection, Resource, Group
-
-
-logger = logging.getLogger(__name__)
-
-class Medium[R: Resource]:
-    '''
-    Medium base class. 
-
-    A Resource space
-    '''
-    _engine_cls: type[Engine] = Engine
- 
-    def __init__(self, *scope_args, **scope_kwargs):
-        self.engine = self._engine_cls(*engine_args, **engine_kwargs)
-        pass
-        
-    @contextmanager
-    def connect(self, timeout=None) -> Connection:
-        '''
-        Open a connection to the database specified by the resource. Exactly what the
-        returned connection looks like remains relatively unconstrained given the wide
-        variety of possible database interactions. This function should be invoked in
-        with-statement contexts, constituting an "interaction session" with the database
-        (i.e., allowing several actions to be performed using the same connection).
-        '''
-        return self.engine.connect(timeout=timeout)
-
-    def execute(self, query: Query[QL]):
-        pass
-
-
-class ReadableMedium[R: Resource](Medium[R]):
-    def _resolve_relative_uri(self, protocol, value):
-        '''
-        Subclass to implement: fetch child object for supported protocol, i.e., single
-        component subpath
-        '''
-        ...
-
-    def resolve_uri(self, uri: URI) -> ResourceCollection:
-        assert uri.protocols[0] in self.supported_protocols
-
-        obj = self._resolve_relative_uri(uri.protocols[0], uri.components[0])
-
-        # core the uri and recurse
-        cored_uri = uri.core():
-
-        if cored_uri:
-            rc = obj.resolve_uri(cored_uri)
-        else:
-            assert type(obj) is Resource
-            rc = ResourceCollection([obj])
-
-        return rc
-
-    def _to_uri_list(self, uri_like):
-        if type(uri) is not list:
-            uri = [uri]
-        ...
-
-    def browse(
-        self,
-        connection: Connection,
-        uri: str | URI | list[str] | list[URI]
-    ) -> ResourceCollection:
-        '''
-        Analog for Read (CRUD), SELECT (SQL), GET (REST)
-        '''
-        uris = self._to_uri_list(uri)
-
-        rc = ResourceCollection()
-        for uri in uris:
-            rc.extend(self._resolve_uri(uri))
-
-        return rc
-
-
-class WritableMedium[R: Resource](ReadableMedium[R]):
-    def append(self, uri: URI[R], resource: R):
-        '''
-        Analog for Create (CRUD), INSERT (SQL), POST/PUT (REST)
-        '''
-        pass
-
-    def change(self, uri: URI[R], resource: R):
-        '''
-        Analog for Update (CRUD), UPDATE (SQL), PUT/PATCH (REST)
-
-        Can a URI be another object? Component for ex; could inherit from URI I guess
-        '''
-        pass
-
-    def delete(self, uri: URI[R]):
-        '''
-        Analog for Delete (CRUD), DELETE (SQL), DELETE (REST)
-        '''
-        pass

co3/mediums/disk.py

@@ -1,6 +0,0 @@
-from co3 import Medium
-from co3.resources import INode
-
-
-class Disk[INode](Medium):
-    pass

co3/resource.py

@@ -1,15 +1,9 @@
-from typing import BinaryIO
+from typing import Protocol
-
-from co3 import URI, Medium
 
 
 class Resource:
-    def __init__(
-        self,
-        context: Medium | list[Medium],
-        uri: URI
-    ):
-        self.uri = uri
-
-    def open(self) -> BinaryIO:
     pass
+
+class SelectableResource(Protocol):
+    def select(self, component, *args, **kwargs):
+        raise NotImplementedError

co3/resources/__init__.py

@@ -0,0 +1 @@
+from co3.resources.disk import DiskResource

co3/resources/disk.py

@@ -1,12 +1,10 @@
-#from co3.resources.disk import DiskResource
-
 from pathlib import Path
 
 from co3.util import paths
-from co3.resource import SelectableDomain
+from co3.resource import SelectableResource
 
 
-class DiskDomain(SelectableDomain):
+class DiskResource(SelectableResource):
     def select(
         self,
         path_list: str | Path | list[str | Path],

co3/resources/inode.py

@@ -1,5 +0,0 @@
-from co3 import Resource
-
-
-class INode(Resource):
-    pass

co3/uri.py

@@ -1,34 +0,0 @@
-from urllib.parse import urlparse
-
-
-class URI:
-    def __init__(self, url_str: str):
-        self.url_str = url_str
-
-class URL(URI):
-    def __init__(self, url_str: str):
-        self.url_str = url_str
-
-class URN(URI):
-    def __init__(self, url_str: str):
-        self.url_str = url_str
-
-
-class CompositeURI(URI):
-    def __init__(self, url_str: str):
-        url_obj = urlparse(url_str)
-
-        self.protocols = url_obj.scheme.split('+')[::-1]
-        self.components = url_obj.scheme.split('+')[::-1]
-
-    def core(self, layers=1) -> 'CompositeURI':
-        '''
-        "Core" the innermost ``layers`` layers of the composite URI.
-        '''
-        pass
-
-    def shed(self, layers=1) -> 'CompositeURI':
-        '''
-        "Shed" the outermost ``layers`` layers of the composite URI.
-        '''
-        pass

docs/reference/uri.md

@@ -1,11 +0,0 @@
-Quick thoughts and ideals:
-
-- Schemes are compositional, "wrapping" super contexts: `c+b+a://a/b/c`
-- The scheme communicates the target type (above is `c`)
-- URIs can be arbitrarily relative so long as they're resolved in the right contexts.
-  Above, `c+b://b/c` can be resolved in the context of `a://a`
-- URIs are resolved by unwrapping schemes and resolving in to out
-- URL params can apply only to the target type (this is the most consistent and probably
-  not too restrictive)
-- Trajectories from one scheme to another can be inferred from the type hierarchy; there
-  may be many

pyproject.toml

@@ -40,7 +40,6 @@ docs = [
     "furo",
     "myst-parser",
 ]
-jupyter = ["ipykernel"]
 
 [project.urls]
 Homepage = "https://doc.olog.io/co3"

tests/co3_medium_demo.py

@@ -1,5 +0,0 @@
-from co3.mediums import Disk
-
-
-disk = Disk('disk:///')
-disk.browse('dir://home')