SQL alchemy models/objects

We use sqlalchemy orm to manage a single target.

from sqlalchemy import Column, Text, Integer, ForeignKey
from flask_appbuilder import Model
from sqlalchemy.orm import relationship

A docModel could be a class, a function or a module (all python)

class docModel(Model):
    __tablename__ = "docs"
    id = Column(Integer, primary_key=True, autoincrement=False)
    name = Column(Text())
    doc = Column(Text(), default="")
    names = Column(Text(), default="")
    level = Column(Integer(), default=-1)
    source = Column(Text(), default="")
    path = Column(Text(), default="")
    code = Column(Text(), default="")
    ctype = Column(Text(), default="")
    alias = Column(Text(), default="")

    def __repr__(self):
        return f"<{self.name}>"

    def new_parent(self, parent):
        if parent:
            self.parents.append(parent)
            parent.kids.append(self)

    def to_dicts(self, *cols):
        return dict((col, getattr(self, col)) for col in cols)

Test a docModel

abcdoc = docModel(name="abc",names="abc,ABC", level=2, ctype="function",alias="abc")
abcdoc

<abc>

abcdoc.to_dicts("name", "names", "level", "ctype", "alias")

{'name': 'abc',
 'names': 'abc,ABC',
 'level': 2,
 'ctype': 'function',
 'alias': 'abc'}

Relations between targets

There are 2 sorts of relation between 2 docModels

Attributes relation

b is the attribute of a, we can get b by a.b, or getattr(a,"b")

class docGraphModel(Model):
    __tablename__ = "doc_graph"
    id = Column(Integer, primary_key=True)
    parent_id = Column(Integer(), ForeignKey("docs.id"))
    parent = relationship(docModel, foreign_keys=[parent_id])
    kid_id = Column(Integer(), ForeignKey("docs.id"))
    kid = relationship(docModel, foreign_keys=[kid_id])
    
# Mapping the self-relate docs on attribute relations
docModel.kids = relationship(docModel,
                             secondary="doc_graph",
                             primaryjoin=(docModel.id == docGraphModel.parent_id),
                             secondaryjoin=(docGraphModel.kid_id == docModel.id)
                             )
docModel.parents = relationship(docModel,
                                secondary="doc_graph",
                                primaryjoin=(docModel.id == docGraphModel.kid_id),
                                secondaryjoin=(docGraphModel.parent_id == docModel.id)
                                )

Inheritance relation

We define inheritance as following

class b(object):
    fromb = "b"
    pass

class a(b):
    froma = "a"
    
class c(a,b):
    fromc = "c"

In this case we can say:

• b is the descendant of object
• a is the descendant of b
• c is the descendant from a and b
• b is the ancestor of a

c.froma, c.fromb,c.fromc

('a', 'b', 'c')

class inhGraphModel(Model):
    __tablename__ = "inh_graph"
    id = Column(Integer, primary_key=True)
    anc_id = Column(Integer(), ForeignKey("docs.id")) # ancestor
    anc = relationship(docModel, foreign_keys=[anc_id])
    des_id = Column(Integer(), ForeignKey("docs.id")) # descendant
    des = relationship(docModel, foreign_keys=[des_id])

# Mappping the self-relate docs on class inheritance
docModel.dess = relationship(docModel,
                             secondary="inh_graph",
                             primaryjoin=(docModel.id == inhGraphModel.anc_id),
                             secondaryjoin=(inhGraphModel.des_id == docModel.id)
                             )
docModel.ancs = relationship(docModel,
                                secondary="inh_graph",
                                primaryjoin=(docModel.id == inhGraphModel.des_id),
                                secondaryjoin=(inhGraphModel.anc_id == docModel.id)
                                )

Parse a class, function, module

of its attributes/being attributes of others, inheritances

Some helper functions

most_frequent[source]

most_frequent(List)

get_source[source]

get_source(obj)

get_path[source]

get_path(obj)

checks[source]

checks(obj)

refresh_table[source]

refresh_table(model, engine)

Drop the existed table from database, if exist In any case create a new table

The core mechanism

The core mechanism of this entire package

class docTour[source]

docTour(root_obj, root_name, sess, load_source=False)

Out facing function

• lib: str, the name you assign to your target, preferably start with a letter, no funny chars except under score
• import_: bool, default True, import the object from lib name?
• obj: could be class, module, function, variable, default None

parse_lib[source]

parse_lib(lib, import_=True, obj=None)

Experiment

Try some example

from torch import nn

dataurl will point to the sqlite file generated from this operation

dt,dataurl = parse_lib("GRU", False, nn.GRU)

creating SQLite db:	 sqlite:////Users/salvor/Downloads/GRU.db

Data sample

Sample of the data we extracted from our operation

dt.df.sample(5)

	id	name	doc	path	names	level	source	ctype	alias
45	5056307712	_get_flat_weights_names	None	/Users/salvor/anaconda3/lib/python3.7/site-pac...	GRU._get_flat_weights._parameter_names_fn,GRU....	3		function	_parameter_names_fn,_get_flat_weights_names
53	5056308576	permute_hidden	None	/Users/salvor/anaconda3/lib/python3.7/site-pac...	GRU.permute_hidden,GRU.permute_hidden	2		function	permute_hidden
11	5055174272	add_module	Adds a child module to the current module.\n\n...	/Users/salvor/anaconda3/lib/python3.7/site-pac...	RNNBase.add_module,GRU.add_module,GRU.add_module	2		function	add_module
5	5055186272	_named_members	Helper method for yielding various names + mem...	/Users/salvor/anaconda3/lib/python3.7/site-pac...	RNNBase._named_members,GRU._named_members,GRU....	2		function	_named_members
13	5055186704	buffers	Returns an iterator over module buffers.\n\n ...	/Users/salvor/anaconda3/lib/python3.7/site-pac...	RNNBase.buffers,GRU.buffers,GRU.buffers	2		function	buffers