extended FSWTabDict class #377

JimHofman · 2021-10-20T16:21:53Z

No description provided.

nttoole · 2021-10-20T16:27:12Z

@JimHofman Looks like you still need to declare your newCmd as an extension in the config.yaml

See: https://ait-core.readthedocs.io/en/master/extensions.html

Config yaml would contain an entry that looks like:

....
extensions:
    ait.core.cmd.Cmd: ait.core.newCmd.NewCmd

Would presumably need something similar for the table extension.

JimHofman · 2021-10-20T16:34:57Z

I'll check that out Nick. I was sure I had added those. Hmmm. Thanks Nick.

MJJoyce · 2021-10-20T20:19:46Z

Hey @JimHofman, replying here with regards to the issue(s) you're running into with the table extension support. Thanks for throwing this branch up so we have something to dig through.

It looks like you got the "general extension" stuff added and working correctly. That's why you can see log message indicating that the extension class has been added. However, it's not getting fully pulled in for a few reasons (I think).

def YAMLCtor_FSWColDefn(loader, node):  # noqa: N802
    fields = loader.construct_mapping(node, deep=True)
    return FSWColDefn(**fields)


def YAMLCtor_FSWTabDefn(loader, node):
    fields = loader.construct_mapping(node, deep=True)
    fields['fswheaderdefns'] = fields.pop('header', None)
    fields['coldefns'] = fields.pop('columns', None)
    return FSWTabDefn(**fields)

These functions are used by the custom YAML class handling stuff when processing the dictionary

yaml.add_constructor("!FSWTable", YAMLCtor_FSWTabDefn)
yaml.add_constructor("!FSWColumn", YAMLCtor_FSWColDefn)

So, if you import the table dictionary you'll see a message indicating that the extension was loaded but it's never going to actually load your class because the above functions (and others in the code) are calling explicit class names instead of the "extensions classes". These createXXX functions that should be used are (poorly) documented in the extension code:
https://github.com/NASA-AMMOS/AIT-Core/blob/master/ait/core/util.py#L112

Looking through the table code really quickly (so you should sanity check that I'm not missing something) I think you need to make the following modifications:

JimHofman · 2021-10-28T21:04:11Z

Table extensions now work. Created a newTable.py to override FSWTabDict.
newTable.py
from ait.core import table, log

class NewFSWTabDict(table.FSWTabDict):
def init(self, *args, **kwargs):
super().init(*args, **kwargs)

def load(self, fname):
    log.info('Starting the table load() from the extended 
    	     FSWTabDict class, using file: {}'.format(self.filename))
    return super(NewFSWTabDict, self).load(fname)

def custom():
    log.info('Test of a unique method defined in an extension.')

Used the following to instantiate and verify:

import ait.core.table as table

table_dict = table.getDefaultFSWTabDict()

jasonmlkang

Please see comments.

I think createFSWxxx methods should be auto generated during the extension process. Otherwise it would throw error when no extension is defined in the config.

Also the branch has conflicts with current master.

jasonmlkang · 2022-03-14T16:56:26Z

ait/core/cmd.py

 def YAMLCtor_include(loader, node):
    # Get the path out of the yaml file
    name = os.path.join(os.path.dirname(loader.name), node.value)
    data = None
    with open(name,'r') as f:
-        data = yaml.load(f)
+        data = yaml.load(f, Loader=yaml.Loader)


Why is this necessary?

I took out loader=yaml.loader.
We might want to get rid of data=None and use a try/except around the open block.

jasonmlkang · 2022-03-14T16:58:03Z

ait/core/dtype.py

@@ -71,7 +71,7 @@
 import sys
 import re

-from ait.core import cmd, dmc, log, util
+from ait.core import cmd, dmc, log, util, table


table is not reference in this module

jasonmlkang · 2022-03-14T17:02:53Z

ait/core/table.py

@@ -427,7 +432,7 @@ def create(self, name, *args):
        tab = None
        defn = self.get(name, None)
        if defn:
-            tab = FSWTab(defn, *args)
+            tab = createFSWTab(defn, *args)


createFSWTab method not defined

jasonmlkang · 2022-03-14T17:03:20Z

ait/core/table.py

@@ -461,7 +466,7 @@ def dirty(self):
    def load(self):
        if self.fswtabdict is None:
            if self.dirty():
-                self.fswtabdict = FSWTabDict(self.filename)
+                self.fswtabdict = createFSWTabDict(self.filename)


createFSWTabDict not defined

jasonmlkang · 2022-03-14T17:03:38Z

ait/core/table.py



 def YAMLCtor_FSWTabDefn(loader, node):
    fields = loader.construct_mapping(node, deep=True)
    fields['fswheaderdefns'] = fields.pop('header', None)
    fields['coldefns'] = fields.pop('columns', None)
-    return FSWTabDefn(**fields)
+    return createFSWTabDefn(**fields)


createFSWTabDict not defined

jasonmlkang · 2022-03-14T17:11:22Z

config/config.yaml

These are testing codes?

Yes that is for testing. I took it out.

nttoole · 2022-03-14T17:26:27Z

Correct, the createXxx() methods are implicitly added here:

AIT-Core/ait/core/util.py

Line 148 in 5b9a247

def __init_extensions__(modname, modsyms): # noqa

jasonmlkang

Please see comment. Thanks!

jasonmlkang · 2022-03-15T00:07:25Z

ait/core/cmd.py

+def YAMLCtor_include(loader, node):  # noqa
+    # Get the path out of the yaml file
+    name = os.path.join(os.path.dirname(loader.name), node.value)
+    data = None
+    with open(name, "r") as f:
+        data = yaml.load(f)
    return data


This overwrites previous definition of YAMLCtor_include. Is this done on purpose?

jasonmlkang · 2022-03-15T00:58:55Z

ait/core/table.py

@@ -434,7 +433,7 @@ def create(self, name, *args):
        tab = None
        defn = self.get(name, None)
        if defn:
-            tab = createFSWTab(defn, *args)
+            tab = FSWTab(defn, *args)


Sorry, my comment from last review was incorrect. The createFSWxxx method gets created in run-time during the extension process (i.e. util.__init_extensions__). So it is okay to call createFSWxxx method here. Please revert back to before this commit.

jasonmlkang · 2022-03-15T00:59:06Z

ait/core/table.py

@@ -468,7 +467,7 @@ def dirty(self):
    def load(self):
        if self.fswtabdict is None:
            if self.dirty():
-                self.fswtabdict = createFSWTabDict(self.filename)
+                self.fswtabdict = FSWTabDict(self.filename)


Sorry, my comment from last review was incorrect. The createFSWxxx method gets created in run-time during the extension process (i.e. util.__init_extensions__). So it is okay to call createFSWxxx method here. Please revert back to before this commit.

jasonmlkang · 2022-03-15T00:59:14Z

ait/core/table.py

@@ -504,14 +503,14 @@ def getDefaultDict():  # noqa: N802

 def YAMLCtor_FSWColDefn(loader, node):  # noqa: N802
    fields = loader.construct_mapping(node, deep=True)
-    return createFSWColDefn(**fields)
+    return FSWColDefn(**fields)


Sorry, my comment from last review was incorrect. The createFSWxxx method gets created in run-time during the extension process (i.e. util.__init_extensions__). So it is okay to call createFSWxxx method here. Please revert back to before this commit.

jasonmlkang · 2022-03-15T00:59:26Z

ait/core/table.py

-    return createFSWTabDefn(**fields)
+    fields["fswheaderdefns"] = fields.pop("header", None)
+    fields["coldefns"] = fields.pop("columns", None)
+    return FSWTabDefn(**fields)


Sorry, my comment from last review was incorrect. The createFSWxxx method gets created in run-time during the extension process (i.e. util.__init_extensions__). So it is okay to call createFSWxxx method here. Please revert back to before this commit.

Ok, I made the changes back and update utll.py

extended FSWTabDict class

eacbb9f

JimHofman requested review from a team as code owners October 20, 2021 16:21

issue-376: update new branch

a6dd09c

item-376 extensions for table.py

9fbae8b

jasonmlkang requested changes Mar 14, 2022

View reviewed changes

modifications based on rewiew

763ce15

jasonmlkang requested changes Mar 15, 2022

View reviewed changes

JHofman728 and others added 3 commits March 24, 2022 13:57

reverts in table, and update util

7def8d4

conflict with a comment

b0daf40

Merge branch 'master' into issue-376

3ed3b43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

extended FSWTabDict class #377

extended FSWTabDict class #377

JimHofman commented Oct 20, 2021

nttoole commented Oct 20, 2021 •

edited

Loading

JimHofman commented Oct 20, 2021

MJJoyce commented Oct 20, 2021

JimHofman commented Oct 28, 2021

jasonmlkang left a comment •

edited

Loading

jasonmlkang Mar 14, 2022

JimHofman Mar 14, 2022 •

edited

Loading

jasonmlkang Mar 14, 2022

jasonmlkang Mar 14, 2022

JimHofman Mar 14, 2022

jasonmlkang Mar 14, 2022

JimHofman Mar 14, 2022

jasonmlkang Mar 14, 2022

JimHofman Mar 14, 2022

jasonmlkang Mar 14, 2022

JimHofman Mar 14, 2022

nttoole commented Mar 14, 2022

jasonmlkang left a comment

jasonmlkang Mar 15, 2022

jasonmlkang Mar 15, 2022

jasonmlkang Mar 15, 2022

jasonmlkang Mar 15, 2022

jasonmlkang Mar 15, 2022

JimHofman Mar 24, 2022

extended FSWTabDict class #377

Are you sure you want to change the base?

extended FSWTabDict class #377

Conversation

JimHofman commented Oct 20, 2021

nttoole commented Oct 20, 2021 • edited Loading

JimHofman commented Oct 20, 2021

MJJoyce commented Oct 20, 2021

JimHofman commented Oct 28, 2021

jasonmlkang left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JimHofman Mar 14, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nttoole commented Mar 14, 2022

jasonmlkang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nttoole commented Oct 20, 2021 •

edited

Loading

jasonmlkang left a comment •

edited

Loading

JimHofman Mar 14, 2022 •

edited

Loading