Jupyter compatible analysis frontend
To experiment, run the choost_data() first
Visualize with scatter plot¶
Ugly prototype¶
To run interactive filter steps, I can use function returning function
def choose_data():
latest_tasks = list(i["name"] for i in get_ember_record())[:20]
def choose_task(task = latest_tasks):
er = emberReader(task)
def choose_log(log_file = er.t.log_files):
html(f"<div>Log file:\t<code>{log_file}</code> selected</div>")
df = pd.DataFrame(json.loads(er.read_log(log_file)))
def choose_plot(module = df.module.unique(),
ttype = df.ttype.unique(),
):
cols = ["mean","std","max","min","zero_pct"]
module_df = df[df.module == module]
global ember_sub_df
ember_sub_df = module_df[module_df.ttype == ttype]
html(f"<h3>Data selected</h3>")
html(f"<h4>Module:\t{module}</h4>")
html(f"<h4>Tensor Type:\t{ttype}</h4>")
if len(ember_sub_df)>0:
scatter_cols(ember_sub_df,cols)
else:
html(f"""<div>No such selection under <strong>{module}</Strong>
<br><strong>{ttype}</Strong></div>""")
return ember_sub_df
interact_manual(choose_plot)
sub_df = interact_manual(choose_log)
html("<h2>Choose from task names</h2>")
interact(choose_task)
Testing on the crude prototype
choose_data()
It works, just the way of doing it is beyond a self-pretentious python coder can take
Visualize Statictics¶
OOP improved
Having test
vis = VisualByTensor()
Useful attributes¶
vis.df.sample(5)
vis.ember_sub_df.sample(5)
Visualize Movement¶
vis_m = VisualMovement()