Data/provenance browser
Revision as of 16:21, 13 December 2010 by Tommy (talk | contribs) (→Add ideas/suggestions/goals here)
Dave's ideas
- We need a better mechanism that allows users to search/query for workflows and provenance, for example search by project, parameter names and values
- One way to view this is to allow the creation of mappings between the vistrails/projects into different structures---like "smart folders", e.g., /projectN/vt1/wf3/exec23/O1....
- Support editing and synchronization in both ways (if edit smart folder, should update vt)
- Should be able to script the use of the browser, e.g., to test a new version of a workflow against a saved test suite
- Scientists like a directory structure and can go browse there like they browse files with finder/explorer
- Keep mappings from our datatypes to file structures
- Need an exportable archive (to send with a paper, allow browsing outside of VisTrails)
Juliana's suggestions
As for the steps, here's what I suggest:
- try and get multiple vistrails accessible through the browser
- allow users to sort the information in different ways, based on the metadata stored in SQLLite.
- support for (simple) queries over the actual provenance info (version tree, workflows and logs), e.g., we could support the layered queries we described in the First Provenance Challenge.
- design and implement the map-to-directory functionality.
As we implement these, we should make sure we have a clean API that can be used without a GUI.
Tommy's notes
- How to add project annotations?
- Keep a synchronized index
- "Monitor" a specified directory for changes?
- "add" vistrail or directory to index
- Index vistrail when opened/saved
- How to efficiently index the execution log?
- Browsing the index
- Create smart directory structure using project/vistrail/workflow
- If we allow a vistrail to belong to multiple projects, the directory structure will be a DAG and not a tree.
- Querying the index
- Keyword searches
- Search on specific types e.g. "vistrail:proj3*"
- Export functionality
- Export all vistrails in a specific project in a directory structure
- Also handle workflow data?