From 5923036214bbd182dc79ca5f6ad21626a46803b9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 12:26:17 +0000 Subject: [PATCH 01/52] Initial plan From ae86059a9dc66a331d6fb439eee6a08f0cef3968 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 12:31:14 +0000 Subject: [PATCH 02/52] Add v3 UI/UX roadmap documentation with user stories Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/README.md | 9 ++ doc/v3/README.md | 63 ++++++++++++++ doc/v3/user-story-01-search.md | 81 +++++++++++++++++ doc/v3/user-story-02-account.md | 87 +++++++++++++++++++ doc/v3/user-story-03-submission.md | 121 ++++++++++++++++++++++++++ doc/v3/user-story-04-review.md | 129 +++++++++++++++++++++++++++ doc/v3/user-story-05-admin.md | 134 +++++++++++++++++++++++++++++ doc/v3/user-story-06-technical.md | 117 +++++++++++++++++++++++++ doc/v3/user-story-07-governance.md | 131 ++++++++++++++++++++++++++++ 9 files changed, 872 insertions(+) create mode 100644 doc/v3/README.md create mode 100644 doc/v3/user-story-01-search.md create mode 100644 doc/v3/user-story-02-account.md create mode 100644 doc/v3/user-story-03-submission.md create mode 100644 doc/v3/user-story-04-review.md create mode 100644 doc/v3/user-story-05-admin.md create mode 100644 doc/v3/user-story-06-technical.md create mode 100644 doc/v3/user-story-07-governance.md diff --git a/doc/README.md b/doc/README.md index 554ce3976..6f131d9c2 100644 --- a/doc/README.md +++ b/doc/README.md @@ -30,6 +30,14 @@ Historical documentation that is no longer relevant for current deployments: - **[INSTALL.md](archive/INSTALL.md)** - Original data loading environment setup (deprecated) - **[LOADING.md](archive/LOADING.md)** - TreeBASE v.1 to v.2 data migration procedures (deprecated) +### [v3/](v3/) + +UI/UX roadmap documentation for TreeBASE v3: + +- **[README.md](v3/README.md)** - Overview and links to all v3 user stories +- User stories covering search, account management, data submission, review, administration, + technical documentation, and governance + ## Quick Links ### I want to... @@ -41,6 +49,7 @@ Historical documentation that is no longer relevant for current deployments: - **Understand DWR integration** → [technical-notes/DWR.md](technical-notes/DWR.md) - **Learn about recent upgrades** → [technical-notes/UPGRADES.md](technical-notes/UPGRADES.md) - **Find historical data loading info** → [archive/](archive/) +- **See the v3 UI/UX roadmap** → [v3/](v3/) ## Contributing to Documentation diff --git a/doc/v3/README.md b/doc/v3/README.md new file mode 100644 index 000000000..8c3d717b7 --- /dev/null +++ b/doc/v3/README.md @@ -0,0 +1,63 @@ +# TreeBASE v3 UI/UX Roadmap + +This directory contains the UI/UX roadmap documentation for TreeBASE v3. The goal is to define +clear user stories that map to current application pages, identify navigation flows, and +ensure all existing pages are accounted for. + +## Purpose + +The v3 roadmap aims to: +1. Document user stories for all major user types +2. Map each story to specific pages in the current application +3. Define navigation paths through the application +4. Prepare for future wireframes and mockups + +## User Stories + +### Public Users + +- **[User Story 1: Search](user-story-01-search.md)** - A user or agent visits TreeBASE to search + for study, tree, matrix, taxon or analysis data + +### Authenticated Users + +- **[User Story 2: Account Management](user-story-02-account.md)** - A user visits TreeBASE to + create or update their user account and log in + +- **[User Story 3: Data Submission](user-story-03-submission.md)** - A user visits TreeBASE to + submit study, tree, matrix, taxon or analysis data + +### Reviewers + +- **[User Story 4: Review](user-story-04-review.md)** - A reviewer visits TreeBASE to review a + study with tree, matrix, taxon and analysis data + +### Administrators + +- **[User Story 5: Administration](user-story-05-admin.md)** - An admin visits TreeBASE to + administer the submission and review queue + +### Stakeholders + +- **[User Story 6: Technical Documentation](user-story-06-technical.md)** - A technical + stakeholder visits TreeBASE to learn about the technology of the web app, database, data + standards, APIs, or other tools + +- **[User Story 7: Governance](user-story-07-governance.md)** - A governance stakeholder visits + TreeBASE to learn about the project, its history, contributors, and role in phylogenetic data + management + +## Next Steps + +After these user stories are complete and all current pages are accounted for: + +1. Create wireframes/mockups for each page +2. Define widgets and interactions +3. Design the updated navigation structure +4. Plan the implementation phases + +## Related Documentation + +- [Main Documentation](../README.md) +- [API Documentation](../API.md) +- [OAI-PMH Documentation](../OAI-PMH.md) diff --git a/doc/v3/user-story-01-search.md b/doc/v3/user-story-01-search.md new file mode 100644 index 000000000..97ceb0963 --- /dev/null +++ b/doc/v3/user-story-01-search.md @@ -0,0 +1,81 @@ +# User Story 1: Search + +## Overview + +**As a** user or automated agent, +**I want to** search for study, tree, matrix, taxon or analysis data, +**So that** I can find and access phylogenetic data relevant to my research or application. + +## User Types + +- Anonymous visitors (researchers, students, general public) +- Automated agents (harvesters, API clients, search engines) +- Registered users looking for data + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] Home page search interface +- [ ] Advanced search page +- [ ] Search results page +- [ ] Study detail page +- [ ] Tree detail page +- [ ] Matrix detail page +- [ ] Taxon detail page +- [ ] Analysis detail page + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Entry Point] --> [Search Interface] --> [Search Results] --> [Detail Page] + | | + v v + [Advanced Search] [Related Data] +``` + +## Search Capabilities + +### Search Types +- Full-text search +- Search by study ID +- Search by author +- Search by taxon +- Search by publication +- Search by tree characteristics +- Search by matrix characteristics + +### Data Types Searchable +1. **Studies** - Published research studies with phylogenetic data +2. **Trees** - Phylogenetic trees associated with studies +3. **Matrices** - Character matrices used in analyses +4. **Taxa** - Taxonomic names and their occurrences +5. **Analyses** - Analysis metadata and parameters + +## API/Agent Access + +*TODO: Document programmatic access points for this user story* + +- PhyloWS API endpoints +- OAI-PMH harvesting interface +- Direct URL patterns + +## Pages to Account For + +*TODO: Complete inventory of pages related to search functionality* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What search refinement options should be available? +- How should results be sorted and paginated? +- What metadata should be shown in search results? diff --git a/doc/v3/user-story-02-account.md b/doc/v3/user-story-02-account.md new file mode 100644 index 000000000..1dff0777c --- /dev/null +++ b/doc/v3/user-story-02-account.md @@ -0,0 +1,87 @@ +# User Story 2: Account Management + +## Overview + +**As a** user, +**I want to** create or update my user account and log in, +**So that** I can access authenticated features like data submission and study management. + +## User Types + +- New users creating an account +- Existing users logging in +- Users updating their profile information +- Users managing their password/credentials + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] Registration page +- [ ] Login page +- [ ] Logout functionality +- [ ] User profile page +- [ ] Password reset page +- [ ] Account settings page + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Any Page] --> [Login Link] --> [Login Page] --> [Authenticated Home] + | | + v v + [Registration] [Forgot Password] + | | + v v + [Verify Email] [Reset Password] +``` + +## Account Features + +### Registration +- Email address +- Username +- Password requirements +- Institutional affiliation +- ORCID integration (if applicable) + +### Authentication +- Username/password login +- Session management +- Remember me functionality +- Logout from all devices + +### Profile Management +- Update personal information +- Change email address +- Update password +- Manage notification preferences +- View submission history + +## Pages to Account For + +*TODO: Complete inventory of pages related to account management* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Security Considerations + +- Password strength requirements +- Account lockout policies +- Session timeout +- HTTPS requirements +- Email verification + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What OAuth/SSO options should be supported? +- What information is required vs optional during registration? +- How long should sessions last? diff --git a/doc/v3/user-story-03-submission.md b/doc/v3/user-story-03-submission.md new file mode 100644 index 000000000..a82c79de1 --- /dev/null +++ b/doc/v3/user-story-03-submission.md @@ -0,0 +1,121 @@ +# User Story 3: Data Submission + +## Overview + +**As a** registered user, +**I want to** submit study, tree, matrix, taxon or analysis data, +**So that** I can share my phylogenetic research with the scientific community. + +## User Types + +- Researchers submitting their own studies +- Students submitting work on behalf of advisors +- Data curators submitting historical data + +## Prerequisites + +- User must be logged in (see [User Story 2: Account Management](user-story-02-account.md)) + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] Submission dashboard +- [ ] New submission form +- [ ] Study metadata entry +- [ ] File upload interface +- [ ] Tree upload/editor +- [ ] Matrix upload/editor +- [ ] Taxa management +- [ ] Analysis metadata entry +- [ ] Submission preview +- [ ] Submission confirmation + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[User Dashboard] --> [New Submission] --> [Study Metadata] + | + v + [File Upload] + | + v + [Tree/Matrix/Taxa Entry] + | + v + [Analysis Metadata] + | + v + [Preview & Validate] + | + v + [Submit] +``` + +## Submission Workflow + +### Step 1: Study Information +- Publication details (journal, DOI, authors) +- Study title and abstract +- Keywords and categories + +### Step 2: Data Upload +- NEXUS file upload +- Newick file upload +- Other supported formats +- File validation + +### Step 3: Tree Data +- Tree visualization +- Tree metadata +- Multiple trees per study + +### Step 4: Matrix Data +- Character matrix display +- Matrix metadata +- Character definitions + +### Step 5: Taxonomic Data +- Taxon name reconciliation +- Taxonomic hierarchy +- OTU mapping + +### Step 6: Analysis Information +- Analysis type/method +- Software used +- Parameters and settings + +### Step 7: Review and Submit +- Validation summary +- Preview all data +- Submit for review + +## Data Validation + +*TODO: Document validation rules and error handling* + +- File format validation +- Required field validation +- Taxonomic name validation +- Consistency checks + +## Pages to Account For + +*TODO: Complete inventory of pages related to data submission* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What file formats should be supported? +- What is the maximum file size? +- Can submissions be saved as drafts? +- What validation happens at each step vs. at submission? diff --git a/doc/v3/user-story-04-review.md b/doc/v3/user-story-04-review.md new file mode 100644 index 000000000..83c51b962 --- /dev/null +++ b/doc/v3/user-story-04-review.md @@ -0,0 +1,129 @@ +# User Story 4: Review + +## Overview + +**As a** reviewer, +**I want to** review a study with tree, matrix, taxon and analysis data, +**So that** I can verify the quality and accuracy of submitted phylogenetic data. + +## User Types + +- Journal editors reviewing submissions +- Peer reviewers assigned to studies +- TreeBASE curators performing quality control + +## Prerequisites + +- User must be logged in with reviewer privileges +- Study must be assigned to the reviewer + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] Review queue/dashboard +- [ ] Study review page +- [ ] Tree review interface +- [ ] Matrix review interface +- [ ] Taxa review interface +- [ ] Analysis review interface +- [ ] Comment/feedback entry +- [ ] Approval/rejection interface + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Reviewer Dashboard] --> [Review Queue] --> [Study Review] + | + +----------------------------+ + | | | + v v v + [Trees] [Matrices] [Taxa] + | | | + v v v + [Review] [Review] [Review] + | | | + +----------------------------+ + | + v + [Analysis Review] --> [Final Decision] + | + +--------------------+--------------------+ + | | | + v v v + [Approve] [Request Changes] [Reject] +``` + +## Review Workflow + +### Step 1: Access Review Queue +- View assigned reviews +- Filter by status, date, study type +- Priority indicators + +### Step 2: Study Overview Review +- Verify publication details +- Check metadata completeness +- Validate citations + +### Step 3: Tree Data Review +- Visualize tree structure +- Check topology +- Verify branch support values +- Validate tree labels + +### Step 4: Matrix Data Review +- Review character matrix +- Check for data completeness +- Validate character definitions +- Review coding consistency + +### Step 5: Taxonomic Review +- Verify taxon names +- Check taxonomic placement +- Review OTU mappings +- Flag nomenclatural issues + +### Step 6: Analysis Review +- Check analysis parameters +- Verify software citations +- Review methodology description + +### Step 7: Final Decision +- Approve for publication +- Request revisions with comments +- Reject with explanation + +## Review Features + +### Comments and Feedback +- Line-level comments on trees/matrices +- General study comments +- Suggestion for corrections +- Communication with submitter + +### Quality Checks +- Automated validation results +- Manual checklist items +- Cross-reference verification + +## Pages to Account For + +*TODO: Complete inventory of pages related to review functionality* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What review statuses are needed? +- How should reviewer assignments work? +- What communication happens between reviewer and submitter? +- Can reviews be collaborative (multiple reviewers)? diff --git a/doc/v3/user-story-05-admin.md b/doc/v3/user-story-05-admin.md new file mode 100644 index 000000000..3552d0812 --- /dev/null +++ b/doc/v3/user-story-05-admin.md @@ -0,0 +1,134 @@ +# User Story 5: Administration + +## Overview + +**As an** administrator, +**I want to** administer the submission and review queue, +**So that** I can ensure efficient processing of phylogenetic data submissions. + +## User Types + +- System administrators +- TreeBASE curators with admin privileges +- Queue managers + +## Prerequisites + +- User must be logged in with administrator privileges + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] Admin dashboard +- [ ] Submission queue management +- [ ] Review queue management +- [ ] User management +- [ ] Reviewer assignment interface +- [ ] System settings +- [ ] Reports and analytics +- [ ] Audit logs + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Admin Dashboard] --> [Submission Queue] + | | + | v + | [Assign Reviewer] + | | + v v +[Review Queue] <--> [Monitor Progress] + | | + v v +[User Management] [Reports/Analytics] + | | + v v +[System Settings] [Audit Logs] +``` + +## Administration Functions + +### Submission Queue Management +- View all pending submissions +- Filter by status, date, submitter +- Prioritize submissions +- Assign to review track +- Bulk actions + +### Review Queue Management +- View all reviews in progress +- Monitor review timelines +- Reassign reviewers +- Escalate overdue reviews +- Track completion rates + +### Reviewer Assignment +- View available reviewers +- Match expertise to submissions +- Balance reviewer workload +- Set review deadlines + +### User Management +- View all users +- Edit user profiles +- Manage user roles (submitter, reviewer, admin) +- Activate/deactivate accounts +- Reset passwords + +### System Settings +- Configure submission requirements +- Set review workflows +- Manage notification templates +- Configure validation rules + +### Reports and Analytics +- Submission statistics +- Review turnaround times +- User activity reports +- Data growth metrics + +### Audit Logs +- User login history +- Data modification history +- Administrative actions +- Security events + +## Admin Features + +### Queue Operations +- Sort and filter options +- Bulk status updates +- Quick actions menu +- Search within queues + +### Communication +- Send notifications to users +- Broadcast announcements +- Email templates + +### Monitoring +- Dashboard widgets +- Alert configurations +- Status indicators + +## Pages to Account For + +*TODO: Complete inventory of pages related to administration* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What admin roles/permission levels are needed? +- What metrics should be tracked on the dashboard? +- How should escalation workflows work? +- What audit requirements exist? diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md new file mode 100644 index 000000000..8b1180b4c --- /dev/null +++ b/doc/v3/user-story-06-technical.md @@ -0,0 +1,117 @@ +# User Story 6: Technical Documentation + +## Overview + +**As a** technical stakeholder, +**I want to** learn about the technology of the web app, database, data standards, APIs, or other +tools, +**So that** I can integrate with TreeBASE, contribute to development, or understand its technical +architecture. + +## User Types + +- Software developers integrating with TreeBASE +- API consumers building applications +- Contributors to TreeBASE development +- Researchers using programmatic access +- System administrators deploying TreeBASE + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] API documentation page +- [ ] Developer guide page +- [ ] Data standards documentation +- [ ] Download/export documentation +- [ ] Integration examples +- [ ] Technical FAQ + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Documentation Home] --> [API Reference] + | | + | v + | [API Examples] + | | + v v +[Data Standards] --> [Data Formats] + | | + v v +[Developer Guide] --> [Contributing] + | | + v v +[Architecture] --> [Deployment Guide] +``` + +## Technical Documentation Areas + +### API Documentation +- PhyloWS API reference +- OAI-PMH interface documentation +- Authentication requirements +- Rate limiting and usage policies +- Example requests and responses + +### Data Standards +- NeXML format documentation +- NEXUS format support +- Newick tree format +- Phylogenetic data interchange standards +- Metadata schemas + +### Developer Resources +- Architecture overview +- Source code repository +- Build instructions +- Development environment setup +- Contribution guidelines + +### Integration Guides +- How to query TreeBASE +- How to harvest data +- How to submit programmatically +- Client libraries and SDKs + +### Database Documentation +- Data model overview +- Entity relationships +- Query patterns +- Data dictionary + +### Deployment +- System requirements +- Installation guide +- Configuration options +- Monitoring and maintenance + +## External Documentation + +*Links to existing technical documentation* + +- [API.md](../API.md) - PhyloWS API documentation +- [OAI-PMH.md](../OAI-PMH.md) - OAI-PMH harvesting interface +- [Development Documentation](../development/) - Build and deployment guides +- [Technical Notes](../technical-notes/) - Implementation details + +## Pages to Account For + +*TODO: Complete inventory of pages related to technical documentation* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- Where should API documentation live (in-app vs external)? +- What interactive API features are needed (playground, testing)? +- How should code examples be maintained? +- What versioning strategy for API documentation? diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md new file mode 100644 index 000000000..d5b75e6f4 --- /dev/null +++ b/doc/v3/user-story-07-governance.md @@ -0,0 +1,131 @@ +# User Story 7: Governance + +## Overview + +**As a** governance stakeholder, +**I want to** learn about the project, its history, contributors, and role in phylogenetic data +management, +**So that** I can understand TreeBASE's position in the scientific data ecosystem and make +informed decisions about its future. + +## User Types + +- Funding agency representatives +- Institutional administrators +- Scientific advisory board members +- Policy makers +- Potential partners and collaborators +- Science journalists and communicators + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] About page +- [ ] History/timeline page +- [ ] Contributors/team page +- [ ] Mission statement +- [ ] Data usage policies +- [ ] Terms of service +- [ ] Privacy policy +- [ ] Contact information +- [ ] Governance structure +- [ ] Funding acknowledgments + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Home] --> [About TreeBASE] + | + +---> [History] + | + +---> [Team/Contributors] + | + +---> [Mission & Vision] + | + +---> [Governance Structure] + | + +---> [Policies] + | | + | +---> [Data Usage] + | | + | +---> [Terms of Service] + | | + | +---> [Privacy Policy] + | + +---> [Funding & Support] + | + +---> [Contact] +``` + +## Governance Information + +### About TreeBASE +- What is TreeBASE +- Mission statement +- Vision for the future +- Value proposition + +### History +- Timeline of development +- Major milestones +- Evolution of the platform +- Historical context in phylogenetics + +### Team and Contributors +- Current team members +- Advisory board +- Past contributors +- Institutional affiliations + +### Governance Structure +- Decision-making processes +- Organizational structure +- Partner institutions +- Community involvement + +### Policies +- Data submission policies +- Data usage and licensing +- Terms of service +- Privacy policy +- Code of conduct + +### Impact and Metrics +- Usage statistics +- Citation information +- Data holdings summary +- Community reach + +### Funding and Support +- Current funders +- Historical funding +- How to support TreeBASE +- Institutional partnerships + +### Contact and Engagement +- Contact information +- Social media presence +- Mailing lists +- Community forums + +## Pages to Account For + +*TODO: Complete inventory of pages related to governance* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What governance information should be prominently featured? +- How should historical information be presented? +- What metrics should be publicly displayed? +- How should contact/support requests be handled? From e19ad7ba59cdda536276baa26c074c29ecb4cd4e Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 16:44:00 +0100 Subject: [PATCH 03/52] Refine TODOs for account documentation and features Updated TODOs to specify documentation requirements and references. --- doc/v3/user-story-02-account.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/doc/v3/user-story-02-account.md b/doc/v3/user-story-02-account.md index 1dff0777c..d4142fa6a 100644 --- a/doc/v3/user-story-02-account.md +++ b/doc/v3/user-story-02-account.md @@ -26,7 +26,7 @@ ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages using a mermaid plot referencing explicit pages and widgets* ``` [Any Page] --> [Login Link] --> [Login Page] --> [Authenticated Home] @@ -41,19 +41,29 @@ ## Account Features ### Registration + +*TODO: Identify the precise requirements for registration from the current implementation including what is required/optional* + - Email address - Username +- Real name +- Phone number - Password requirements - Institutional affiliation -- ORCID integration (if applicable) ### Authentication + +*TODO: Establish the precise details of the authentication implementation* + - Username/password login - Session management - Remember me functionality - Logout from all devices ### Profile Management + +*TODO: Establish the precise functionality of profile management* + - Update personal information - Change email address - Update password @@ -62,7 +72,7 @@ ## Pages to Account For -*TODO: Complete inventory of pages related to account management* +*TODO: Complete inventory of pages related to account management with reference to /treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml* | Page | URL Pattern | Status | |------|-------------|--------| @@ -80,8 +90,3 @@ *To be completed in future PR* -## Open Questions - -- What OAuth/SSO options should be supported? -- What information is required vs optional during registration? -- How long should sessions last? From 8587c95c0dffaca7bcdab0c86472f4ad5826b62f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 15:45:07 +0000 Subject: [PATCH 04/52] Initial plan From f261af16eff283648a15c1ced787638a2588f29b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 15:50:25 +0000 Subject: [PATCH 05/52] Address TODOs in user-story-02-account.md with comprehensive account management documentation Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-02-account.md | 264 ++++++++++++++++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 doc/v3/user-story-02-account.md diff --git a/doc/v3/user-story-02-account.md b/doc/v3/user-story-02-account.md new file mode 100644 index 000000000..1e1c9064b --- /dev/null +++ b/doc/v3/user-story-02-account.md @@ -0,0 +1,264 @@ +# User Story 2: Account Management + +## Overview + +**As a** user, +**I want to** create or update my user account and log in, +**So that** I can access authenticated features like data submission and study management. + +## User Types + +- New users creating an account +- Existing users logging in +- Users updating their profile information +- Users managing their password/credentials + +## Current Pages + +The following pages apply to this user story: + +- [x] Registration page (`/register.html` → `userForm.jsp`) +- [x] Login page (`/login.jsp` → `loginForm.jsp`) +- [x] Logout functionality (`/logout.jsp` → Spring Security logout handler `/logout`) +- [x] User profile page (`/user/updateProfile.html` → `userForm.jsp`) +- [x] Password reset request page (`/passwordForm.html` → `passwordForm.jsp`) +- [x] Password reset form page (`/resetPassword.html` → `resetPasswordForm.jsp`) +- [x] Admin user management pages (`/admin/overrideUserProfile.html`, `/admin/adminSelectUsers.html`, etc.) + +## Navigation Flow + +The following diagram documents how users navigate through account management pages: + +```mermaid +flowchart TD + subgraph Public["Public Pages"] + Home["/home.html
Home Page"] + Login["/login.jsp
Login Page
(loginForm.jsp)"] + Register["/register.html
Registration Page
(userForm.jsp)"] + PwdRequest["/passwordForm.html
Password Reset Request
(passwordForm.jsp)"] + PwdReset["/resetPassword.html
Reset Password Form
(resetPasswordForm.jsp)"] + end + + subgraph Authenticated["Authenticated Pages"] + UserHome["/user/processUser.html
User Landing"] + Profile["/user/updateProfile.html
Update Profile
(userForm.jsp)"] + Submissions["/user/submissionList.html
My Submissions"] + Logout["/logout
Logout Handler"] + end + + subgraph Admin["Admin Pages (Admin/Associate Editor only)"] + AdminPage["/admin/administrationPage.html
Admin Home"] + SelectUsers["/admin/adminSelectUsers.html
Select Users"] + OverrideProfile["/admin/overrideUserProfile.html
Override User Profile"] + MergeUsers["/admin/adminMergingUsers.html
Merge Users"] + DeleteUser["/admin/adminDeletingUserStepOne.html
Delete User"] + end + + Home -->|"Login link"| Login + Login -->|"Sign up link"| Register + Login -->|"Forgot password"| PwdRequest + Login -->|"Submit credentials
(j_security_check)"| UserHome + Register -->|"Submit registration"| Login + PwdRequest -->|"Request sent"| PwdRequest + PwdRequest -->|"Email link"| PwdReset + PwdReset -->|"Password reset success"| Login + + UserHome -->|"Navigation tab"| Profile + UserHome -->|"Navigation tab"| Submissions + Profile -->|"Update success"| Profile + Submissions -->|"Logout link"| Logout + Profile -->|"Logout link"| Logout + Logout --> Login + + UserHome -->|"Admin tab
(if Admin role)"| AdminPage + AdminPage --> SelectUsers + SelectUsers --> OverrideProfile + AdminPage --> MergeUsers + AdminPage --> DeleteUser +``` + +## Account Features + +### Registration + +Registration is handled by `RegisterUserController` with the following requirements from the current implementation: + +**Required Fields:** +- **Username** - Unique identifier for login (required, unique in database) +- **Password** - User's password (required, minimum validation via JavaScript) +- **Re-typed Password** - Password confirmation (required, must match password) +- **First Name** - User's first name (required) +- **Last Name** - User's last name (required) +- **Email Address** - Contact email (required, unique in database) + +**Optional Fields:** +- **Middle Name** - User's middle name +- **Phone Number** - Contact phone number + +**Registration Flow:** +1. User navigates to `/register.html` +2. User fills in required and optional fields +3. Client-side JavaScript validates password match via `checkPasswords()` +4. Server-side validation via `BeanValidator` checks field constraints +5. Password is encoded using BCrypt via `PasswordEncoder` +6. `UserService.createUser()` checks for duplicate username/email +7. On success, redirect to `/login.jsp` +8. On failure (duplicate username/email), error message displayed + +### Authentication + +Authentication is implemented using Spring Security 5.8.15 with the following details: + +**Login Mechanism:** +- Form-based authentication at `/j_security_check` +- Username parameter: `j_username` +- Password parameter: `j_password` +- Login page: `/login.jsp` +- Default success URL: `/user/processUser.html` +- Failure URL: `/login.jsp?error=true` + +**Password Handling:** +- `DelegatingPasswordEncoder` for backward compatibility (supports BCrypt and legacy plain text) +- `PasswordUpgradeAuthenticationSuccessHandler` automatically upgrades plain text passwords to BCrypt on successful login + +**Session Management:** +- Session fixation protection: `newSession` strategy +- CSRF protection: Currently disabled for backward compatibility + +**Authorization (URL-based):** +- `/submit.html` - Requires: User, Admin, or Associate Editor +- `/user/**` - Requires: User, Admin, or Associate Editor +- `/admin/**` - Requires: Admin or Associate Editor + +**Role Configuration:** +- Role prefix: Empty string (no `ROLE_` prefix) +- Existing roles in database: `Admin`, `User`, `Associate Editor` + +**Logout:** +- Logout URL: `/logout` +- Logout success URL: `/login.jsp` +- Session invalidation handled by Spring Security + +### Profile Management + +Profile management is handled by `UserFormController` with the following functionality: + +**View/Update Profile:** +- URL: `/user/updateProfile.html` +- Controller: `UserFormController` +- View: `userForm.jsp` + +**Editable Fields:** +- **Password** - Can update password (leave blank to keep current) +- **Re-typed Password** - Must match new password if changing +- **First Name** - User's first name +- **Middle Name** - User's middle name (optional) +- **Last Name** - User's last name +- **Phone Number** - Contact phone number (optional) +- **Email Address** - Contact email + +**Read-only Fields:** +- **Username** - Cannot be changed after registration + +**Security:** +- Users can only modify their own profile +- Admin users can modify other users' profiles via `/admin/overrideUserProfile.html` +- Password changes are validated and BCrypt-encoded before storage + +**View Submission History:** +- URL: `/user/submissionList.html` +- Shows list of user's submissions with their status +- Each submission links to detailed view and editing options + +**Notification Preferences:** +- Not currently implemented in the system +- Future enhancement opportunity + +## Pages to Account For + +The following is a complete inventory of pages related to account management based on `/treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml`: + +| Page | URL Pattern | Controller | View | Status | +|------|-------------|------------|------|--------| +| Registration | `/register.html` | `registerUserController` | `userForm.jsp` | Active | +| Login | `/login.jsp` (static) | - | `login.jsp` | Active | +| Login Form | `/login.jsp` | - | `loginForm.jsp` (included) | Active | +| Password Reset Request | `/passwordForm.html` | `passwordFormController` | `passwordForm.jsp` | Active | +| Password Reset Form | `/resetPassword.html` | `resetPasswordController` | `resetPasswordForm.jsp` | Active | +| User Profile | `/user/updateProfile.html` | `userFormController` | `userForm.jsp` | Active | +| Process User (Post-login) | `/user/processUser.html` | `processUserController` | - | Active | +| Submission List | `/user/submissionList.html` | `listSubmissionController` | `submissionList.jsp` | Active | +| Admin - Select Users | `/admin/adminSelectUsers.html` | `adminSelectUsersController` | `adminSelectUsers.jsp` | Active | +| Admin - User List | `/admin/userList.html` | `filenameController` | `userList.jsp` | Active | +| Admin - Override Profile | `/admin/overrideUserProfile.html` | `adminOverridingUserFormController` | `overrideUserProfile.jsp` | Active | +| Admin - Update User Info | `/admin/adminUpdatingUserInfo.html` | `adminUpdatingUserInfoController` | `adminUpdatingUserInfo.jsp` | Active | +| Admin - Delete User Step 1 | `/admin/adminDeletingUserStepOne.html` | `adminDeletingUserStepOneController` | `adminDeletingUserStepOne.jsp` | Active | +| Admin - Delete User Step 2 | `/admin/adminDeletingUserStepTwo.html` | `adminDeletingUserStepTwoController` | `adminDeletingUserStepTwo.jsp` | Active | +| Admin - Merge Users | `/admin/adminMergingUsers.html` | `adminMergingUsersController` | `adminMergingUsers.jsp` | Active | +| Admin - User Management | `/admin/userManagement.html` | `userManagementController` | `simpleUserManagement.jsp` | Active | +| Logout | `/logout` | Spring Security | Redirect to `/login.jsp` | Active | + +## Security Considerations + +- **Password strength requirements**: Minimum 8 characters enforced in password reset form; registration form uses JavaScript validation but should be enhanced +- **Account lockout policies**: Not currently implemented; consider adding after N failed attempts +- **Session timeout**: Managed by servlet container (Tomcat) defaults +- **HTTPS requirements**: Should be enforced at server/proxy level; not enforced in application code +- **Email verification**: Not currently implemented; email is validated but not verified via confirmation link +- **Password reset token security**: + - Tokens are cryptographically secure (UUID-based) + - 24-hour expiration + - Single-use (marked as used after successful reset) + - Previous tokens invalidated when new one is generated + - Timing-attack protection via normalized response times +- **CSRF protection**: Currently disabled; should be enabled in future security hardening +- **Role-based access control**: Implemented via Spring Security with URL-based authorization + +## Wireframe Notes + +*To be completed in future PR* + +--- + +## Related Files + +### Controllers +- `RegisterUserController.java` - New user registration +- `UserFormController.java` - Profile updates +- `PasswordFormController.java` - Password reset request +- `ResetPasswordController.java` - Password reset form handling +- `ProcessUserController.java` - Post-login processing +- `AbstractUserController.java` - Base controller for user operations +- `AdminOverridingUserFormController.java` - Admin profile editing +- `AdminDeletingUserStepOneController.java` / `StepTwoController.java` - User deletion +- `AdminMergingUsersController.java` - User account merging +- `AdminSelectUsersController.java` - Admin user selection + +### Domain Objects +- `User.java` - User entity implementing `UserDetails` +- `UserRole.java` - Role entity (Admin, User, Associate Editor) +- `PasswordResetToken.java` - Secure password reset tokens +- `Person.java` - Personal information linked to User + +### Services +- `UserService.java` - User business logic +- `PasswordResetTokenHome.java` - Token persistence + +### Security +- `treebase-security.xml` - Spring Security configuration +- `DelegatingPasswordEncoder.java` - BCrypt + legacy support +- `PasswordUpgradeAuthenticationSuccessHandler.java` - Auto password upgrade + +### Views (JSP) +- `userForm.jsp` - Registration and profile editing +- `login.jsp` - Login page container +- `loginForm.jsp` - Login form widget +- `passwordForm.jsp` - Password reset request +- `resetPasswordForm.jsp` - New password entry +- `overrideUserProfile.jsp` - Admin profile editing +- `userList.jsp` - Admin user list + +### Common Components +- `nav.jsp` - Navigation tabs (Personal Info, Submissions, Search, Admin) +- `header.jsp` - Site header +- `defaultTemplate.jsp` - Main template with login status display From 2388483f1d27f5d16b425b983bdf791109772352 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 15:54:47 +0000 Subject: [PATCH 06/52] Update user-story-02-account.md based on v3-ui-ux branch original with TODOs addressed Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-02-account.md | 95 ++++++++++----------------------- 1 file changed, 28 insertions(+), 67 deletions(-) diff --git a/doc/v3/user-story-02-account.md b/doc/v3/user-story-02-account.md index 1e1c9064b..e1d755b93 100644 --- a/doc/v3/user-story-02-account.md +++ b/doc/v3/user-story-02-account.md @@ -19,7 +19,7 @@ The following pages apply to this user story: - [x] Registration page (`/register.html` → `userForm.jsp`) - [x] Login page (`/login.jsp` → `loginForm.jsp`) -- [x] Logout functionality (`/logout.jsp` → Spring Security logout handler `/logout`) +- [x] Logout functionality (`/logout` → Spring Security logout handler) - [x] User profile page (`/user/updateProfile.html` → `userForm.jsp`) - [x] Password reset request page (`/passwordForm.html` → `passwordForm.jsp`) - [x] Password reset form page (`/resetPassword.html` → `resetPasswordForm.jsp`) @@ -84,16 +84,19 @@ flowchart TD Registration is handled by `RegisterUserController` with the following requirements from the current implementation: **Required Fields:** -- **Username** - Unique identifier for login (required, unique in database) -- **Password** - User's password (required, minimum validation via JavaScript) +- **Username** - Unique identifier for login (required, unique in database, `user.username`) +- **Password** - User's password (required, BCrypt encoded before storage) - **Re-typed Password** - Password confirmation (required, must match password) -- **First Name** - User's first name (required) -- **Last Name** - User's last name (required) -- **Email Address** - Contact email (required, unique in database) +- **First Name** - User's first name (required, `user.firstName`) +- **Last Name** - User's last name (required, `user.lastName`) +- **Email Address** - Contact email (required, unique in database, `user.emailAddressString`) **Optional Fields:** -- **Middle Name** - User's middle name -- **Phone Number** - Contact phone number +- **Middle Name** - User's middle name (`user.middleName`) +- **Phone Number** - Contact phone number (`user.phoneNumber`) + +**Not Implemented:** +- Institutional affiliation - Not present in current User entity **Registration Flow:** 1. User navigates to `/register.html` @@ -107,7 +110,7 @@ Registration is handled by `RegisterUserController` with the following requireme ### Authentication -Authentication is implemented using Spring Security 5.8.15 with the following details: +Authentication is implemented using Spring Security 5.8.15 (`treebase-security.xml`): **Login Mechanism:** - Form-based authentication at `/j_security_check` @@ -123,6 +126,7 @@ Authentication is implemented using Spring Security 5.8.15 with the following de **Session Management:** - Session fixation protection: `newSession` strategy +- Session timeout: Managed by servlet container (Tomcat) defaults - CSRF protection: Currently disabled for backward compatibility **Authorization (URL-based):** @@ -134,10 +138,9 @@ Authentication is implemented using Spring Security 5.8.15 with the following de - Role prefix: Empty string (no `ROLE_` prefix) - Existing roles in database: `Admin`, `User`, `Associate Editor` -**Logout:** -- Logout URL: `/logout` -- Logout success URL: `/login.jsp` -- Session invalidation handled by Spring Security +**Not Implemented:** +- Remember me functionality - Not present in current security configuration +- Logout from all devices - Single session logout only via `/logout` ### Profile Management @@ -161,7 +164,7 @@ Profile management is handled by `UserFormController` with the following functio - **Username** - Cannot be changed after registration **Security:** -- Users can only modify their own profile +- Users can only modify their own profile (enforced in `UserFormController`) - Admin users can modify other users' profiles via `/admin/overrideUserProfile.html` - Password changes are validated and BCrypt-encoded before storage @@ -170,24 +173,25 @@ Profile management is handled by `UserFormController` with the following functio - Shows list of user's submissions with their status - Each submission links to detailed view and editing options -**Notification Preferences:** -- Not currently implemented in the system -- Future enhancement opportunity +**Not Implemented:** +- Change email address - Email is editable but not re-verified +- Manage notification preferences - Not present in current implementation ## Pages to Account For -The following is a complete inventory of pages related to account management based on `/treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml`: +Complete inventory of pages related to account management from `/treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml`: | Page | URL Pattern | Controller | View | Status | |------|-------------|------------|------|--------| | Registration | `/register.html` | `registerUserController` | `userForm.jsp` | Active | -| Login | `/login.jsp` (static) | - | `login.jsp` | Active | -| Login Form | `/login.jsp` | - | `loginForm.jsp` (included) | Active | +| Login | `/login.jsp` | (static JSP) | `login.jsp` | Active | +| Login Form | (included in login.jsp) | - | `loginForm.jsp` | Active | | Password Reset Request | `/passwordForm.html` | `passwordFormController` | `passwordForm.jsp` | Active | | Password Reset Form | `/resetPassword.html` | `resetPasswordController` | `resetPasswordForm.jsp` | Active | | User Profile | `/user/updateProfile.html` | `userFormController` | `userForm.jsp` | Active | | Process User (Post-login) | `/user/processUser.html` | `processUserController` | - | Active | -| Submission List | `/user/submissionList.html` | `listSubmissionController` | `submissionList.jsp` | Active | +| Submission List | `/user/submissionList.html` | `listSubmissionController` | - | Active | +| Logout | `/logout` | Spring Security | Redirect to `/login.jsp` | Active | | Admin - Select Users | `/admin/adminSelectUsers.html` | `adminSelectUsersController` | `adminSelectUsers.jsp` | Active | | Admin - User List | `/admin/userList.html` | `filenameController` | `userList.jsp` | Active | | Admin - Override Profile | `/admin/overrideUserProfile.html` | `adminOverridingUserFormController` | `overrideUserProfile.jsp` | Active | @@ -196,11 +200,12 @@ The following is a complete inventory of pages related to account management bas | Admin - Delete User Step 2 | `/admin/adminDeletingUserStepTwo.html` | `adminDeletingUserStepTwoController` | `adminDeletingUserStepTwo.jsp` | Active | | Admin - Merge Users | `/admin/adminMergingUsers.html` | `adminMergingUsersController` | `adminMergingUsers.jsp` | Active | | Admin - User Management | `/admin/userManagement.html` | `userManagementController` | `simpleUserManagement.jsp` | Active | -| Logout | `/logout` | Spring Security | Redirect to `/login.jsp` | Active | ## Security Considerations -- **Password strength requirements**: Minimum 8 characters enforced in password reset form; registration form uses JavaScript validation but should be enhanced +Current implementation status: + +- **Password strength requirements**: Minimum 8 characters enforced in password reset form; registration form uses JavaScript validation but could be enhanced with server-side enforcement - **Account lockout policies**: Not currently implemented; consider adding after N failed attempts - **Session timeout**: Managed by servlet container (Tomcat) defaults - **HTTPS requirements**: Should be enforced at server/proxy level; not enforced in application code @@ -218,47 +223,3 @@ The following is a complete inventory of pages related to account management bas *To be completed in future PR* ---- - -## Related Files - -### Controllers -- `RegisterUserController.java` - New user registration -- `UserFormController.java` - Profile updates -- `PasswordFormController.java` - Password reset request -- `ResetPasswordController.java` - Password reset form handling -- `ProcessUserController.java` - Post-login processing -- `AbstractUserController.java` - Base controller for user operations -- `AdminOverridingUserFormController.java` - Admin profile editing -- `AdminDeletingUserStepOneController.java` / `StepTwoController.java` - User deletion -- `AdminMergingUsersController.java` - User account merging -- `AdminSelectUsersController.java` - Admin user selection - -### Domain Objects -- `User.java` - User entity implementing `UserDetails` -- `UserRole.java` - Role entity (Admin, User, Associate Editor) -- `PasswordResetToken.java` - Secure password reset tokens -- `Person.java` - Personal information linked to User - -### Services -- `UserService.java` - User business logic -- `PasswordResetTokenHome.java` - Token persistence - -### Security -- `treebase-security.xml` - Spring Security configuration -- `DelegatingPasswordEncoder.java` - BCrypt + legacy support -- `PasswordUpgradeAuthenticationSuccessHandler.java` - Auto password upgrade - -### Views (JSP) -- `userForm.jsp` - Registration and profile editing -- `login.jsp` - Login page container -- `loginForm.jsp` - Login form widget -- `passwordForm.jsp` - Password reset request -- `resetPasswordForm.jsp` - New password entry -- `overrideUserProfile.jsp` - Admin profile editing -- `userList.jsp` - Admin user list - -### Common Components -- `nav.jsp` - Navigation tabs (Personal Info, Submissions, Search, Admin) -- `header.jsp` - Site header -- `defaultTemplate.jsp` - Main template with login status display From 1e769dd64117b519602619b109f2b7e085cfd325 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 17:24:12 +0100 Subject: [PATCH 07/52] Update navigation flow documentation to include mermaid plot --- doc/v3/user-story-06-technical.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index 8b1180b4c..fec84cbe4 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -29,7 +29,7 @@ architecture. ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages in the form of a mermaid plot* ``` [Documentation Home] --> [API Reference] From 5cb5158fe885623b28c29c6a087e5dbb2c1dd6d9 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 17:25:35 +0100 Subject: [PATCH 08/52] Update user stories to include checkboxes --- doc/v3/README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 8c3d717b7..209bbcffd 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -16,34 +16,34 @@ The v3 roadmap aims to: ### Public Users -- **[User Story 1: Search](user-story-01-search.md)** - A user or agent visits TreeBASE to search +- [ ] **[User Story 1: Search](user-story-01-search.md)** - A user or agent visits TreeBASE to search for study, tree, matrix, taxon or analysis data ### Authenticated Users -- **[User Story 2: Account Management](user-story-02-account.md)** - A user visits TreeBASE to +- [x] **[User Story 2: Account Management](user-story-02-account.md)** - A user visits TreeBASE to create or update their user account and log in -- **[User Story 3: Data Submission](user-story-03-submission.md)** - A user visits TreeBASE to +- [ ] **[User Story 3: Data Submission](user-story-03-submission.md)** - A user visits TreeBASE to submit study, tree, matrix, taxon or analysis data ### Reviewers -- **[User Story 4: Review](user-story-04-review.md)** - A reviewer visits TreeBASE to review a +- [ ] **[User Story 4: Review](user-story-04-review.md)** - A reviewer visits TreeBASE to review a study with tree, matrix, taxon and analysis data ### Administrators -- **[User Story 5: Administration](user-story-05-admin.md)** - An admin visits TreeBASE to +- [ ] **[User Story 5: Administration](user-story-05-admin.md)** - An admin visits TreeBASE to administer the submission and review queue ### Stakeholders -- **[User Story 6: Technical Documentation](user-story-06-technical.md)** - A technical +- [ ] **[User Story 6: Technical Documentation](user-story-06-technical.md)** - A technical stakeholder visits TreeBASE to learn about the technology of the web app, database, data standards, APIs, or other tools -- **[User Story 7: Governance](user-story-07-governance.md)** - A governance stakeholder visits +- [ ] **[User Story 7: Governance](user-story-07-governance.md)** - A governance stakeholder visits TreeBASE to learn about the project, its history, contributors, and role in phylogenetic data management From c7d8b01f41724bb12616833c1464e7bd611df8e5 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 17:28:44 +0100 Subject: [PATCH 09/52] Expand technical documentation structure Added sections for various technical documentation areas with placeholders for future content. --- doc/v3/user-story-06-technical.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index fec84cbe4..22a9f1413 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -50,6 +50,9 @@ architecture. ## Technical Documentation Areas ### API Documentation + +*TODO: Specify what pages are available in this category* + - PhyloWS API reference - OAI-PMH interface documentation - Authentication requirements @@ -57,6 +60,9 @@ architecture. - Example requests and responses ### Data Standards + +*TODO: Specify what pages are available in this category* + - NeXML format documentation - NEXUS format support - Newick tree format @@ -64,6 +70,9 @@ architecture. - Metadata schemas ### Developer Resources + +*TODO: Specify what pages are available in this category* + - Architecture overview - Source code repository - Build instructions @@ -71,18 +80,27 @@ architecture. - Contribution guidelines ### Integration Guides + +*TODO: Specify what pages are available in this category* + - How to query TreeBASE - How to harvest data - How to submit programmatically - Client libraries and SDKs ### Database Documentation + +*TODO: Specify what pages are available in this category* + - Data model overview - Entity relationships - Query patterns - Data dictionary ### Deployment + +*TODO: Specify what pages are available in this category* + - System requirements - Installation guide - Configuration options @@ -99,7 +117,7 @@ architecture. ## Pages to Account For -*TODO: Complete inventory of pages related to technical documentation* +*TODO: Complete inventory of JSP pages and controllers related to technical documentation* | Page | URL Pattern | Status | |------|-------------|--------| From f4bc80f5d4613ec06a0b9d94489b146a714ee965 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 16:29:42 +0000 Subject: [PATCH 10/52] Initial plan From d3d960123e4555682bb45656f97f25e1e447ca29 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 16:35:35 +0000 Subject: [PATCH 11/52] Address TODO items in user-story-06-technical.md with comprehensive documentation inventory Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-06-technical.md | 272 +++++++++++++++++++++++------- 1 file changed, 209 insertions(+), 63 deletions(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index 22a9f1413..205e8c128 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -18,93 +18,206 @@ architecture. ## Current Pages -*TODO: Identify and list all current pages that apply to this user story* - -- [ ] API documentation page -- [ ] Developer guide page -- [ ] Data standards documentation -- [ ] Download/export documentation -- [ ] Integration examples -- [ ] Technical FAQ +The following JSP pages and documentation files serve technical stakeholders: + +### Web Application Pages (JSP) +- [x] **urlAPI.jsp** (`/urlAPI.html`) - Data Access page with PhyloWS API overview, URI patterns, and RSS feeds +- [x] **technology.jsp** (`/technology.html`) - Implementation technologies and architecture overview +- [x] **about.jsp** (`/about.html`) - Background, history, funding, and related resources +- [x] **contact.jsp** (`/contact.html`) - Helpdesk contact and GitHub issue tracker links +- [x] **submitTutorial.jsp** (`/submitTutorial.html`) - NEXUS file preparation and submission process +- [x] **help.jsp** (`/help.html`) - Dynamic help text system + +### Documentation Files (Markdown) +- [x] **doc/API.md** - Comprehensive PhyloWS API documentation with CQL search syntax +- [x] **doc/OAI-PMH.md** - OAI-PMH metadata harvesting service documentation +- [x] **doc/development/BUILDING.md** - Build prerequisites and instructions (Java 17, Maven) +- [x] **doc/development/DEPLOYING.md** - Tomcat deployment guide with troubleshooting +- [x] **doc/technical-notes/DWR.md** - DWR (Direct Web Remoting) integration for AJAX +- [x] **doc/technical-notes/UPGRADES.md** - Dependency upgrade history (Spring, Jersey, etc.) + +### External Resources +- [ ] GitHub Wiki - API documentation (linked from urlAPI.jsp) +- [ ] GitHub Wiki - OAI-PMH documentation (linked from urlAPI.jsp) ## Navigation Flow -*TODO: Document how users navigate through these pages in the form of a mermaid plot* - -``` -[Documentation Home] --> [API Reference] - | | - | v - | [API Examples] - | | - v v -[Data Standards] --> [Data Formats] - | | - v v -[Developer Guide] --> [Contributing] - | | - v v -[Architecture] --> [Deployment Guide] +Technical users navigate through documentation pages based on their integration needs: + +```mermaid +flowchart TD + A[Entry Points] --> B{User Goal} + + B -->|API Integration| C[urlAPI.html
Data Access] + B -->|Understand System| D[technology.html
Implementation] + B -->|Contribute Code| E[GitHub Repository] + B -->|Deploy Instance| F[doc/development/] + + C --> G[doc/API.md
PhyloWS Reference] + C --> H[doc/OAI-PMH.md
Harvesting Interface] + + G --> I[/phylows/study/**] + G --> J[/phylows/tree/**] + G --> K[/phylows/matrix/**] + G --> L[/phylows/taxon/**] + + D --> M[about.html
Background] + D --> N[Architecture Diagram] + + E --> O[doc/development/BUILDING.md] + E --> P[doc/technical-notes/] + + F --> O + F --> Q[doc/development/DEPLOYING.md] + + H --> R[OAI-PMH Verbs] + + style C fill:#e1f5fe + style D fill:#e8f5e9 + style E fill:#fff3e0 + style F fill:#fce4ec ``` ## Technical Documentation Areas ### API Documentation -*TODO: Specify what pages are available in this category* +Available documentation for API consumers: + +| Resource | Location | Description | +|----------|----------|-------------| +| **urlAPI.jsp** | `/urlAPI.html` | Overview of programmatic access, URI patterns, RSS feeds | +| **doc/API.md** | Repository | PhyloWS API reference with CQL search syntax and examples | +| **doc/OAI-PMH.md** | Repository | OAI-PMH harvesting verbs and usage | +| **GitHub Wiki** | External | Extended API documentation (linked from urlAPI.jsp) | + +**PhyloWS Endpoints:** +- `/phylows/study/**` - Study records (PhyloWSStudyController) +- `/phylows/tree/**` - Phylogenetic trees (PhyloWSTreeController) +- `/phylows/matrix/**` - Character matrices (PhyloWSMatrixController) +- `/phylows/taxon/**` - Taxonomic data (PhyloWSTaxonController) +- `/phylows/classification/**` - Classification data (PhyloWSClassificationController) -- PhyloWS API reference -- OAI-PMH interface documentation -- Authentication requirements -- Rate limiting and usage policies -- Example requests and responses +**Output Formats:** HTML, NeXML, NEXUS, RDF, RSS 1.0, JSON ### Data Standards -*TODO: Specify what pages are available in this category* +Documentation for phylogenetic data formats is distributed across in-app pages and external links: -- NeXML format documentation -- NEXUS format support -- Newick tree format -- Phylogenetic data interchange standards -- Metadata schemas +| Format | Coverage | Location | +|--------|----------|----------| +| **NEXUS** | Submission format | submitTutorial.jsp, Mesquite documentation | +| **NeXML** | Primary output format | doc/API.md, nexml.org (external) | +| **Newick** | Tree representation | Implicit in tree endpoints | +| **RDF/CDAO** | Semantic web output | doc/API.md, evolutionaryontology.org | +| **RSS 1.0** | Search result feeds | doc/API.md | -### Developer Resources +**Data Preparation:** +- **submitTutorial.jsp** (`/submitTutorial.html`) - NEXUS file preparation using Mesquite +- YouTube instructional videos embedded in submitTutorial.jsp + +**Metadata Standards:** +- Dublin Core (dcterms) predicates for bibliographic metadata +- PRISM for publication dates +- TreeBASE-specific predicates (tb:) documented in API.md -*TODO: Specify what pages are available in this category* +### Developer Resources -- Architecture overview -- Source code repository -- Build instructions -- Development environment setup -- Contribution guidelines +Resources for contributors and developers: + +| Resource | Location | Description | +|----------|----------|-------------| +| **technology.jsp** | `/technology.html` | Architecture overview, technology stack diagram | +| **BUILDING.md** | `doc/development/` | Build prerequisites (Java 17, Maven), compilation steps | +| **DWR.md** | `doc/technical-notes/` | AJAX integration with Spring 5 compatibility | +| **UPGRADES.md** | `doc/technical-notes/` | Dependency upgrade history and migration notes | +| **GitHub Repository** | github.com/TreeBASE/treebase | Source code under BSD license | +| **GitHub Issues** | Linked from contact.jsp | Bug reports and feature requests | + +**Technology Stack (from technology.jsp):** +- Database: PostgreSQL +- ORM: Hibernate +- Framework: Spring +- File Parsing: Mesquite +- Tree Visualization: phylotree.js + +**Project Structure:** +- `treebase-core` - ORM API for PostgreSQL database access +- `treebase-web` - MVC web application with JSP/HTML GUI +- `oai-pmh_data_provider` - OAI-PMH interface functionality ### Integration Guides -*TODO: Specify what pages are available in this category* +Programmatic integration documentation: + +| Integration Type | Documentation | Key Information | +|-----------------|---------------|-----------------| +| **REST API Queries** | doc/API.md | PhyloWS URL patterns, CQL search syntax | +| **Metadata Harvesting** | doc/OAI-PMH.md | OAI-PMH verbs: Identify, ListRecords, GetRecord, etc. | +| **Data Download** | urlAPI.jsp | NEXUS, NeXML, JSON format parameters | +| **RSS Feeds** | doc/API.md, urlAPI.jsp | RSS 1.0 for search results and alerts | -- How to query TreeBASE -- How to harvest data -- How to submit programmatically -- Client libraries and SDKs +**URI Patterns (from urlAPI.jsp):** +- Study: `{purlBase}study/TB2:S{id}` +- Matrix: `{purlBase}matrix/TB2:M{id}` +- Tree: `{purlBase}tree/TB2:Tr{id}` + +**Search Examples (from API.md):** +- `/study/find?query=dcterms.contributor=Huelsenbeck` +- `/taxon/find?query=dcterms.title=="Homo sapiens"&format=rss1&recordSchema=tree` + +**Note:** No official client libraries or SDKs are currently provided. ### Database Documentation -*TODO: Specify what pages are available in this category* +Database and data model documentation: + +| Resource | Location | Description | +|----------|----------|-------------| +| **technology.jsp** | `/technology.html` | Data content overview, architecture diagram | +| **BUILDING.md** | `doc/development/` | Database connection configuration | +| **Hibernate Mappings** | `treebase-core` source | Entity relationships via ORM | + +**Data Model Overview (from technology.jsp):** +1. Studies contain bibliographic references to published phylogenetic research +2. Each study has one or more analyses; each analysis has steps associating matrices, trees, algorithms, and software +3. Character matrices contain taxon labels mapped to tree leaf nodes +4. Row segments link to specimen, tissue, or gene sequence metadata +5. Taxon names map to uBio NameBank and NCBI taxonomy + +**Key Entity Types:** +- Study, Citation, Analysis, AnalysisStep +- Matrix, Tree, TreeBlock +- Taxon, TaxonLabel, TaxonVariant -- Data model overview -- Entity relationships -- Query patterns -- Data dictionary +**Note:** Formal data dictionary and entity-relationship diagrams are not currently published. ### Deployment -*TODO: Specify what pages are available in this category* +Deployment and operations documentation: -- System requirements -- Installation guide -- Configuration options -- Monitoring and maintenance +| Resource | Location | Description | +|----------|----------|-------------| +| **BUILDING.md** | `doc/development/` | Prerequisites, compilation, WAR packaging | +| **DEPLOYING.md** | `doc/development/` | Tomcat configuration, JVM arguments, verification | +| **UPGRADES.md** | `doc/technical-notes/` | Dependency versions, security fixes, migration paths | +| **DOCKER.md** | Repository root | Docker deployment option | +| **docker-compose.yml** | Repository root | Container orchestration | + +**System Requirements (from BUILDING.md/DEPLOYING.md):** +- Java 17 LTS +- Maven 3.8+ +- PostgreSQL database +- Tomcat 7+ servlet container +- Headless Mesquite for file parsing + +**Build Artifacts:** +- `treebase-web.war` (~59MB) - Main web application +- `data_provider_web.war` - OAI-PMH interface + +**Configuration Files:** +- `jdbc.properties` - Database connection +- `context.xml` - Tomcat context with database and Mesquite paths ## External Documentation @@ -117,11 +230,44 @@ architecture. ## Pages to Account For -*TODO: Complete inventory of JSP pages and controllers related to technical documentation* - -| Page | URL Pattern | Status | -|------|-------------|--------| -| | | | +Complete inventory of JSP pages and controllers related to technical documentation: + +| Page | URL Pattern | Controller | Status | +|------|-------------|------------|--------| +| urlAPI.jsp | /urlAPI.html | filenameController | ✅ Active | +| technology.jsp | /technology.html | filenameController | ✅ Active | +| about.jsp | /about.html | filenameController | ✅ Active | +| contact.jsp | /contact.html | filenameController | ✅ Active | +| submitTutorial.jsp | /submitTutorial.html | filenameController | ✅ Active | +| help.jsp | /help.html | helpController | ✅ Active | +| sitemap.jsp | /sitemap.xml | sitemapController | ✅ Active | + +**PhyloWS Controllers:** + +| Controller | URL Pattern | Description | +|------------|-------------|-------------| +| PhyloWSStudyController | /phylows/study/** | Study data API | +| PhyloWSTreeController | /phylows/tree/** | Tree data API | +| PhyloWSMatrixController | /phylows/matrix/** | Matrix data API | +| PhyloWSTaxonController | /phylows/taxon/** | Taxon data API | +| PhyloWSClassificationController | /phylows/classification/** | Classification API | + +**OAI-PMH Interface:** + +| Endpoint | URL Pattern | Description | +|----------|-------------|-------------| +| OAI-PMH Provider | /top/oai | Metadata harvesting service | + +**JSON Output Pages (for API responses):** + +| Page | Purpose | +|------|---------| +| json/studyToJSON.jsp | Study JSON serialization | +| json/treeToJSON.jsp | Tree JSON serialization | +| json/matrixToJSON.jsp | Matrix JSON serialization | +| json/taxonToJSON.jsp | Taxon JSON serialization | +| anyObjectAsRDF.jsp | RDF output generation | +| searchResultsAsRDF.jsp | Search results as RDF | ## Wireframe Notes From 1359522666b2fdbee6d4e529c6a7cb0a48a682e3 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 16:37:11 +0000 Subject: [PATCH 12/52] Add clarifying note for external GitHub Wiki resources Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-06-technical.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index 205e8c128..ca4ab7be8 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -37,8 +37,8 @@ The following JSP pages and documentation files serve technical stakeholders: - [x] **doc/technical-notes/UPGRADES.md** - Dependency upgrade history (Spring, Jersey, etc.) ### External Resources -- [ ] GitHub Wiki - API documentation (linked from urlAPI.jsp) -- [ ] GitHub Wiki - OAI-PMH documentation (linked from urlAPI.jsp) +- [ ] GitHub Wiki - API documentation (linked from urlAPI.jsp) - *External, not maintained in repository* +- [ ] GitHub Wiki - OAI-PMH documentation (linked from urlAPI.jsp) - *External, not maintained in repository* ## Navigation Flow From 5be0e6a71210055581ca85640e982fd679685184 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 16:54:21 +0000 Subject: [PATCH 13/52] Fix mermaid diagram syntax by quoting labels with special characters Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-06-technical.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index ca4ab7be8..a650ad1e5 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -48,27 +48,27 @@ Technical users navigate through documentation pages based on their integration flowchart TD A[Entry Points] --> B{User Goal} - B -->|API Integration| C[urlAPI.html
Data Access] - B -->|Understand System| D[technology.html
Implementation] + B -->|API Integration| C["urlAPI.html (Data Access)"] + B -->|Understand System| D["technology.html (Implementation)"] B -->|Contribute Code| E[GitHub Repository] - B -->|Deploy Instance| F[doc/development/] + B -->|Deploy Instance| F["doc/development/"] - C --> G[doc/API.md
PhyloWS Reference] - C --> H[doc/OAI-PMH.md
Harvesting Interface] + C --> G["doc/API.md (PhyloWS Reference)"] + C --> H["doc/OAI-PMH.md (Harvesting Interface)"] - G --> I[/phylows/study/**] - G --> J[/phylows/tree/**] - G --> K[/phylows/matrix/**] - G --> L[/phylows/taxon/**] + G --> I["phylows/study endpoint"] + G --> J["phylows/tree endpoint"] + G --> K["phylows/matrix endpoint"] + G --> L["phylows/taxon endpoint"] - D --> M[about.html
Background] + D --> M["about.html (Background)"] D --> N[Architecture Diagram] - E --> O[doc/development/BUILDING.md] - E --> P[doc/technical-notes/] + E --> O["doc/development/BUILDING.md"] + E --> P["doc/technical-notes/"] F --> O - F --> Q[doc/development/DEPLOYING.md] + F --> Q["doc/development/DEPLOYING.md"] H --> R[OAI-PMH Verbs] From 5059bb791a11ef763e28095ca680e5605d6ab112 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 17:57:24 +0100 Subject: [PATCH 14/52] Update User Story 6 status to completed --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 209bbcffd..7956270d6 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -39,7 +39,7 @@ The v3 roadmap aims to: ### Stakeholders -- [ ] **[User Story 6: Technical Documentation](user-story-06-technical.md)** - A technical +- [x] **[User Story 6: Technical Documentation](user-story-06-technical.md)** - A technical stakeholder visits TreeBASE to learn about the technology of the web app, database, data standards, APIs, or other tools From 5094d48e0ac15a928501f93fb7dd86baca6f2ca7 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 18:10:46 +0000 Subject: [PATCH 15/52] Initial plan From ac68441ff8981726aa62c220eac33c62808ea067 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 18:20:42 +0000 Subject: [PATCH 16/52] Initial plan for addressing TODO items in governance user story Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-07-governance.md | 131 +++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 doc/v3/user-story-07-governance.md diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md new file mode 100644 index 000000000..d5b75e6f4 --- /dev/null +++ b/doc/v3/user-story-07-governance.md @@ -0,0 +1,131 @@ +# User Story 7: Governance + +## Overview + +**As a** governance stakeholder, +**I want to** learn about the project, its history, contributors, and role in phylogenetic data +management, +**So that** I can understand TreeBASE's position in the scientific data ecosystem and make +informed decisions about its future. + +## User Types + +- Funding agency representatives +- Institutional administrators +- Scientific advisory board members +- Policy makers +- Potential partners and collaborators +- Science journalists and communicators + +## Current Pages + +*TODO: Identify and list all current pages that apply to this user story* + +- [ ] About page +- [ ] History/timeline page +- [ ] Contributors/team page +- [ ] Mission statement +- [ ] Data usage policies +- [ ] Terms of service +- [ ] Privacy policy +- [ ] Contact information +- [ ] Governance structure +- [ ] Funding acknowledgments + +## Navigation Flow + +*TODO: Document how users navigate through these pages* + +``` +[Home] --> [About TreeBASE] + | + +---> [History] + | + +---> [Team/Contributors] + | + +---> [Mission & Vision] + | + +---> [Governance Structure] + | + +---> [Policies] + | | + | +---> [Data Usage] + | | + | +---> [Terms of Service] + | | + | +---> [Privacy Policy] + | + +---> [Funding & Support] + | + +---> [Contact] +``` + +## Governance Information + +### About TreeBASE +- What is TreeBASE +- Mission statement +- Vision for the future +- Value proposition + +### History +- Timeline of development +- Major milestones +- Evolution of the platform +- Historical context in phylogenetics + +### Team and Contributors +- Current team members +- Advisory board +- Past contributors +- Institutional affiliations + +### Governance Structure +- Decision-making processes +- Organizational structure +- Partner institutions +- Community involvement + +### Policies +- Data submission policies +- Data usage and licensing +- Terms of service +- Privacy policy +- Code of conduct + +### Impact and Metrics +- Usage statistics +- Citation information +- Data holdings summary +- Community reach + +### Funding and Support +- Current funders +- Historical funding +- How to support TreeBASE +- Institutional partnerships + +### Contact and Engagement +- Contact information +- Social media presence +- Mailing lists +- Community forums + +## Pages to Account For + +*TODO: Complete inventory of pages related to governance* + +| Page | URL Pattern | Status | +|------|-------------|--------| +| | | | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What governance information should be prominently featured? +- How should historical information be presented? +- What metrics should be publicly displayed? +- How should contact/support requests be handled? From e3955298c98ef15ec01044961f20991160bda026 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 18:24:17 +0000 Subject: [PATCH 17/52] Address TODO items in user-story-07-governance.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-07-governance.md | 126 ++++++++++++++++++++--------- 1 file changed, 87 insertions(+), 39 deletions(-) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index d5b75e6f4..13b940a55 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -19,47 +19,80 @@ informed decisions about its future. ## Current Pages -*TODO: Identify and list all current pages that apply to this user story* - -- [ ] About page -- [ ] History/timeline page -- [ ] Contributors/team page -- [ ] Mission statement -- [ ] Data usage policies -- [ ] Terms of service -- [ ] Privacy policy -- [ ] Contact information -- [ ] Governance structure -- [ ] Funding acknowledgments +The following pages in the current TreeBASE web application relate to this user story: + +- [x] About page (`/about.html`) - Background information, history, funding, governance, and related resources +- [ ] History/timeline page - *Not a separate page; history content is included in the About page* +- [x] Contributors/team page (`/people.html`) - Current and past contributors with photos and roles +- [x] Mission statement - *Included in About page and Home page* +- [x] Data usage policies (`/dataMan.html`) - NSF Data Management Plan covering submission, integrity, standards, and persistence +- [ ] Terms of service - *Not currently available as a dedicated page* +- [ ] Privacy policy - *Not currently available as a dedicated page* +- [x] Contact information (`/contact.html`) - Help desk email and bug reporting via GitHub issues +- [ ] Governance structure - *Basic information included in About page; no dedicated governance page* +- [x] Funding acknowledgments - *Included in About page with NSF grant numbers* +- [x] Partnerships page (`/partnership.html`) - Current and historical institutional partners +- [x] References/Citations page (`/reference.html`) - How to cite TreeBASE publications +- [x] Technology page (`/technology.html`) - Implementation details and source code information +- [x] Journals page (`/journal.html`) - Partner journals and publication policies ## Navigation Flow -*TODO: Document how users navigate through these pages* +Users navigate to governance-related pages primarily through the left sidebar menu available on the home page and all main pages. The current navigation structure is: ``` -[Home] --> [About TreeBASE] - | - +---> [History] - | - +---> [Team/Contributors] - | - +---> [Mission & Vision] - | - +---> [Governance Structure] - | - +---> [Policies] - | | - | +---> [Data Usage] - | | - | +---> [Terms of Service] - | | - | +---> [Privacy Policy] - | - +---> [Funding & Support] - | - +---> [Contact] +[Home] (/home.html) + | + +---> Left Sidebar Navigation + | + +---> [Search TreeBASE] (/search/studySearch.html) + | + +---> [Submit Data] (/user/processUser.html) + | + +---> Documentation + | | + | +---> [Technology] (/technology.html) + | | + | +---> [Submit Tutorial] (/submitTutorial.html) + | | + | +---> [Data Access/API] (/urlAPI.html) + | + +---> About + | | + | +---> [Overview] (/about.html) + | | - Background + | | - History, Funding, and Governance + | | - Related resources + | | - Logo + | | + | +---> [People] (/people.html) + | | - Current contributors + | | - Past developers + | | + | +---> [Partnerships] (/partnership.html) + | | - Naturalis Biodiversity Center + | | - NESCent (historical) + | | - CIPRES + | | - Dryad + | | + | +---> [References] (/reference.html) + | - How to cite TreeBASE + | - Further reading + | + +---> [NSF Data Management] (/dataMan.html) + | + +---> [Journals] (/journal.html) + | + +---> [Contact] (/contact.html) + - Help desk email + - GitHub issue tracker ``` +**External Links in Footer:** +- Mendeley group for TreeBASE publications +- Twitter @TreeBASE +- Data Citation Index badge + ## Governance Information ### About TreeBASE @@ -113,11 +146,26 @@ informed decisions about its future. ## Pages to Account For -*TODO: Complete inventory of pages related to governance* - -| Page | URL Pattern | Status | -|------|-------------|--------| -| | | | +The following is a complete inventory of pages related to governance: + +| Page | URL Pattern | Status | Notes | +|------|-------------|--------|-------| +| Home | `/home.html` | ✅ Exists | Welcome message, mission overview, recent studies | +| About/Overview | `/about.html` | ✅ Exists | Background, history, funding, governance, related resources | +| People/Team | `/people.html` | ✅ Exists | Current and past contributors with photos | +| Partnerships | `/partnership.html` | ✅ Exists | Institutional partners (Naturalis, NESCent, CIPRES, Dryad) | +| References | `/reference.html` | ✅ Exists | How to cite TreeBASE | +| Technology | `/technology.html` | ✅ Exists | Implementation stack, source code links | +| Submit Tutorial | `/submitTutorial.html` | ✅ Exists | Submission process documentation | +| Data Access/API | `/urlAPI.html` | ✅ Exists | Web service and programmatic access | +| NSF Data Management | `/dataMan.html` | ✅ Exists | Data management plan details | +| Journals | `/journal.html` | ✅ Exists | Partner journals requiring TreeBASE submissions | +| Contact | `/contact.html` | ✅ Exists | Help desk email, GitHub issue tracker | +| Terms of Service | N/A | ❌ Missing | No dedicated page | +| Privacy Policy | N/A | ❌ Missing | No dedicated page | +| Governance Structure | N/A | ❌ Missing | Basic info in About page only | +| Usage Statistics | N/A | ❌ Missing | Some stats on home page, no dedicated page | +| Impact Metrics | N/A | ❌ Missing | No dedicated page | ## Wireframe Notes From e9041ba7e67df85a71c73bd9a5309ec2885cf55f Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 19:30:54 +0100 Subject: [PATCH 18/52] Add TODOs for governance information sections Added placeholders for governance information sections. --- doc/v3/user-story-07-governance.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index d5b75e6f4..f68aea778 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -63,30 +63,46 @@ informed decisions about its future. ## Governance Information ### About TreeBASE + +*TODO: find out and specify whether and where this information is covered* + - What is TreeBASE - Mission statement - Vision for the future - Value proposition ### History + +*TODO: find out and specify whether and where this information is covered* + - Timeline of development - Major milestones - Evolution of the platform - Historical context in phylogenetics ### Team and Contributors + +*TODO: find out and specify whether and where this information is covered* + - Current team members - Advisory board - Past contributors - Institutional affiliations ### Governance Structure + +*TODO: find out and specify whether and where this information is covered* + - Decision-making processes - Organizational structure - Partner institutions - Community involvement +- Relationship with journals ### Policies + +*TODO: find out and specify whether and where this information is covered* + - Data submission policies - Data usage and licensing - Terms of service @@ -94,18 +110,27 @@ informed decisions about its future. - Code of conduct ### Impact and Metrics + +*TODO: find out and specify whether and where this information is covered* + - Usage statistics - Citation information - Data holdings summary - Community reach ### Funding and Support + +*TODO: find out and specify whether and where this information is covered* + - Current funders - Historical funding - How to support TreeBASE - Institutional partnerships ### Contact and Engagement + +*TODO: find out and specify whether and where this information is covered* + - Contact information - Social media presence - Mailing lists From 2b576431abac60f9a977df31a65e277e71e8c323 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 19:31:20 +0100 Subject: [PATCH 19/52] Update navigation flow documentation to include mermaid plot --- doc/v3/user-story-07-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index f68aea778..fe2e0eb10 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -34,7 +34,7 @@ informed decisions about its future. ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages as a mermaid plot* ``` [Home] --> [About TreeBASE] From 73075cda93b534160517400fb85ef4689db53c73 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 18:44:44 +0000 Subject: [PATCH 20/52] Address all TODOs including mermaid diagram and coverage analysis for governance sections Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-07-governance.md | 163 +++++++++++++++-------------- 1 file changed, 84 insertions(+), 79 deletions(-) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index 13b940a55..9d20f2cbe 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -40,52 +40,32 @@ The following pages in the current TreeBASE web application relate to this user Users navigate to governance-related pages primarily through the left sidebar menu available on the home page and all main pages. The current navigation structure is: -``` -[Home] (/home.html) - | - +---> Left Sidebar Navigation - | - +---> [Search TreeBASE] (/search/studySearch.html) - | - +---> [Submit Data] (/user/processUser.html) - | - +---> Documentation - | | - | +---> [Technology] (/technology.html) - | | - | +---> [Submit Tutorial] (/submitTutorial.html) - | | - | +---> [Data Access/API] (/urlAPI.html) - | - +---> About - | | - | +---> [Overview] (/about.html) - | | - Background - | | - History, Funding, and Governance - | | - Related resources - | | - Logo - | | - | +---> [People] (/people.html) - | | - Current contributors - | | - Past developers - | | - | +---> [Partnerships] (/partnership.html) - | | - Naturalis Biodiversity Center - | | - NESCent (historical) - | | - CIPRES - | | - Dryad - | | - | +---> [References] (/reference.html) - | - How to cite TreeBASE - | - Further reading - | - +---> [NSF Data Management] (/dataMan.html) - | - +---> [Journals] (/journal.html) - | - +---> [Contact] (/contact.html) - - Help desk email - - GitHub issue tracker +```mermaid +flowchart TD + Home[Home /home.html] --> Sidebar[Left Sidebar Navigation] + + Sidebar --> Search[Search TreeBASE /search/studySearch.html] + Sidebar --> Submit[Submit Data /user/processUser.html] + + Sidebar --> Documentation + Documentation --> Technology[Technology /technology.html] + Documentation --> Tutorial[Submit Tutorial /submitTutorial.html] + Documentation --> API[Data Access/API /urlAPI.html] + + Sidebar --> About + About --> Overview[Overview /about.html] + About --> People[People /people.html] + About --> Partnerships[Partnerships /partnership.html] + About --> References[References /reference.html] + + Sidebar --> DataMan[NSF Data Management /dataMan.html] + Sidebar --> Journals[Journals /journal.html] + Sidebar --> Contact[Contact /contact.html] + + Sidebar --> Footer[Footer Links] + Footer --> Mendeley[Mendeley Group] + Footer --> Twitter[Twitter @TreeBASE] + Footer --> DCI[Data Citation Index] ``` **External Links in Footer:** @@ -96,53 +76,78 @@ Users navigate to governance-related pages primarily through the left sidebar me ## Governance Information ### About TreeBASE -- What is TreeBASE -- Mission statement -- Vision for the future -- Value proposition + +**Coverage:** `/about.html` and `/home.html` + +- What is TreeBASE - ✅ Covered in About page ("Background" section) and Home page welcome text +- Mission statement - ✅ Covered in About page and Home page (TreeBASE is a repository of phylogenetic information) +- Vision for the future - ⚠️ Partially covered; About page mentions current version but no explicit future vision +- Value proposition - ✅ Covered in About page (applications list) and Home page (new features list) ### History -- Timeline of development -- Major milestones -- Evolution of the platform -- Historical context in phylogenetics + +**Coverage:** `/about.html` ("History, Funding, and Governance" section) + +- Timeline of development - ✅ Covered (prototype 1994, redevelopment with CIPRES, v2.0 March 2010) +- Major milestones - ✅ Covered (NSF grants, institutional hosts over time) +- Evolution of the platform - ✅ Covered (mentions previous hosts: NESCent, Yale Peabody, SDSC, UB, Harvard, Leiden, UC Davis) +- Historical context in phylogenetics - ✅ Covered (references to Sanderson et al., Piel et al. publications) ### Team and Contributors -- Current team members -- Advisory board -- Past contributors -- Institutional affiliations + +**Coverage:** `/people.html` + +- Current team members - ✅ Covered with photos, names, and roles +- Advisory board - ❌ Not explicitly listed +- Past contributors - ✅ Covered ("Past developers" section) +- Institutional affiliations - ✅ Covered (institutions shown with contributor entries) ### Governance Structure -- Decision-making processes -- Organizational structure -- Partner institutions -- Community involvement + +**Coverage:** `/about.html`, `/partnership.html`, `/journal.html` + +- Decision-making processes - ⚠️ Partially covered; mentions Phyloinformatics Research Foundation governance +- Organizational structure - ✅ Covered in About page (Phyloinformatics Research Foundation, Inc.) +- Partner institutions - ✅ Covered in Partnerships page (Naturalis, NESCent, CIPRES, Dryad) +- Community involvement - ⚠️ Limited coverage; GitHub issues for bug reports mentioned in Contact +- Relationship with journals - ✅ Covered in Journals page (`/journal.html`) - lists partner journals that recommend/require TreeBASE submission, provides PhyloWS URLs for each journal's studies ### Policies -- Data submission policies -- Data usage and licensing -- Terms of service -- Privacy policy -- Code of conduct + +**Coverage:** `/dataMan.html`, `/submitTutorial.html` + +- Data submission policies - ✅ Covered in NSF Data Management page and Submit Tutorial +- Data usage and licensing - ✅ Covered in NSF Data Management page (public domain, no restrictions on reuse) +- Terms of service - ❌ No dedicated page +- Privacy policy - ❌ No dedicated page +- Code of conduct - ❌ No dedicated page ### Impact and Metrics -- Usage statistics -- Citation information -- Data holdings summary -- Community reach + +**Coverage:** `/home.html`, `/reference.html` + +- Usage statistics - ⚠️ Partially covered; Home page shows counts (4,076 publications, 8,777 authors, etc. as of April 2014 - needs update) +- Citation information - ✅ Covered in References page (how to cite TreeBASE) +- Data holdings summary - ✅ Covered in Home page (matrices, trees, taxon labels counts) +- Community reach - ⚠️ Limited; Twitter and Mendeley links in sidebar, Data Citation Index badge ### Funding and Support -- Current funders -- Historical funding -- How to support TreeBASE -- Institutional partnerships + +**Coverage:** `/about.html`, `/partnership.html` + +- Current funders - ⚠️ Limited; current host (Naturalis) mentioned but no current funding details +- Historical funding - ✅ Covered in About page (NSF grants DEB 9318325, EF 0331654) +- How to support TreeBASE - ❌ Not covered +- Institutional partnerships - ✅ Covered in Partnerships page ### Contact and Engagement -- Contact information -- Social media presence -- Mailing lists -- Community forums + +**Coverage:** `/contact.html`, sidebar links + +- Contact information - ✅ Covered in Contact page (help@treebase.org) +- Social media presence - ✅ Twitter link in sidebar +- Mailing lists - ❌ Not covered +- Community forums - ❌ Not covered (GitHub issues for bugs only) ## Pages to Account For From 824042747a45bf5e89b8fdc63db2c438dc4a563d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:07:45 +0000 Subject: [PATCH 21/52] Add GitHub issues and discussions links to community engagement section Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-07-governance.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index 9d20f2cbe..66a5444ea 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -142,12 +142,14 @@ flowchart TD ### Contact and Engagement -**Coverage:** `/contact.html`, sidebar links +**Coverage:** `/contact.html`, sidebar links, GitHub - Contact information - ✅ Covered in Contact page (help@treebase.org) - Social media presence - ✅ Twitter link in sidebar - Mailing lists - ❌ Not covered -- Community forums - ❌ Not covered (GitHub issues for bugs only) +- Community forums - ✅ Available via GitHub: + - Issues/bugs: https://github.com/TreeBASE/treebase/issues + - Discussions: https://github.com/TreeBASE/treebase/discussions ## Pages to Account For From d74b4a8f9a7fe3cfc490f64943b9f0f8c60d4641 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:15:20 +0000 Subject: [PATCH 22/52] Remove mailing lists item since community forum exists via GitHub Discussions Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-07-governance.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/v3/user-story-07-governance.md b/doc/v3/user-story-07-governance.md index 66a5444ea..70eb8e3f7 100644 --- a/doc/v3/user-story-07-governance.md +++ b/doc/v3/user-story-07-governance.md @@ -146,7 +146,6 @@ flowchart TD - Contact information - ✅ Covered in Contact page (help@treebase.org) - Social media presence - ✅ Twitter link in sidebar -- Mailing lists - ❌ Not covered - Community forums - ✅ Available via GitHub: - Issues/bugs: https://github.com/TreeBASE/treebase/issues - Discussions: https://github.com/TreeBASE/treebase/discussions From 75fd7113f40c12dd3b895d96dda2000a3a71d457 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 20:20:25 +0100 Subject: [PATCH 23/52] Mark User Story 7 as completed in README --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 7956270d6..e40ebdeba 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -43,7 +43,7 @@ The v3 roadmap aims to: stakeholder visits TreeBASE to learn about the technology of the web app, database, data standards, APIs, or other tools -- [ ] **[User Story 7: Governance](user-story-07-governance.md)** - A governance stakeholder visits +- [x] **[User Story 7: Governance](user-story-07-governance.md)** - A governance stakeholder visits TreeBASE to learn about the project, its history, contributors, and role in phylogenetic data management From 3554008308328606707f4228e9e8545f7bfcbfda Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 20:24:31 +0100 Subject: [PATCH 24/52] Enhance user story documentation and verification tasks Updated navigation flow documentation to specify use of a mermaid plot and added verification tasks for required fields in the submission workflow. --- doc/v3/user-story-03-submission.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/doc/v3/user-story-03-submission.md b/doc/v3/user-story-03-submission.md index a82c79de1..84ae76a80 100644 --- a/doc/v3/user-story-03-submission.md +++ b/doc/v3/user-story-03-submission.md @@ -33,7 +33,7 @@ ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages as a mermaid plot* ``` [User Dashboard] --> [New Submission] --> [Study Metadata] @@ -57,37 +57,58 @@ ## Submission Workflow ### Step 1: Study Information + +*TODO: verify the exact fields that are currently required and the pages that record this information* + - Publication details (journal, DOI, authors) - Study title and abstract - Keywords and categories ### Step 2: Data Upload + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - NEXUS file upload - Newick file upload - Other supported formats - File validation ### Step 3: Tree Data + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - Tree visualization - Tree metadata - Multiple trees per study ### Step 4: Matrix Data + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - Character matrix display - Matrix metadata - Character definitions ### Step 5: Taxonomic Data + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - Taxon name reconciliation - Taxonomic hierarchy - OTU mapping ### Step 6: Analysis Information + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - Analysis type/method - Software used - Parameters and settings ### Step 7: Review and Submit + +*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* + - Validation summary - Preview all data - Submit for review From de067d461652a2dc2dff887bef01be9901f339a1 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:27:50 +0000 Subject: [PATCH 25/52] Initial plan From c060505350b6cc41d8095f2d2aa53ee0027a25bb Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:35:27 +0000 Subject: [PATCH 26/52] Initial plan From 341e8cf13aed530248bb191e8a2edfe7173f7a7b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:39:06 +0000 Subject: [PATCH 27/52] Address all TODO items in user-story-03-submission.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-03-submission.md | 481 ++++++++++++++++++++++++----- 1 file changed, 405 insertions(+), 76 deletions(-) diff --git a/doc/v3/user-story-03-submission.md b/doc/v3/user-story-03-submission.md index 84ae76a80..f9287fa4d 100644 --- a/doc/v3/user-story-03-submission.md +++ b/doc/v3/user-story-03-submission.md @@ -18,117 +18,422 @@ ## Current Pages -*TODO: Identify and list all current pages that apply to this user story* - -- [ ] Submission dashboard -- [ ] New submission form -- [ ] Study metadata entry -- [ ] File upload interface -- [ ] Tree upload/editor -- [ ] Matrix upload/editor -- [ ] Taxa management -- [ ] Analysis metadata entry -- [ ] Submission preview -- [ ] Submission confirmation +The following pages apply to this user story: + +- [x] Submission dashboard (`/user/submissionList.html` → `submissionList.jsp`) +- [x] New submission form (`/user/studyForm.html?form=new` → `studyForm.jsp`) +- [x] Study summary/metadata (`/user/summary.html` → `submissionSummaryView.jsp`) +- [x] Citation form (`/user/citationForm.html` → `citationForm.jsp`) +- [x] Author search/add (`/user/authorSearchForm.html`, `/user/addAuthor.html` → `peopleSearchForm.jsp`, `addPersonForm.jsp`) +- [x] File upload interface (`/user/uploadFile.html` → `uploadFile.jsp`) +- [x] Upload summary (`/user/uploadFileSummary.html` → `uploadFileSummary.jsp`) +- [x] Tree blocks list (`/user/treeBlockList.html` → `treeBlockList.jsp`) +- [x] Tree list/editor (`/user/treeList.html` → `treeList.jsp`) +- [x] Tree viewer/editor (`/user/directMapToPhyloWidget.html` → `treeViewer.jsp`) +- [x] Matrix list/editor (`/user/matrixList.html` → `matrixList.jsp`) +- [x] Matrix row list (`/user/matrixRowList.html` → `matrixRowList.jsp`) +- [x] Row segment data (`/user/viewAllRowSegmentData.html` → `viewAllRowSegmentData.jsp`) +- [x] Taxa management (`/user/taxaList.html` → `taxonLabels.jsp`) +- [x] Edit taxon label (`/user/editTaxonLabel.html` → `editTaxonLabel.jsp`) +- [x] Analysis display (`/user/analysisDisplay.html` → `analysisList.jsp`) +- [x] Analysis form (`/user/analysisForm.html` → `analysisForm.jsp`) +- [x] Analysis step form (`/user/analysisStepForm.html` → `analysisStepForm.jsp`) +- [x] Analyzed data form (`/user/analyzedDataForm.html` → `analyzedDataForm.jsp`) +- [x] Ready state confirmation (`/user/readyState.html` → `readyState.jsp`) +- [x] Delete study (`/user/deleteStudy.html` → `deleteStudy.jsp`) +- [x] Submission tutorial (`/submitTutorial.html` → `submitTutorial.jsp`) ## Navigation Flow -*TODO: Document how users navigate through these pages as a mermaid plot* - -``` -[User Dashboard] --> [New Submission] --> [Study Metadata] - | - v - [File Upload] - | - v - [Tree/Matrix/Taxa Entry] - | - v - [Analysis Metadata] - | - v - [Preview & Validate] - | - v - [Submit] +The following diagram documents how users navigate through the submission pages: + +```mermaid +flowchart TD + subgraph Dashboard["Submission Dashboard"] + SubmitList["/user/submissionList.html
My Submissions
(submissionList.jsp)"] + end + + subgraph Create["Create Submission"] + NewStudy["/user/studyForm.html?form=new
New Submission Form
(studyForm.jsp)"] + end + + subgraph StudyMeta["Study Metadata"] + Summary["/user/summary.html
Submission Summary
(submissionSummaryView.jsp)"] + Citation["/user/citationForm.html
Citation Form
(citationForm.jsp)"] + AuthorSearch["/user/authorSearchForm.html
Author Search
(peopleSearchForm.jsp)"] + AddAuthor["/user/addAuthor.html
Add Author
(addPersonForm.jsp)"] + end + + subgraph Upload["File Upload"] + UploadFile["/user/uploadFile.html
NEXUS File Upload
(uploadFile.jsp)"] + UploadSummary["/user/uploadFileSummary.html
Upload Summary
(uploadFileSummary.jsp)"] + end + + subgraph Trees["Tree Management"] + TreeBlocks["/user/treeBlockList.html
Tree Blocks
(treeBlockList.jsp)"] + TreeList["/user/treeList.html
Trees in Block
(treeList.jsp)"] + TreeViewer["/user/directMapToPhyloWidget.html
Tree Viewer
(treeViewer.jsp)"] + end + + subgraph Matrices["Matrix Management"] + MatrixList["/user/matrixList.html
Matrices
(matrixList.jsp)"] + MatrixRows["/user/matrixRowList.html
Matrix Rows
(matrixRowList.jsp)"] + RowSegments["/user/viewAllRowSegmentData.html
Row Segments
(viewAllRowSegmentData.jsp)"] + end + + subgraph Taxa["Taxon Management"] + TaxaList["/user/taxaList.html
Taxon Labels
(taxonLabels.jsp)"] + EditTaxon["/user/editTaxonLabel.html
Edit Taxon
(editTaxonLabel.jsp)"] + end + + subgraph Analysis["Analysis Management"] + AnalysisList["/user/analysisDisplay.html
Analyses
(analysisList.jsp)"] + AnalysisForm["/user/analysisForm.html
Analysis Form
(analysisForm.jsp)"] + AnalysisStep["/user/analysisStepForm.html
Analysis Step
(analysisStepForm.jsp)"] + AnalyzedData["/user/analyzedDataForm.html
Analyzed Data
(analyzedDataForm.jsp)"] + end + + subgraph Submit["Review & Submit"] + ReadyState["/user/readyState.html
Change to Ready State
(readyState.jsp)"] + DeleteStudy["/user/deleteStudy.html
Delete Study
(deleteStudy.jsp)"] + end + + %% Main flow + SubmitList -->|"New Submission"| NewStudy + NewStudy -->|"Submit"| Summary + SubmitList -->|"Edit submission"| Summary + + %% Summary navigation (via Tool Box menu) + Summary -->|"Citation"| Citation + Summary -->|"Upload"| UploadFile + Summary -->|"Trees"| TreeBlocks + Summary -->|"Matrices"| MatrixList + Summary -->|"Taxa"| TaxaList + Summary -->|"Analysis"| AnalysisList + + %% Citation flow + Citation -->|"Add/search authors"| AuthorSearch + AuthorSearch -->|"Add new"| AddAuthor + AddAuthor --> AuthorSearch + Citation -->|"Done"| Summary + + %% Upload flow + UploadFile -->|"Upload complete"| UploadSummary + UploadSummary -->|"Continue"| Summary + + %% Tree flow + TreeBlocks -->|"View trees"| TreeList + TreeList -->|"Edit tree"| TreeViewer + TreeViewer --> TreeList + + %% Matrix flow + MatrixList -->|"View rows"| MatrixRows + MatrixRows -->|"View segments"| RowSegments + + %% Taxa flow + TaxaList -->|"Edit taxon"| EditTaxon + EditTaxon --> TaxaList + TaxaList -->|"Validate labels"| TaxaList + + %% Analysis flow + AnalysisList -->|"Add analysis"| AnalysisForm + AnalysisForm -->|"Add step"| AnalysisStep + AnalysisStep -->|"Add data"| AnalyzedData + AnalyzedData --> AnalysisStep + AnalysisStep --> AnalysisForm + AnalysisForm --> AnalysisList + + %% Submission status + SubmitList -->|"Change status"| ReadyState + ReadyState -->|"Success"| SubmitList + SubmitList -->|"Delete"| DeleteStudy + DeleteStudy -->|"Confirmed"| SubmitList ``` ## Submission Workflow ### Step 1: Study Information -*TODO: verify the exact fields that are currently required and the pages that record this information* - -- Publication details (journal, DOI, authors) -- Study title and abstract -- Keywords and categories +Study information is recorded on multiple pages managed by `StudyFormController` and `CitationFormController`: + +**Study Form** (`/user/studyForm.html` → `studyForm.jsp`): +- **Study Name** (required) - Brief title, usually same as publication title +- **Study Notes** - Private notes for submitter/staff communication, not public + +**Citation Form** (`/user/citationForm.html` → `citationForm.jsp`): +- **Citation Type** (required) - Article, Book, or Book Section +- **Publish Year** (required) - Year of publication (1900 to current+1) +- **Publication Status** - Status of the publication + +For Article citations (`citationForm-article.jsp`): +- **Article Title** (required) +- **Keywords** - Comma-separated keywords +- **Abstract** - Publication abstract +- **PubMed ID** (PMID) - Maximum 10 characters +- **DOI** - Digital Object Identifier +- **URL** - Link to publication +- **Journal** (required) - Autocomplete from existing journals +- **Volume** +- **Issue** +- **Pages** - In format like "11-28" + +**Author Management** (`/user/authorSearchForm.html`, `/user/addAuthor.html`): +- Search for existing authors or add new ones +- Authors linked to citation via `AuthorFormController` ### Step 2: Data Upload -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +File upload is handled by `UploadFileController` (`/user/uploadFile.html` → `uploadFile.jsp`): + +**Supported Formats:** +- **NEXUS** - Primary format, parsed by Mesquite +- Files must be parseable by Mesquite before upload + +**Upload Interface:** +- Multiple file upload support via "Attach another file" +- Progress bar during upload +- **Maximum upload size**: 250MB (configured in `treebase-servlet.xml`) -- NEXUS file upload -- Newick file upload -- Other supported formats -- File validation +**Upload Constraints:** +- Only first ~30 trees are parsed to avoid overwhelming the interface +- Users should place preferred/consensus trees first in the tree block + +**Upload Summary** (`/user/uploadFileSummary.html` → `uploadFileSummary.jsp`): +- Shows count of matrices uploaded +- Shows count of trees uploaded +- Parser log available via "Show Parser Log" button ### Step 3: Tree Data -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +Tree management is handled by `ListTreeBlockController` and `ListTreeController`: + +**Tree Block List** (`/user/treeBlockList.html` → `treeBlockList.jsp`): +- **Block Title** - Editable title for the tree block +- **Tree Count** - Number of trees in the block +- **Analysis Status** - Icon showing if block is linked to an analysis +- Actions: View trees, View in PhyloWidget, Download, Delete + +**Tree List** (`/user/treeList.html` → `treeList.jsp`): +- **Tree Label** - Editable label +- **Tree Title** - Editable title +- **Tree Type** - Dropdown selection (e.g., Consensus, Single) +- **Tree Kind** - Dropdown selection (e.g., Species Tree, Gene Tree) +- **Tree Quality** - Dropdown selection +- **Analysis Status** - Icon showing if tree is linked to an analysis +- Actions: Edit in PhyloWidget, Download reconstructed, Download original NEXUS, Delete -- Tree visualization -- Tree metadata -- Multiple trees per study +**Tree Viewer** (`/user/directMapToPhyloWidget.html` → `treeViewer.jsp`): +- PhyloWidget-based tree visualization and editing +- Opens in popup window (1000x900 pixels) ### Step 4: Matrix Data -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +Matrix management is handled by `ListMatrixController`: + +**Matrix List** (`/user/matrixList.html` → `matrixList.jsp`): +- **Matrix Title** - Editable title +- **Matrix Description** - Editable description +- **Matrix Kind** - Dropdown selection (e.g., DNA, Protein, Morphological) +- **Analysis Status** - Icon showing if matrix is linked to an analysis +- Actions: View rows, Download reconstructed, Download original NEXUS, Delete + +**Matrix Row List** (`/user/matrixRowList.html` → `matrixRowList.jsp`): +- View matrix rows +- Navigate to row segments -- Character matrix display -- Matrix metadata -- Character definitions +**Row Segment Data** (`/user/viewAllRowSegmentData.html` → `viewAllRowSegmentData.jsp`): +- View detailed character state data +- Export functionality available ### Step 5: Taxonomic Data -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +Taxon management is handled by `ListTaxaController` and `EditTaxonLabelController`: -- Taxon name reconciliation -- Taxonomic hierarchy -- OTU mapping +**Taxon Labels List** (`/user/taxaList.html` → `taxonLabels.jsp`): +- **Taxon Label** - Original label from data file +- **Taxon Name** - Mapped scientific name +- **NCBI Tax ID** - Link to NCBI Taxonomy Browser +- **uBio Tax ID** - Link to uBio NameBank +- **Validation Status** - Icon showing if linking was attempted +- Actions: Edit, Bulk update, Validate selected labels -### Step 6: Analysis Information +**Edit Taxon Label** (`/user/editTaxonLabel.html` → `editTaxonLabel.jsp`): +- **Taxon Label** - Editable text, guidelines provided for formatting +- **External Taxonomy** - Radio selection from suggested matches: + - NCBI taxonomy suggestions with IDs + - uBio lookup option with manual NamebankID entry + - "No association" option if no match exists + +**Validation Process:** +- Labels validated against uBio and NCBI taxonomies +- Guidelines: Use full binomials (no abbreviations), separate codes with spaces +- "Validate Taxon Labels" button triggers bulk validation -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +### Step 6: Analysis Information -- Analysis type/method -- Software used -- Parameters and settings +Analysis management is handled by `AnalysisFormController` and `AnalysisStepFormController`: + +**Analysis Form** (`/user/analysisForm.html` → `analysisForm.jsp`): +- **Analysis Name** - Descriptive name for the analysis +- **Analysis Notes** - Additional notes about the analysis + +**Analysis Step Form** (`/user/analysisStepForm.html` → `analysisStepForm.jsp`): +- **Step Name** - Name for this analysis step +- **Step Notes** - Additional notes +- **Commands** - Actual commands/settings used +- **Software Name** - Autocomplete from known software +- **Software Version** - Version number +- **Software URL** - Link to software +- **Software Description** - Brief description +- **Algorithm Type** - Dropdown: Maximum Likelihood, Parsimony, Distance, Bayesian, Other +- **New Algorithm** - Text field when "Other Algorithm" selected + +**Analyzed Data** (`/user/analyzedDataForm.html` → `analyzedDataForm.jsp`): +- Multi-page wizard for linking inputs/outputs: + 1. Select data type and input/output designation + 2. Select matrices (for input) + 3. Select trees (for output) + 4. Select tree blocks ### Step 7: Review and Submit -*TODO: verify the exact fields and interactions that are currently required and the pages that record this information* +The review and submission process validates data completeness before allowing status change: + +**Submission Summary** (`/user/summary.html` → `submissionSummaryView.jsp`): +- Displays submission number and status +- Shows study accession URL (permanent identifier) +- Shows reviewer access URL with access code +- Citation summary if entered +- Abstract if provided +- Analysis list embedded + +**Ready State Confirmation** (`/user/readyState.html` → `readyState.jsp`): -- Validation summary -- Preview all data -- Submit for review +**Pre-submission Validation Checks:** +The following must be satisfied before changing to "Ready State": +1. Citation entered with minimum: authors, year, title, journal name +2. No orphaned matrices (all matrices must be inputs to at least one analysis) +3. No orphaned trees (all trees must be outputs from at least one analysis) +4. Tree taxon labels must match input matrix taxon labels +5. Taxon labels validation attempted (via "Validate Taxon Labels" button) + +**Submission Confirmation:** +- Warning message about read-only status after submission +- Exception: Citation metadata can still be updated (volume, issue, pages, DOI) +- Submit button completes the status change ## Data Validation -*TODO: Document validation rules and error handling* +Data validation occurs at multiple stages in the submission process: -- File format validation -- Required field validation -- Taxonomic name validation -- Consistency checks +### File Format Validation -## Pages to Account For +**At Upload Time:** +- NEXUS files are parsed using Mesquite library +- Invalid syntax results in parse errors shown in Parser Log +- Recommendation: Test files in Mesquite before upload + +**File Size Validation:** +- Maximum upload size: 250MB (configurable in `treebase-servlet.xml`) +- Tree block limit: Only first ~30 trees parsed per block + +### Required Field Validation + +**Study Form:** +- Study Name: Required, validated server-side -*TODO: Complete inventory of pages related to data submission* +**Citation Form:** +- Citation Type: Required (Article, Book, Book Section) +- Publish Year: Required, validated 1900 ≤ year ≤ current+1 +- Title: Required for articles +- Journal: Required for articles +- At least one author required (validated at Ready State) -| Page | URL Pattern | Status | -|------|-------------|--------| -| | | | +**Analysis Step Form:** +- Algorithm Type validation when "Other" selected - New Algorithm field required + +### Taxonomic Name Validation + +**Automated Validation:** +- Triggered by "Validate Taxon Labels" button +- Matches labels against uBio and NCBI taxonomies +- Visual indicators show validation status: + - Green checkmark: Linking attempted + - Red X: Not yet validated + +**Manual Validation:** +- Edit individual labels via Edit Taxon Label page +- Search uBio manually and enter NamebankID +- Select "no association" for unmatched taxa + +### Consistency Checks (at Ready State) + +The system validates the following before allowing status change: + +1. **Citation Completeness:** + - Authors present + - Year present + - Title present + - Journal name present (for articles) + +2. **Data Completeness:** + - No orphaned matrices (each matrix linked to ≥1 analysis as input) + - No orphaned trees (each tree linked to ≥1 analysis as output) + +3. **Taxon Label Matching:** + - Output tree taxon labels must exist in input matrix + - Mismatched labels prevent Ready State + +4. **Validation Attempted:** + - Taxon label validation must be attempted + +**Error Display:** +- Errors shown in red at top of Ready State page +- Tool Box items highlighted in yellow indicate problems +- Hover over highlighted items for error details + +## Pages to Account For + +Complete inventory of pages related to data submission from `/treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml`: + +| Page | URL Pattern | Controller | View | Status | +|------|-------------|------------|------|--------| +| Submission List | `/user/submissionList.html` | `listSubmissionController` | `submissionList.jsp` | Active | +| New Study Form | `/user/studyForm.html` | `studyFormController` | `studyForm.jsp` | Active | +| Submission Summary | `/user/summary.html` | `summaryController` | `submissionSummaryView.jsp` | Active | +| Citation Form | `/user/citationForm.html` | `citationFormController` | `citationForm.jsp` | Active | +| Author Search | `/user/authorSearchForm.html` | `authorSearchFormController` | `peopleSearchForm.jsp` | Active | +| Add Author | `/user/addAuthor.html` | `addAuthorController` | `addPersonForm.jsp` | Active | +| Upload File | `/user/uploadFile.html` | `uploadFileController` | `uploadFile.jsp` | Active | +| Upload Summary | `/user/uploadFileSummary.html` | `uploadFileSummaryController` | `uploadFileSummary.jsp` | Active | +| Tree Block List | `/user/treeBlockList.html` | `listTreeBlockController` | `treeBlockList.jsp` | Active | +| Tree List | `/user/treeList.html` | `listTreeController` | `treeList.jsp` | Active | +| Tree Viewer | `/user/directMapToPhyloWidget.html` | `directMapToPhyloWidgetController` | `treeViewer.jsp` | Active | +| Delete Tree | `/user/deleteATree.html` | `deleteATreeController` | `generalDeletePage.jsp` | Active | +| Delete Tree Block | `/user/deleteATreeBlock.html` | `deleteATreeBlockController` | `generalDeletePage.jsp` | Active | +| Matrix List | `/user/matrixList.html` | `listMatrixController` | `matrixList.jsp` | Active | +| Matrix Row List | `/user/matrixRowList.html` | `listMatrixRowController` | `matrixRowList.jsp` | Active | +| Row Segment List | `/user/matrixRowSegmentList.html` | `listMatrixRowSegmentController` | `matrixRowSegmentList.jsp` | Active | +| View All Row Segments | `/user/viewAllRowSegmentData.html` | `viewAllRowSegmentDataController` | `viewAllRowSegmentData.jsp` | Active | +| Delete Matrix | `/user/deleteAMatrix.html` | `deleteAMatrixController` | `generalDeletePage.jsp` | Active | +| Taxa List | `/user/taxaList.html` | `listTaxaController` | `taxonLabels.jsp` | Active | +| Edit Taxon Label | `/user/editTaxonLabel.html` | `editTaxonLabelController` | `editTaxonLabel.jsp` | Active | +| Edit Set Taxon Labels | `/user/editSetTaxonLabel.html` | `editSetTaxonLabelController` | `editSetTaxonLabel.jsp` | Active | +| Analysis List | `/user/analysisList.html` | `listAnalysisController` | `analysisList.jsp` | Active | +| Analysis Display | `/user/analysisDisplay.html` | `displayAnalysisController` | `analysisList.jsp` | Active | +| Analysis Form | `/user/analysisForm.html` | `analysisFormController` | `analysisForm.jsp` | Active | +| Analysis Step List | `/user/analysisStepList.html` | `listAnalysisStepController` | `analysisStepList.jsp` | Active | +| Analysis Step Form | `/user/analysisStepForm.html` | `analysisStepFormController` | `analysisStepForm.jsp` | Active | +| Analyzed Data List | `/user/analyzedDataList.html` | `listAnalyzedDataController` | `analyzedDataList.jsp` | Active | +| Analyzed Data Form | `/user/analyzedDataForm.html` | `analyzedDataFormController` | `analyzedDataForm.jsp` | Active | +| Add Analyzed Data | `/user/addAnalyzedData.html` | `addAnalyzedDataController` | - | Active | +| Ready State | `/user/readyState.html` | `readyStateController` | `readyState.jsp` | Active | +| Delete Study | `/user/deleteStudy.html` | `deleteStudyController` | `deleteStudy.jsp` | Active | +| Submit Tutorial | `/submitTutorial.html` | `filenameController` | `submitTutorial.jsp` | Active | +| Download Tree | `/user/downloadATree.html` | `downloadATreeController` | - | Active | +| Download Tree Block | `/user/downloadATreeBlock.html` | `downloadATreeBlockController` | - | Active | +| Download Matrix | `/user/downloadAMatrix.html` | `downloadAMatrixController` | - | Active | +| Download NEXUS File | `/user/downloadANexusFile.html` | `downloadANexusFileController` | - | Active | ## Wireframe Notes @@ -136,7 +441,31 @@ ## Open Questions -- What file formats should be supported? -- What is the maximum file size? -- Can submissions be saved as drafts? -- What validation happens at each step vs. at submission? +The following questions have been addressed based on current implementation: + +- **What file formats should be supported?** + - NEXUS is the primary supported format + - Files must be parseable by Mesquite + - TreeBASE does not directly support Newick-only files; they should be embedded in NEXUS format + +- **What is the maximum file size?** + - 250MB maximum upload size (configured in `treebase-servlet.xml`) + - Only first ~30 trees per block are parsed + +- **Can submissions be saved as drafts?** + - Yes, submissions in "In Progress" status function as drafts + - Users can continue editing until they change status to "Ready" + - "In Progress" submissions are not visible to the public + +- **What validation happens at each step vs. at submission?** + - File validation: At upload time (NEXUS parsing) + - Field validation: On form submission (server-side Spring validators) + - Comprehensive validation: When changing to "Ready State" (orphan checks, taxon matching, etc.) + +## Security Considerations + +- **File Upload Security**: File uploads handled by `AjaxMultipartResolver` with size limits +- **Access Control**: Submission pages require User, Admin, or Associate Editor role +- **Study Ownership**: Users can only modify their own submissions (enforced in controllers) +- **Status Transitions**: Only study owners and admins can change submission status +- **Read-only After Ready**: Once in "Ready" status, only citation metadata can be updated From 01dc154e7d3758a4aab239f8fb24ec1a61a4defc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:39:55 +0000 Subject: [PATCH 28/52] Create doc/v3/user-story-01-search.md with search and projection documentation Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-01-search.md | 108 +++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 doc/v3/user-story-01-search.md diff --git a/doc/v3/user-story-01-search.md b/doc/v3/user-story-01-search.md new file mode 100644 index 000000000..394dbd635 --- /dev/null +++ b/doc/v3/user-story-01-search.md @@ -0,0 +1,108 @@ +# User Story 01: Search + +## Overview + +This document describes the search functionality in TreeBASE v3, including the searchable data types and result projection features. + +--- + +## Searchable Data Types + +TreeBASE supports searching across five primary data types. Selecting any of these tabs in the interface displays the corresponding results page: + +1. **Studies** - Search for phylogenetic studies by metadata, authors, citations, etc. +2. **Matrices** - Search for character matrices used in phylogenetic analyses +3. **Trees** - Search for phylogenetic trees by properties, topology, or associated metadata +4. **Taxa** - Search for taxonomic names and their occurrences across studies +5. **Tree Topologies** - Search for trees based on their structural topology patterns + +### Tab Navigation + +Each search type is accessible via a dedicated tab in the search interface. When a user selects a tab: + +- The search form updates to show relevant search fields for that data type +- The results page displays items of the selected type +- Pagination and sorting options are specific to the selected data type + +--- + +## Result Set Projection + +Users can project (transform) a result set from one data type into another. This allows users to explore related data across different dimensions. + +### How Result Projection Works + +After performing a search and obtaining a result set, users can switch to a different data type tab to see the related items. The system computes the union of all items of the new type that are associated with the original result set. + +### Example: Studies → Taxa + +1. **Initial Search**: User performs a study search and obtains a list of matching studies +2. **Projection**: User switches to the "Taxa" tab +3. **Result**: The system displays the union of all taxa that appear in any of the studies from the original result set + +### Supported Projections + +| From | To | Description | +|------|-----|-------------| +| Studies | Taxa | All taxa referenced in the selected studies | +| Studies | Matrices | All matrices associated with the selected studies | +| Studies | Trees | All trees generated by the selected studies | +| Studies | Tree Topologies | All tree topologies found in the selected studies | +| Matrices | Taxa | All taxa included in the selected matrices | +| Matrices | Studies | All studies that contain the selected matrices | +| Matrices | Trees | All trees derived from the selected matrices | +| Matrices | Tree Topologies | All tree topologies derived from the selected matrices | +| Trees | Taxa | All taxa present in the selected trees | +| Trees | Studies | All studies that produced the selected trees | +| Trees | Matrices | All matrices used to generate the selected trees | +| Trees | Tree Topologies | Topology patterns of the selected trees | +| Taxa | Studies | All studies that reference the selected taxa | +| Taxa | Matrices | All matrices containing the selected taxa | +| Taxa | Trees | All trees that include the selected taxa | +| Taxa | Tree Topologies | All tree topologies that include the selected taxa | +| Tree Topologies | Trees | All trees matching the selected topologies | +| Tree Topologies | Studies | All studies containing trees with the selected topologies | +| Tree Topologies | Matrices | All matrices that produced trees matching the selected topologies | +| Tree Topologies | Taxa | All taxa present in trees matching the selected topologies | + +### Use Cases + +- **Taxonomic Analysis**: Start with a study search, then project to taxa to understand which organisms are covered +- **Data Discovery**: Search for taxa, then project to studies to find relevant research +- **Tree Comparison**: Search for tree topologies, then project to studies to compare methodologies +- **Cross-Study Analysis**: Search for specific matrices, then project to taxa to see overlapping organisms + +--- + +## User Interface + +### Search Tabs + +``` +┌─────────┬──────────┬────────┬────────┬─────────────────┐ +│ Studies │ Matrices │ Trees │ Taxa │ Tree Topologies │ +└─────────┴──────────┴────────┴────────┴─────────────────┘ +``` + +Each tab is clickable and: +- Projects the current result set into the selected type when results from a previous search exist +- Performs a new search of that type when starting fresh (no previous results) + +A "Clear Results" option allows users to start a new search without projection. + +### Result Projection Indicator + +When viewing projected results, the interface should indicate: +- The original search type and query +- The current projection type +- The total number of items in both the original and projected result sets + +--- + +## Technical Notes + +- Result projection is computed dynamically based on database relationships +- Large result sets (over 1,000 items) may trigger a warning to users about potential delay when projecting +- Pagination applies to displayed results; projection computes against the complete result set +- Cache strategies may be employed to improve projection performance for frequently accessed result sets +- Maximum result set size for projection is configurable by administrators From 83919ade4152b0f9a926ccd4e3df2b391efa66f9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 19:40:57 +0000 Subject: [PATCH 29/52] Add delete confirmation and restriction details to security section Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-03-submission.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/v3/user-story-03-submission.md b/doc/v3/user-story-03-submission.md index f9287fa4d..4a8495c14 100644 --- a/doc/v3/user-story-03-submission.md +++ b/doc/v3/user-story-03-submission.md @@ -469,3 +469,5 @@ The following questions have been addressed based on current implementation: - **Study Ownership**: Users can only modify their own submissions (enforced in controllers) - **Status Transitions**: Only study owners and admins can change submission status - **Read-only After Ready**: Once in "Ready" status, only citation metadata can be updated +- **Delete Confirmation**: Study deletion requires explicit confirmation via `deleteStudy.jsp` confirmation page with warning message about permanent data loss +- **Delete Restrictions**: Studies can only be deleted when in "In Progress" status; all trees and matrices must be deleted first before the study can be removed From 02cd7ab3aab0fc497c991d236c0233d285a55c84 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 20:50:40 +0100 Subject: [PATCH 30/52] Mark User Story 3 as completed in README --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index e40ebdeba..1ddc835ca 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -24,7 +24,7 @@ The v3 roadmap aims to: - [x] **[User Story 2: Account Management](user-story-02-account.md)** - A user visits TreeBASE to create or update their user account and log in -- [ ] **[User Story 3: Data Submission](user-story-03-submission.md)** - A user visits TreeBASE to +- [x] **[User Story 3: Data Submission](user-story-03-submission.md)** - A user visits TreeBASE to submit study, tree, matrix, taxon or analysis data ### Reviewers From 746f5baeb1d2fa2da4ce84b505e870faffcf7091 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 20:14:12 +0000 Subject: [PATCH 31/52] Initial plan From 34e272181cd606fc312a14a873fa7b57de70ca22 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 20:21:07 +0000 Subject: [PATCH 32/52] Address TODO items in user-story-01-search.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-01-search.md | 360 +++++++++++++++++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 doc/v3/user-story-01-search.md diff --git a/doc/v3/user-story-01-search.md b/doc/v3/user-story-01-search.md new file mode 100644 index 000000000..24617e30b --- /dev/null +++ b/doc/v3/user-story-01-search.md @@ -0,0 +1,360 @@ +# User Story 01: Search + +## Overview + +**As a** user or automated agent, +**I want to** search for study, tree, matrix, taxon or analysis data, +**So that** I can find and access phylogenetic data relevant to my research or application. + +## User Types + +- Anonymous visitors (researchers, students, general public) +- Automated agents (harvesters, API clients, search engines) +- Registered users looking for data + +## Current Pages + +The following search pages are currently available in the TreeBASE web application: + +- [x] **Study Search Page** (`/search/studySearch.html`) - Search for phylogenetic studies +- [x] **Matrix Search Page** (`/search/matrixSearch.html`) - Search for character matrices +- [x] **Tree Search Page** (`/search/treeSearch.html`) - Search for phylogenetic trees +- [x] **Taxon Search Page** (`/search/taxonSearch.html`) - Search for taxonomic names +- [x] **Tree Topology Search Page** (`/search/treeTopSearch.html`) - Search by tree structure + +## Navigation Flow + +The following diagram shows how users navigate through search pages: + +```mermaid +graph TB + subgraph Entry Points + HOME[Home Page] + NAV[Navigation Bar] + DIRECT[Direct URL / API] + end + + subgraph Search Tabs + STUDY[Study Search
/search/studySearch.html] + MATRIX[Matrix Search
/search/matrixSearch.html] + TREE[Tree Search
/search/treeSearch.html] + TAXON[Taxon Search
/search/taxonSearch.html] + TREETOP[Tree Topology Search
/search/treeTopSearch.html] + end + + subgraph Search Forms + SIMPLE[Simple Search Form] + ADV[Advanced CQL Search] + end + + subgraph Results + RESULTS[Search Results List] + end + + subgraph Detail Pages + STUDY_DETAIL[Study Summary
/search/study/summary.html] + MATRIX_DETAIL[Matrix Detail
/search/study/matrix.html] + TREE_DETAIL[Tree Viewer
/search/study/tree.html] + TAXA_DETAIL[Taxa List
/search/study/taxa.html] + end + + subgraph Downloads + NEXUS[NEXUS Format] + NEXML[NeXML Format] + end + + HOME --> NAV + NAV --> STUDY + NAV --> MATRIX + NAV --> TREE + NAV --> TAXON + NAV --> TREETOP + DIRECT --> STUDY + DIRECT --> MATRIX + DIRECT --> TREE + DIRECT --> TAXON + DIRECT --> TREETOP + + STUDY --> SIMPLE + MATRIX --> SIMPLE + TREE --> SIMPLE + TAXON --> SIMPLE + TREETOP --> SIMPLE + + SIMPLE --> ADV + SIMPLE --> RESULTS + ADV --> RESULTS + + RESULTS --> STUDY_DETAIL + RESULTS --> MATRIX_DETAIL + RESULTS --> TREE_DETAIL + RESULTS --> TAXA_DETAIL + + STUDY_DETAIL --> NEXUS + STUDY_DETAIL --> NEXML + MATRIX_DETAIL --> NEXUS + TREE_DETAIL --> NEXUS + + %% Tab switching for result projection + RESULTS -.->|Project Results| STUDY + RESULTS -.->|Project Results| MATRIX + RESULTS -.->|Project Results| TREE + RESULTS -.->|Project Results| TAXON +``` + +### Navigation Notes + +- Users can switch between search tabs at any time via the navigation bar +- When results exist, switching tabs projects the current result set to the new data type +- Each search page provides both simple and advanced (CQL) search options +- Search results include links to detail pages and download options + +## Search Capabilities + +## Searchable Data Types + +TreeBASE supports searching across five primary data types. Each data type has specific searchable fields and display options. + +### 1. Studies + +Search for phylogenetic studies by metadata, authors, citations, etc. + +**Searchable Fields:** +| Field | Description | CQL Predicate | +|-------|-------------|---------------| +| Study ID | TreeBASE study identifier (S####) | `tb.identifier.study` | +| Legacy Study ID | TreeBASE 1.x study identifier | `tb.identifier.study.tb1` | +| Author | Author name(s) | `dcterms.contributor` | +| Title | Study/publication title | `tb.title.study` | +| Abstract | Publication abstract | `dcterms.abstract` | +| Citation | Full bibliographic citation | `dcterms.bibliographicCitation` | +| Keywords | Subject keywords | `dcterms.subject` | +| DOI | Digital Object Identifier | `prism.doi` | + +**Result Display Columns:** +- Study ID (with link to summary) +- Authors (first 3, then "et al.") +- Year +- Title +- Journal/Publisher +- DOI link (external) +- Download icons (NEXUS, NeXML) + +### 2. Matrices + +Search for character matrices used in phylogenetic analyses. + +**Searchable Fields:** +| Field | Description | CQL Predicate | +|-------|-------------|---------------| +| Matrix ID | TreeBASE matrix identifier (M####) | `tb.identifier.matrix` | +| Title | Matrix title/name | `tb.title.matrix` | +| Type | Matrix data type | `tb.type.matrix` | +| NTAX | Number of taxa | `tb.ntax.matrix` | +| NCHAR | Number of characters | `tb.nchar.matrix` | + +**Result Display Columns:** +- Matrix ID (with link to detail) +- Title +- Description +- Data Type +- NTAX +- NCHAR +- Download icons (original, reconstructed) +- View rows icon + +### 3. Trees + +Search for phylogenetic trees by properties, topology, or associated metadata. + +**Searchable Fields:** +| Field | Description | CQL Predicate | +|-------|-------------|---------------| +| Tree ID | TreeBASE tree identifier (Tr####) | `tb.identifier.tree` | +| Title | Tree title/name | `tb.title.tree` | +| Type | Tree type (e.g., Species Tree) | `tb.type.tree` | +| Kind | Tree kind (e.g., Consensus) | `tb.kind.tree` | +| Quality | Tree quality assessment | `tb.quality.tree` | +| NTAX | Number of taxa | `tb.ntax.tree` | + +**Result Display Columns:** +- Tree ID (with link to viewer) +- Label +- Title +- Tree Type +- Tree Kind +- Tree Quality +- NTAX +- View Taxa link +- Download icons (reconstructed, original) +- Tree viewer icon + +### 4. Taxa + +Search for taxonomic names and their occurrences across studies. + +**Searchable Fields:** +| Field | Description | CQL Predicate | +|-------|-------------|---------------| +| Taxon ID | TreeBASE taxon identifier (Tx####) | `tb.identifier.taxon` | +| Legacy Taxon ID | TreeBASE 1.x taxon identifier | `tb.identifier.taxon.tb1` | +| NCBI ID | NCBI Taxonomy identifier | `tb.identifier.ncbi` | +| uBio ID | uBio Namebank identifier | `tb.identifier.ubio` | +| Taxon Name | Scientific name | `tb.title.taxon` | +| Taxon Label | Label as used in study | `tb.title.taxonLabel` | +| Taxon Variant | Name variant | `tb.title.taxonVariant` | + +**Result Display Columns:** +- Taxon ID +- Taxon Name +- uBio ID (with external link) +- NCBI ID (with external link) + +### 5. Tree Topologies + +Search for trees based on their structural topology patterns. + +**Search Types:** +| Search Type | Description | Input Fields | +|-------------|-------------|--------------| +| 3-Taxon Topology | Find trees containing a specific 3-taxon relationship | taxon_a, taxon_b, taxon_c | +| 4-Taxon Asymmetric | Find trees with asymmetric 4-taxon topology | taxon_a, taxon_b, taxon_c, taxon_d | +| 4-Taxon Symmetric | Find trees with symmetric 4-taxon topology | taxon_a, taxon_b, taxon_c, taxon_d | + +**Result Display:** Same as Trees (returns matching tree records) + +### Tab Navigation + +Each search type is accessible via a dedicated tab in the search interface. When a user selects a tab: + +- The search form updates to show relevant search fields for that data type +- An advanced search option is available for more complex queries using CQL +- The results page displays items of the selected type +- Pagination and sorting options are specific to the selected data type + +## Result Set Projection + +Users can project (transform) a result set from one data type into another. This allows users to explore related data +across different dimensions. + +### How Result Projection Works + +After performing a search and obtaining a result set, users can switch to a different data type tab to see the related +items. The system computes the union of all items of the new type that are associated with the original result set. + +### Example: Studies → Taxa + +1. **Initial Search**: User performs a study search and obtains a list of matching studies +2. **Projection**: User switches to the "Taxa" tab +3. **Result**: The system displays the union of all taxa that appear in any of the studies from the original result set + +### Supported Projections + +| From | To | Description | +|------|-----|-------------| +| Studies | Taxa | All taxa referenced in the selected studies | +| Studies | Matrices | All matrices associated with the selected studies | +| Studies | Trees | All trees generated by the selected studies | +| Studies | Tree Topologies | All tree topologies found in the selected studies | +| Matrices | Taxa | All taxa included in the selected matrices | +| Matrices | Studies | All studies that contain the selected matrices | +| Matrices | Trees | All trees derived from the selected matrices | +| Matrices | Tree Topologies | All tree topologies derived from the selected matrices | +| Trees | Taxa | All taxa present in the selected trees | +| Trees | Studies | All studies that produced the selected trees | +| Trees | Matrices | All matrices used to generate the selected trees | +| Trees | Tree Topologies | Topology patterns of the selected trees | +| Taxa | Studies | All studies that reference the selected taxa | +| Taxa | Matrices | All matrices containing the selected taxa | +| Taxa | Trees | All trees that include the selected taxa | +| Taxa | Tree Topologies | All tree topologies that include the selected taxa | +| Tree Topologies | Trees | All trees matching the selected topologies | +| Tree Topologies | Studies | All studies containing trees with the selected topologies | +| Tree Topologies | Matrices | All matrices that produced trees matching the selected topologies | +| Tree Topologies | Taxa | All taxa present in trees matching the selected topologies | + +### Use Cases + +- **Taxonomic Analysis**: Start with a study search, then project to taxa to understand which organisms are covered +- **Data Discovery**: Search for taxa, then project to studies to find relevant research +- **Tree Comparison**: Search for tree topologies, then project to studies to compare methodologies +- **Cross-Study Analysis**: Search for specific matrices, then project to taxa to see overlapping organisms + +## User Interface + +### Search Tabs + +``` +┌─────────┬──────────┬────────┬────────┬─────────────────┐ +│ Studies │ Matrices │ Trees │ Taxa │ Tree Topologies │ +└─────────┴──────────┴────────┴────────┴─────────────────┘ +``` + +Each tab is clickable and: +- Projects the current result set into the selected type when results from a previous search exist +- Performs a new search of that type when starting fresh (no previous results) + +A "Clear Results" option allows users to start a new search without projection. + +### Result Projection Indicator + +When viewing projected results, the interface should indicate: +- The original search type and query +- The current projection type +- The total number of items in both the original and projected result sets + +## Technical Notes + +- Result projection is computed dynamically based on database relationships +- Large result sets (over 1,000 items) may trigger a warning to users about potential delay when projecting +- Pagination applies to displayed results; projection computes against the complete result set +- Cache strategies may be employed to improve projection performance for frequently accessed result sets +- Maximum result set size for projection is configurable by administrators + +## Pages to Account For + +Complete inventory of pages related to search functionality: + +| Page | URL Pattern | Description | Status | +|------|-------------|-------------|--------| +| Study Search | `/search/studySearch.html` | Main study search interface | Active | +| Matrix Search | `/search/matrixSearch.html` | Matrix search interface | Active | +| Tree Search | `/search/treeSearch.html` | Tree search interface | Active | +| Taxon Search | `/search/taxonSearch.html` | Taxon search interface | Active | +| Tree Topology Search | `/search/treeTopSearch.html` | Topology-based tree search | Active | +| Classification Search | `/search/classificationSearch.html` | Hierarchical classification search | Active | +| Study Summary | `/search/study/summary.html` | Study detail/summary view | Active | +| Study Matrices | `/search/study/matrices.html` | Matrices within a study | Active | +| Study Trees | `/search/study/trees.html` | Trees within a study | Active | +| Study Taxa | `/search/study/taxa.html` | Taxa within a study | Active | +| Study Analyses | `/search/study/analyses.html` | Analyses within a study | Active | +| Study Analysis Detail | `/search/study/analysis.html` | Single analysis detail | Active | +| Matrix Detail | `/search/study/matrix.html` | Matrix detail view | Active | +| Tree Viewer | `/search/study/tree.html` | Interactive tree viewer | Active | +| Tree Block Viewer | `/search/study/treeBlock.html` | Tree block viewer | Active | +| Tree Blocks List | `/search/study/treeBlocks.html` | List of tree blocks | Active | +| Row Segments | `/search/study/rowSegments.html` | Matrix row segments | Active | +| Row Segments TSV | `/search/study/rowSegmentsTSV.html` | Row segments as TSV download | Active | +| RDF Export | `/search/study/anyObjectAsRDF.rdf` | RDF export for any object | Active | +| Download Tree | `/search/downloadATree.html` | Download single tree | Active | +| Download Matrix | `/search/downloadAMatrix.html` | Download single matrix | Active | +| Download NEXUS | `/search/downloadANexusFile.html` | Download original NEXUS file | Active | + +### PhyloWS API Endpoints + +| Endpoint | Description | +|----------|-------------| +| `/phylows/study/find` | Study search API | +| `/phylows/matrix/find` | Matrix search API | +| `/phylows/tree/find` | Tree search API | +| `/phylows/taxon/find` | Taxon search API | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What search refinement options should be available? +- How should results be sorted and paginated? +- What metadata should be shown in search results? \ No newline at end of file From cc2c13b058fe36e40a12ced6313c4c50cee27554 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 20:22:59 +0000 Subject: [PATCH 33/52] Address code review feedback for user-story-01-search.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-01-search.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/doc/v3/user-story-01-search.md b/doc/v3/user-story-01-search.md index 24617e30b..92d183ff9 100644 --- a/doc/v3/user-story-01-search.md +++ b/doc/v3/user-story-01-search.md @@ -214,6 +214,9 @@ Search for taxonomic names and their occurrences across studies. Search for trees based on their structural topology patterns. +> **Note:** Tree Topology searches use a form-based interface rather than CQL queries. Users specify taxon names +> in a visual tree structure diagram, and the system finds trees matching that relationship pattern. + **Search Types:** | Search Type | Description | Input Fields | |-------------|-------------|--------------| @@ -223,6 +226,13 @@ Search for trees based on their structural topology patterns. **Result Display:** Same as Trees (returns matching tree records) +### 6. Classification Search (Secondary) + +The Classification Search (`/search/classificationSearch.html`) provides an alternative way to browse taxonomic +hierarchies. This is a specialized search interface that is not part of the main search tab navigation. + +**Note:** This feature is accessible via direct URL but is not prominently featured in the main search navigation. + ### Tab Navigation Each search type is accessible via a dedicated tab in the search interface. When a user selects a tab: @@ -315,6 +325,8 @@ When viewing projected results, the interface should indicate: Complete inventory of pages related to search functionality: +### Main Search Pages + | Page | URL Pattern | Description | Status | |------|-------------|-------------|--------| | Study Search | `/search/studySearch.html` | Main study search interface | Active | @@ -322,7 +334,17 @@ Complete inventory of pages related to search functionality: | Tree Search | `/search/treeSearch.html` | Tree search interface | Active | | Taxon Search | `/search/taxonSearch.html` | Taxon search interface | Active | | Tree Topology Search | `/search/treeTopSearch.html` | Topology-based tree search | Active | -| Classification Search | `/search/classificationSearch.html` | Hierarchical classification search | Active | + +### Secondary Search Pages + +| Page | URL Pattern | Description | Status | +|------|-------------|-------------|--------| +| Classification Search | `/search/classificationSearch.html` | Hierarchical classification browse (not in main nav) | Active | + +### Study Detail Pages + +| Page | URL Pattern | Description | Status | +|------|-------------|-------------|--------| | Study Summary | `/search/study/summary.html` | Study detail/summary view | Active | | Study Matrices | `/search/study/matrices.html` | Matrices within a study | Active | | Study Trees | `/search/study/trees.html` | Trees within a study | Active | @@ -334,6 +356,11 @@ Complete inventory of pages related to search functionality: | Tree Block Viewer | `/search/study/treeBlock.html` | Tree block viewer | Active | | Tree Blocks List | `/search/study/treeBlocks.html` | List of tree blocks | Active | | Row Segments | `/search/study/rowSegments.html` | Matrix row segments | Active | + +### Download Endpoints + +| Page | URL Pattern | Description | Status | +|------|-------------|-------------|--------| | Row Segments TSV | `/search/study/rowSegmentsTSV.html` | Row segments as TSV download | Active | | RDF Export | `/search/study/anyObjectAsRDF.rdf` | RDF export for any object | Active | | Download Tree | `/search/downloadATree.html` | Download single tree | Active | From b70c90f80cbb273cdb0728b44bb52ffe4497895c Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 21:27:01 +0100 Subject: [PATCH 34/52] Mark User Story 1 as completed --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 1ddc835ca..010f19bfc 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -16,7 +16,7 @@ The v3 roadmap aims to: ### Public Users -- [ ] **[User Story 1: Search](user-story-01-search.md)** - A user or agent visits TreeBASE to search +- [x] **[User Story 1: Search](user-story-01-search.md)** - A user or agent visits TreeBASE to search for study, tree, matrix, taxon or analysis data ### Authenticated Users From 9b4ffb78feafffb5fba913d595662e781207cd23 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 21:54:19 +0100 Subject: [PATCH 35/52] Revise user story for review process and navigation Updated user story for reviewer privileges and navigation workflow. --- doc/v3/user-story-04-review.md | 46 +++++++++------------------------- 1 file changed, 12 insertions(+), 34 deletions(-) diff --git a/doc/v3/user-story-04-review.md b/doc/v3/user-story-04-review.md index 83c51b962..a517df062 100644 --- a/doc/v3/user-story-04-review.md +++ b/doc/v3/user-story-04-review.md @@ -14,25 +14,24 @@ ## Prerequisites -- User must be logged in with reviewer privileges -- Study must be assigned to the reviewer +- Data submitter has provided the reviewer with a URL with a special access + token in the query string +- Reviewer visits the website via this URL to access the unpublished study ## Current Pages *TODO: Identify and list all current pages that apply to this user story* -- [ ] Review queue/dashboard -- [ ] Study review page +- [ ] Reviewer popup +- [ ] Study review page - [ ] Tree review interface - [ ] Matrix review interface - [ ] Taxa review interface - [ ] Analysis review interface -- [ ] Comment/feedback entry -- [ ] Approval/rejection interface ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages in a mermaid plot* ``` [Reviewer Dashboard] --> [Review Queue] --> [Study Review] @@ -58,57 +57,36 @@ ## Review Workflow -### Step 1: Access Review Queue -- View assigned reviews -- Filter by status, date, study type -- Priority indicators +*TODO: Document how the reviewer navigates the unlocked pages to review the study* -### Step 2: Study Overview Review +### Step 1: Study Overview Review - Verify publication details - Check metadata completeness - Validate citations -### Step 3: Tree Data Review +### Step 2: Tree Data Review - Visualize tree structure - Check topology - Verify branch support values - Validate tree labels -### Step 4: Matrix Data Review +### Step 3: Matrix Data Review - Review character matrix - Check for data completeness - Validate character definitions - Review coding consistency -### Step 5: Taxonomic Review +### Step 4: Taxonomic Review - Verify taxon names - Check taxonomic placement - Review OTU mappings - Flag nomenclatural issues -### Step 6: Analysis Review +### Step 5: Analysis Review - Check analysis parameters - Verify software citations - Review methodology description -### Step 7: Final Decision -- Approve for publication -- Request revisions with comments -- Reject with explanation - -## Review Features - -### Comments and Feedback -- Line-level comments on trees/matrices -- General study comments -- Suggestion for corrections -- Communication with submitter - -### Quality Checks -- Automated validation results -- Manual checklist items -- Cross-reference verification - ## Pages to Account For *TODO: Complete inventory of pages related to review functionality* From b6152d60e1543938ae6e7487d3ca08c3a0786b1a Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 20:59:52 +0000 Subject: [PATCH 36/52] Initial plan From a14540f17f205f701e71a6ce1b9b96bc6230436e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 21:07:27 +0000 Subject: [PATCH 37/52] Address TODO items in user-story-04-review.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-04-review.md | 177 +++++++++++++++++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 doc/v3/user-story-04-review.md diff --git a/doc/v3/user-story-04-review.md b/doc/v3/user-story-04-review.md new file mode 100644 index 000000000..9329d62ae --- /dev/null +++ b/doc/v3/user-story-04-review.md @@ -0,0 +1,177 @@ +# User Story 4: Review + +## Overview + +**As a** reviewer, +**I want to** review a study with tree, matrix, taxon and analysis data, +**So that** I can verify the quality and accuracy of submitted phylogenetic data. + +## User Types + +- Journal editors reviewing submissions +- Peer reviewers assigned to studies +- TreeBASE curators performing quality control + +## Prerequisites + +- Data submitter has provided the reviewer with a URL with a special access + token in the query string (`x-access-code` parameter) +- Reviewer visits the website via this URL to access the unpublished study + +## Current Pages + +The following pages are involved in the review workflow: + +- [x] Reviewer access agreement popup (`nav.jsp` lines 191-257) - Modal dialog that appears on first access requiring agreement to confidentiality terms +- [x] Study summary page (`search/study/summary.html`) - Displays citation, authors, abstract, keywords, and external links +- [x] Trees list page (`search/study/trees.html`) - Lists all trees in the study with download options +- [x] Tree viewer (`treeViewer.jsp`) - Interactive phylogenetic tree visualization using phylotree.js +- [x] Matrices list page (`search/study/matrices.html`) - Lists all character matrices with metadata +- [x] Matrix detail page (`search/study/matrix.html`) - Shows matrix row data +- [x] Taxa list page (`search/study/taxa.html`) - Shows taxon labels with NCBI and uBio taxonomy links +- [x] Analyses list page (`search/study/analyses.html`) - Shows analysis steps, software, and data connections + +## Navigation Flow + +```mermaid +flowchart TD + A[Reviewer receives URL
with x-access-code] --> B{First Visit?} + B -->|Yes| C[Reviewer Access
Agreement Popup] + C -->|Accept| D[Study Summary Page] + C -->|Cancel| Z[Access Denied] + B -->|No| D + + D --> E[Navigation Menu] + E --> F[Citation/Summary] + E --> G[Taxa] + E --> H[Matrices] + E --> I[Trees] + E --> J[Analyses] + + F --> K[View Authors] + F --> L[View Abstract] + F --> M[Download Options
Nexus/NeXML] + + G --> N[Taxon Labels List] + N --> O[NCBI Taxonomy Links] + N --> P[uBio Links] + + H --> Q[Matrix List] + Q --> R[Matrix Detail View] + Q --> S[Download Matrix
Nexus/NeXML] + Q --> T[View Taxa for Matrix] + + I --> U[Tree List] + U --> V[Tree Viewer
phylotree.js] + U --> W[Download Tree
Nexus/NeXML] + U --> X[View Taxa for Tree] + + J --> Y[Analysis Steps] + Y --> AA[Software Info] + Y --> AB[Input/Output Data Links] +``` + +## Review Workflow + +The reviewer navigates through study data using the x-access-code URL parameter which grants temporary read-only access to unpublished submissions. + +### Step 1: Initial Access & Agreement +1. Reviewer receives URL with `x-access-code` parameter from journal editor or submitter +2. URL format: `/treebase-web/phylows/study/TB2:S{id}?x-access-code={token}&format=html` +3. On first access, a modal agreement dialog appears requiring acceptance of: + - Confidentiality of data + - Agreement not to retain data after review + - Agreement not to use data for own research until published + - Agreement to keep URL confidential +4. Clicking "OK" stores session acceptance and allows access +5. A red banner "You are in reviewer mode" appears on all pages + +### Step 2: Study Overview Review +- View study summary page with citation information +- Check publication details (authors, year, journal, title) +- Verify metadata completeness (DOI, PMID, abstract, keywords) +- Review author contact information +- Access BibTeX and RIS reference formats + +### Step 3: Tree Data Review +- Navigate to Trees tab to see all trees in the study +- For each tree, verify: + - Tree label and title + - Tree type (e.g., species tree, gene tree) + - Tree kind (e.g., consensus, bootstrap) + - Number of taxa +- Use interactive tree viewer (phylotree.js) to: + - Visualize tree topology + - Hover over nodes for branch length information + - Verify taxon labels match expected names +- Download trees in Nexus or NeXML format for local verification + +### Step 4: Matrix Data Review +- Navigate to Matrices tab +- For each matrix, review: + - Matrix title and description + - Data type (DNA, RNA, protein, morphological, etc.) + - Number of taxa (NTAX) and characters (NCHAR) +- Access matrix row view to check: + - Character data completeness + - Taxon label consistency +- Download matrices for local software verification + +### Step 5: Taxonomic Review +- Navigate to Taxa tab (accessible from study level or per-tree/matrix) +- Verify taxon labels are properly mapped +- Check NCBI Taxonomy ID links for validation +- Check uBio NameBank ID links +- Flag any nomenclatural issues or unmapped labels + +### Step 6: Analysis Review +- Navigate to Analyses tab +- For each analysis, verify: + - Software used (name and version) + - Algorithm type (parsimony, likelihood, Bayesian, etc.) + - Command strings (PAUP blocks, MrBayes commands) + - Input data (matrices) linked correctly + - Output data (trees) linked correctly +- Ensure analysis steps are complete and reproducible + +## Pages to Account For + +| Page | URL Pattern | Status | +|------|-------------|--------| +| Study Summary | `/search/study/summary.html?id={studyId}` | ✅ Implemented | +| Trees List | `/search/study/trees.html?id={studyId}` | ✅ Implemented | +| Tree Viewer | `/search/study/tree.html?treeid={treeId}` | ✅ Implemented | +| Matrices List | `/search/study/matrices.html?id={studyId}` | ✅ Implemented | +| Matrix Detail | `/search/study/matrix.html?id={studyId}&matrixid={matrixId}` | ✅ Implemented | +| Taxa List | `/search/study/taxa.html?id={studyId}` | ✅ Implemented | +| Taxa for Tree | `/search/study/taxa.html?id={studyId}&treeid={treeId}` | ✅ Implemented | +| Taxa for Matrix | `/search/study/taxa.html?id={studyId}&matrixid={matrixId}` | ✅ Implemented | +| Analyses List | `/search/study/analyses.html?id={studyId}` | ✅ Implemented | +| Analysis Detail | `/search/study/analysis.html?id={studyId}&analysisid={analysisId}` | ✅ Implemented | +| Download Study (Nexus) | `/phylows/study/TB2:S{id}?format=nexus&x-access-code={token}` | ✅ Implemented | +| Download Study (NeXML) | `/phylows/study/TB2:S{id}?format=nexml&x-access-code={token}` | ✅ Implemented | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- What review statuses are needed? +- How should reviewer assignments work? +- What communication happens between reviewer and submitter? +- Can reviews be collaborative (multiple reviewers)? + +## Technical Implementation Notes + +### Access Control +- Access tokens are generated from `study.namespacedGUID.hashedIDString` +- Token is passed via `x-access-code` URL parameter +- Token is stored in session after first successful access +- Constant defined in `Constants.java`: `X_ACCESS_CODE = "x-access-code"` + +### Key Files +- `nav.jsp` - Navigation and reviewer agreement popup +- `PhyloWSController.java` - Handles access code validation and URL generation +- `submissionSummaryView.jsp` - Shows reviewer access URL to submitters +- `Constants.java` - Defines access code constant From 163bcc968c3bf7297514632c44079146f0b16279 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 21:08:45 +0000 Subject: [PATCH 38/52] Address code review feedback in user-story-04-review.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-04-review.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/v3/user-story-04-review.md b/doc/v3/user-story-04-review.md index 9329d62ae..ea20c2a54 100644 --- a/doc/v3/user-story-04-review.md +++ b/doc/v3/user-story-04-review.md @@ -137,7 +137,7 @@ The reviewer navigates through study data using the x-access-code URL parameter ## Pages to Account For | Page | URL Pattern | Status | -|------|-------------|--------| +| ---- | ----------- | ------ | | Study Summary | `/search/study/summary.html?id={studyId}` | ✅ Implemented | | Trees List | `/search/study/trees.html?id={studyId}` | ✅ Implemented | | Tree Viewer | `/search/study/tree.html?treeid={treeId}` | ✅ Implemented | @@ -165,7 +165,8 @@ The reviewer navigates through study data using the x-access-code URL parameter ## Technical Implementation Notes ### Access Control -- Access tokens are generated from `study.namespacedGUID.hashedIDString` +- Access tokens are MD5 hashes generated from the study's namespaced GUID (e.g., "TB2:S1234") +- The token is obtained via `study.namespacedGUID.getHashedIDString()` method in Java, or `${submission.study.namespacedGUID.hashedIDString}` in JSP - Token is passed via `x-access-code` URL parameter - Token is stored in session after first successful access - Constant defined in `Constants.java`: `X_ACCESS_CODE = "x-access-code"` From 8630801dc3715958e9a12f58ee9db3bdaecf4994 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 22:11:18 +0100 Subject: [PATCH 39/52] Revise admin user story with new management features Updated the admin user story to include additional management functions and pages. --- doc/v3/user-story-05-admin.md | 92 ++++++++++------------------------- 1 file changed, 26 insertions(+), 66 deletions(-) diff --git a/doc/v3/user-story-05-admin.md b/doc/v3/user-story-05-admin.md index 3552d0812..6cc85272a 100644 --- a/doc/v3/user-story-05-admin.md +++ b/doc/v3/user-story-05-admin.md @@ -3,7 +3,7 @@ ## Overview **As an** administrator, -**I want to** administer the submission and review queue, +**I want to** administer studies and users, **So that** I can ensure efficient processing of phylogenetic data submissions. ## User Types @@ -20,18 +20,21 @@ *TODO: Identify and list all current pages that apply to this user story* -- [ ] Admin dashboard -- [ ] Submission queue management -- [ ] Review queue management -- [ ] User management -- [ ] Reviewer assignment interface -- [ ] System settings -- [ ] Reports and analytics -- [ ] Audit logs +- [ ] Admin landing page +- [ ] Study management for a user +- [ ] Management of ready studies +- [ ] Search submissions +- [ ] Select studies +- [ ] Select users +- [ ] Update user info +- [ ] Delete user +- [ ] Merge users +- [ ] Select persons +- [ ] Merge persons ## Navigation Flow -*TODO: Document how users navigate through these pages* +*TODO: Document how users navigate through these pages in a mermaid plot* ``` [Admin Dashboard] --> [Submission Queue] @@ -52,67 +55,24 @@ ## Administration Functions ### Submission Queue Management + +*Todo: identify all pages and their respective form fields for the operations* + - View all pending submissions -- Filter by status, date, submitter -- Prioritize submissions -- Assign to review track +- Filter by status, submitter +- Search submissions - Bulk actions -### Review Queue Management -- View all reviews in progress -- Monitor review timelines -- Reassign reviewers -- Escalate overdue reviews -- Track completion rates +### User Management -### Reviewer Assignment -- View available reviewers -- Match expertise to submissions -- Balance reviewer workload -- Set review deadlines +*Todo: identify all pages and their respective form fields for the operations* -### User Management -- View all users -- Edit user profiles -- Manage user roles (submitter, reviewer, admin) -- Activate/deactivate accounts -- Reset passwords - -### System Settings -- Configure submission requirements -- Set review workflows -- Manage notification templates -- Configure validation rules - -### Reports and Analytics -- Submission statistics -- Review turnaround times -- User activity reports -- Data growth metrics - -### Audit Logs -- User login history -- Data modification history -- Administrative actions -- Security events - -## Admin Features - -### Queue Operations -- Sort and filter options -- Bulk status updates -- Quick actions menu -- Search within queues - -### Communication -- Send notifications to users -- Broadcast announcements -- Email templates - -### Monitoring -- Dashboard widgets -- Alert configurations -- Status indicators +- Select users +- Update user info +- Delete user +- Merge users +- Select persons +- Merge persons ## Pages to Account For From 54313fc7af191b2701e8dca7b876bf9aada43889 Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 22:16:26 +0100 Subject: [PATCH 40/52] Update User Story 4 review status to completed --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 010f19bfc..4f62fe959 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -29,7 +29,7 @@ The v3 roadmap aims to: ### Reviewers -- [ ] **[User Story 4: Review](user-story-04-review.md)** - A reviewer visits TreeBASE to review a +- [x] **[User Story 4: Review](user-story-04-review.md)** - A reviewer visits TreeBASE to review a study with tree, matrix, taxon and analysis data ### Administrators From fdc3d9339dc81a9c4f22c19ead023e1cdbc62fe6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 21:19:37 +0000 Subject: [PATCH 41/52] Initial plan From 48c0407d4820d7076387a6aaa36dc86fcf8e354d Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 22:22:53 +0100 Subject: [PATCH 42/52] Revise next steps and clean up documentation Updated next steps and removed related documentation section. --- doc/v3/README.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index 4f62fe959..ea8935601 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -51,13 +51,8 @@ The v3 roadmap aims to: After these user stories are complete and all current pages are accounted for: -1. Create wireframes/mockups for each page -2. Define widgets and interactions -3. Design the updated navigation structure -4. Plan the implementation phases +1. Create wireframes, mockups, widgets and interactions for each user story +2. Design the updated house style and design principles +3. Plan the implementation phases +4. Phased implementation of user stories -## Related Documentation - -- [Main Documentation](../README.md) -- [API Documentation](../API.md) -- [OAI-PMH Documentation](../OAI-PMH.md) From a656c7357469fc9abe93e2cb4dc18955bc2654e7 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 21:38:34 +0000 Subject: [PATCH 43/52] Address TODO items in user-story-05-admin.md with comprehensive admin documentation Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-05-admin.md | 256 +++++++++++++++++++++++++++------- 1 file changed, 203 insertions(+), 53 deletions(-) diff --git a/doc/v3/user-story-05-admin.md b/doc/v3/user-story-05-admin.md index 6cc85272a..b73860b2b 100644 --- a/doc/v3/user-story-05-admin.md +++ b/doc/v3/user-story-05-admin.md @@ -9,78 +9,227 @@ ## User Types - System administrators -- TreeBASE curators with admin privileges -- Queue managers +- TreeBASE curators with admin privileges (Admin role) +- Associate Editors (Associate Editor role) ## Prerequisites -- User must be logged in with administrator privileges +- User must be logged in with Admin or Associate Editor role +- Access is controlled via Spring Security configuration in `treebase-security.xml` ## Current Pages -*TODO: Identify and list all current pages that apply to this user story* - -- [ ] Admin landing page -- [ ] Study management for a user -- [ ] Management of ready studies -- [ ] Search submissions -- [ ] Select studies -- [ ] Select users -- [ ] Update user info -- [ ] Delete user -- [ ] Merge users -- [ ] Select persons -- [ ] Merge persons +The following pages are involved in the administration workflow: + +- [x] Admin landing page (`administrationPage.jsp`) - Main dashboard with links to Study Management and User Management menus +- [x] Study management for a user (`simpleUserManagement.jsp`) - List studies by submitter with filters +- [x] Management of ready studies (`changeStudyStatus.jsp`) - View and change status of Ready state studies +- [x] Search submissions (`searchBySubmissionID.jsp`) - Search by TreeBASE2, TB1 legacy, or TB2 Study ID +- [x] Select studies (`selectStudies.jsp`) - Select studies by status type +- [x] Select users (`adminSelectUsers.jsp`) - Search users by email, username, last name, or role +- [x] Update user info (`adminUpdatingUserInfo.jsp`) - Enter username to update +- [x] Override user profile (`overrideUserProfile.jsp`) - Modify user details after lookup +- [x] Delete user Step 1 (`adminDeletingUserStepOne.jsp`) - Enter username to delete +- [x] Delete user Step 2 (`adminDeletingUserStepTwo.jsp`) - Confirm deletion +- [x] Merge users (`adminMergingUsers.jsp`) - Merge source user into target user +- [x] Select persons (`adminSelectPersons.jsp`) - Find potential duplicate person records +- [x] Merge persons (`adminMergingPersons.jsp`) - Merge person records (affects user accounts and citations) +- [x] Merge persons Step 2 (`adminMergingPersonsStepTwo.jsp`) - Confirm person merge ## Navigation Flow -*TODO: Document how users navigate through these pages in a mermaid plot* - -``` -[Admin Dashboard] --> [Submission Queue] - | | - | v - | [Assign Reviewer] - | | - v v -[Review Queue] <--> [Monitor Progress] - | | - v v -[User Management] [Reports/Analytics] - | | - v v -[System Settings] [Audit Logs] +```mermaid +flowchart TD + A[Login with Admin/Associate Editor Role] --> B[Administration Page] + + subgraph StudyMgmt["Study Management Menu"] + B --> C[For a User] + B --> D[Ready State Studies] + B --> E[Search by Submission ID] + B --> F[Select Studies] + end + + subgraph UserMgmt["User Management Menu"] + B --> G[Select Users] + B --> H[Update User Info] + B --> I[Delete User] + B --> J[Merge Users] + B --> K[Select Persons] + B --> L[Merge Persons] + end + + C --> M[Change Study Status Page] + D --> M + E --> M + F --> M + + M --> N{Action} + N -->|Change Status| O[Update Status Radio] + N -->|Delete| P[Delete Study] + N -->|View| Q[Study Summary] + + G --> R[User List] + H --> S[Override User Profile] + I --> T[Delete User Step 1] + T --> U[Delete User Step 2] + U --> V[Message After Action] + + J --> W[Merge Users Wizard] + W --> V + + K --> X[Person List - Duplicates] + L --> Y[Merge Persons Step 1] + Y --> Z[Merge Persons Step 2] + Z --> V ``` ## Administration Functions -### Submission Queue Management +### Study Management + +#### For a User (`/admin/userManagement.html`) +List all studies submitted by a particular user. + +| Field | Type | Options/Values | +|-------|------|----------------| +| Search by | Radio | Email Address, Username (default), Last Name | +| User info | Text | Search term matching selected criteria | +| Study type | Dropdown | All, In Progress, Ready (default), Published | + +#### Ready State Studies (`/admin/readyStateStudies.html`) +Shows all studies currently in "Ready" state for curation review. -*Todo: identify all pages and their respective form fields for the operations* +#### Search by Submission ID (`/admin/searchBySubmissionID.html`) +Search for submissions using various identifier types. -- View all pending submissions -- Filter by status, submitter -- Search submissions -- Bulk actions +| Field | Type | Options/Values | +|-------|------|----------------| +| Identifier Type | Radio | TreeBASE2 Submission ID (default), TreeBASE1 Legacy Study ID, TreeBase2 Study ID | +| Submission Accession | Text | The ID value to search (max 25 chars) | + +#### Select Studies (`/admin/selectStudies.html`) +Filter studies by their status. + +| Field | Type | Options/Values | +|-------|------|----------------| +| Study type | Dropdown | All, In Progress, Ready (default), Published | + +#### Change Study Status (`/admin/changeStudyStatus.html`) +View and modify study statuses. Displayed as a table with columns: +- ID (link to study summary) +- Submitter (email link) +- Study Name +- Study Notes +- Created Date +- Last Modified Date +- Change Status (radio buttons: In Progress, Ready, Published) +- Delete link ### User Management -*Todo: identify all pages and their respective form fields for the operations* +#### Select Users (`/admin/adminSelectUsers.html`) +Search for users based on various criteria. -- Select users -- Update user info -- Delete user -- Merge users -- Select persons -- Merge persons +| Field | Type | Options/Values | +|-------|------|----------------| +| Search by | Radio | Email Address, Username (default), Last Name, User Role | +| User info | Text | Search term (for Email/Username/Last Name) | +| User Role | Dropdown | (Available roles from system, e.g., Admin, User, Associate Editor) | -## Pages to Account For +#### Update User Info (`/admin/adminUpdatingUserInfo.html`) +Enter a username to look up and modify. + +| Field | Type | Description | +|-------|------|-------------| +| Username | Text | Username of the account to update | + +Redirects to Override User Profile page on success. -*TODO: Complete inventory of pages related to administration* +#### Delete User (`/admin/adminDeletingUserStepOne.html` → `adminDeletingUserStepTwo.html`) +Two-step process to delete a user account. + +**Step 1:** +| Field | Type | Description | +|-------|------|-------------| +| Username | Text | Username of the account to delete | + +**Step 2:** Confirmation page before deletion. + +#### Merge Users (`/admin/adminMergingUsers.html`) +Combine two user accounts into one. + +| Field | Type | Description | +|-------|------|-------------| +| Source Username | Text | User account to be merged (will be removed) | +| Target Username | Text | User account to receive merged data | + +#### Select Persons (`/admin/adminSelectPersons.html`) +Audit tool to find potential duplicate person records. + +| Field | Type | Options/Values | +|-------|------|----------------| +| Search criteria | Radio | With Same First and Last Name (default), With Same Last Name | + +#### Merge Persons (`/admin/adminMergingPersons.html` → `adminMergingPersonsStepTwo.html`) +Merge duplicate person records. This affects: +- User accounts associated with the person records +- Citation author/editor records + +**Step 1:** +| Field | Type | Description | +|-------|------|-------------| +| Source Person ID | Text | ID of person record to be merged (will be deleted) | +| Target Person ID | Text | ID of person record to receive merged data | + +**Step 2:** Confirmation page showing what will be affected. + +## Pages to Account For -| Page | URL Pattern | Status | -|------|-------------|--------| -| | | | +| Page | URL Pattern | JSP File | Status | +|------|-------------|----------|--------| +| Administration Landing | `/admin/administrationPage.html` | `administrationPage.jsp` | ✅ Implemented | +| User Management (Studies) | `/admin/userManagement.html` | `simpleUserManagement.jsp` | ✅ Implemented | +| Ready State Studies | `/admin/readyStateStudies.html` | `changeStudyStatus.jsp` | ✅ Implemented | +| Search by Submission ID | `/admin/searchBySubmissionID.html` | `searchBySubmissionID.jsp` | ✅ Implemented | +| Select Studies | `/admin/selectStudies.html` | `selectStudies.jsp` | ✅ Implemented | +| Change Study Status | `/admin/changeStudyStatus.html` | `changeStudyStatus.jsp` | ✅ Implemented | +| Select Users | `/admin/adminSelectUsers.html` | `adminSelectUsers.jsp` | ✅ Implemented | +| User List | `/admin/userList.html` | `userList.jsp` | ✅ Implemented | +| Update User Info | `/admin/adminUpdatingUserInfo.html` | `adminUpdatingUserInfo.jsp` | ✅ Implemented | +| Override User Profile | `/admin/overrideUserProfile.html` | `overrideUserProfile.jsp` | ✅ Implemented | +| Delete User Step 1 | `/admin/adminDeletingUserStepOne.html` | `adminDeletingUserStepOne.jsp` | ✅ Implemented | +| Delete User Step 2 | `/admin/adminDeletingUserStepTwo.html` | `adminDeletingUserStepTwo.jsp` | ✅ Implemented | +| Merge Users | `/admin/adminMergingUsers.html` | `adminMergingUsers.jsp` | ✅ Implemented | +| Select Persons | `/admin/adminSelectPersons.html` | `adminSelectPersons.jsp` | ✅ Implemented | +| Person List | `/admin/personList.html` | `personList.jsp` | ✅ Implemented | +| Merge Persons | `/admin/adminMergingPersons.html` | `adminMergingPersons.jsp` | ✅ Implemented | +| Merge Persons Step 2 | `/admin/adminMergingPersons.html` | `adminMergingPersonsStepTwo.jsp` | ✅ Implemented | +| Message After Action | `/admin/messageToAdminAfterAction.html` | `messageToAdminAfterAction.jsp` | ✅ Implemented | + +## Technical Implementation Notes + +### Access Control +- Admin pages are protected by Spring Security (`treebase-security.xml`) +- URL pattern `/admin/**` requires `Admin` or `Associate Editor` role +- Menu visibility controlled via `menu-config.xml` with `roles="Admin,Associate Editor"` + +### Key Configuration Files +- `menu-config.xml` - Defines Study Management and User Management menus +- `treebase-servlet.xml` - URL to controller mappings (lines 1020-1037) +- `treebase-security.xml` - Security intercept rules + +### Controllers +- `userManagementController` - For a User page +- `selectStudiesController` - Select Studies page +- `changeStudyStatusController` - Ready State and Change Status pages +- `searchBySubmissionIDController` - Search by Submission ID +- `adminSelectUsersController` - Select Users +- `adminUpdatingUserInfoController` - Update User Info +- `adminOverridingUserFormController` - Override User Profile +- `adminDeletingUserStepOneController` / `adminDeletingUserStepTwoController` - Delete User +- `adminMergingUsersController` - Merge Users +- `adminSelectPersonsController` - Select Persons +- `adminMergingPersonsController` - Merge Persons ## Wireframe Notes @@ -88,7 +237,8 @@ ## Open Questions -- What admin roles/permission levels are needed? +- What admin roles/permission levels are needed beyond Admin and Associate Editor? - What metrics should be tracked on the dashboard? -- How should escalation workflows work? -- What audit requirements exist? +- How should escalation workflows work for stuck submissions? +- What audit logging requirements exist for admin actions? +- Should there be bulk operations for study status changes? From ce244960b83d095a2df0a3e753c634b358e06dc2 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 21:39:40 +0000 Subject: [PATCH 44/52] Mark user-story-05-admin.md as completed in README.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/v3/README.md b/doc/v3/README.md index ea8935601..b2cfc9999 100644 --- a/doc/v3/README.md +++ b/doc/v3/README.md @@ -34,7 +34,7 @@ The v3 roadmap aims to: ### Administrators -- [ ] **[User Story 5: Administration](user-story-05-admin.md)** - An admin visits TreeBASE to +- [x] **[User Story 5: Administration](user-story-05-admin.md)** - An admin visits TreeBASE to administer the submission and review queue ### Stakeholders From bc45f018ab3a66741edf362a6795903adc49846d Mon Sep 17 00:00:00 2001 From: Rutger Vos Date: Mon, 12 Jan 2026 22:44:07 +0100 Subject: [PATCH 45/52] Revise user roles and prerequisites in documentation Updated user roles and prerequisites for admin access. --- doc/v3/user-story-05-admin.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/v3/user-story-05-admin.md b/doc/v3/user-story-05-admin.md index b73860b2b..bfc8c17f4 100644 --- a/doc/v3/user-story-05-admin.md +++ b/doc/v3/user-story-05-admin.md @@ -10,11 +10,10 @@ - System administrators - TreeBASE curators with admin privileges (Admin role) -- Associate Editors (Associate Editor role) ## Prerequisites -- User must be logged in with Admin or Associate Editor role +- User must be logged in with Admin role - Access is controlled via Spring Security configuration in `treebase-security.xml` ## Current Pages From ee81cbd640d458d076bae456921ccecc47a8627c Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 22:24:44 +0000 Subject: [PATCH 46/52] Initial plan From 4531b5c64c1166ee64272f2f96b959c357a91b80 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 22:32:06 +0000 Subject: [PATCH 47/52] Add row segment functionality to user story 03 submission documentation Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-03-submission.md | 611 +++++++++++++++++++++++++++++ 1 file changed, 611 insertions(+) create mode 100644 doc/v3/user-story-03-submission.md diff --git a/doc/v3/user-story-03-submission.md b/doc/v3/user-story-03-submission.md new file mode 100644 index 000000000..632af4f50 --- /dev/null +++ b/doc/v3/user-story-03-submission.md @@ -0,0 +1,611 @@ +# User Story 3: Data Submission + +## Overview + +**As a** registered user, +**I want to** submit study, tree, matrix, taxon or analysis data, +**So that** I can share my phylogenetic research with the scientific community. + +## User Types + +- Researchers submitting their own studies +- Students submitting work on behalf of advisors +- Data curators submitting historical data + +## Prerequisites + +- User must be logged in (see [User Story 2: Account Management](user-story-02-account.md)) + +## Current Pages + +The following pages apply to this user story: + +- [x] Submission dashboard (`/user/submissionList.html` → `submissionList.jsp`) +- [x] New submission form (`/user/studyForm.html?form=new` → `studyForm.jsp`) +- [x] Study summary/metadata (`/user/summary.html` → `submissionSummaryView.jsp`) +- [x] Citation form (`/user/citationForm.html` → `citationForm.jsp`) +- [x] Author search/add (`/user/authorSearchForm.html`, `/user/addAuthor.html` → `peopleSearchForm.jsp`, `addPersonForm.jsp`) +- [x] File upload interface (`/user/uploadFile.html` → `uploadFile.jsp`) +- [x] Upload summary (`/user/uploadFileSummary.html` → `uploadFileSummary.jsp`) +- [x] Tree blocks list (`/user/treeBlockList.html` → `treeBlockList.jsp`) +- [x] Tree list/editor (`/user/treeList.html` → `treeList.jsp`) +- [x] Tree viewer/editor (`/user/directMapToPhyloWidget.html` → `treeViewer.jsp`) +- [x] Matrix list/editor (`/user/matrixList.html` → `matrixList.jsp`) +- [x] Matrix row list (`/user/matrixRowList.html` → `matrixRowList.jsp`) +- [x] Row segment list for matrix row (`/user/matrixRowSegmentList.html` → `matrixRowSegmentList.jsp`) +- [x] Row segment form (`/user/matrixRowSegmentForm.html` → `matrixRowSegmentForm.jsp`) +- [x] Upload row segment data (`/user/uploadRowSegmentData.html` → `uploadRowSegmentData.jsp`) +- [x] Row segment data table (`/user/rowSegmentDataTable.html` → `rowSegmentDataTable.jsp`) +- [x] View all row segment data (`/user/viewAllRowSegmentData.html` → `viewAllRowSegmentData.jsp`) +- [x] Export row segment data (`/user/exportRowSegmentData.html` → file download) +- [x] Export row segment template (`/user/exportRowSegmentTemplate.html` → file download) +- [x] Delete row segment (`/user/deleteARowSegment.html` → `generalDeletePage.jsp`) +- [x] Taxa management (`/user/taxaList.html` → `taxonLabels.jsp`) +- [x] Edit taxon label (`/user/editTaxonLabel.html` → `editTaxonLabel.jsp`) +- [x] Analysis display (`/user/analysisDisplay.html` → `analysisList.jsp`) +- [x] Analysis form (`/user/analysisForm.html` → `analysisForm.jsp`) +- [x] Analysis step form (`/user/analysisStepForm.html` → `analysisStepForm.jsp`) +- [x] Analyzed data form (`/user/analyzedDataForm.html` → `analyzedDataForm.jsp`) +- [x] Ready state confirmation (`/user/readyState.html` → `readyState.jsp`) +- [x] Delete study (`/user/deleteStudy.html` → `deleteStudy.jsp`) +- [x] Submission tutorial (`/submitTutorial.html` → `submitTutorial.jsp`) + +## Navigation Flow + +The following diagram documents how users navigate through the submission pages: + +```mermaid +flowchart TD + subgraph Dashboard["Submission Dashboard"] + SubmitList["/user/submissionList.html
My Submissions
(submissionList.jsp)"] + end + + subgraph Create["Create Submission"] + NewStudy["/user/studyForm.html?form=new
New Submission Form
(studyForm.jsp)"] + end + + subgraph StudyMeta["Study Metadata"] + Summary["/user/summary.html
Submission Summary
(submissionSummaryView.jsp)"] + Citation["/user/citationForm.html
Citation Form
(citationForm.jsp)"] + AuthorSearch["/user/authorSearchForm.html
Author Search
(peopleSearchForm.jsp)"] + AddAuthor["/user/addAuthor.html
Add Author
(addPersonForm.jsp)"] + end + + subgraph Upload["File Upload"] + UploadFile["/user/uploadFile.html
NEXUS File Upload
(uploadFile.jsp)"] + UploadSummary["/user/uploadFileSummary.html
Upload Summary
(uploadFileSummary.jsp)"] + end + + subgraph Trees["Tree Management"] + TreeBlocks["/user/treeBlockList.html
Tree Blocks
(treeBlockList.jsp)"] + TreeList["/user/treeList.html
Trees in Block
(treeList.jsp)"] + TreeViewer["/user/directMapToPhyloWidget.html
Tree Viewer
(treeViewer.jsp)"] + end + + subgraph Matrices["Matrix Management"] + MatrixList["/user/matrixList.html
Matrices
(matrixList.jsp)"] + MatrixRows["/user/matrixRowList.html
Matrix Rows
(matrixRowList.jsp)"] + end + + subgraph RowSegments["Row Segment Management"] + RowSegmentList["/user/matrixRowSegmentList.html
Row Segments for Row
(matrixRowSegmentList.jsp)"] + RowSegmentForm["/user/matrixRowSegmentForm.html
Edit Row Segment
(matrixRowSegmentForm.jsp)"] + UploadRowSegment["/user/uploadRowSegmentData.html
Upload Row Segment Data
(uploadRowSegmentData.jsp)"] + RowSegmentDataTable["/user/rowSegmentDataTable.html
Map Columns
(rowSegmentDataTable.jsp)"] + ViewAllRowSegments["/user/viewAllRowSegmentData.html
All Row Segments
(viewAllRowSegmentData.jsp)"] + ExportRowSegment["/user/exportRowSegmentData.html
Export TSV
(file download)"] + ExportTemplate["/user/exportRowSegmentTemplate.html
Export Template
(file download)"] + DeleteRowSegment["/user/deleteARowSegment.html
Delete Row Segment
(generalDeletePage.jsp)"] + end + + subgraph Taxa["Taxon Management"] + TaxaList["/user/taxaList.html
Taxon Labels
(taxonLabels.jsp)"] + EditTaxon["/user/editTaxonLabel.html
Edit Taxon
(editTaxonLabel.jsp)"] + end + + subgraph Analysis["Analysis Management"] + AnalysisList["/user/analysisDisplay.html
Analyses
(analysisList.jsp)"] + AnalysisForm["/user/analysisForm.html
Analysis Form
(analysisForm.jsp)"] + AnalysisStep["/user/analysisStepForm.html
Analysis Step
(analysisStepForm.jsp)"] + AnalyzedData["/user/analyzedDataForm.html
Analyzed Data
(analyzedDataForm.jsp)"] + end + + subgraph Submit["Review & Submit"] + ReadyState["/user/readyState.html
Change to Ready State
(readyState.jsp)"] + DeleteStudy["/user/deleteStudy.html
Delete Study
(deleteStudy.jsp)"] + end + + %% Main flow + SubmitList -->|"New Submission"| NewStudy + NewStudy -->|"Submit"| Summary + SubmitList -->|"Edit submission"| Summary + + %% Summary navigation (via Tool Box menu) + Summary -->|"Citation"| Citation + Summary -->|"Upload"| UploadFile + Summary -->|"Trees"| TreeBlocks + Summary -->|"Matrices"| MatrixList + Summary -->|"Taxa"| TaxaList + Summary -->|"Analysis"| AnalysisList + + %% Citation flow + Citation -->|"Add/search authors"| AuthorSearch + AuthorSearch -->|"Add new"| AddAuthor + AddAuthor --> AuthorSearch + Citation -->|"Done"| Summary + + %% Upload flow + UploadFile -->|"Upload complete"| UploadSummary + UploadSummary -->|"Continue"| Summary + + %% Tree flow + TreeBlocks -->|"View trees"| TreeList + TreeList -->|"Edit tree"| TreeViewer + TreeViewer --> TreeList + + %% Matrix flow + MatrixList -->|"View rows"| MatrixRows + MatrixRows -->|"View segments"| RowSegmentList + + %% Row Segment flow (individual row) + RowSegmentList -->|"Add segment"| RowSegmentForm + RowSegmentList -->|"Edit segment"| RowSegmentForm + RowSegmentForm -->|"Done"| RowSegmentList + + %% Row Segment bulk operations flow + MatrixList -->|"Upload row segment data"| UploadRowSegment + UploadRowSegment -->|"Upload complete"| RowSegmentDataTable + RowSegmentDataTable -->|"Submit"| ViewAllRowSegments + ViewAllRowSegments -->|"Export"| ExportRowSegment + ViewAllRowSegments -->|"Upload more"| UploadRowSegment + ViewAllRowSegments -->|"Delete segment"| DeleteRowSegment + ViewAllRowSegments -->|"Back to rows"| MatrixRows + MatrixList -->|"Export template"| ExportTemplate + DeleteRowSegment -->|"Confirmed"| ViewAllRowSegments + + %% Taxa flow + TaxaList -->|"Edit taxon"| EditTaxon + EditTaxon --> TaxaList + TaxaList -->|"Validate labels"| TaxaList + + %% Analysis flow + AnalysisList -->|"Add analysis"| AnalysisForm + AnalysisForm -->|"Add step"| AnalysisStep + AnalysisStep -->|"Add data"| AnalyzedData + AnalyzedData --> AnalysisStep + AnalysisStep --> AnalysisForm + AnalysisForm --> AnalysisList + + %% Submission status + SubmitList -->|"Change status"| ReadyState + ReadyState -->|"Success"| SubmitList + SubmitList -->|"Delete"| DeleteStudy + DeleteStudy -->|"Confirmed"| SubmitList +``` + +## Submission Workflow + +### Step 1: Study Information + +Study information is recorded on multiple pages managed by `StudyFormController` and `CitationFormController`: + +**Study Form** (`/user/studyForm.html` → `studyForm.jsp`): +- **Study Name** (required) - Brief title, usually same as publication title +- **Study Notes** - Private notes for submitter/staff communication, not public + +**Citation Form** (`/user/citationForm.html` → `citationForm.jsp`): +- **Citation Type** (required) - Article, Book, or Book Section +- **Publish Year** (required) - Year of publication (1900 to current+1) +- **Publication Status** - Status of the publication + +For Article citations (`citationForm-article.jsp`): +- **Article Title** (required) +- **Keywords** - Comma-separated keywords +- **Abstract** - Publication abstract +- **PubMed ID** (PMID) - Maximum 10 characters +- **DOI** - Digital Object Identifier +- **URL** - Link to publication +- **Journal** (required) - Autocomplete from existing journals +- **Volume** +- **Issue** +- **Pages** - In format like "11-28" + +**Author Management** (`/user/authorSearchForm.html`, `/user/addAuthor.html`): +- Search for existing authors or add new ones +- Authors linked to citation via `AuthorFormController` + +### Step 2: Data Upload + +File upload is handled by `UploadFileController` (`/user/uploadFile.html` → `uploadFile.jsp`): + +**Supported Formats:** +- **NEXUS** - Primary format, parsed by Mesquite +- Files must be parseable by Mesquite before upload + +**Upload Interface:** +- Multiple file upload support via "Attach another file" +- Progress bar during upload +- **Maximum upload size**: 250MB (configured in `treebase-servlet.xml`) + +**Upload Constraints:** +- Only first ~30 trees are parsed to avoid overwhelming the interface +- Users should place preferred/consensus trees first in the tree block + +**Upload Summary** (`/user/uploadFileSummary.html` → `uploadFileSummary.jsp`): +- Shows count of matrices uploaded +- Shows count of trees uploaded +- Parser log available via "Show Parser Log" button + +### Step 3: Tree Data + +Tree management is handled by `ListTreeBlockController` and `ListTreeController`: + +**Tree Block List** (`/user/treeBlockList.html` → `treeBlockList.jsp`): +- **Block Title** - Editable title for the tree block +- **Tree Count** - Number of trees in the block +- **Analysis Status** - Icon showing if block is linked to an analysis +- Actions: View trees, View in PhyloWidget, Download, Delete + +**Tree List** (`/user/treeList.html` → `treeList.jsp`): +- **Tree Label** - Editable label +- **Tree Title** - Editable title +- **Tree Type** - Dropdown selection (e.g., Consensus, Single) +- **Tree Kind** - Dropdown selection (e.g., Species Tree, Gene Tree) +- **Tree Quality** - Dropdown selection +- **Analysis Status** - Icon showing if tree is linked to an analysis +- Actions: Edit in PhyloWidget, Download reconstructed, Download original NEXUS, Delete + +**Tree Viewer** (`/user/directMapToPhyloWidget.html` → `treeViewer.jsp`): +- PhyloWidget-based tree visualization and editing +- Opens in popup window (1000x900 pixels) + +### Step 4: Matrix Data + +Matrix management is handled by `ListMatrixController`: + +**Matrix List** (`/user/matrixList.html` → `matrixList.jsp`): +- **Matrix Title** - Editable title +- **Matrix Description** - Editable description +- **Matrix Kind** - Dropdown selection (e.g., DNA, Protein, Morphological) +- **Analysis Status** - Icon showing if matrix is linked to an analysis +- Actions: View rows, Download reconstructed, Download original NEXUS, Delete + +**Matrix Row List** (`/user/matrixRowList.html` → `matrixRowList.jsp`): +- View matrix rows +- Navigate to row segments for each row + +### Step 4a: Row Segment Data + +Row segments represent parts of sequences within character state matrices. They allow users to annotate specific portions of matrix rows with specimen information, accession numbers, and geographic data. + +**Example Use Case:** A researcher has a concatenated DNA matrix where positions 1-500 are 16S rRNA and positions 501-1200 are COI. Using row segments, they can annotate these regions separately for each taxon, recording different GenBank accession numbers and specimen voucher codes for each gene fragment. + +**Row segment management provides two approaches:** + +#### Individual Row Segment Entry + +Via `ListMatrixRowSegmentController` and `SingleRowSegmentController`: + +**Row Segment List** (`/user/matrixRowSegmentList.html` → `matrixRowSegmentList.jsp`): +- Lists all row segments for a specific matrix row +- Shows segment title, start index, and end index +- Actions: Add new segment, Edit segment + +**Row Segment Form** (`/user/matrixRowSegmentForm.html` → `matrixRowSegmentForm.jsp`): +- **Taxon Label** - Display only, inherited from matrix row +- **Title** - Segment title/name +- **Start Index** / **End Index** - Character positions defining the segment +- **Segment Data** - Selected portion of the sequence (auto-filled from selection) + +Specimen Label fields: +- **Institution Acronym** - Museum/institution code +- **Collection Code** - Collection identifier +- **Catalog Number** - Specimen catalog number +- **GenBank Accession** - GenBank accession number +- **Other Accession** - Other database accession numbers +- **Sample Date** - Collection date (format: YYYY-MM-DD) +- **Collector** - Name of collector + +Geographic fields: +- **Latitude** / **Longitude** - Decimal coordinates +- **Elevation** - Elevation in meters +- **Country** / **State** / **Locality** - Administrative location + +- **Notes** - Free-text notes about the specimen + +Interactive segment selection: Users can highlight text in the sequence display and click "Select" to auto-populate start/end indices. + +#### Bulk Row Segment Upload + +Via `UploadRowSegmentDataController`, `RowSegmentDataTableController`, and `ViewAllRowSegmentDataController`: + +**Upload Row Segment Data** (`/user/uploadRowSegmentData.html` → `uploadRowSegmentData.jsp`): +- Upload tab-delimited files containing row segment data for multiple rows +- File should contain columns matching row segment fields + +**Row Segment Data Table** (`/user/rowSegmentDataTable.html` → `rowSegmentDataTable.jsp`): +- Preview of uploaded data (first 10 rows displayed) +- Column mapping interface to match file columns to row segment fields +- Available field mappings: + - Taxon Label + - Segment Title + - Start Index / End Index + - Institution Acronym, Collection Code, Catalog Number + - GenBank Accession, Other Accession + - Sample Date, Collector + - Latitude, Longitude, Elevation + - Country, State, Locality + - Notes + - Ignore (skip column) +- Checkbox to indicate if file includes header row +- Validates for duplicate column selections + +**View All Row Segments** (`/user/viewAllRowSegmentData.html` → `viewAllRowSegmentData.jsp`): +- Displays all row segments for the current matrix +- Inline editing of all segment fields +- Bulk update: Modify multiple segments and save changes +- Bulk delete: Select segments via checkboxes and delete +- Links to: + - Export Row Segment Data + - Upload Another Row Segment Data File + - Go Back To Matrix Rows + +**Export Row Segment Data** (`/user/exportRowSegmentData.html`): +- Downloads current row segment data as tab-separated values (TSV) file +- Filename: `RowSegmentData_{matrixId}.tsv` +- Handled by `ExportRowSegmentDataController` + +**Export Row Segment Template** (`/user/exportRowSegmentTemplate.html`): +- Downloads template file with headers for bulk upload +- Filename: `RowSegmentTemplate_{matrixId}.txt` +- Handled by `ExportRowSegmentTemplateController` + +**Delete Row Segment** (`/user/deleteARowSegment.html` → `generalDeletePage.jsp`): +- Confirmation page before deleting individual row segment +- Requires WRITE permission on the submission +- Handled by `DeleteARowSegmentController` + +### Step 5: Taxonomic Data + +Taxon management is handled by `ListTaxaController` and `EditTaxonLabelController`: + +**Taxon Labels List** (`/user/taxaList.html` → `taxonLabels.jsp`): +- **Taxon Label** - Original label from data file +- **Taxon Name** - Mapped scientific name +- **NCBI Tax ID** - Link to NCBI Taxonomy Browser +- **uBio Tax ID** - Link to uBio NameBank +- **Validation Status** - Icon showing if linking was attempted +- Actions: Edit, Bulk update, Validate selected labels + +**Edit Taxon Label** (`/user/editTaxonLabel.html` → `editTaxonLabel.jsp`): +- **Taxon Label** - Editable text, guidelines provided for formatting +- **External Taxonomy** - Radio selection from suggested matches: + - NCBI taxonomy suggestions with IDs + - uBio lookup option with manual NamebankID entry + - "No association" option if no match exists + +**Validation Process:** +- Labels validated against uBio and NCBI taxonomies +- Guidelines: Use full binomials (no abbreviations), separate codes with spaces +- "Validate Taxon Labels" button triggers bulk validation + +### Step 6: Analysis Information + +Analysis management is handled by `AnalysisFormController` and `AnalysisStepFormController`: + +**Analysis Form** (`/user/analysisForm.html` → `analysisForm.jsp`): +- **Analysis Name** - Descriptive name for the analysis +- **Analysis Notes** - Additional notes about the analysis + +**Analysis Step Form** (`/user/analysisStepForm.html` → `analysisStepForm.jsp`): +- **Step Name** - Name for this analysis step +- **Step Notes** - Additional notes +- **Commands** - Actual commands/settings used +- **Software Name** - Autocomplete from known software +- **Software Version** - Version number +- **Software URL** - Link to software +- **Software Description** - Brief description +- **Algorithm Type** - Dropdown: Maximum Likelihood, Parsimony, Distance, Bayesian, Other +- **New Algorithm** - Text field when "Other Algorithm" selected + +**Analyzed Data** (`/user/analyzedDataForm.html` → `analyzedDataForm.jsp`): +- Multi-page wizard for linking inputs/outputs: + 1. Select data type and input/output designation + 2. Select matrices (for input) + 3. Select trees (for output) + 4. Select tree blocks + +### Step 7: Review and Submit + +The review and submission process validates data completeness before allowing status change: + +**Submission Summary** (`/user/summary.html` → `submissionSummaryView.jsp`): +- Displays submission number and status +- Shows study accession URL (permanent identifier) +- Shows reviewer access URL with access code +- Citation summary if entered +- Abstract if provided +- Analysis list embedded + +**Ready State Confirmation** (`/user/readyState.html` → `readyState.jsp`): + +**Pre-submission Validation Checks:** +The following must be satisfied before changing to "Ready State": +1. Citation entered with minimum: authors, year, title, journal name +2. No orphaned matrices (all matrices must be inputs to at least one analysis) +3. No orphaned trees (all trees must be outputs from at least one analysis) +4. Tree taxon labels must match input matrix taxon labels +5. Taxon labels validation attempted (via "Validate Taxon Labels" button) + +**Submission Confirmation:** +- Warning message about read-only status after submission +- Exception: Citation metadata can still be updated (volume, issue, pages, DOI) +- Submit button completes the status change + +## Data Validation + +Data validation occurs at multiple stages in the submission process: + +### File Format Validation + +**At Upload Time:** +- NEXUS files are parsed using Mesquite library +- Invalid syntax results in parse errors shown in Parser Log +- Recommendation: Test files in Mesquite before upload + +**File Size Validation:** +- Maximum upload size: 250MB (configurable in `treebase-servlet.xml`) +- Tree block limit: Only first ~30 trees parsed per block + +### Required Field Validation + +**Study Form:** +- Study Name: Required, validated server-side + +**Citation Form:** +- Citation Type: Required (Article, Book, Book Section) +- Publish Year: Required, validated 1900 ≤ year ≤ current+1 +- Title: Required for articles +- Journal: Required for articles +- At least one author required (validated at Ready State) + +**Analysis Step Form:** +- Algorithm Type validation when "Other" selected - New Algorithm field required + +**Row Segment Form:** +- Start Index: Must be 1 or greater and must not exceed the total sequence length +- End Index: Must be greater than or equal to Start Index and must not exceed the total sequence length +- Segment coordinates validated against matrix row data + +### Taxonomic Name Validation + +**Automated Validation:** +- Triggered by "Validate Taxon Labels" button +- Matches labels against uBio and NCBI taxonomies +- Visual indicators show validation status: + - Green checkmark: Linking attempted + - Red X: Not yet validated + +**Manual Validation:** +- Edit individual labels via Edit Taxon Label page +- Search uBio manually and enter NamebankID +- Select "no association" for unmatched taxa + +### Consistency Checks (at Ready State) + +The system validates the following before allowing status change: + +1. **Citation Completeness:** + - Authors present + - Year present + - Title present + - Journal name present (for articles) + +2. **Data Completeness:** + - No orphaned matrices (each matrix linked to ≥1 analysis as input) + - No orphaned trees (each tree linked to ≥1 analysis as output) + +3. **Taxon Label Matching:** + - Output tree taxon labels must exist in input matrix + - Mismatched labels prevent Ready State + +4. **Validation Attempted:** + - Taxon label validation must be attempted + +**Error Display:** +- Errors shown in red at top of Ready State page +- Tool Box items highlighted in yellow indicate problems +- Hover over highlighted items for error details + +## Pages to Account For + +Complete inventory of pages related to data submission from `/treebase-web/src/main/webapp/WEB-INF/treebase-servlet.xml`: + +| Page | URL Pattern | Controller | View | Status | +|------|-------------|------------|------|--------| +| Submission List | `/user/submissionList.html` | `listSubmissionController` | `submissionList.jsp` | Active | +| New Study Form | `/user/studyForm.html` | `studyFormController` | `studyForm.jsp` | Active | +| Submission Summary | `/user/summary.html` | `summaryController` | `submissionSummaryView.jsp` | Active | +| Citation Form | `/user/citationForm.html` | `citationFormController` | `citationForm.jsp` | Active | +| Author Search | `/user/authorSearchForm.html` | `authorSearchFormController` | `peopleSearchForm.jsp` | Active | +| Add Author | `/user/addAuthor.html` | `addAuthorController` | `addPersonForm.jsp` | Active | +| Upload File | `/user/uploadFile.html` | `uploadFileController` | `uploadFile.jsp` | Active | +| Upload Summary | `/user/uploadFileSummary.html` | `uploadFileSummaryController` | `uploadFileSummary.jsp` | Active | +| Tree Block List | `/user/treeBlockList.html` | `listTreeBlockController` | `treeBlockList.jsp` | Active | +| Tree List | `/user/treeList.html` | `listTreeController` | `treeList.jsp` | Active | +| Tree Viewer | `/user/directMapToPhyloWidget.html` | `directMapToPhyloWidgetController` | `treeViewer.jsp` | Active | +| Delete Tree | `/user/deleteATree.html` | `deleteATreeController` | `generalDeletePage.jsp` | Active | +| Delete Tree Block | `/user/deleteATreeBlock.html` | `deleteATreeBlockController` | `generalDeletePage.jsp` | Active | +| Matrix List | `/user/matrixList.html` | `listMatrixController` | `matrixList.jsp` | Active | +| Matrix Row List | `/user/matrixRowList.html` | `listMatrixRowController` | `matrixRowList.jsp` | Active | +| Row Segment List | `/user/matrixRowSegmentList.html` | `listMatrixRowSegmentController` | `matrixRowSegmentList.jsp` | Active | +| Row Segment Form | `/user/matrixRowSegmentForm.html` | `matrixRowSegmentFormController` | `matrixRowSegmentForm.jsp` | Active | +| Upload Row Segment Data | `/user/uploadRowSegmentData.html` | `uploadRowSegmentDataController` | `uploadRowSegmentData.jsp` | Active | +| Row Segment Data Table | `/user/rowSegmentDataTable.html` | `rowSegmentDataTableController` | `rowSegmentDataTable.jsp` | Active | +| View All Row Segments | `/user/viewAllRowSegmentData.html` | `viewAllRowSegmentDataController` | `viewAllRowSegmentData.jsp` | Active | +| Export Row Segment Data | `/user/exportRowSegmentData.html` | `exportRowSegmentDataController` | (file download) | Active | +| Export Row Segment Template | `/user/exportRowSegmentTemplate.html` | `exportRowSegmentTemplateController` | (file download) | Active | +| Delete Row Segment | `/user/deleteARowSegment.html` | `deleteARowSegmentController` | `generalDeletePage.jsp` | Active | +| Delete Matrix | `/user/deleteAMatrix.html` | `deleteAMatrixController` | `generalDeletePage.jsp` | Active | +| Taxa List | `/user/taxaList.html` | `listTaxaController` | `taxonLabels.jsp` | Active | +| Edit Taxon Label | `/user/editTaxonLabel.html` | `editTaxonLabelController` | `editTaxonLabel.jsp` | Active | +| Edit Set Taxon Labels | `/user/editSetTaxonLabel.html` | `editSetTaxonLabelController` | `editSetTaxonLabel.jsp` | Active | +| Analysis List | `/user/analysisList.html` | `listAnalysisController` | `analysisList.jsp` | Active | +| Analysis Display | `/user/analysisDisplay.html` | `displayAnalysisController` | `analysisList.jsp` | Active | +| Analysis Form | `/user/analysisForm.html` | `analysisFormController` | `analysisForm.jsp` | Active | +| Analysis Step List | `/user/analysisStepList.html` | `listAnalysisStepController` | `analysisStepList.jsp` | Active | +| Analysis Step Form | `/user/analysisStepForm.html` | `analysisStepFormController` | `analysisStepForm.jsp` | Active | +| Analyzed Data List | `/user/analyzedDataList.html` | `listAnalyzedDataController` | `analyzedDataList.jsp` | Active | +| Analyzed Data Form | `/user/analyzedDataForm.html` | `analyzedDataFormController` | `analyzedDataForm.jsp` | Active | +| Add Analyzed Data | `/user/addAnalyzedData.html` | `addAnalyzedDataController` | - | Active | +| Ready State | `/user/readyState.html` | `readyStateController` | `readyState.jsp` | Active | +| Delete Study | `/user/deleteStudy.html` | `deleteStudyController` | `deleteStudy.jsp` | Active | +| Submit Tutorial | `/submitTutorial.html` | `filenameController` | `submitTutorial.jsp` | Active | +| Download Tree | `/user/downloadATree.html` | `downloadATreeController` | - | Active | +| Download Tree Block | `/user/downloadATreeBlock.html` | `downloadATreeBlockController` | - | Active | +| Download Matrix | `/user/downloadAMatrix.html` | `downloadAMatrixController` | - | Active | +| Download NEXUS File | `/user/downloadANexusFile.html` | `downloadANexusFileController` | - | Active | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +The following questions have been addressed based on current implementation: + +- **What file formats should be supported?** + - NEXUS is the primary supported format + - Files must be parseable by Mesquite + - TreeBASE does not directly support Newick-only files; they should be embedded in NEXUS format + +- **What is the maximum file size?** + - 250MB maximum upload size (configured in `treebase-servlet.xml`) + - Only first ~30 trees per block are parsed + +- **Can submissions be saved as drafts?** + - Yes, submissions in "In Progress" status function as drafts + - Users can continue editing until they change status to "Ready" + - "In Progress" submissions are not visible to the public + +- **What validation happens at each step vs. at submission?** + - File validation: At upload time (NEXUS parsing) + - Field validation: On form submission (server-side Spring validators) + - Comprehensive validation: When changing to "Ready State" (orphan checks, taxon matching, etc.) + +- **How are row segments used?** + - Row segments annotate portions of character state matrix sequences + - They link specimen information (museum codes, GenBank accessions) to specific sequence regions + - Can be entered individually per row or uploaded in bulk via tab-delimited files + - Templates can be exported to facilitate bulk data entry + +## Security Considerations + +- **File Upload Security**: File uploads handled by `AjaxMultipartResolver` with size limits +- **Access Control**: Submission pages require User, Admin, or Associate Editor role +- **Study Ownership**: Users can only modify their own submissions (enforced in controllers) +- **Status Transitions**: Only study owners and admins can change submission status +- **Read-only After Ready**: Once in "Ready" status, only citation metadata can be updated +- **Delete Confirmation**: Study deletion requires explicit confirmation via `deleteStudy.jsp` confirmation page with warning message about permanent data loss +- **Delete Restrictions**: Studies can only be deleted when in "In Progress" status; all trees and matrices must be deleted first before the study can be removed +- **Row Segment Authorization**: Row segment deletion requires WRITE permission on the containing submission. WRITE permission is granted to the study owner and administrators. Users can verify their permission by checking if edit/delete buttons appear on the row segment pages. From acf7af540dd0c714c99edc4f9d4a77f0194d101b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 22:37:47 +0000 Subject: [PATCH 48/52] Initial plan From c5bbf7117d7ac28fae4e955cae6336228da2d084 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 22:40:14 +0000 Subject: [PATCH 49/52] Initial plan Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-06-technical.md | 281 ++++++++++++++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 doc/v3/user-story-06-technical.md diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md new file mode 100644 index 000000000..a650ad1e5 --- /dev/null +++ b/doc/v3/user-story-06-technical.md @@ -0,0 +1,281 @@ +# User Story 6: Technical Documentation + +## Overview + +**As a** technical stakeholder, +**I want to** learn about the technology of the web app, database, data standards, APIs, or other +tools, +**So that** I can integrate with TreeBASE, contribute to development, or understand its technical +architecture. + +## User Types + +- Software developers integrating with TreeBASE +- API consumers building applications +- Contributors to TreeBASE development +- Researchers using programmatic access +- System administrators deploying TreeBASE + +## Current Pages + +The following JSP pages and documentation files serve technical stakeholders: + +### Web Application Pages (JSP) +- [x] **urlAPI.jsp** (`/urlAPI.html`) - Data Access page with PhyloWS API overview, URI patterns, and RSS feeds +- [x] **technology.jsp** (`/technology.html`) - Implementation technologies and architecture overview +- [x] **about.jsp** (`/about.html`) - Background, history, funding, and related resources +- [x] **contact.jsp** (`/contact.html`) - Helpdesk contact and GitHub issue tracker links +- [x] **submitTutorial.jsp** (`/submitTutorial.html`) - NEXUS file preparation and submission process +- [x] **help.jsp** (`/help.html`) - Dynamic help text system + +### Documentation Files (Markdown) +- [x] **doc/API.md** - Comprehensive PhyloWS API documentation with CQL search syntax +- [x] **doc/OAI-PMH.md** - OAI-PMH metadata harvesting service documentation +- [x] **doc/development/BUILDING.md** - Build prerequisites and instructions (Java 17, Maven) +- [x] **doc/development/DEPLOYING.md** - Tomcat deployment guide with troubleshooting +- [x] **doc/technical-notes/DWR.md** - DWR (Direct Web Remoting) integration for AJAX +- [x] **doc/technical-notes/UPGRADES.md** - Dependency upgrade history (Spring, Jersey, etc.) + +### External Resources +- [ ] GitHub Wiki - API documentation (linked from urlAPI.jsp) - *External, not maintained in repository* +- [ ] GitHub Wiki - OAI-PMH documentation (linked from urlAPI.jsp) - *External, not maintained in repository* + +## Navigation Flow + +Technical users navigate through documentation pages based on their integration needs: + +```mermaid +flowchart TD + A[Entry Points] --> B{User Goal} + + B -->|API Integration| C["urlAPI.html (Data Access)"] + B -->|Understand System| D["technology.html (Implementation)"] + B -->|Contribute Code| E[GitHub Repository] + B -->|Deploy Instance| F["doc/development/"] + + C --> G["doc/API.md (PhyloWS Reference)"] + C --> H["doc/OAI-PMH.md (Harvesting Interface)"] + + G --> I["phylows/study endpoint"] + G --> J["phylows/tree endpoint"] + G --> K["phylows/matrix endpoint"] + G --> L["phylows/taxon endpoint"] + + D --> M["about.html (Background)"] + D --> N[Architecture Diagram] + + E --> O["doc/development/BUILDING.md"] + E --> P["doc/technical-notes/"] + + F --> O + F --> Q["doc/development/DEPLOYING.md"] + + H --> R[OAI-PMH Verbs] + + style C fill:#e1f5fe + style D fill:#e8f5e9 + style E fill:#fff3e0 + style F fill:#fce4ec +``` + +## Technical Documentation Areas + +### API Documentation + +Available documentation for API consumers: + +| Resource | Location | Description | +|----------|----------|-------------| +| **urlAPI.jsp** | `/urlAPI.html` | Overview of programmatic access, URI patterns, RSS feeds | +| **doc/API.md** | Repository | PhyloWS API reference with CQL search syntax and examples | +| **doc/OAI-PMH.md** | Repository | OAI-PMH harvesting verbs and usage | +| **GitHub Wiki** | External | Extended API documentation (linked from urlAPI.jsp) | + +**PhyloWS Endpoints:** +- `/phylows/study/**` - Study records (PhyloWSStudyController) +- `/phylows/tree/**` - Phylogenetic trees (PhyloWSTreeController) +- `/phylows/matrix/**` - Character matrices (PhyloWSMatrixController) +- `/phylows/taxon/**` - Taxonomic data (PhyloWSTaxonController) +- `/phylows/classification/**` - Classification data (PhyloWSClassificationController) + +**Output Formats:** HTML, NeXML, NEXUS, RDF, RSS 1.0, JSON + +### Data Standards + +Documentation for phylogenetic data formats is distributed across in-app pages and external links: + +| Format | Coverage | Location | +|--------|----------|----------| +| **NEXUS** | Submission format | submitTutorial.jsp, Mesquite documentation | +| **NeXML** | Primary output format | doc/API.md, nexml.org (external) | +| **Newick** | Tree representation | Implicit in tree endpoints | +| **RDF/CDAO** | Semantic web output | doc/API.md, evolutionaryontology.org | +| **RSS 1.0** | Search result feeds | doc/API.md | + +**Data Preparation:** +- **submitTutorial.jsp** (`/submitTutorial.html`) - NEXUS file preparation using Mesquite +- YouTube instructional videos embedded in submitTutorial.jsp + +**Metadata Standards:** +- Dublin Core (dcterms) predicates for bibliographic metadata +- PRISM for publication dates +- TreeBASE-specific predicates (tb:) documented in API.md + +### Developer Resources + +Resources for contributors and developers: + +| Resource | Location | Description | +|----------|----------|-------------| +| **technology.jsp** | `/technology.html` | Architecture overview, technology stack diagram | +| **BUILDING.md** | `doc/development/` | Build prerequisites (Java 17, Maven), compilation steps | +| **DWR.md** | `doc/technical-notes/` | AJAX integration with Spring 5 compatibility | +| **UPGRADES.md** | `doc/technical-notes/` | Dependency upgrade history and migration notes | +| **GitHub Repository** | github.com/TreeBASE/treebase | Source code under BSD license | +| **GitHub Issues** | Linked from contact.jsp | Bug reports and feature requests | + +**Technology Stack (from technology.jsp):** +- Database: PostgreSQL +- ORM: Hibernate +- Framework: Spring +- File Parsing: Mesquite +- Tree Visualization: phylotree.js + +**Project Structure:** +- `treebase-core` - ORM API for PostgreSQL database access +- `treebase-web` - MVC web application with JSP/HTML GUI +- `oai-pmh_data_provider` - OAI-PMH interface functionality + +### Integration Guides + +Programmatic integration documentation: + +| Integration Type | Documentation | Key Information | +|-----------------|---------------|-----------------| +| **REST API Queries** | doc/API.md | PhyloWS URL patterns, CQL search syntax | +| **Metadata Harvesting** | doc/OAI-PMH.md | OAI-PMH verbs: Identify, ListRecords, GetRecord, etc. | +| **Data Download** | urlAPI.jsp | NEXUS, NeXML, JSON format parameters | +| **RSS Feeds** | doc/API.md, urlAPI.jsp | RSS 1.0 for search results and alerts | + +**URI Patterns (from urlAPI.jsp):** +- Study: `{purlBase}study/TB2:S{id}` +- Matrix: `{purlBase}matrix/TB2:M{id}` +- Tree: `{purlBase}tree/TB2:Tr{id}` + +**Search Examples (from API.md):** +- `/study/find?query=dcterms.contributor=Huelsenbeck` +- `/taxon/find?query=dcterms.title=="Homo sapiens"&format=rss1&recordSchema=tree` + +**Note:** No official client libraries or SDKs are currently provided. + +### Database Documentation + +Database and data model documentation: + +| Resource | Location | Description | +|----------|----------|-------------| +| **technology.jsp** | `/technology.html` | Data content overview, architecture diagram | +| **BUILDING.md** | `doc/development/` | Database connection configuration | +| **Hibernate Mappings** | `treebase-core` source | Entity relationships via ORM | + +**Data Model Overview (from technology.jsp):** +1. Studies contain bibliographic references to published phylogenetic research +2. Each study has one or more analyses; each analysis has steps associating matrices, trees, algorithms, and software +3. Character matrices contain taxon labels mapped to tree leaf nodes +4. Row segments link to specimen, tissue, or gene sequence metadata +5. Taxon names map to uBio NameBank and NCBI taxonomy + +**Key Entity Types:** +- Study, Citation, Analysis, AnalysisStep +- Matrix, Tree, TreeBlock +- Taxon, TaxonLabel, TaxonVariant + +**Note:** Formal data dictionary and entity-relationship diagrams are not currently published. + +### Deployment + +Deployment and operations documentation: + +| Resource | Location | Description | +|----------|----------|-------------| +| **BUILDING.md** | `doc/development/` | Prerequisites, compilation, WAR packaging | +| **DEPLOYING.md** | `doc/development/` | Tomcat configuration, JVM arguments, verification | +| **UPGRADES.md** | `doc/technical-notes/` | Dependency versions, security fixes, migration paths | +| **DOCKER.md** | Repository root | Docker deployment option | +| **docker-compose.yml** | Repository root | Container orchestration | + +**System Requirements (from BUILDING.md/DEPLOYING.md):** +- Java 17 LTS +- Maven 3.8+ +- PostgreSQL database +- Tomcat 7+ servlet container +- Headless Mesquite for file parsing + +**Build Artifacts:** +- `treebase-web.war` (~59MB) - Main web application +- `data_provider_web.war` - OAI-PMH interface + +**Configuration Files:** +- `jdbc.properties` - Database connection +- `context.xml` - Tomcat context with database and Mesquite paths + +## External Documentation + +*Links to existing technical documentation* + +- [API.md](../API.md) - PhyloWS API documentation +- [OAI-PMH.md](../OAI-PMH.md) - OAI-PMH harvesting interface +- [Development Documentation](../development/) - Build and deployment guides +- [Technical Notes](../technical-notes/) - Implementation details + +## Pages to Account For + +Complete inventory of JSP pages and controllers related to technical documentation: + +| Page | URL Pattern | Controller | Status | +|------|-------------|------------|--------| +| urlAPI.jsp | /urlAPI.html | filenameController | ✅ Active | +| technology.jsp | /technology.html | filenameController | ✅ Active | +| about.jsp | /about.html | filenameController | ✅ Active | +| contact.jsp | /contact.html | filenameController | ✅ Active | +| submitTutorial.jsp | /submitTutorial.html | filenameController | ✅ Active | +| help.jsp | /help.html | helpController | ✅ Active | +| sitemap.jsp | /sitemap.xml | sitemapController | ✅ Active | + +**PhyloWS Controllers:** + +| Controller | URL Pattern | Description | +|------------|-------------|-------------| +| PhyloWSStudyController | /phylows/study/** | Study data API | +| PhyloWSTreeController | /phylows/tree/** | Tree data API | +| PhyloWSMatrixController | /phylows/matrix/** | Matrix data API | +| PhyloWSTaxonController | /phylows/taxon/** | Taxon data API | +| PhyloWSClassificationController | /phylows/classification/** | Classification API | + +**OAI-PMH Interface:** + +| Endpoint | URL Pattern | Description | +|----------|-------------|-------------| +| OAI-PMH Provider | /top/oai | Metadata harvesting service | + +**JSON Output Pages (for API responses):** + +| Page | Purpose | +|------|---------| +| json/studyToJSON.jsp | Study JSON serialization | +| json/treeToJSON.jsp | Tree JSON serialization | +| json/matrixToJSON.jsp | Matrix JSON serialization | +| json/taxonToJSON.jsp | Taxon JSON serialization | +| anyObjectAsRDF.jsp | RDF output generation | +| searchResultsAsRDF.jsp | Search results as RDF | + +## Wireframe Notes + +*To be completed in future PR* + +## Open Questions + +- Where should API documentation live (in-app vs external)? +- What interactive API features are needed (playground, testing)? +- How should code examples be maintained? +- What versioning strategy for API documentation? From b4ea86a844d0f6de9d15b4256b43a275b272581c Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 12 Jan 2026 22:41:38 +0000 Subject: [PATCH 50/52] Add /rss.xml and /feed.xml endpoints to technical documentation Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-06-technical.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/doc/v3/user-story-06-technical.md b/doc/v3/user-story-06-technical.md index a650ad1e5..3fc9d01ca 100644 --- a/doc/v3/user-story-06-technical.md +++ b/doc/v3/user-story-06-technical.md @@ -98,7 +98,11 @@ Available documentation for API consumers: - `/phylows/taxon/**` - Taxonomic data (PhyloWSTaxonController) - `/phylows/classification/**` - Classification data (PhyloWSClassificationController) -**Output Formats:** HTML, NeXML, NEXUS, RDF, RSS 1.0, JSON +**RSS/Atom Feed Endpoints:** +- `/rss.xml` - RSS 2.0 feed of recent studies and updates +- `/feed.xml` - Atom feed of recent studies and updates + +**Output Formats:** HTML, NeXML, NEXUS, RDF, RSS 1.0, RSS 2.0, Atom, JSON ### Data Standards @@ -111,6 +115,8 @@ Documentation for phylogenetic data formats is distributed across in-app pages a | **Newick** | Tree representation | Implicit in tree endpoints | | **RDF/CDAO** | Semantic web output | doc/API.md, evolutionaryontology.org | | **RSS 1.0** | Search result feeds | doc/API.md | +| **RSS 2.0** | Recent studies feed | /rss.xml endpoint | +| **Atom** | Recent studies feed | /feed.xml endpoint | **Data Preparation:** - **submitTutorial.jsp** (`/submitTutorial.html`) - NEXUS file preparation using Mesquite @@ -258,6 +264,13 @@ Complete inventory of JSP pages and controllers related to technical documentati |----------|-------------|-------------| | OAI-PMH Provider | /top/oai | Metadata harvesting service | +**RSS/Atom Feed Endpoints:** + +| Endpoint | URL Pattern | Output Format | Description | +|----------|-------------|---------------|-------------| +| RSS Feed | /rss.xml | RSS 2.0 | Recent studies and updates feed | +| Atom Feed | /feed.xml | Atom | Recent studies and updates feed | + **JSON Output Pages (for API responses):** | Page | Purpose | From 6aae1474cdf94db3e4313614a9fe12f099409921 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 13 Jan 2026 07:49:35 +0000 Subject: [PATCH 51/52] Initial plan From 1e6a0ba738e76d0f31755ba4d26fa343f6ddcc85 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 13 Jan 2026 07:54:52 +0000 Subject: [PATCH 52/52] Add missing download endpoints to user-story-01-search.md Co-authored-by: rvosa <106490+rvosa@users.noreply.github.com> --- doc/v3/user-story-01-search.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/v3/user-story-01-search.md b/doc/v3/user-story-01-search.md index 92d183ff9..537aa25ac 100644 --- a/doc/v3/user-story-01-search.md +++ b/doc/v3/user-story-01-search.md @@ -363,9 +363,13 @@ Complete inventory of pages related to search functionality: |------|-------------|-------------|--------| | Row Segments TSV | `/search/study/rowSegmentsTSV.html` | Row segments as TSV download | Active | | RDF Export | `/search/study/anyObjectAsRDF.rdf` | RDF export for any object | Active | +| Download Study | `/search/downloadAStudy.html` | Download complete study | Active | | Download Tree | `/search/downloadATree.html` | Download single tree | Active | +| Download Tree Block | `/search/downloadATreeBlock.html` | Download tree block | Active | | Download Matrix | `/search/downloadAMatrix.html` | Download single matrix | Active | +| Download Analysis Step | `/search/downloadAnAnalysisStep.html` | Download analysis step data | Active | | Download NEXUS | `/search/downloadANexusFile.html` | Download original NEXUS file | Active | +| Download NEXUS RCT | `/search/downloadANexusRCTFile.html` | Download reconstructed NEXUS file | Active | ### PhyloWS API Endpoints