ncbi issueshttps://gitlab.coko.foundation/ncbi/ncbi/-/issues2024-02-21T16:17:18Zhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1504Book part chapter title not being correctly written in converted XML metadata...2024-02-21T16:17:18ZKin Man NgBook part chapter title not being correctly written in converted XML metadata (instead front matter part title in body of that chapter is by mistake)<!-- Required. Provide a general summary of the issue in the title above -->
## Expected behaviour
\<title\> from book-part book-part-type="chapter" should be captured as the title for the chapter and not the title from the front matter ...<!-- Required. Provide a general summary of the issue in the title above -->
## Expected behaviour
\<title\> from book-part book-part-type="chapter" should be captured as the title for the chapter and not the title from the front matter nested in the body of that book part.
## Current behaviour
In
the example provided (see link below), the title captured in the metadata is 'Preface' (title of hte front matter nested within the body of the book part) instead of the title in the parent book part metadata for the entire book part - as tagged in book-part chapter.
Tagging from converted XML:
`<title-group>
<title>Quantitative Synthesis—An Update</title>
</title-group>`
<!-- Required. Tell us what happens instead of the expected behaviour -->
The title should be 'Quantitative Synthesis—An Update' in the chapter list page and also in the chapter metadata page instead of 'Preface'
## Steps to reproduce
<!-- Required. Provide a link to a live example or screenshots, and the steps to reproduce this bug.]-->
Sample chapter loaded with chapter title issue:
https://ncbi.cloud68.co/organizations/4dd138c1-1baa-437b-8a0e-d25b91319687/bookmanager/f516c180-3155-427a-97d2-277b4d91e8cf
## Environment
<!-- Required. Provide relevant information such as browser name and version, PC or Mac use, internet speed, etc.]-->
## Possible solution
The chapter in the example has an front-matter/preface within the book-part, see attached screenshot.
The preface has `<title>Preface</title>` so that might be where the 'Preface' title is coming from.
![cerguidequantup_preface_title](/uploads/e9e9a68b4c4a66dd1d1048b1ee92bdc5/cerguidequantup_preface_title.png)
<!-- If known, provide details on how to fix the bug.-->
<!-- After creating this issue you can link other related or blocking issues with the Gitlab's Linked issues functionality. -->
## QA Steps
[To be completed by Coko once dev is done]
## Scheduling
<!-- Select all the relevant options -->Stacy LathropStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/783Book versioning to support changes in workflow and processing2024-01-17T06:50:26ZErin SchmiederBook versioning to support changes in workflow and processing
## Context
<details><summary>Click to expand</summary>
Changing some major processing issues may cause problems for the BCMS and potentially PMCBook database / processes. We need to understand these technical issues to make informed ...
## Context
<details><summary>Click to expand</summary>
Changing some major processing issues may cause problems for the BCMS and potentially PMCBook database / processes. We need to understand these technical issues to make informed decisions about not permitting minimally Bookshelf staff the ability to change processing settings as NCBI requested and is documented. If Bookshelf staff can't change these settings we must agree what Bookshelf staff / development staff would do in the event a change must be made to one of them. Bookshelf staff has recent memory of such changes happening for good reason. Not having the ability to make these changes poses risks to the migration as there are unknowns to testing all migration uses cases fully as not all BCMS modules are built out.
Summary why we need a way to change these settings and in what cases:
- Book content was migrated by error under the **wrong conversion type**. We get an update to that book and can't process it because the source type can't be converted. Until we can support the ability in a later phase, we need either 1) a way in the BCMS backend to change the setting or 2) still be able to upload source files AND process the content outside the BCMS or in a **test book** and then load the converted files to the "real" book.
- If a book was migrated under the **wrong conversion type** that requires chapter-processing (e.g., Word conversion), then this setting must be changed too.
- Content provider changes their publishing model and asks us to change their workflow handling. This happens several times a year and can include existing, updated domains. This use case can be postponed to after deployment.
Risk of using **test book** is it might get published to Bookshelf site by mistake. Possible workaround in Phase 1, have setting that something is a **test book** that does not permit publication. Note, this is an existing domain attribute in PMCBook:
![image](/uploads/0a29e98b0b280de4f42d52c83fc564d2/image.png)
**NCBI Agreement: Anything migrated as chapter-processed will NEVER change to be book processed.**__
![image](/uploads/0327406313ac059c916d2cf4e1ebce21/image.png)
</details>
## Proposal (updated June 2023)
### Option 1: Change in conversion workflow type
This proposal update considers a new request from NCBI:
>create a new version before the current version is published in order to change the processing workflow. The complication here is that this use case does not constitute a new scientific record (published version) so it would not be correct to change either the `content type` (version name) or `version number`. For example, the following would be valid in this use case:
book version 1:
* conversion workflow: PDF
* content type: final-full-text
* version number: V1
* flag: locked
book version 2:
* conversion workflow: XML
* content type: final-full-text
* version number: V1
* flag: current
#### workflow only change
| Change from | Change to |
|--------------------|--------------------|
| XML | PDF |
| XML | Word |
| PDF | XML |
| PDF | Word |
| Word | XML |
| Word | PDF |
There is no difference in complexity between the possible conversion workflow options, if the processing type remains the same.
#### Minimal workflow for Option 1
1. User publishes OR locks the current book version
2. User chooses to create a new version
3. User chooses the workflow type for the new version
4. BCMS shows confrimation that previous versions will be locked (because of the conversion workflow change)
5. User confirms decision
* new book version is created
* previous version is locked
* user is redirected to the new version
6. User uploads the relevant files to new version (i.e. the BCMS does *not* attempt to match files from the previous version to the current one)
#### Caveats
1. On initial review we don't see a technical issue for books created in the BCMS, however since repeating `content type` and `version number` across BCMS book versions is a departure from the original scope, there may be implications we're not thinking of now. We're adding 2 days for code review and scope sign-off.
1. The [updated request above (option 1)](https://gitlab.coko.foundation/ncbi/ncbi/-/issues/783#option-1-change-in-conversion-workflow-type) asks the BCMS to accommodate a scenario that results in two book versions with the same `book-submit-id` + `version-name` + `version-number`. Therefore the current FTP spec would need to be reconsidered if NCBI plans to introduce FTP support at a later date. Since we also have the `workflow` information in the meta.xml file, a reasonable solution is to add `workflow` as a require value to match book versions. This is the best proposal we can give at this time and it would need to be validated by Coko and NCBI at the start of the project. As an integration point, this spec change requires input from NCBI and Coko teams.
2. We think it would be confusing to users to see two versions in the BCMS with the same name. Ideally we should hide all locked versions from all users except Sys Amins. This is not included in this estimate.
Estimate: 8 days
### Extended option 2: Change in processing type:
| Change from | Change to |
|--------------------|--------------------|
| Wholebook | Chapter-processed |
| Complete book | Wholebook |
There is a difference in complexity between the possible options. Some considerations include:
#### files
Source file requirements:
* wholebooks can have more than one source file
* Each chapter in a chapter-processsed book can only have one source with a consistent filename.
* complete books must only have one source file that is named `toc.doc` (or `docx)
Bookshelf display PDF files:
* Wholebooks can have multiple PDF uploaded
* User identifies the `book_pdf` in XML wholebooks only
* The BCMS automatically renames the `book_pdf` in XML wholebooks only
Instead of building the UI for the user to communicate these changes, the simplest development approach is to let the user upload whichever files the need to the new version.
#### UI navigation between versions
User experience when changing processing type:
* The UI of a chapter-processed book is significantly different to that of a wholebook or complete document.
* Suggest providing a download link to get files for previous version, instead of allowing navigation back and forth between these versions.
#### Change book settings template
In the current implementation of versions, book settings are copied from the current version to the new version. This won't be possible if the processing type changes. Instead, the BCMS should apply relevant book setting template from the organization.
### Minimal workflow for estimate for option 2
(Estimate: add 3 days for items shown in Bold)
1. User publishes OR locks a book version
2. User chooses to create a new version **(in the case of chapter-processed books and Word complete books this could be limited to System to allow changing the processing type).**
3. User chooses the workflow and **processing type** for the new version
4. BCMS shows confrimation that previous versions will be locked (because of combination of workflow and processing type change)
5. User confirms decision
* new book version is created **(using the book template from the organisation)**
* previous version is locked
* user is redirected to the new version **(in the case of changing from wholebook to chapter-processed, the user cannot navigate to previous version, only files can be downloaded)**
6. User uploads the relevant files to new version (i.e. the BCMS does *not* attempt to match files from the previous version to the current one)
## Design
Pending
## Implementation (if applicable)
[A description of the steps to implement the feature.]
## Alternative approaches (if applicable)
Recommend NCBI consider an alternative solution to these changes, especially if they are uncommon. Can these be supported by archiving the book and linking it to new book?
| Change from | Change to | Notes |
|--------------------|--------------------|----------------------------------------------|
| Chapter-processed | Wholebook | The UI for this use case will be complex. Suggest rather supporting "Delete" book in status "new book". |
| Supports Multiple published versions | Does not supports Multiple published versions | If this use case includes books that already have 2+ book component versions, then the UI for this case could be complex. |
## Open issues (if applicable)
[Links to and a discussion of related issues, if applicable.]Stacy LathropStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1444Unable to submit files2024-01-17T06:33:24ZStacy LathropUnable to submit files
## Expected behaviour
User should be able to submit files.
## Current behaviour
User is not able to submit files. The case originally reported in this issue, shown in the video below, was confirmed as temporary issue due to the serve...
## Expected behaviour
User should be able to submit files.
## Current behaviour
User is not able to submit files. The case originally reported in this issue, shown in the video below, was confirmed as temporary issue due to the server restarting in https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1444#note_105129
See: ![unablesubmitvideo1079642470](/uploads/809b7e987d70f8494a0b7cccd6799a67/unablesubmitvideo1079642470.mp4)
Additional cases to review were provided in https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1444#note_105154, copied below:
We still are getting reports that in some cases users can't submit files.
**Case 1** from today of inability to bulk upload (that I cannot reproduce):
User uses latest version of Firefox and a Lenovo ThinkPad - he also reproduced issue in Microsoft Edge
![image](/uploads/b7ff4f267544ff00d33e1477acdd6966/image.png)
(I am confused though because when I go that record it says there is a New Upload, so how was it successful unless he later did it via that chapter). I will try to get on a webex with him to see what he does, not get email.
**Case 2** - some chapters don't bulk upload (this one I could reproduce):
Try uploading two chapter below to this book:
https://ncbi.cloud68.co/organizations/6c4d37c4-b9f2-4962-9080-b3e29a37dfbd/bookmanager/7f099bb1-3379-4beb-bc4b-aeca26884a47
[cf.docx](/uploads/f8558cde562ea96df6842bfbb1468734/cf.docx)
[mdel15q13_3.docx](/uploads/1346ebe28b8ffd75c8548b1cb73fa590/mdel15q13_3.docx)
## Steps to reproduce
**Case 1** -- NCBI team could not reproduce
**Case 2** -- Coko reproduced in https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1444#note_109118 below showing upload works as expected.
**Updated April 11, 2023**: See additional cases that must be met for BCMS MVP:
- #1487
- #1488
## QA Steps
[To be completed by Coko once dev is done]
## Scheduling
<!-- Select all the relevant options -->
- [x] This issue is blocking current user testing
- [x] This issue is blocking current migration testing
- [x] Fixing this issue is required for [Priority 1: Deploy MVP](https://gitlab.coko.foundation/groups/ncbi/-/epics/56)
- [ ] This issue should be triaged into later epics in the PM weekly meetingDione Mentisdione@coko.foundationStacy LathropDione Mentisdione@coko.foundationhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1461BCMS site continually crashes and NCBI needs confidence these issues are bein...2024-01-17T06:29:49ZStacy LathropBCMS site continually crashes and NCBI needs confidence these issues are being monitored and addressed
## User Story
BCMS users experience frequent instances of the application giving a red pop up error and/or then going blank and/or giving a Gateway error, which usually causes our users to refresh their browser repeatedly to try to res...
## User Story
BCMS users experience frequent instances of the application giving a red pop up error and/or then going blank and/or giving a Gateway error, which usually causes our users to refresh their browser repeatedly to try to resurrect what they were doing before this experience. We need to ensure we have a reliable and systematic way to detect the root causes of such experiences resulting from suspected instability of the application and that our changes to the application to address such issues and their root causes are effective.
## Acceptance Criteria
- [ ] Reduction of reports from BCMS users about issues described in this bug ticket
- [ ] Logs created in #1493 demonstrate the BCMS application is stable without any crashes when we try to reproduce the user behaviors reported in this issue
-----
<!-- Required. Provide a general summary of the issue in the title above -->
## Bug Description: Expected behaviour
- BCMS site should not crash
- Coko should monitor logs for crashes and have a documented process to identify the source of the crash and to fix that source problem
## Bug Description: Current behaviour
We are repeatedly experienced the BCMS site crash.
First we get a lot of popup red errors that I could not get a screenshot of, but it said, "Can not reload preview"
Then we see this:
![stuck_logging_in](/uploads/b6cb067a335cf9cfa88d6a89fd0c72cf/stuck_logging_in.PNG)
And then we get this page:
![image](/uploads/f3d0f19375679b90cdd491c7774e09db/image.png)
But when we try to log in - add our credentials and Login, nothing happens.
Usually in 15-20 minutes it is back up, but we get many complaints and this can't happen in production. **Update 2023-02-06:** Coko confirmed in #1469 that this temporary log in issue is because we're automatically restarting the server which can take some time.
## Steps to reproduce
We don't know what is causing this so can't say how to reproduce it, and we would like Coko to propose a process for how they will monitor and troubleshoot such issues past, current, and future.
### Case 1
Additional information from @lathrops1 in https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1461#note_107648
I did ask these questions and the users could be doing anything:
- uploading files
- submitting, loading, publishing files
OR as with myself when I have experienced crashes, even just scrolling and navigating the BCMS to find something.
These issues have been happening regularly so hard to track, but I know it happened exactly when I reported it on Mattermost on December 19, 2022 slightly before 3pm ET time.
I will ask folks to report to me exactly what they were doing and at what time, but I think everyone is now pretty trained to ignore and try again, so that behavior might be hard to change.
The site did not fully crash, but in our instance on Tuesday Jan 17 between 4 and 5pm, I was trying to publish a chapter and got a JSON red error that would not permit me to finish my transaction. Denis thought this might be because the site crashed, but the behavior is different than things going blank, getting a login page, and not being able to log in.
### Case 2
From https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1155#note_108472
@John.kopanas @DioneMentis We do still have frequent crashes:
For example, today we had crashes at these times: (19:19, 18:12, 14:44, 10:57, 00:32, 23:32)
They typically have the same error that postgres `Connection terminated unexpectedly`
It could be related to the (excessive pg connections but no query) issue;
Here is some of the logs surrounding these crashes
[January_25th_2023__19.19.14.582_2023-01-.txt](/uploads/a729f60a3e78f7e46fe7e388b23b04e9/January_25th_2023__19.19.14.582_2023-01-.txt)
[January_25th_2023__18.12.32.208_2023-01-.txt](/uploads/c36f11eeff0e736f40299b108cc1108b/January_25th_2023__18.12.32.208_2023-01-.txt)
[January_25th_2023__18.12.29.144_2023-01-.txt](/uploads/28edd61a0117a87f354a339d09f537a4/January_25th_2023__18.12.29.144_2023-01-.txt)
[January_17th_2023__09.18.33.551_2023-01-.txt](/uploads/2bb90ffd8a8305aa59c8a966109e08c2/January_17th_2023__09.18.33.551_2023-01-.txt)
## Bug Description: Environment
<!-- Required. Provide relevant information such as browser name and version, PC or Mac use, internet speed, etc.]-->
## Possible solutions to resolving bug / technical proposal
<!-- If known, provide details on how to fix the bug.-->
Coko will receive an email notification whenever the server goes down and will investigate the logs. Known investigations are listed below:
- [x] increase idleTimeoutMillis !1099 and monitor
- [ ] refactor permissions query (estimated in #1471)
## QA Steps
[To be completed by Coko once dev is done]
## SchedulingP1a: Book Manager redesign: HighStacy LathropStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1482Extend “bulk move” functionality to support books without parts2024-01-17T06:29:43ZDione Mentisdione@coko.foundationExtend “bulk move” functionality to support books without parts<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The context is provided in the Epic https://gitlab.coko.foundation/groups/ncbi/-/epics/2 and Table of contents building user stories in #1458.
Table of contents list can be very long. Drag and drop functionality is not suitable for moving items to a position that is far from the default position (top of the body division). For books that do not have parts, there is no alternative to drag and drop currently.
## Proposal
<!-- A precise statement of the proposed feature. -->
Extend “bulk move” functionality to support:
* bulk moving on Front and Back tabs
* bulk move chapters in books without parts.
* bulk move parts and chapters in books with parts
## Design
### Current UI
Currently users can only move one or more chapters into a part
![Screenshot_2023-08-03_at_19.00.45](/uploads/249a3dbd07dac7fcdf747edc73a82437/Screenshot_2023-08-03_at_19.00.45.png)
<!-- Include sketches or wireframes of the UI suggested for this feature -->
### New Design
#### Book without parts
*Users can move chapters above or below another chapter*
![The_move_to_moda__no_parts](/uploads/3335670b2e60c7256b0a4d5d247e6703/The_move_to_moda__no_parts.jpeg)
#### Book with parts
*Users can move chapters and parts above or below another chapter or part, or inside another part*
![The_move_to_modal_V1__with_parts](/uploads/37d1254d30eef7dbfb0063df3378d228/The_move_to_modal_V1__with_parts.jpeg)
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
**Applies to All**
- [ ] All criteria apply to System Admin, Org Admin and Editors
**Select Functionality to Bulk Move Records from Book Manager and Search Results pages**
- [ ] On each paginated page in the Front, Body, and Back tabs of the main Book Manager Dashboard, users can select one or multiple records using the select box next to the records, or select all records using the Select All function.
- [ ] On each paginated page in the Search Results view, users can select one or multiple records using the select box next to the records, or select all records using the Select All function.
- [ ] The select boxes will be responsive within 5 seconds
**Display and Functionality of Move action button on Book Manager and Search Results pages**
- [ ] In all cases, users can only move records on Book Manager pages in a status defined as "Yes" in Column T of the [ documented rules rules in the Status/Actions sheet](https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit#gid=0)
- [ ] In all cases, if there are no selected components in a status that can be moved, the Move button will not be clickable on the paginated pages
_Move from Book Manager Page_
**- Front Matter**
- [ ] IF the "Front" tab contains any front matter documents in a status that may be moved, it will have a clickable Move action button at the bottom of every paginated page. The "Move" button allows the user to:
- [ ] Move selected Front Matter documents before or after another Front Matter document in the Table of Contents
**- Body without Parts, Manual**
- [ ] IF a user has the “Group chapters into parts” setting toggled OFF AND the "Manual" value selected in the "Order chapters by" dropdown in the “Landing Page: Table of Contents” section of the book's Settings AND the "Body" tab contains Chapters in a status that may be moved, they will see a clickable Move action button at the bottom of every paginated page. The "Move" button allows the user to:
- [ ] Move selected Chapters before or after another Chapter in the Table of Contents
**- Body without Parts, Auto-sorted**
- [ ] IF a user has the “Group chapters into parts” setting toggled OFF AND any value other than "Manual" selected in the "Order chapters by" dropdown in the “Landing Page: Table of Contents” section of the book's Settings, they will NOT see a clickable Move action button at the bottom of every paginated page.
**- Body with Parts, with Manual Chapters**
- [ ] IF a user has the “Group chapters into parts” setting toggled ON AND the "Manual" value selected in the "Order chapters by" dropdown in the “Landing Page: Table of Contents” section of the book's Settings AND the "Body" tab contains chapters in a status that may be moved, they will see a clickable Move action button at the bottom of every paginated page. The "Move" button allows the user to:
- [ ] Move selected Chapters before or after another Chapter or Part in the Table of Contents
- [ ] Move selected Parts before or after another Part or Chapter in the Table of Contents
- [ ] Move selected Chapters within any Part in the Table of Contents
- [ ] Move selected Parts and all of their descendent Parts and Chapters within any other Part in the Table of Contents
**- Body with Parts, with Auto-sorted Chapters**
- [ ] IF a user has the “Group chapters into parts” setting toggled ON AND any value other than "Manual" selected in the "Order chapters by" dropdown in the “Landing Page: Table of Contents” section of the book's Settings AND the "Body" tab contains chapters in a status that may be moved, they will see a clickable Move action button at the bottom of every paginated page. The "Move" button allows the user to:
- [ ] Move selected Parts before or after another Part or Chapter in the Table of Contents
- [ ] Move selected Chapters within any Part in the Table of Contents
- [ ] Move selected Parts and all of their descendent Parts and Chapters within any other Part in the Table of Contents
**- Back Matter**
- [ ] If the "Back" tab contains back matter documents in a status that may be moved, it will have a clickable Move action button at the bottom of every paginated page. The "Move" button allows the user to
- [ ] Move selected Back Matter documents before or after another Back Matter document in the Table of Contents managed from the Back tab of the Book Manager page
_Move from Search Results Page_
- [ ] The Move button on the paginated Search Results pages will display and function according to the rules outlined in the Acceptance Criteria for #1467
**Display and Functionality of Move Modal**
- [ ] In all cases, when a user clicks on the Move button it will open a modal window, which:
- [ ] Opens within 5 sec
- [ ] Opens in a position from which the user can still see the checkboxes selected by a user
- [ ] Lists the titles (title and subtitle) of the components selected to be moved at the top of the Move Modal
_Move Modal opened from Book Manager Page_
**- From Front Tab**
- [ ] After a user opens the Move modal from within a Book Manager's Front tab, the user will be able to search for the Front Matter document title of the record they wish to move their selected item(s) before or after, and then the
- [ ] User can then click on a positioning button within the modal to execute the move.
- [ ] User will see "Move Above" or "Move Below" buttons. Clicking one of these buttons will move the Front Matter document to Above or below the specified title.
- [ ] After a user has clicked a positioning button, the modal closes, and the user will be on the paginated page in the Front of the Book Manager Page which the parts and/or chapters moved to, not the page they move from, so that the user can see the move was successful, assuming this does not cause performance issues for the BCMS (i.e., the BCMS must continue to meet performance user stories), in which alternative design choices will be offered by Coko to NCBI for approval of implementation before development.
**- From Body Tab with Book without Parts, Manual**
- [ ] After a user opens the Move modal from within a Book Manager's Body tab AND if that Book's “Landing Page: Table of Contents” Settings are set to “Group chapters into parts” OFF AND "Order chapters by" Manual, then the:
- [ ] User will be able to search for the Chapter title of the record they wish to move their selected item(s) before or after, and then the
- [ ] User can click on a positioning button within the modal to execute the move.
- [ ] User will see "Move Above" or "Move Below" buttons. Clicking one of these buttons will move the Chapter to Above or below the specified title.
- [ ] After a user has clicked a positioning button, the modal closes, and the user will be on the paginated page in the Body of the Book Manager Page which the parts and/or chapters moved to, not the page they move from, so that the user can see the move was successful, assuming this does not cause performance issues for the BCMS (i.e., the BCMS must continue to meet performance user stories), in which alternative design choices will be offered by Coko to NCBI for approval of implementation before development.
**- From Body Tab with Book with Parts, with Manual Chapters**
- [ ] After a user opens the Move modal from within a Book Manager's Body tab when a book's “Landing Page: Table of Contents” Settings are set to “Group chapters into parts” ON AND "Order chapters by" Manual, then
- [ ] Users will see a "Move Inside Part, "Move Above" or "Move below" buttons, and
- [ ] Clicking the "Move above" or "Move below" button will move selected Part(s) and all of their descendants, and/or Chapter(s), above or below the specified title.
- [ ] Clicking "Move Inside Part" will move selected Part(s) and all of their descendants, and/or Chapter(s) within the specified title
- [ ] by nesting them as the first children of that Part Title if manually ordered in the order in which they were selected to be moved
- [ ] After a user has clicked a positioning button, the modal closes, and the user will be on the paginated page of the Body of the Book Manager in which the parts and/or chapters moved to, not the page they move from, so that the user can see the move was successful, assuming this does not cause performance issues for the BCMS (i.e., the BCMS must continue to meet performance user stories), in which case, alternative design choices will be offered by Coko to NCBI for approval of implementation before development.
- [ ] The action of moving selected items Before or After must be done separately from the action of moving selected items within a Part
- [ ] For instance, if the user adds a new chapter X and wants to both manually move chapter X (placed by default at the top of the Body) so it appears in the Table of Contents after chapter Y, then move the part C with chapters X and Y within a part D, the user would need to:
- [ ] First select chapter X on the Book Manager page, click Move, search for chapter Y, select chapter Y, click Move Below, and then after that action is complete and the Move modal closes,
- [ ] Then select part C on the Book Manager page, click Move, search for part D, click Move inside part, which will move part C with chapters Y and X (and any other within part C) as a child directly nested at the very top of part D
**- Body with Parts, with Auto-sorted Chapters**
- [ ] After a user opens the Move modal from within a Book Manager's Body tab when a book's “Landing Page: Table of Contents” Settings are set to “Group chapters into parts” ON AND "Order chapters by" and value OTHER THAN Manual, then
- [ ] Users will see a "Move Inside Part, "Move Above" or "Move below" buttons, and
- [ ] Clicking the "Move above" or "Move below" button will move selected Part(s) and all of their descendants above or below the specified Part or Chapter title.
- [ ] Clicking "Move Inside Part" will move selected Part(s) and all of their descendants within the specified title
- [ ] by nesting them as the first children of that Part Title if manually ordered in the order in which they were selected to be moved
- [ ] After a user has clicked a positioning button, the modal closes, and the user will be on the paginated page of the Body of the Book Manager in which the parts and/or chapters moved to, not the page they move from, so that the user can see the move was successful, assuming this does not cause performance issues for the BCMS (i.e., the BCMS must continue to meet performance user stories), in which case, alternative design choices will be offered by Coko to NCBI for approval of implementation before development.
- [ ] The action of moving selected items Before or After must be done separately from the action of moving selected items within a Part
- [ ] For instance, if the user adds a new part X and wants to both manually move part X (placed by default at the top of the Body) so it appears in the Table of Contents after part Y, then move parts X and Y within part A, the user would need to:
- [ ] First select part X on the Book Manager page, click Move, search for part Y, select part Y, click Move Below, and then after that action is complete and the Move modal closes,
- [ ] Then select parts X and Y on the Book Manager page, click Move, search for part A, click Move inside part, which will move parts X and Y (and any of their associated descendants) as direct children at the top of part A ordered by last updated date
**- Back Matter**
- [ ] After a user opens the Move modal from within a Book Manager's Back tab, the user will be able to search for the Back Matter document title of the record they wish to move their selected item(s) before or after, and then the
- [ ] User can then click on a positioning button within the modal to execute the move.
- [ ] User will see "Move Above" or "Move Below" buttons. Clicking one of these buttons will move the Back Matter document to Above or below the specified title.
- [ ] After a user has clicked a positioning button, the modal closes, and the user will be on the paginated page in the Back of the Book Manager Page which the parts and/or chapters moved to, not the page they move from, so that the user can see the move was successful, assuming this does not cause performance issues for the BCMS (i.e., the BCMS must continue to meet performance user stories), in which case, alternative design choices will be offered by Coko to NCBI for approval of implementation before development.
_Move Modal opened from Search Results Page_
- [ ] A user may only open the Move modal from within a Book Manager's Search Results page if all selected components are in the same division (Front | Body | Back) according to the acceptance criteria outlined in #1467 for the Search Results View page functionality
**- All Front Matter Components**
- [ ] Per the same acceptance criteria for opening a Modal from a Book Manager Front tab
**- All Body Components**
- [ ] Per the same acceptance criteria for opening a Modal from a Book Manager in the case of From Body Tab with”All Chapters without Parts, Manual,” “Book with Parts, with Manual Chapter,s” or “Body with Parts, with Auto-sorted Chapters”
**- All Back Matter Components**
- [ ] Per the same acceptance criteria for opening a Modal from a Book Manager Back tab
**Search Functionality**
- [ ] The BCMS will only search the text strings in the `<title-group>` elements of the `<book-part-meta>` of each` <book-part-wrapper>` converted XML file in a chapter-processed book
- [ ] The BCMS search will be case insensitive and return results that contain any part of a search query, if that Title:
- [ ] Is not one of the records to be moved, and
- [ ] If it is a Title that the user can move a Front Matter document, Back Matter document, Part or Chapter before or after, including:
- [ ] A user can move a manually ordered Part before or after a Chapter which is automatically ordered according to a supported sort order, for instance as in this case: https://ncbi.cloud68.co/organizations/d400f251-44e8-44f4-9c27-58d25a09f849/bookmanager/22545acf-246e-41cf-8621-cbaf4f61dd60
- [ ] All relevant results will be returned on the modal page.
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [ ] BCMS User Story / Context has been well defined
- [ ] The priority of the user story is specified and agreed
- [ ] Digital assets added (design, database scheme, mockups etc if relevant)
- [ ] Coko Technical Proposal approved by NCBI
- [ ] Testable Acceptance Criteria approved by NCBI
- [ ] Estimate of effort to complete (time or points)
- [ ] The issue has been broken down into development tasks (if necessary)
- [ ] Requirements Clarified
- [ ] The product owner and development team agree that the user story is ready for development
- [ ] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done” -->
- [ ] All coding tasks are finished and implemented by Coko
- [ ] All end-to-end tests written and run by Coko
- [ ] QA approved by Coko
- [ ] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Deployed and tested by NCBI in NCBI deployed instance and tested by all necessary users in their native work environments
- [ ] NCBI approves that Acceptance Criteria Met
## Implementation (if applicable)
<!-- A description of the steps to implement the feature.-->
Development tasks broken down in Tasks below: #1510, #1511, #1512
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## SchedulingP1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1467Improve Search / Findability on Book Manager pages2024-01-17T06:28:35ZStacy LathropImprove Search / Findability on Book Manager pages<!-- Required. Provide a general summary of the issue in the title above -->
## Context
The Silverlight CMS permits its users to search and find content by search by domain name, title, collection, publisher. It also permits users to s...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
The Silverlight CMS permits its users to search and find content by search by domain name, title, collection, publisher. It also permits users to sort and filter its dashboard tables by critical details, such as filename, filetype, last updated, last published, and status of the content. The BCMS should ideally permit the same level of search / findability by its users so that they may easily find the content they are required to manage.
Issues that need to be solved for the MVP: Significant chapter-processed book NIH-funded stakeholder editorial teams report they cannot easily find the chapters they need to download or process individually or in bulk to update, fix errors, or QA by title, filename, history details, and status.
Impact if this issue is not solved: If this issue is not resolved, it puts at risk Bookshelf’s ability to onboard its users to the BCMS without great resistance from them and adds significant time in adding new and updated content to Bookshelf. Bookshelf staff get weekly / monthly complaints from NIH Word users who are doing conversion in BCMS now.
## Proposal
Provide following search on Book Manager pages without compromising Files Management, TOC Building, and Processing Integration User Stories:
- By title
- By filename
- By BCMS ID
And filter by:
- By status
- Last updated
- Last published
- Division
## Design
### Design option 1: support search with Book manager redesign (chosen)
* Users type text in the Search input to search by:
- By title
- By filename
- By BCMS ID
* Users make selection from the filters dropdown to search by:
- Status (multiple select)
- Division (single select)
- Last updated (date range picker)
- Last published (date range picker)
* UI indicates how may results there are, for example "Showing 100 results of 500".
* UI indicates different states for the filter buttons (options selected/no options selected).
*View prior to search: filters shown*
![Book_manager-pagination_V1_-_show_filters](/uploads/0e55c8b5f1aa0b3555295feb3bcb4589/Book_manager-pagination_V1_-_show_filters.jpeg)
#### Users actions and results
* The user chooses filter options and/or types in the search text area, then selects "search" button
* Results are shown in a flat structure, paginated, and include results from all book divisions (front/body/back).
* Users can select some or all of the results to perform bulk actions from the bottom bar: Publish, Reload Preview, Submit
* User can clear and re-enter text from the Search input area, and see the results change accordingly.
* User can clear and reselect options from the filer dropdowns, Select "Search" button, and see the results change accordingly
* To clear the results completely, the user selects the "Clear" button which clears the search results, and shows the Book Manager in the Tabs(Front/Body/Back) view.
*Search results view*
![Book_manager-search_results_V1](/uploads/ed89c30b966e735555ea4775b70eff88/Book_manager-search_results_V1.jpeg)
*Search results view: with warning notification*
![Book_manager-search_results_V2](/uploads/e75bc6e76b4e776d30bae40638b534e4/Book_manager-search_results_V2.jpeg)
<!-- Include sketches or wireframes of the UI suggested for this feature -->
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [ ] When users load a Book Manager page the default view will be the main Book Manager Dashboard view with Front | Body | Back tabs, pagination, and all contents ordered according to manual and automatic sorting and grouping settings and rules
- [x] Users will see on the top right of the Book Manager Dashboard view, filters to filter these contents by Division, Status, Last Updated, Last Published, a Search box to search the contents by title, filename, or BCMS ID, and the ability to Show filter and Clear filters
- [ ] When users select any Filter or conduct a search, the view will change to a Search Results view that shows only record titles (without any nesting) from most recent to oldest according to the last updated date
- [x] If a user wishes to return to a main Book Manager Dashboard view, they will click the "Clear" button on the top right of any page of the search results
**Filters Functionality**
- [ ] Users may filter their contents on the main Book Manager Dashboard by:
- [x] Select only one BCMS supported division in which a component belongs in a Table of Contents (i.e., parent structural units of a book): Front | Body | Back
- [x] Select one or more BCMS supported statuses documented at: https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit#gid=0
- [x] The last updated date, which is the last time any user has upload files or conducted any action listed at: https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit#gid=0
- [x] The last published date, which is the last time any user successfully "published" a component on Bookshelf (meaning it has been released to the Bookshelf live site)
- [x] Users may select one or more statuses in a Status dropdown filter
- [x] To filter by Last updated, users may select a date range by which they wish to filter their contents, which will include a first date and a last date, which may be the same date, styled according to clear YYYY/MM/DD standards, where that format is clearly communicated to the user in the UI
- [x] To filter by Last published, users may select a date range by which they wish to filter their contents, which will include a first date and a last date, which may be the same date, styled according to clear YYYY/MM/DD standards, where that format is clearly communicated to the user in the UI
- [x] A user may filter a page of already filtered or searched results
- [x] If a filter has option(s) selected, then it is shown as active by a different colour
**Search Functionality**
- [ ] Users may search by the title and subtitle of their Front Matter document, Back Matter document, Chapter, or Part
- [ ] The BCMS will only search what is equivalent to the text strings in the `<title>`and `<subtitle>` elements in the `<title-group>` elements of the `<book-part-meta>` of each `<book-part-wrapper>` converted XML file in a chapter-processed book for Front Matter documents, Back Matter documents, Chapters, and Parts with Body (when supported), and
- [x] The BCMS will search the Title field in the UI for Parts maintained in the BCMS UI
- [x] The BCMS search will be case insensitive
- [ ] The BCMS search will return results that contain any part of a search query
- [ ] Users may search by the filename of their source and converted Front Matter documents, Back Matter documents, Chapters, and Parts with Body (when supported) files
- [ ] The BCMS search will be case insensitive
- [ ] The BCMS search will return results that contain any part of a search query
- [ ] Users may search by the BCMS ID for their Front Matter documents, Back Matter documents, Chapters, and Parts with Body (when supported)
- [ ] The BCMS search will require an exact match of the base ID, anything before the period,
- [ ] For instance for a record with a BCMS ID of bcms2100.1, user must at least enter bcms2100 to retrieve it
- [x] A user may further search a page of already filtered or searched results
**Search Results View Functionality**
- [ ] A user can apply a filter or conduct a search from any tab on the Book Manager page (Front | Body | Back) or any paginated page within those tabs
- [ ] Regardless of what tab/page a user was viewing when they did a filter and/or search action, that filter and/or search applies across all tabs and paginated pages within a book
- [ ] After a user completes a filter and/or search action, they will see a single Search Results View page. This means:
- [ ] There is not a separate Results view page for each tab.
- [ ] Users will no longer see the Front | Body | Back division tabs while on a Search Results page
- [ ] To bring back the the Front | Body | Back division tabs, a user must clear all searches and filters by clicking the Clear button at the top right of any page
- [ ] All search results on a Search Results page will be paginated, and each paginated page will match the number of results per page according to the rules specified in #1480 for the main book manager page, where a record may be:
- [ ] A front matter document title, back matter document title, chapter title, part title, and in which
- [ ] Part records may not be expanded or collapsed
- [ ] The number of results that meet the selected filters and/or search will be indicated at the top of the Search Results page before the first results (e.g., "Showing 100 results of 500")
- [ ] All search results, including Parts without files, will be displayed in order of most recent to oldest according to the last updated date, where this is clearly indicated at the top of the Search Results page before the first results (e.g., (e.g., "Showing 100 results of 500, sorted by most recent")
- [ ] Users can select, one or more, or all results on a paginated page and click to Move (i.e. Move a Front Matter document, Back Matter document, Part and all of its descendant Parts and Chapter), or Chapter in the Table of Contents)
- [ ] User will not be able to “Move” selected items if all selected items do not belong to the same Front, Body, or Back division, in which case
- [ ] User will see a gray informational warning them why they cannot move these items with a link for more information, which the
- [ ] User can acknowledge and tell the system to not show them again within any user session
- [ ] User will only be able to "Move" selected items if they are in a book with a “Landing Page: Table of Contents” Setting that permits "Manual" movement within a part or before or after another part or chapter, including
- [ ] Parts and Chapters in books with “Group chapters into parts” AND/OR
- [ ] Chapters in books with "Order chapters by" Manual
- [ ] Users can Upload new book files from a Search Results view page
- [ ] Search Results View pages and their paginated results will "stick" and not change until a user does the following:
- [ ] Conducts a new filter or search
- [ ] Clicks on the Next, Previous, or a Page number at the bottom of the page to see the next set of search results
- [ ] Clicks Clear to clear the results and return the user to the main Book Manager view
- [ ] Clicks the Back to Book title link at the top of the page
- [ ] Leaves that book's chapter-processed Book Manager UI
- [ ] Users will see the label, title, file name, last updated date, last published date for each retrieved search results
- [ ] Users can select one, some, or all of the results on a page to perform bulk actions from the bottom bar: Move, Publish, Reload Preview, Submit
- [ ] User will not be able to conduct any action if all selected items are not in a status that would permit them to conduct this action, according to the rules at: https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit#gid=0
**Performance of Search Results Page**
- [ ] A search results page will load within 20sec
- [ ] All select boxes will be responsive within 5sec
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] All unit and end-to-end tests written, implemented, and run by Coko
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Deployed and tested by NCBI in NCBI deployed instance and tested by all necessary user
- [ ] NCBI approves Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature.-->
## Alternative approaches (not chosen)
<details><summary>Alternative approaches (not chosen)</summary>
<!-- Include any alternatives to meet this use case. -->
### Proposal A: minimal improvements to current search
Currently the Book manager page supports search by
- By title
- By filename
- Search by Status possible, but users have to use a hyphen to search by the full status =name because search is current done on the status name saved in the database. For example: Users can find chapters in "Loading Errors Status if they search with:
* one word `loading` or `errors`)
* two words `loading-errors`
To add full support for search by Status and search by BCMS ID we estimate 1 day developer time.
OR, with redesign:
## Design option 2
**Development estimate 3 days**
This is compatible with Design option 3 in the Design options for resolving chapter-processed book manager performance issues (#1480)
* Search functions as described in design option 1
* Results are shown, paginated.
![Book_manager-option3](/uploads/6b372d600812dae7d8336eeb45cf0d01/Book_manager-option3.jpeg)
</details>
## SchedulingP1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1458Separate the front/body/back divisions into three tabs2024-01-17T06:27:05ZStacy LathropSeparate the front/body/back divisions into three tabs
## Context / User Story
Table of Contents Building Story: Because of an improvement made to the BCMS data model from the existing Silverlight CMS to support editorial teams to manage their book part contents at the same time themselves...
## Context / User Story
Table of Contents Building Story: Because of an improvement made to the BCMS data model from the existing Silverlight CMS to support editorial teams to manage their book part contents at the same time themselves, Bookshelf requires the BCMS to have functionality in the UI for users to build their TOC by communicating, through automated settings and/or manual placement, its structural place in the book (front, body, back), what part in the body it belongs if there are parts, and where in the order of contents a book part of any structural attribute falls; the BCMS must then read this communicated information and write valid book Table of Contents (TOC) XML files (see TOC.xml user story). Original acceptance criteria are here: https://gitlab.coko.foundation/groups/ncbi/-/epics/2
TOC Building issue that needs to be solved for the MVP: Currently, BCMS users cannot manually drag and drop their book parts on the existing chapter-processed Book Manager pages to order their front matter, body contents, part contents, or back matter. This is because lately introduced lazy load scroll bars interfere with the user also being able to see all their contents to be ordered in context of an entire structural unit, and then to drag and drop them where they belong in the context of that entire structural unit. Also, the performance of the page is slow and the drag and drop functionality is not responsive to users’ attempts to move their contents to the right structural place and for those place parts to "stick" in place; that is even after NCBI has removed lazy load to see all parts to validate assumptions.
Impact if this TOC Building issue is not solved: Based on a Bookshelf analysis of all its chapter processed TOCs and discussion with some participants, the results of which were shared with Coko, nearly all these TOCs have some level of required manual curation, according to the participants’ editorial policies that Bookshelf does not have the power to change. These are either the manual ordering of front matter, back matter, or of the placement of content within a part title, and to order those part titles by the historical and professional understanding of those editorial teams. Unless these teams can manage their TOCs in this way, Bookshelf will not be able to display TOC landing pages on its Bookshelf websites per terms of its current participant agreements.
This issue also relates to the proposal Design options for resolving chapter-processed book manager performance issues (#1480) and supporting parts with Body content in #1455.
## Proposal
<!-- A precise statement of the proposed feature. -->
NCBI has chosen the table of content building improvements below, along with Design option 1.
- #1482
The combination of very long lists and managing the books divisions (front/body/back) with multi-nested structures makes it difficult for users of these kinds of books to curate their TOCs because the space to do this work on the page is limited.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
### Current UI
This is the current UI, an example from [this book](https://ncbidev.cloud68.co/organizations/73814fa2-d9fc-44f8-a12a-d3e52c589230/bookmanager/8e1ce032-ddc1-41fd-9088-7b9a75162f23)
### Design option 1 (chosen)
* Provide more space for TOC curation by separating the front/body/back divisions into three tabs.
* Tabs are positioned below the 'Upload Chapters' button (left of page) and Search input area and buttons (right of page) because the Tabs UI will be replaced with the search results UI (#1467) when users search.
* 'Body' is the default tab when user arrives at the Book manager
![Book_manager-pagination_V1](/uploads/779d1669ccad10473a66a370d5b6db70/Book_manager-pagination_V1.jpeg)
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] All chapter-processed book contents on the Book Manager page is split across three divisions: Front, Body, and Back. These three division can be accesses via tabs at the top left of the page.
- [ ] The Front|Body|Back pages will load within 20 sec (the Silverlight CMS current baseline) in their native environments (within institutional networks or home systems; benchmark is Bookshelf staff and GeneReviews, LiverTox, LactMed editorial team's environments in which they manage their contents)
- [ ] When a supported user navigates to the Book manager page from any other page in the BCMS, the Book Manager page will open with the "Body" tab and its page(s) contents in view.
- [ ] If a user changes a Body view to a Front or Back view using the Front or Back tab, the Front or Back view will remain in view until the user navigates away from their current Book Manager page or changes from a Front view to a Body or Back view or from a Back view to a Body or Front view.
- [ ] If a user changes a Front view to a Body or Back view using the Front or Back tab, the Body or Back view will remain in view until the user navigates away from their current Book Manager page or changes from a Body view to a Front or Back view or from a Back view to a Front or Body view.
- [ ] If a user changes a Back view to a Body or Front view using the Body or Front tab, the Body or Front view will remain in view until the user navigates away from their current Book Manager page or changes from a Body view to a Front or Back view or from a Front view to a Body or Back view.
- [ ] A user will see the paginated design in #1480 when implemented on all Front|Body|Back pages.
- [x] Users will not see a Repeat button on Front|Back pages as this function is not possible in those divisions (Pending in task #1544)
- [x] The design will not change the current rules for how the BCMS automatically moves components to the correct book division after conversion. The current functionality remains: Components are placed in 'body' when uploaded, and move to the the corrected division after conversion:
| Type of Document You Wish to Create | Value in Word template | Tagging from conversion | Section placement |
|-------------------------------------|------------------------|-------------------------|-------------------|
| appendix | appendix | `<book-part-wrapper content-type="appendix" id="">`; `<book-app book-part-type="appendix">` or `<book-part book-part-type="appendix">` | back |
| chapter | chapter | `<book-part-wrapper content-type="chapter" id="">`; `<book-part book-part-type="chapter">` | body |
| dedication page | dedication | `<book-part-wrapper content-type="dedication" id="">`; `<dedication>` | front |
| foreword | foreword | `<book-part-wrapper content-type="foreword" id="">`; `<foreword>` | front |
| frontmatter part | front-matter-part | `<book-part-wrapper content-type="front-matter-part" id="">`; `<front-matter-part>` | front |
| glossary | glossary | `<book-part-wrapper content-type="glossary" id="">`; `<book-part book-part-type="glossary">` | back |
| preface | preface | `<book-part-wrapper content-type="preface" id="">`; `<preface>` | front |
| reference list | ref-list | `<book-part-wrapper content-type="ref-list" id="">`; `<book-part book-part-type="ref-list">` | back |
| section | section | `<book-part-wrapper content-type="section" id="">`; `<book-part book-part-type="section">` | body |
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] All end to end tests are finished and implemented by Coko: the tests will make sure the design is implemented correctly by checking the tabs as well the test will check that the components are positioned in the right division, by uploading the converted files to them (so we don't wait for conversion in pipeline).
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” of cloud68 hosted site, by Coko
- [ ] Deployed and tested on “ncbi” of cloud68 hosted site, by NCBI team
- [ ] Deployed and tested in NCBI-hosted environment by NCBI team and any necessary external editorial teams
- [ ] NCBI approves that all Acceptance Criteria have been Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
Development broken down in tasks below: #1514
## Alternative approaches (if applicable)
<details><summary>Alternative approaches (not chosen)</summary>
## Alternative approaches (not chosen)
NCBI and Coko will agree on one clear development path of least effort to minimally address this issue for MVP that will still meet user stories for a stable data model, performance of the application, and file management, as well as other user stories such as metadata, parts, and complete documents. Possible technical solutions to evaluate least effort for a stable MVP solution include:
### Design option 2
* Maintain full TOC divisions in one page in a separate view for TOC building actions only. -- **Development estimate 3 days**
* This design is compatible with Design option 3 proposed in the Design options for resolving chapter-processed book manager performance issues (#1480)
* The Order UI is only necessary when manual ordering is required
* The Order UI is placed with exiting TOC Live view and Errors reporting
![New_TOC_ordening](/uploads/80074bbdd5117a0389b3fe0d3cdf0fb9/New_TOC_ordening.jpeg)
### Approach A
- BCMS displays position number of book part in UI relative to structural division (front | body | back) and any parts, and permits a right click of that book part to provide a reposition modal for the user to reposition that book part. The modal would be pre-filled by the UI with the current position. Note, there is no restriction on how many parts a user can create. Parts (and how many levels of Parts) dropdown would only appear if those Parts already exist. See proposed mockup from @deniskar:
![image__18_](/uploads/931a20427248465200d2adaa099db044/image__18_.png)
BCMS will not permit loading of TOC if any published book part does not have a position or has a duplicate position. This would be in addition to automated ordering, including all labels, and all publication and publication history dates. NCBI and Coko will provide a clear technical proposal for the estimates of this work to successfully meet these user stories and issues before a full development agreement can commence, so its deliverables are understood by all parties , OR
**Development estimate 5 days**
Note: this approach does not account for moving items in bulk.
### Approach B
- Bookshelf and PMC developers support the provision by a submitter in source files, sorting information within front matter, book body, a book part, or back matter that is passed in a consistent BITS element and attribute to the converted BITS XML stored in the BCMS, and which the BCMS can read to build a TOC or communicate submitter errors to fix if this needed information is missing or problematic. These errors will block publication of a TOC. Coko will read ALL supported BITS elements, not only explicit ordering information, but also other metadata for automated ordering, including all labels, and all publication and publication history dates. NCBI and Coko will provide a clear technical proposal for the estimates of this work to successfully meet these user stories and issues before a full development agreement can commence, so its deliverables are understood by all parties , OR
**Development estimate 3 days**
Note: Coko does not recommend this approach because it will place a lot of burden on users.
### Approach C (This is estimated in Design Option 2 above)
- Separate out ALL or PARTS of the TOC Building from the chapter-processed Book Dashboard for Files Management and Processing Integration. Possible design:
![image](/uploads/5eab2e4b9d84c89fbca5f9ca423b3f73/image.png)
OR
### Approach D
- Coko will add functionality for Bookshelf staff to download from the BCMS TOC.XML to adjust the order of contents in the XML and upload the file and reload it to PMC Books successfully. Note, it would be important for user to have BCMS create a TOC, download it, and just update it and reupload it with some modifications if too difficult using UI. User story will not support all users supplying a TOC from scratch. Possible design:
![image](/uploads/19c373b90a8aa6b6b3801b5dd81a099f/image.png)
If this last option is chosen and this could be extended with least effort to resolve TOC.xml issues, THEN add functionality for Bookshelf staff to download from the BCMS TOC.XML written by BCMS with loading errors to fix those errors in the XML and upload the file and reload it to PMC Books successfully
**Development estimate 7 days**
Note:
* This design is compatible with Design option 3 proposed in the Design options for resolving chapter-processed book manager performance issues (#1480)
* if this approach is taken our assumption is that uploading a TOC replaces the need for any manual ordering, but we continue to support automatic ordering and building TOC.xml for the auto-ordered cases.
Therefore all toc.xml writing issues must be addressed.
</details>
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* Milestone: linked
* Iteration: linked
* Dependencies: no
* Development estimate (hours): added to time trackingP1a: Book Manager redesign: HighStacy LathropDione Mentisdione@coko.foundationStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1471Revise user permissions rule for assigning book and book component editors an...2024-01-17T06:25:19ZDione Mentisdione@coko.foundationRevise user permissions rule for assigning book and book component editors and authors<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
This issues describes the required changes to user permissions on books and book components to resolve book manager performance issues. The BCMS currently has permissions rules that say:
* when a user creates a book, make that user the editor of the book
* when a user creates a chapter (by uploading a file) make that user the author of the chapter. (this is done on the backend only at this stage)
However, as NCBI has reported (see epic &106 and [457#note_108306](https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1457#note_108306))
>Stacy's account is editor to migrated books and chapters. Having a reference list of all books and chapters that she's an editor for, and checking books against these lists causes severe performance degradation and consistent browser crashes for Stacy's account. Instead, permissions should probably be part of the getBook call lists should be removed.
![image-1](https://gitlab.coko.foundation/groups/ncbi/-/uploads/2d630c0654e9324d599f19a03b7981c8/image-1.png)
![image-2](https://gitlab.coko.foundation/groups/ncbi/-/uploads/46a1d908e9aad050e1e9d8b68b26e010/image-2.png)
![image-3](https://gitlab.coko.foundation/groups/ncbi/-/uploads/c295273da003a6600dac244bb4d24fcb/image-3.png)
## Proposal
<!-- A precise statement of the proposed feature. -->
An editor or author role should not be required. Therefore, revise the permissions rules:
1. When a user creates a book, make that user the editor of the book *unless* the user is a System Admin of the BCMS or an Org Admin of the Organisation
2. When a user creates a chapter (by uploading a file) make that user the author of the chapter *unless* the user is a System Admin of the BCMS; or Org Admin of the Organisation; or an Editor of the Organisation
On the frontend the change is: A system admin or Org Admin will not be listed on the "Team" tab in the "Editor" section for a book or chapter and an Editor of an Org will not be listed on the "Team" tab in the "Editor" section for a chapter.
Note: the user who creates the book or chapter component is recorded as the `owner`. In future we can expose this information for activity tracking in the UI. E.g. "Stacy created 'book title' on 'date'."
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
* A system admin or Org Admin will **not** be listed on the "team" tab in the "Editor" section in the Team modal of a book. (see screenshot example below)
* A system admin, Org Admin, and Editor will still have the relevant permissions as per [BCMS permissions rules](https://docs.google.com/spreadsheets/d/1cJeKIUhkkM_kwQ8Fzh9d2Vw9h2IJcgT3_87sWLiNdNQ/edit?usp=sharing)
![Screenshot_2023-08-02_at_15.53.27](/uploads/3ccdbf9e66b56e9b8c9a8994c9a9fd5e/Screenshot_2023-08-02_at_15.53.27.png)
Note: the performance from the users perspective will be tested after all linked items are complete
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [ ] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [ ] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature.-->
* Update the permissions rules above
* Any time a user is assigned Org Admin or Sys Admin role, revoke their role-specific privileges on collections, books, and chapters.
* Write migration script for all current data to remove Sys admins and Org Admins users as "editors" of book and chapters that they created
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [x] Milestone is linked
* [x] Iteration is linked
* [x] Dependencies: none
* [x] Development estimate is added to issue time trackingP1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1472Remove query to display {book-component-version}.{file-version} number on the...2024-01-17T06:25:09ZDione Mentisdione@coko.foundationRemove query to display {book-component-version}.{file-version} number on the chapters rows of the book manager<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The book manager displays a list of book components. Each component may have multiple versions -- each of which may have been published before. A current requirement of the book manager is: For each book component listed, display the published date and `{book-component-version}.{file-version}` number of whichever `{book-component-version}.{file-version}` was most recently published.
When books have a high number of book components, the queries to get this data is affecting the performance and therefore usability of the book manager page.
## Proposal
<!-- A precise statement of the proposed feature. -->
Remove the query to get the `{book-component-version}.{file-version}` value for "last published". Get "last published" date only. NCBI has agreed to this in a meeting (1 Feb '23) since it does not negatively impact users ability to perform their tasks on the book manager page.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
Only design change is the removal of the version number in brackets after the "last published" date. Here is the current design:
![image](/uploads/6ee80196c334bfd6183d1141091159e8/image.png)
Here is the design after this implementation:
![image](/uploads/512fc8572182a65d60f46b039417e567/image.png)
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] Each chapter listed on the book manager page lists the "Last published" date if the chapter has been published
- [x] The component.file version number will no longer display after the "Last published date" on the book manager page - see screenshot below of what will be removed (crossed out in blue) in the highlighted information. **Note:** Existing users may need to hard refresh their browsers to see this change.
Screenshot from: https://ncbi.cloud68.co/organizations/6c4d37c4-b9f2-4962-9080-b3e29a37dfbd/bookmanager/7f099bb1-3379-4beb-bc4b-aeca26884a47
![image](/uploads/9271a3bb44e10af981215b3a59bef97d/image.png)
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” of cloud68 (by Coko team)
- [x] Deployed and tested on “ncbi” of cloud68 (by NCBI team)
- [x] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* Milestone:
* Iteration:
* Dependencies: (list issue numbers if relevant)
* Development estimate (hours):P1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1480Implement pagination on main chapter-processed Book Manager dashboard page to...2024-01-17T06:25:03ZDione Mentisdione@coko.foundationImplement pagination on main chapter-processed Book Manager dashboard page to resolve performance issues<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The context for the performance issues are in the Epic description: https://gitlab.coko.foundation/groups/ncbi/-/epics/106. This issue also relates to the proposal TOC building in #1458 and supporting parts with Body content in #1455. Also see [NCBI's TOC analysis of their current chapter content](https://docs.google.com/spreadsheets/d/1LhQ1oKz9cl3pqHLB4E4nwsV8oFgoMlsz/edit?usp=sharing&ouid=110829906581357764134&rtpof=true&sd=true) and [TOC analysis of current part content](https://docs.google.com/spreadsheets/d/13MfGk5JoUCwqHlWjz0ASvDH2LYK8n3UK/edit#gid=941617861)
## Proposal
<!-- A precise statement of the proposed feature. -->
NCBI has chosen the performance improvements below, in combination with Design option 1.
- #1471
- #1472
- #1479
For future-proofing beyond MVP we can look to known large book examples that NCBI has provided and confirmed we should not rule out as a possibility. The largest example provided to date is a book with 35 405 chapters. It is therefore a requirement to design for very large lists of data.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
### Design option 1 (chosen)
* Improve performance by splitting long lists over pages -- **Development estimate 3 days**
* The amount of items per page can be experimented with, taking into account:
* the expected number of files being processed at one time (50 components).
* the expected **highest** number of chapters within a part
* the expected **highest** number of chapters within a sub-part, and **highest** number number of sub-parts within a part
* A good starting point is having 100 top-level components per page.
* Having the same amount of items per page introduces consistency which provides a short-hand for users to quickly jump to a particular section on the book.
![Book_manager-pagination_V1](/uploads/a7b259396633b68096713559ae612e24/Book_manager-pagination_V1.jpeg)
## Data from NCBI on TOC structures
1. The **highest** known number of chapters within a part is currently **271 chapters** in a part for the domain **mtb** and that number **gets larger about \~1-5 new chapters every month**. This book will be processed in the BCMS as chapter-processed. The parts are Languages, so you get the same drug record in every part.-- see comment reference https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1480#note_119260
2. The **highest** known **nested levels of parts is three**, with the **highest** known number of parts within parts about **~20**, and the **highest** known entire set of a top level part with all its descendant parts and chapters **613** components. This is for the domain **micad**.
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [ ] On the main Book Manager dashboard page for chapter-processed books, on the Front, Body, and Back tabs, users will see a pagination at the bottom of the page that displays at least the first and last page number (and however many page numbers in between the design specifies). Clicking on one of these page numbers brings the user to that page within whichever division they are in (Front, Body, Back). The pagination navigation also includes a button for the user to navigate to the previous page, and a button for the user to navigate to the next page.
**Pagination Rules and Functionality**
- [ ] The initial implementation of the pagination will display 100 records per paginated page on the main Book Manager page for chapter-processed books. If we find during implementation or review that the pages do not load within 20 sec (the Silverlight CMS current baseline) in their native environments (within institutional networks or home systems; benchmark is Bookshelf staff and GeneReviews, LiverTox, LactMed editorial team's environments in which they manage their contents) or select and expand/collapse functionality on each paginated page is not responsive in transactions within 5 seconds, then this default number may need to be changed.
- [ ] The default view of a paginated Book Manager page is that all parts are collapsed.
- [ ] When a part is collapsed either when a user clicks on collapse all or using the collapse toggle for a particular part, only the top-level part counts toward the 100-record limit (not the subparts or chapters contained within that top-level part).
- [ ] For instance, in the case of 'mtb` (as of 20230809) a user when entering the Book Manager page would see two collapsed parts, which count as two records toward the 100-record limit per page. When they expand these two parts, they would see all ~ 542 chapters of that book on the first page, with each chapter nested within its associated parent part.
- [ ] For instance, in the case of 'micad` (as of 20230809) a user when entering the Book Manager page would see 7 collapsed top-level parts, which count as 7 records toward the 100-record limit per page. When they "Expand All", they would see all ~ 148 children parts grouped within one of those 7 parent parents they belong, and all ~2888 chapters grouped within one of the 148 second-level parts they belong. The user would see all of this content on page one of the Book Manager.
- [ ] For instance, in the case of `gtrbook` (as of 20230809) a user when entering the Book Manager page would see the Introduction chapter component that doesn't belong to any part, followed by two collapsed top-level parts ("Genetic variants and drug responses" AND "Genetic variants and disease"), which count as 3 records toward the 100-record limit per page. When they "Expand All", they would see all other ~ 70 chapters grouped within their parent part. The user would see all of this content on page one of the Book Manager, even if another batch of ~ 25 new chapter records were added and moved within one of the top-level parts.
- [ ] Each standalone chapter that is not grouped within a part always counts as one record on the page.
- [ ] Anything past the 100 record limit per page according to these pagination rules will get pushed to the next page.
- [ ] When a System Admin, Org Admin, Editor clicks on the Previous button it will go to the page previous to the one they are currently on and display that page's 100 results
- [ ] When a System Admin, Org Admin, Editor clicks on the Next button it will go to the page next to the one they are currently on and display that page's 100 results
- [ ] When a System Admin, Org Admin, Editor clicks on a page number it will go to that page and display that page's 100 results
**Performance of Paginated Pages**
- [ ] Each paginated page in the main, unfiltered / searched Book Manager dashboard page will load within 20 sec (the Silverlight CMS current baseline) in their native environments (within institutional networks or home systems; benchmark is Bookshelf staff and GeneReviews, LiverTox, LactMed editorial team's environments in which they manage their contents)
- [ ] System Admin, Org Admin, Editors should expect select functionality and expand/collapse functionality on each paginated page to be responsive for transactions within 5 seconds
**Related Book Manager page functionality to test at the same time**
- [ ] A user will have the capability to expand all parts via a link at the top right of each paginated page. This link will only expand all parts on the paginated page the user is situated.
- [ ] A user will have the capability to collapse all parts via a link at the top right of each paginated page. This link will only collapse all parts on the paginated page the user is situated.
- [ ] In a Manual Sort Order for chapter components, System Admin, Org Admin, Editors should expect to see content in a New Upload status at the top of the first paginated page
- [ ] In a Manual Sort Order for Parts, System Admin, Org Admin, Editors should expect to see newly created parts at the top of the first paginated page
- [ ] Chapters and parts sorted and grouped according automated rules will remain where it has been placed according to those automated rules in the structure of the book regardless of which paginated page that might be
- [ ] Each paginated page will show all supported BCMS bulk action buttons, currently permitting System Admin, Org Admin, Editors to successfully:
- [ ] Bulk Upload files
- [ ] Add a part title
- [ ] Bulk Submit files
- [ ] Bulk Reload Preview files
- [ ] Bulk Publish files
- [ ] Move components in a TOC or within a Part
- [ ] Repeat components in a TOC
- [ ] The Select all functionality on each paginated page of the Book Manager dashboard will only select all component items only on the paginated page the user is currently situated.
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] All end-to-end tests written and integrated in pipeline by Coko
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Deployed and tested to NCBI hosted sites and tested by NCBI power users in their native environments
- [ ] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
1. Introduce pagination -- https://gitlab.coko.foundation/ncbi/ncbi/-/work_items/1508
2. Test with data from NCBI (see above)
3. Review testing results with NCBI before going any further
## Alternative approaches (if applicable)
<details><summary>Alternative approaches (not chosen)</summary>
## Alternative approaches (not chosen)
### Design option 2
* Improve performance by allowing user to load more chapters into view when needed. -- **Development estimate 3 days**
* The amount of items shown prior to the user having to select 'load more' can be experienced with, taking into account:
* the expected number of files being processed at one time (50 components).
* the expected number of chapters with a part in most cases (?)
* suggest starting with 100 top-level components per page.
There's an assumption here that users are unlikely to load all the chapters within very large because they are automatically ordered and they can find what they're looking for via the search functionality. Eventually we'll reach a threshold where loading too many chapters will cause a performance issue so we should users reason for loading high amount of chapters at one time when considering this design option.
![Book_manager-pagination_V1-load](/uploads/e11707c4959806ca572cc08491086b08/Book_manager-pagination_V1-load.jpeg)
### Design option 3
The third option combines two approaches and splits the file processing actions from the table of contents building actions
* Improve performance by splitting long lists over pages in "Dashboard style" (for file processing) -- **Development estimate 5 days**
* Items are ordered by same default as the Dashboard: last updated
* Nesting is removed
* The amount of items per page can be experienced with, taking into account the expected number of files being processed at one time (50 components).
* suggest starting with 100 components per page.
* Improve performance by simplifying list requirements (for TOC building).
* page accessed via "TOC" button in top right
* only be necessary when manual ordering is required
* requires only `id` and `title`
![Book_manager_option_3](/uploads/3f7cf89a7538663751a0cf2ba0e04583/Book_manager_option_3.jpeg)
![New_TOC_ordening](/uploads/80074bbdd5117a0389b3fe0d3cdf0fb9/New_TOC_ordening.jpeg)
</details>
## Scheduling
* Milestone:
* Iteration:
* Dependencies: (list issue numbers if relevant)
* Development estimate (hours):P1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1481Body → Chapter (required) Descending date order2024-01-17T06:24:49ZDione Mentisdione@coko.foundationBody → Chapter (required) Descending date order<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The context is provided in the Epic https://gitlab.coko.foundation/groups/ncbi/-/epics/2 and Table of contents building user stories in #1458.
[NCBI did an analysis of their TOCs](https://docs.google.com/spreadsheets/d/1LhQ1oKz9cl3pqHLB4E4nwsV8oFgoMlsz/edit?usp=sharing&ouid=110829906581357764134&rtpof=true&sd=true) to find out what automatic ordering cases could be developed to reduce the need for users to manually order books. Each new ordering option is a separate issue linked below -- each to be developed in the priority order specified.
## Proposal
<!-- A precise statement of the proposed feature. -->
Extend the options supported by including:
| Book division | Ordering rule |
|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Body → Chapter (required) | Descending date order/most recent date at top, in the following priority order: 1) Publication date; 2) If ‘publication date’ does not exist use ‘updated date’; 3) If ‘updated date’ does not exist, use ‘created date’. |
See Tab 2 of [BCMS_TOC_Chapters_Sorting_Anaysis](https://docs.google.com/spreadsheets/d/1LhQ1oKz9cl3pqHLB4E4nwsV8oFgoMlsz/edit?usp=sharing&ouid=110829906581357764134&rtpof=true&sd=true) for details on the metadata nodes that are relevant to get the order values.
### Sample content
**NOTE WE NEED A RICHER DATA SET WITH ALL PERMUTATIONS OF 1-3 IN ONE BOOK LOGIC, BUT PROVIDING JUST ONE EXISTING EXAMPLE FOR EACH ELEMENT AND ATTRIBUTE TO KEY OFF**
#### Publication Date
* Final Rendered TOC Example with pub-date: https://www.ncbi.nlm.nih.gov/books/NBK512958/
* Sample converted chapter XML file with pub-date: [RB-0032-Williams.xml](/uploads/46ed7a3ee037d003a7a0dd8424b43b48/RB-0032-Williams.xml)
#### Publication history - updated date
* Final Rendered TOC Example with updated date in publication history: https://www.ncbi.nlm.nih.gov/books/NBK570439/
* Sample converted chapter XML with updated date in publication history:[indicators-monitor.xml](/uploads/f2a90cf21a43797832bf6f2717f763fe/indicators-monitor.xml)
#### Publication history - created date`
* Final Rendered TOC Example with created date in publication history: https://www.ncbi.nlm.nih.gov/books/NBK62348/
* Sample converted chapter XML with created date in publication history:[consinsomnia.xml](/uploads/d9b2878f8348b5f0e441e5aa0d538401/consinsomnia.xml)
### Ordering spec
In the following priority order:
1. Publication date
2. If ‘publication date’ does not exist use ‘updated date’
3. If ‘updated date’ does not exist, use ‘created date’
XML pub-date element used to order content and target to write in TOC.XML
`<pub-date publication-format="electronic" date-type="pub"><month>12</month><year>2022</year></pub-date>`
where publication-format can be any and date-type can be any
Example TOC.XML entry
`<toc-entry><title>COVID-19&#x2019;s Impact on Clinical Research</title><?ppub 20221201?>`
XML publication history element used to order content and target to write in TOC.XML
`<pub-history><date date-type="updated"><day>11</day><month>8</month><year>2022</year></date></pub-history>`
Example TOC.XML entry
`<toc-entry>
<title>Science Brief: Indicators for Monitoring COVID-19 Community Levels and Making Public Health Recommendations</title>
<?date-updated 20220811?>
XML publication history element used to order content and target to write in TOC.XML
`<pub-history>
<date iso-8601-date="2017-08" date-type="created">
<month>08</month>
<year>2017</year>
</date>
</pub-history>`
Example TOC.XML entry
```
<toc-entry><title>Managing Insomnia Disorder</title><subtitle>A Review of the Research for Adults</subtitle>
<?date-created 20170801?>
```
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
### Current UI
The following currently exist in the organisation Book Templates for chapter-processed books, which gets applied to Book Settings.
![Screenshot_2023-08-04_at_14.38.30](/uploads/fcf9cd65363be4b8fbcf3522b0dacc1a/Screenshot_2023-08-04_at_14.38.30.png)
### Design amendment
There is no design amendment needed to support additional chapter ordering options. The new option will be listed in the current dropdown. The text for the dropdown is:
>Most recent
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] System Admin, Org Admin, and Editors will have the ability to automatically sort their chapters in the body of their book by date order (most recent to oldest) to reduce the level of effort to manualy manage the table of contents of large books across multiple paginated pages.
**Settings**
- [x] On the Book Settings page under the "Landing Page: Table of Contents" section, System Admin, Org Admin, and Editors will have the ability to select the drop down option under "Order Chapter by" to sort by "Most recent."
- [x] The "Most recent" option will appear after the "Chapter label - descending" option in the dropdown menu.
**Book Manager Page**
- [x] On the chapter-processed Book Manager dashboard page, users who have selected to automatically order their settings by "Most recent" will see all of their chapters that have successfully converted appear in descending date order (most recent date at top), according to this order of priority of date information in the converted XML book part metadata: 1) publication date `<pub-date>`; 2) if ‘publication date’ does not exist use ‘updated date’ `<pub-history><date date-type="updated">`; 3) If ‘updated date’ does not exist, use ‘created date’ `<pub-history><date date-type="created">`.
- [x] If the BCMS cannot identify a document label in the converted XML, a red TOC? indicator label will appear at the far right on the row of that chapter component to alert the user that it cannot be properly sorted for lack of information.
**Bookshelf Table of Contents**
- [x] On the TOC page of the BCMS, users who have selected to automatically order their settings by "Most recent" and who have updated the published version of the TOC, will see all of their published chapters sorted in date order from most recent to oldest.
- [ ] If users have selected to "Display on TOC" the "Chapter-level dates" in the "Landing Page: Table of Contents" section of Settings, these dates will appear on the published Bookshelf Table of Contents according Bookshelf rendering rules.
- [ ] The TOC.xml that the user downloads via the Download button on the bottom right of the TOC page and which is sent to NCBI as a chapter ingest job will be valid, meet all Bookshelf tagging guidelines and TOC XML building specification in #24, and display to the same level of quality as the Table of Contents managed when loaded from the Silverlight CMS.
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] All unit tests are written and run by Coko
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] NCBI approves that Acceptance Criteria Met
## Implementation
Some background info for this to make sense :
We have two tables Divisions and BookComponents . A book has 3 divisions (Front , Body, Back) To each record there is an array field bookComponents that saves the id of the bookComponents that leave in the
bookComponent table This is the field where we save the ordering of the bookComponents. The same happens for the BookComponents (Parts) that have other bookComponents under them, there is a field called bookComponents that keeps the order of the ids of other bookComponents (referenced to the same table).
So in our case this is the flow that is happening already:
1. user opens the settings modal of a book and updates the order settings to the value he wants for example to `Arabic numeral label descending`.
2. That means will trigger an update to the settings of that Book which will in turns execute the the orderService functionality. (This works already nothing to change )
You can check that in this file : server/api/graphql/bookComponent/bookComponent.resolvers.js at line 524.
3. By executing this Service an update will happen automatically for that specific Record (Model) and will update the list (in our above example that will update the bookComponents column of either the division or the bookComponent / part).
Order Service Overview
So For this issue to work we need to make use of the OrderService class. which is here `server/services/orderService/orderService.js` . The orderService takes as arguments the model
(Book, BookComponent , etc), and based on the configuration we have implemented already in the Model does the ordering. So for example in the model BookComponent `(server/models/bookComponent/bookComponent.js)` you will see a static function `orderMappings` this takes care of the whole ordering . in That function we configure the identifiers of the Model the fields with the list we want to order (see above bookComponents field) and the field that should be checked for the ordering we want to do for each case (Arabic numeral, Title A-z, etc). Sometimes that field needs some manipulation before can be ordered and for that we have a custom function that we can run before the ordering, For example we want to order by Number 1,2,3, etc, but the field is in the following format: `Part 1, Part 2, Part 3 etc.`
This custom function is located in line 46 of the BookComponent Model (And this where we need to add the extra code for this feature to work).
What we need to add :
This the predicate function :
```
predicate: {
modelClass: Book,
idColumn: 'id',
foreignColumn: 'bookId',
field: book => {
const value = (book.settings.toc || {}).order_chapters_by || 'manual'
if (value === 'number')
return {
field: 'metadata.chapter_number',
type: 'Number',
value: v => (v ? v.replace(/\D/g, '') : 0),
}
if (value === 'title')
return {
field: 'title',
type: 'String',
value: v => {
return v === 'Untitled' ? null : v
},
}
return {
field: value,
type: 'String',
}
},
},
```
The value variable is the value that the user saves and wants to perform the ordering . So for example when the user saves the order as `title`
the service will trigger this function and will do the ordering based the return object
```
return {
field: 'title',
type: 'String',
value: v => {
return v === 'Untitled' ? null : v
},
}
```
and before performing the ordering will execute the value custom function that will normalize a little bit the values (example with Part 1, Part 2, etc.).
So all we need is to add at the field, the ordering cases that come from the ui (title, number) and return the field we want to compare for the ordering and the type.
Currently we support two Types at the OrderService 'String' , "Number" that will be enough for this issue.
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* Milestone:
* Iteration:
* Dependencies: (list issue numbers if relevant)
* Development estimate (hours): 24P1a: Book Manager redesign: Highhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1493Create Grafana server to store and search BCMS application logs2024-01-17T06:24:34ZDione Mentisdione@coko.foundationCreate Grafana server to store and search BCMS application logs## Context
The BCMS has the following staging deployments managed by the Coko team:
1. ncbidev.cloud68.co: used by the Coko team for testing development in progress
2. ncbi.cloud68.co: used by the NCBI team for testing releases
NC...## Context
The BCMS has the following staging deployments managed by the Coko team:
1. ncbidev.cloud68.co: used by the Coko team for testing development in progress
2. ncbi.cloud68.co: used by the NCBI team for testing releases
NCBI has requested another deployment with a dump of the PMC database so that testing can be done on real content. See new instance “bcms-pmc.cloud68.co” in diagram below.
NCBI’s has requested a technical proposal and estimate to integrate the Elastic Stack (or similar) to store and search application logs for all Coko-managed deployments so that developers can search logs by crashes by time, frequency, BCMS URL, any other relevant variables (to be defined).
![The_BCMS_-_BCMS_deployments](/uploads/25bdcee8c73d622d37a26355a2bc0665/The_BCMS_-_BCMS_deployments.jpeg)
## Proposal: Log management
For log management, we will create and deploy a [Grafana Loki server](https://grafana.com/oss/loki/), which in turn will aggregate logs from all of our BCMS deployments and display them in a dashboard. To see what a Grafana interface would look like, check their [website](https://grafana.com/grafana/).
## Design
Not applicable
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] A dashboard user interface is provided through Grafana to allow Coko and NCBI development team (developers, project management, product owner) to search applications logs (with details according to the acceptance criteria for those logs in #1542) from all supported deployed BCMS instances, currently the ncbidev.cloud68.co and ncbi.cloud68.co deployments
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- ~Digital assets added (design, database scheme, mockups etc if relevant)~
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented
- [x] Acceptance Criteria Met
## Scheduling
* Development estimate for the Grafana server: 1 week
* Development estimate for connecting BCMS server to Grafana: 3 daysP1a: Book Manager redesign: HighEvgenyDenis KaramyshevStacy LathropEvgenyhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1486GeneReviews Book Manager Performance Issues with NCBI MVC Changes to Producti...2024-01-17T06:24:20ZStacy LathropGeneReviews Book Manager Performance Issues with NCBI MVC Changes to Production BCMS Deployment@DioneMentis @John.kopanas - cc @deniskar @ErinS
I am providing clips of our user testing with GeneReviews users who must be supported when we deploy the MVP BCMS. They are our most active Word users. This user testing was done on our ...@DioneMentis @John.kopanas - cc @deniskar @ErinS
I am providing clips of our user testing with GeneReviews users who must be supported when we deploy the MVP BCMS. They are our most active Word users. This user testing was done on our Production BCMS Deployment that has some MVC changes by Denis, where he removed several backend calls AND the lazy load which our users found problematic and unusable.
In addition to this, there were at least the two GeneReviews users accessing the Production BCMS at the same time of the recorded user testing. They were logged in from their own home networks in the Seattle, Washington area, which is where they work now and will work in the future. They will always both be using the system at the same time in their Book Manager pages as Editors and Org Admin, likely with a third person, NCBI Bookshelf System Admin, in their Book Manager area when troubleshooting.
These are the issues that they noted would cause them problems working and we recorded, some difficult to see and measure the user frustration, in the slowness and lack of responsivity of even moving up and down the page and navigating and clicking between pages and tabs. Such lags are very hard to record in a way they can be perceived by another as they are tactile in nature.
We could not test any actual processing since we tested on our Production Deployment that is not integrated with our Production databases and file systems.
The clips of the testing I am sharing with issues that I think can be clearly observed and need to be met by the MVC and Design Option A improvements are as follows:
(1) _Problems with bulk upload of chapters_
Includes:
- User sees "Uploading files" when tries to upload files and doesn't understand why as they hadn't uploaded them and doesn't see them on Book Manager page
- User gets kicked out of the modal where it shows the files uploading by the system without choosing to close out of it
- User repeatedly says "it is not responding"
Note in all instances of the BCMS users report they cannot do more than any 1-2 bulk actions right now on ANY Book Manager pages, including those with Denis' optimization of reducing calls to backend. We can provide and record more instances if necessary. In meetings the consensus is that the MVC and Design Option A changes should address these problems doing all bulk actions, but we will need to test this and have GeneReviews redo these usability tests recorded after those changes to confirm they do address these issues with bulk actions.
See clip: ![genebulkuploadproblems](/uploads/08e9508a1367e48e5b152c3967e56156/genebulkuploadproblems.mp4)
(2) _Problems with the search not showing the search results_
Includes:
- User at first tries to type search term and the UI does not show the search term being entered
- Second time, takes 20 seconds (their count while doing it) before user sees their typed text appear
- User cannot use next to navigate to instance(s) of their search terms
- User does not see a filtered list of their search results
Note, this problem I am able to reproduce from my testing instance as well and was reported in additional user testing with LactMed and LiverTox users. We shared the Design Option A search design mockups with these users and theoretically they agree that should address their concerns.
See clip: ![genesearchissues](/uploads/e6691994d5d96d6613fbafd0fcd2bfa2/genesearchissues.mp4)
(3) _Problems with drag and drop of their Appendix items that they will need to order in the UI_
Includes:
- Slow to navigate to the Back matter - feels to drag
- Checking boxes is slow - they counted at about 5-6 seconds
- Can't control or see where you are moving a component - it feels to drag and jump around and never stick
- Item never moves to where you are trying to place it
See clip: ![dragdropissues](/uploads/335bda9c48e7dacee5953d32073e5d80/dragdropissues.mp4)
(4) _Don't see saved Edited Book Metadata changes in the BCMS UI form_
Includes:
- User enters two edits to the book metadata: changing text in abstract and adding a license, but when user saves these changes and closes out of the book metadata and reopens it, they do not see those changes recorded in the forms
See clip: ![metadatadoesnotsave](/uploads/7a7804f049cb7166e250c85861be1ccf/metadatadoesnotsave.mp4)P1a: Book Manager redesign: HighStacy LathropStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1479Pre-calculate values to improve book manager performance and remove unneeded ...2024-01-17T06:23:06ZDione Mentisdione@coko.foundationPre-calculate values to improve book manager performance and remove unneeded query<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The context is described in the Epic description: https://gitlab.coko.foundation/groups/ncbi/-/epics/106. Coko team investigated additional backend improvements to improve book manager performance.
## Proposal
<!-- A precise statement of the proposed feature. -->
Pre-calculate the following:
1. The value for TOC tags: 'repeat' and 'exclude'
2. Error detail for values: 'publisher' or 'PMC'
Remove:
The In review status – we should close this query since it’s not supported for MVP. Currently the functionality is only hidden in the frontend.
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [ ] Compare the current logs to the new logs to show the improvement on the postgres queries.
- [ ] Save "In Review" query in separate branch for future use.
Note: the performance from the users perspective will be tested after all linked items are complete
### Log for NCBI developers' review:
Logs of postgresql queries and graphql Requests before and after removing the unneeded queries:
[postgresql-before-remove-queries.log](/uploads/c14b9cc4e9d55371d05d55ba4561b2eb/postgresql-before-remove-queries.log)
[loading-time-after-removing-queries](/uploads/05edf34eeed46df748c42239bda5e4bd/loading-time-after-removing-queries.png)
[postgresql-after-remove-queries.log](/uploads/6d80ba44f6aa400df1b558f11bf12ed2/postgresql-after-remove-queries.log)
[loading-time-before-remove-queries](/uploads/bac0664c0df81e49134e7ee860125556/loading-time-before-remove-queries.png)
### Brach with saved "In review" code
https://gitlab.coko.foundation/ncbi/ncbi/-/tree/remove-review-backend
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [x] Milestone is linked
* [x] Iteration is linked
* [x] Dependencies: none
* [x] Development estimate is added to issue time trackingP1a: Book Manager redesign: Highhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/676Support Proper Display of Funded Content: Selected Chapters; Author MS; Prep...2024-01-16T18:39:55ZDione Mentisdione@coko.foundationSupport Proper Display of Funded Content: Selected Chapters; Author MS; Prepublication draft; Published PDFHi @lathrops1, @Kireev, @latternm
cc @John.kopanas @andynicholson
## When a chapter-processed book belongs to a "funded" collection
All chapter-processed books within a "funded" collection should have `<?select_chapters?>` added to ...Hi @lathrops1, @Kireev, @latternm
cc @John.kopanas @andynicholson
## When a chapter-processed book belongs to a "funded" collection
All chapter-processed books within a "funded" collection should have `<?select_chapters?>` added to the `<book-meta>` node
### Funded content type: Author Manuscript value
This use case applies to chapter-processed and wholebooks in the PDF workflow.
In the chapter-processed case where BCMS writes book metadata:
1. NCBI sys admins should upload the manuscript note text (that Apex should write into the `book-part-meta`) to the "support" folder of the relevant chapter
2. BCMS writes chapter-processed book metadata with the processing instruction `<?book-chapter-manuscript?>` PI as child of `<book-meta>` into the converted.xml file received from Apex. For example:
```xml
<book-part-wrapper xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" id="ch5" content-type="chapter" from-where="body">
<collection-meta collection-type="ncbi-books-collection">
<collection-id collection-id-type="pmcid">wtcollect</collection-id>
<title-group>
<title>Wellcome Trust–Funded Monographs and Book Chapters</title>
</title-group>
</collection-meta>
<book-meta>
<?select_chapters ?>
<?book-chapter-manuscript ?>
<book-id book-id-type="pmcid">wt605993</book-id>
...
</book-meta>
```
In the wholebook use case or chapter-processed where BCMS does not write XML:
* BCMS **does nothing**, there is not integration for this setting in phase 1
* NCBI sys admins should upload the manuscript note text (that Apex should write into the `book-meta`) to the "support" folder of the book. This note should include the instruction to write the processing instruction `<?book-chapter-manuscript?>` as a child of (`<book>`)
### Funded content type: Prepublication Draft and Published PDF value
This applies to wholebooks and chapter-processed books BCMS does not write book metadata in the XML and PDF workflow.
* BCMS **does nothing**, there is not integration for this setting in phase 1
* In the PDF use case: NCBI sys admins should upload a note to the "support" folder of the book, with instruction to APEX to writes the PI `<book-meta><?da-pubstatus pdf-prepub?>` or `"funded-content-type: "pdf-prepub"` as child of `<book-meta>`.
* In the XML use case: NCBI sys admins writes the PI `<book-meta><?da-pubstatus pdf-prepub?>` or `"funded-content-type: "pdf-prepub"` as child of `<book-meta>` before uploading the converted XML
## Acceptance criteria
- [ ] When BCMS writes book metadata XML, BCMS communicates to NCBI per provided specifications if submitted content is only a selected chapter so that Selected Chapters displays on Collection pages of the NCBI Bookshelf site(s) like this: https://www.ncbi.nlm.nih.gov/books/NBK327155/
- [ ] When BCMS writes book metadata XML, BCMS communicates to NCBI per provided specifications if submitted content is an author manuscript so that it displays on the NCBI Bookshelf site(s) like this: https://www.ncbi.nlm.nih.gov/books/NBK572928/
- [ ] When BCMS does not write book metadata XML, BCMS supports NCBI to communicate content is an author manuscript so that it displays on the NCBI Bookshelf site(s) like this: https://www.ncbi.nlm.nih.gov/books/NBK572928/
- [ ] BCMS supports NCBI to add prepublication draft PIs to converted files so that it displays on the NCBI Bookshelf site(s) like this:
![image](/uploads/4ecffc7b43a325257691fcbd469e06fe/image.png)
- [ ] BCMS supports NCBI to add final published PDF first PIs to converted files so that it displays on the NCBI Bookshelf site(s) like this: https://www.ncbi.nlm.nih.gov/books/NBK570371/P06: MVP Data Model and Integrity Issues to Ensure Data Quality in Bookshelf and NLM Databaseshttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1436Draft: Mitigate risk of Review and Help chat messages without @ mentions, whi...2024-01-16T18:39:54ZChristinaTrompDraft: Mitigate risk of Review and Help chat messages without @ mentions, which won't notify users<!-- Required. Provide a general summary of the issue in the title above -->
## Context
If a message in the Review and Errors chat doesn't @ someone, there is a risk that the message won't be seen by the relevant people since they won'...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
If a message in the Review and Errors chat doesn't @ someone, there is a risk that the message won't be seen by the relevant people since they won't be notified.
## Proposal
Show a dismissible warning before sending a message if the message does not contain an @ mention, to mitigate the risk of a user sending a message without an addressee, while reducing the number of unnecessary notifications a user receives (as there are valid cases in which a user might send a message without an @ mention and the team might be very large). The message would say 'Are you sure you want to send this message without @ mentioning a member of the team? Only team members who are mentioned will be notified of this message.'
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
## Implementation (if applicable)
<!-- A description of the steps to implement the feature.-->
## Alternative approaches (if applicable)
If a message in the Review or Help chats doesn't @ someone, send the message to all members of the team and the Sys admin assigned.
## Scheduling
<!-- Select the relevant option -->
- [x] This feature is required for [Priority 1: Deploy MVP](https://gitlab.coko.foundation/groups/ncbi/-/epics/56)
- [ ] This feature is required for [Priority 2: Deploy Fully Production-Ready BCMS with All content migrated, some power users onboarded](https://gitlab.coko.foundation/groups/ncbi/-/epics/60)
<!-- Assignee and labels are added automatically. After creating this issue you can link other related or blocking issues with the Gitlab's Linked issues functionality. -->https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1165Changes to when we show Display and Create PDF settings2024-01-16T18:39:54ZDiana jordanChanges to when we show Display and Create PDF settings@lathrops1
## Expected behaviour / Acceptance Criteria
### All Word books:
Should show:
- Create book-level PDF for download
- Display book-level PDF for download
- Create chapter-level PDF for download
- Display chapter-level PDF f...@lathrops1
## Expected behaviour / Acceptance Criteria
### All Word books:
Should show:
- Create book-level PDF for download
- Display book-level PDF for download
- Create chapter-level PDF for download
- Display chapter-level PDF for download
### All XML books:
Should show:
- Display book-level PDF for download
- Display chapter-level PDF for download
### All PDF books:
Should show:
1. Create book-level PDF for download
1. Display book-level PDF for download
1. Create chapter-level PDF for download
1. Display chapter-level PDF for download
## Current behaviour
### Word one doc shows:
- Create book-level PDF for download
- Display book-level PDF for download
### Word CPB shows:
- Create book-level PDF for download
- Display book-level PDF for download
- Create chapter-level PDF for download
- Display chapter-level PDF for download
### For Word, One docs and chapter processed books will show, after the changes (therefore change required to Word One doc case):
- Create book-level PDF for download
- Display book-level PDF for download
- Create chapter-level PDF for download
- Display chapter-level PDF for download
### XML wholebook currently shows:
- Display book-level PDF for download
- Display chapter-level PDF for download
### XML CPB currently shows:
- Display book-level PDF for download
- Display chapter-level PDF for download
### For XML wholebooks and chapter processed books, after this change will show (therefore no changes required for XML):
- Display book-level PDF for download
- Display chapter-level PDF for download
### PDF wholebooks currently show:
- Display book-level PDF for download
- Display chapter-level PDF for download
### PDF CPB shows:
- Display book-level PDF for download
- Display chapter-level PDF for download
### After this change, PDF wholebooks and chapter processed books will show (therefore change required to both PDF wholebooks and chapter processed books):
- Create book-level PDF for download
- Display book-level PDF for download
- Create chapter-level PDF for download
- Display chapter-level PDF for download
## Steps to reproduce
1. go to https://ncbi.cloud68.co/organizations/8867a9b9-5629-4d10-89ce-1baeb59d7acb/bookmanager/5ec03c59-d53d-4ba1-97c2-8186c9043380
2. Go to Settings - look at the Landing Page: Downloads section:
![image](/uploads/ae5f06683dd45583921463ec20e27876/image.png)
## NCBI's priority feedback
Y, need for accurate migration
## QA Steps
[To be completed by Coko once dev is done]P06: MVP Data Model and Integrity Issues to Ensure Data Quality in Bookshelf and NLM Databaseshttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1366Able to copy bold style in collection title2024-01-16T18:39:54ZStacy LathropAble to copy bold style in collection title## Expected behaviour
Bold is not a supported style for collection title
## Current behaviour
User is able to introduce bold into a collection title by copy and paste
## Steps to reproduce
See: https://ncbi.cloud68.co/organizations/...## Expected behaviour
Bold is not a supported style for collection title
## Current behaviour
User is able to introduce bold into a collection title by copy and paste
## Steps to reproduce
See: https://ncbi.cloud68.co/organizations/aa8b10ab-a0ea-41f2-897a-1cae9a632296/collectionmanager/b84235d0-784d-4e20-b5e2-7ad09474c41b
## NCBI's priority feedback
Y, if the <bold> is getting written into collection metadata
## QA Steps
[To be completed by Coko once dev is done]https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1317Upload buttons appear inactive for books that are not the latest version2024-01-16T18:39:54ZErin SchmiederUpload buttons appear inactive for books that are not the latest version## Expected behaviour
Based on BCMS Roles Permission Spreadsheet spreadsheet ("PDF and XML Wholebook versions" tab), it looks like these upload buttons should be active for sys admin, org admin, and editor users.
## Current behaviour
...## Expected behaviour
Based on BCMS Roles Permission Spreadsheet spreadsheet ("PDF and XML Wholebook versions" tab), it looks like these upload buttons should be active for sys admin, org admin, and editor users.
## Current behaviour
- Upload buttons for “Bookshelf Display PDFs”, “Supplementary”, “Images”, and “Support” under Files tab are inactive
- Only seeing this issue if the book is not the latest version (Upload buttons are active in latest versions)
- All books where I'm seeing this issue are in Published status
## Steps to reproduce
Organization: Erin Book Versions Permissions Testing
https://ncbi.cloud68.co/organizations/03f79aa4-1577-43eb-99e8-8b9b827f0ba5
Books:
Coronavirus Disease 2019 (COVID-19) Treatment Guidelines - Prepublication Draft v1
Coronavirus Disease 2019 (COVID-19) Treatment Guidelines - Published PDF v36
Coronavirus Disease 2019 (COVID-19) Treatment Guidelines - Final Full Text v37
1. Open book
2. Navigate to Files tab
3. Upload buttons for “Bookshelf Display PDFs”, “Supplementary”, “Images”, and “Support” under Files tab are inactive
## Environment
PC, Microsoft Edge
## Possible solution
N/A
## NCBI's priority feedback
[Select "Y" or "N" and provide an explanation]
1. This bug is blocking NCBI's work on migration planning for deployment 1 (Y/N)
2. This can be prioritised after deployment 1 (Y/N)
cc @lathrops1
## QA Steps
[To be completed by Coko once dev is done]Stacy LathropChristinaTrompStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1138Users Should be able to Upload Display PDFs for Word One Doc Books2024-01-16T18:39:53ZStacy LathropUsers Should be able to Upload Display PDFs for Word One Doc Books@DioneMentis @ChristinaTromp -
cc @jordandc @douglassue
## Context
In working on migration and testing the new feature to support our Word One Doc Books, we see we have the use cases of some of our providers submitting this kind of b...@DioneMentis @ChristinaTromp -
cc @jordandc @douglassue
## Context
In working on migration and testing the new feature to support our Word One Doc Books, we see we have the use cases of some of our providers submitting this kind of book having their own Display PDFs that we display on the Bookshelf website.
Example: https://www.ncbi.nlm.nih.gov/books/NBK577724/ - see the PDF is from the publisher, not one created by NCBI: https://www.ncbi.nlm.nih.gov/books/NBK577724/pdf/Bookshelf_NBK577724.pdf
## Proposal
Add the ability for Word One Doc users to Upload a Display PDF AND decouple the automatic setting of Display and Create book-level PDF settings, so user can set these according for their content, or set them up in a Publisher Template so they are set properly for new books.
## Design
Available on request.
## Implementation (if applicable)
[A description of the steps to implement the feature.]
## Alternative approaches (if applicable)
[Include any alternatives to meet this use case.]
## Open issues (if applicable)
See related issue #1137