ncbi issueshttps://gitlab.coko.foundation/ncbi/ncbi/-/issues2021-12-06T13:03:04Zhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/685Check abstract parsing in case of conversion Error2021-12-06T13:03:04ZGiannis Kopanasjkopanas@gmail.comCheck abstract parsing in case of conversion Error@andynicholson I was checking the issue https://gitlab.coko.foundation/ncbi/ncbi/-/issues/679. I realized the following things:
1) At line https://gitlab.coko.foundation/ncbi/ncbi/-/blob/develop/server/services/handleNcbiMessage/bookCom...@andynicholson I was checking the issue https://gitlab.coko.foundation/ncbi/ncbi/-/issues/679. I realized the following things:
1) At line https://gitlab.coko.foundation/ncbi/ncbi/-/blob/develop/server/services/handleNcbiMessage/bookComponentConversion.js#L72 we don't do the same checks as we do higher in the code for the abstract field.
2) Checking the bookComponent of issue #679 we can see that two upload versions happened. At the first one, we had a successful conversion but for the second one, a conversion error occurred. The second time failed because the abstract was an array (this should be a string) and it was coming from the DB. (which means that the first version saved the abstract inside the metadata column as an array which it shouldn't )
We need to simulate a chapter with the same files and check that abstract is saved correctly in case of a conversion error.
@andynicholson this is a tricky case. Please let's discuss it if the issue is not clear.Nov 02.Sidorela UkuSidorela Ukuhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/191Login and register pages Refactor2021-03-26T13:53:59ZDanjela Shehidanjelashehi@gmail.comLogin and register pages RefactorWe need to refactor code in the login and register page before we go on with other features. Here is a list of what i think needs to be changed:
* [x] Use the `TextFieldComponent` for all text fields , in case the current component do...We need to refactor code in the login and register page before we go on with other features. Here is a list of what i think needs to be changed:
* [x] Use the `TextFieldComponent` for all text fields , in case the current component does not fit, improve that component to be adjustable for all cases we use it.
* [x] `RemeberMeCheckbox` is redundant, as we already have on checkbox
* [x] `SubmitButton` is just a normal button, no need for new component
* [x] graphql folder inside Login folder should only contain index.js file with all mutations and queries (as there is no sense separating them when we don't have many queries and mutations)
* [x] Same issues in Register page
* [x] if the Register is working fine we can delete the Signup folder
* [x] Check all styled components in login and register folders to make sure we have used theme variables. ex fixed numbers in pixels should be only inn `gridUnit` * or / a number to where we need it.
Functionality issues :
~~* [ ] If you are in a page and the session has expired , it should redirect to Login page, in this state where the app is, it just hides the menu making it impossible for a normal user to go to login page, the only way is by adding /login to url~~
* [x] Can you try to do remember me functionality (see [this link](http://www.webstudypoint.com/remember-me-functionality-in-react-js/))
* [x] in login page , after you type password, when pressing enter key (from keyboard) should trigger the submit button, it used to work like that before , and it usually does in all apps
@DioneMentis the list of UI components we share in different pages is :
* Form elements
* Toggle, Input fields, Selects, Checkbox, Icon buttons
* Composite Elements
* Status (book component status,book status and file statuses ), Source File Icons, Date component (in human readable format as in dashboard example),Pages Filter, Tabs and Accordions
* App bar, Second navigation bar, Footer Action bar
I think once these ui elements as finished and cover all our cases, we will be just re-using them and won't have conflicts working in different pages, some of them may just need changes in the styles, some may need to be redone as some elements that we use pubsweet may not be extendable enough for our needs2020-09-25https://gitlab.coko.foundation/ncbi/ncbi/-/issues/582Statuses BookComponent - Source - Converted Files2021-08-10T09:55:54ZGiannis Kopanasjkopanas@gmail.comStatuses BookComponent - Source - Converted Files
Based on the wiki [here ](https://gitlab.coko.foundation/ncbi/ncbi/-/wikis/Workflows/BCMS-states-and-user-actions). We need to update Statuses of Book, Book Component, and Files.
For this we need two basic changes:
- [x] fix the side e...
Based on the wiki [here ](https://gitlab.coko.foundation/ncbi/ncbi/-/wikis/Workflows/BCMS-states-and-user-actions). We need to update Statuses of Book, Book Component, and Files.
For this we need two basic changes:
- [x] fix the side effects of not having a source file
- [x] fix statuses based on the tables of the wiki. Are 3-4 cases that should change
- [x] Update the restriction rules based on the table. this means there that there are some rules when we should change status or not.
@danjela's questions from #565 are resolved by this issue, and the decisions below:
1. [x] On the book manager row for each chapter we should show the **book component version number only**. It would be confusing to show a file version because this should always reflect whatever is the latest -- sometimes this would be the source file version and sometimes this would be the converted file version.
2. [x] When the book component status is "In Review" we also show the source file status on the book component row. The change here is: show the status of whichever file is **most recent**: either source or converted. That's meaningful to the user because whichever file is most recent is the one being processed.
~~3. [ ] Finally the file version for the Last published value on the book component row should change to display the Book Component Version Number + most recent published converted file version~~
I've also linked related issues that will be resolved by this work
---
### Work-in-progress corrections:
1. For the wholebook use case, when status = new book
- [x] the 'Reload preview' button should be inactive on the files tab
- [x] I upload a converted file, the file status is correct but the book component and book status does not change
2. [x] When I request a review on a wholebook or chapter within a chapter-brocessed book, the book component status does not change to In Review -- this is blocking me from testing other use cases. See screen recording in [#note_63789](https://gitlab.coko.foundation/ncbi/ncbi/-/issues/582#note_63789)July.02: Refactoring from 554 | Bulk download | Mock NCBI service | BugsDione Mentisdione@coko.foundationGiannis Kopanasjkopanas@gmail.comDanjela Shehidanjelashehi@gmail.comDione Mentisdione@coko.foundationhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/602Submit during In review status bugs2021-08-09T13:41:00ZDione Mentisdione@coko.foundationSubmit during In review status bugs@John.kopanas @danjela
This came up in my testing of #582.
I first had this issue in [an XML chapter](http://ncbi-staging.coko.foundation/organizations/1e2937d2-6389-441e-8b0b-e4b5f85b53c8/bookmanager/bfd98d77-027e-4ab7-b31d-20fca1b...@John.kopanas @danjela
This came up in my testing of #582.
I first had this issue in [an XML chapter](http://ncbi-staging.coko.foundation/organizations/1e2937d2-6389-441e-8b0b-e4b5f85b53c8/bookmanager/bfd98d77-027e-4ab7-b31d-20fca1bf88dc/d8a0bb33-d323-403c-a8c6-3e824e4c5a46) and I reproduced it in a WORD chapter.
My book component is in status "In review" and I have 1 source file that has previewed successfully. I upload another source file (v1.2 New upload).
![Screenshot_2021-07-23_at_16.45.49](/uploads/767b258e08d21327b1cff0d0eb23e580/Screenshot_2021-07-23_at_16.45.49.png)
I select "submit", I see the "submitting files" loader, but the file status does not change to "Converting", and the "Submit" but is still active. From the UI perspective, it looks like the submit action failed, so I try again about a minute later. After a while a come back to the chapter and I see ...
![Screenshot_2021-07-23_at_17.15.28](/uploads/b0b041ab45b470c2d5b4d5ae2d1750d4/Screenshot_2021-07-23_at_17.15.28.png)
**Expected behaviour**
After I uploaded the source file (1.2) and selected "submit":
1. Source file status changes to "Converting"
2. Submit button is inactive because source file is "Converting"
3. Converted file (1.2) displays when received from NCBI and both source and converted file have status "loading preview"July.02: Refactoring from 554 | Bulk download | Mock NCBI service | BugsGiannis Kopanasjkopanas@gmail.comGiannis Kopanasjkopanas@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/631Add last pub date from converted file version2021-10-20T01:21:17ZDione Mentisdione@coko.foundationAdd last pub date from converted file versionThis last item for #582 requires a decision to point 3 in #603
Currently we show the `bookComponentVersionNumber.SourceFileVersion` in the last pub date. This should change to `bookComponent versionNumber.ConvertedFileVersion` because ...This last item for #582 requires a decision to point 3 in #603
Currently we show the `bookComponentVersionNumber.SourceFileVersion` in the last pub date. This should change to `bookComponent versionNumber.ConvertedFileVersion` because it's possible to publish a converted file when a source file does not exist.
**Display in UI**
Use simplified date followed by the version number. For example `3 days ago (V1.4)`. Show the full date on hover, as already dine for the "last updated" date.
* book manager and book component page: `3 days ago (V1.4)`
* Dashboard: `3 days ago` (note: if it's a chapter-processed book then show date for whichever chapter was published most recently)Sept 02Danjela Shehidanjelashehi@gmail.comGiannis Kopanasjkopanas@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/644warnings in console2024-01-16T18:19:05ZDanjela Shehidanjelashehi@gmail.comwarnings in console- [ ] Missing keys in lists (`Warning: Each child in a list should have a unique "key" prop.`)
- [ ] `Warning: Failed prop type: Invalid prop `tabItems[0]` of type `object` supplied to `Tabs`, expected a single ReactElement.`
- [x] `W...- [ ] Missing keys in lists (`Warning: Each child in a list should have a unique "key" prop.`)
- [ ] `Warning: Failed prop type: Invalid prop `tabItems[0]` of type `object` supplied to `Tabs`, expected a single ReactElement.`
- [x] `Warning: Failed prop type: You provided a `checked` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultChecked`. Otherwise, set either `onChange` or `readOnly`.`
- [x] ` Warning: Failed prop type: Invalid prop `rowStyle` supplied to `Table`. `
- [x] `Warning: Failed prop type: Invalid prop `width` of type `string` supplied to `Column`, expected `number`.`
- [ ] `Warning: Failed prop type: You provided a `checked` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultChecked`. Otherwise, set either `onChange` or `readOnly`.`
- [ ] `Warning: Failed prop type: The prop `itemsList[0].id` is marked as required in `Dropdown`, but its value is `undefined`.`
- [ ] `Warning: `value` prop on `input` should not be null. Consider using an empty string to clear the component or `undefined` for uncontrolled components.`
- [ ] `Warning: Function components cannot be given refs. Attempts to access this ref will fail. Did you mean to use React.forwardRef()?
Check the render method of 'CoverComponent'. in Button (created by CoverComponent)`
- [x] `Warning: Failed prop type: Invalid prop `size` of type `string` supplied to Colorized, expected number.`
- [ ] `Warning: Failed prop type: The prop `name` is marked as required in `Checkbox`, but its value is `undefined`.`
- [x] `Warning: Failed prop type: You provided a `checked` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultChecked`. Otherwise, set either `onChange` or `readOnly`.`Danjela Shehidanjelashehi@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/648Updates to Org level settings2021-08-31T17:57:17ZDione Mentisdione@coko.foundationUpdates to Org level settings@danjela
Here is an updated UI for the Org setting tab:
![org-settings-20210819](/uploads/5cb92526330011ec0df1a22dd9c51c97/org-settings-20210819.png)
Changes here:
1. [x] remove email field (#585)
2. [x] categorise collections in...@danjela
Here is an updated UI for the Org setting tab:
![org-settings-20210819](/uploads/5cb92526330011ec0df1a22dd9c51c97/org-settings-20210819.png)
Changes here:
1. [x] remove email field (#585)
2. [x] categorise collections into two types: Book Series collection and Funded collection
3. [x] Organization type should be toggles to allow more than one typeAug 01.Danjela Shehidanjelashehi@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/668Chapter-processed books should have same "part" functionality2021-10-20T14:51:48ZDione Mentisdione@coko.foundationChapter-processed books should have same "part" functionality@danjela
Originally NCBI's brief included different use cases for adding parts in the PDF and XML workflow, and we partially developed that in #401 with the understanding that user would not submit content to the *same* book by FTP and...@danjela
Originally NCBI's brief included different use cases for adding parts in the PDF and XML workflow, and we partially developed that in #401 with the understanding that user would not submit content to the *same* book by FTP and via the UI. However, since then that use case has changed (https://gitlab.coko.foundation/ncbi/ncbi/-/issues/475#note_58180)
In this initial release, parts will only be supported by adding them in the UI bu the "add part", as currently developed for Word and PDF workflow.
This should be the same for XML workflow. You can see in [this book](https://ncbidev.cloud68.co/organizations/a2c1e381-f348-41a9-9d76-e4f11f761033/bookmanager/7bc0a5c2-0b97-43ac-9543-97bb98d0d7ba/), when I create a book that supports parts, the "add part" button ; "move to" ; and "Preview TOC" button don't display.
The variation of creating a part by file upload is still valid in the PDF and XML workflow but supporting this will only be scoped after the initial release.Oct.02.Danjela Shehidanjelashehi@gmail.comSidorela UkuDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/693Amendments to new book step 12021-10-21T15:54:07ZDione Mentisdione@coko.foundationAmendments to new book step 1@yannis
Here's the updated UI based on the spec for the settings template:
*Step 1*
![New-book-modal-202100909](/uploads/a0d049e4017b4d87b4b18df5583ef665/New-book-modal-202100909.png)
Form logic:
* If conversion workflow = Word,...@yannis
Here's the updated UI based on the spec for the settings template:
*Step 1*
![New-book-modal-202100909](/uploads/a0d049e4017b4d87b4b18df5583ef665/New-book-modal-202100909.png)
Form logic:
* If conversion workflow = Word, only 'chapter-level' submission is relevant.
* If funded content type = 'Prepublication Draft' or 'Published PDF', only 'book-level' submission is relevant.
After user selects "next" there should be a confrimation modal (current used at step 2) that reads: Are you sure? Only NCBI System Admins can change these settings after the book is created. -- This addresses feedback in #651
*Step 2*
Show relevant template settings based on the selections the user made in step 1.Oct.01.Dione Mentisdione@coko.foundationYannis BarlasDione Mentisdione@coko.foundationhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/713Add source icon for collection based on collection members2021-10-05T12:21:41ZDione Mentisdione@coko.foundationAdd source icon for collection based on collection members@danjela
The source workflow (PDF/WORD/XML) of a collection is dependent on the books in the collection, so:
1. When collection is created, the "source" is null (show no icon in "source" column in the dashboard)
2. When a book is ad...@danjela
The source workflow (PDF/WORD/XML) of a collection is dependent on the books in the collection, so:
1. When collection is created, the "source" is null (show no icon in "source" column in the dashboard)
2. When a book is added, match collection source to book source
3. If books in collection have more than one source, show no icon (this can be improved later with a icon to show multiple sources)Sept 02Danjela Shehidanjelashehi@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/715updated folder names in dowloaded packaged to match folders in Apex/PDF conve...2021-10-05T12:21:41ZDione Mentisdione@coko.foundationupdated folder names in dowloaded packaged to match folders in Apex/PDF conversion@andynicholson
FYI @John.kopanas
Currently folder names in download packages are as below:
![Screenshot_2021-09-24_at_15.21.11](/uploads/72804c1a9e2ecc21382cfac685d77fd2/Screenshot_2021-09-24_at_15.21.11.png)
Please update the nam...@andynicholson
FYI @John.kopanas
Currently folder names in download packages are as below:
![Screenshot_2021-09-24_at_15.21.11](/uploads/72804c1a9e2ecc21382cfac685d77fd2/Screenshot_2021-09-24_at_15.21.11.png)
Please update the names of the folders to the below:
1. `pdf` --> `display-pdfs`
1. `image` --> `images`
1. `cover` --> `metadata`
1. `comment` --> `review` // only relevant to UI downloads
1. `supplement` --> `suppl`Sept 02Andy NicholsonAndy Nicholsonhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/761Refactor DateFields Component2021-11-02T19:28:45ZDanjela Shehidanjelashehi@gmail.comRefactor DateFields ComponentDateFields Component needs to be refactored since it is buggy in some scenarios. This resolves the UI issue in point 1 described in the testing ticket: https://gitlab.coko.foundation/ncbi/ncbi/-/issues/763#note-for-your-testing, and copi...DateFields Component needs to be refactored since it is buggy in some scenarios. This resolves the UI issue in point 1 described in the testing ticket: https://gitlab.coko.foundation/ncbi/ncbi/-/issues/763#note-for-your-testing, and copied here:
>1. Since the "Book publication date format" setting has been removed, the format is now controlled as part of the metadata form logic by selecting the type:
![Screenshot_2021-10-21_at_22.16.14](/uploads/6b87d25e30b423bcfd6211eb009f1bba/Screenshot_2021-10-21_at_22.16.14.png)
>However this functionality wasn't intended when this UI component was originally developed. Unfortunately it needs some refactoring to allow `DD` and and `MM` to be optional. This means you cannot create a book without completing these fields. This will be fixed in the next release.
![Screenshot_2021-10-21_at_22.19.16](/uploads/5593f11d7af369884f4151bf8732d757/Screenshot_2021-10-21_at_22.19.16.png)Oct.02.Danjela Shehidanjelashehi@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/809refactor search filters on Dashboard to list all error status types2021-12-06T20:24:13ZDione Mentisdione@coko.foundationrefactor search filters on Dashboard to list all error status types@andynicholson
As @sidorelauku reported in here https://gitlab.coko.foundation/ncbi/ncbi/-/issues/494#note_64731:
![image](/uploads/be01d3c174ca0c03dbc17515f74206da/image.png)
Previously we had one "error" status. There are now thre...@andynicholson
As @sidorelauku reported in here https://gitlab.coko.foundation/ncbi/ncbi/-/issues/494#note_64731:
![image](/uploads/be01d3c174ca0c03dbc17515f74206da/image.png)
Previously we had one "error" status. There are now three:
1. Conversion errors
2. Loading errors
3. Submission errors (not yet developed)
The user should be able to filter by each of these.Nov 02.Sidorela UkuSidorela Ukuhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/837Less strict rules for when users can move book components in the TOC2021-12-14T21:33:12ZDione Mentisdione@coko.foundationLess strict rules for when users can move book components in the TOC@danjela
Please see [changed rules in Column T](https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit?usp=sharing) for when a user can move a book component in the TOC. This applies to the "Move" butt...@danjela
Please see [changed rules in Column T](https://docs.google.com/spreadsheets/d/1M4ZdBbzr2s4-PUXqEblfsKoPBUuk4aG343u9ZAUn_Tk/edit?usp=sharing) for when a user can move a book component in the TOC. This applies to the "Move" button and Drag and Drop for the manual ordering use case.
- [x] Conversion errors --> Yes
- [x] Publishing Failed --> Yes
- [x] Published --> YesDec .01Danjela Shehidanjelashehi@gmail.comDione Mentisdione@coko.foundationDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/883Refactor errors table in the database so it can relate to bcms-id2022-02-01T12:48:03ZDione Mentisdione@coko.foundationRefactor errors table in the database so it can relate to bcms-idRefactor errors table in the database so it can relate to bcms-id (current only relates to a book component)
Required for Collection TOCsRefactor errors table in the database so it can relate to bcms-id (current only relates to a book component)
Required for Collection TOCs2022-Jan-AGiannis Kopanasjkopanas@gmail.comGiannis Kopanasjkopanas@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/972Refactoring WholeBook Conversion case2022-07-19T15:56:06ZGiannis Kopanasjkopanas@gmail.comRefactoring WholeBook Conversion caseRefactoring the conversion that updates the book and bookComponent, for the whole book scenario.
Right now we update the bookComponent and book metadata and we should decouple this to just update the book metadata.
This may involve chan...Refactoring the conversion that updates the book and bookComponent, for the whole book scenario.
Right now we update the bookComponent and book metadata and we should decouple this to just update the book metadata.
This may involve changes at the frontEnd,So i will need a day to investigate the exact estimation for this change.2022-May-AGiannis Kopanasjkopanas@gmail.comGiannis Kopanasjkopanas@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1010Refactor bookMeta to use objects for child tags2024-01-16T18:20:59ZAndy NicholsonRefactor bookMeta to use objects for child tags
Refactor TOC XML building class BookMeta to use classes to hold child XML tags , such as notes, and pub history
Refactor TOC XML building class BookMeta to use classes to hold child XML tags , such as notes, and pub historyhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1048Use trigger events of postgres for updating collections and book status2024-01-16T18:20:58ZGiannis Kopanasjkopanas@gmail.comUse trigger events of postgres for updating collections and book statusRefactor the way we update the status of books and Collections.
Books and Collections statuses were calculated from the status of each chapter or book that belong to them.
The update function is here : `https://gitlab.coko.foundation/n...Refactor the way we update the status of books and Collections.
Books and Collections statuses were calculated from the status of each chapter or book that belong to them.
The update function is here : `https://gitlab.coko.foundation/ncbi/ncbi/-/blob/develop/server/models/book/book.js#L526`
We are calling this function every time we have an update at the book of chapters, there is an easier way of doing this by writing a postgresql trigger event on every update of book or chapter and by that the postgres function can do the update for the book status.
- [ ] Remove the function that is responsible now for updating the book status. Remove any call that we do that function.
- [ ] Create a postgresql trigger event and a new function that hanldes the status update of the bookPokhiPokhihttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1049Refactor Book Metadata builder Xml2024-01-16T18:39:50ZGiannis Kopanasjkopanas@gmail.comRefactor Book Metadata builder XmlRefactor And Minimize the bookMeta.js File by spliting the code to new files or to files that already exist.
This still need some thinking for the exact points that need to be refactored, before we start it.Refactor And Minimize the bookMeta.js File by spliting the code to new files or to files that already exist.
This still need some thinking for the exact points that need to be refactored, before we start it.https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1050Refactor the way Abstract and Book Covers are being managed.2022-03-31T16:43:28ZGiannis Kopanasjkopanas@gmail.comRefactor the way Abstract and Book Covers are being managed.Refactor the way we save the Cover to the backend . Seperate the logic of saving the file and removing it from update Book and Collection.
- [x] Decouple the fileCover and fileAbstract saving part in the files : `server/services/Command...Refactor the way we save the Cover to the backend . Seperate the logic of saving the file and removing it from update Book and Collection.
- [x] Decouple the fileCover and fileAbstract saving part in the files : `server/services/CommandService/domainCommands/commands/updateCommand.js` , `server/services/CommandService/domainCommands/commands/updateCollectionCommand.js`
```
data.fileCover = FileService.saveFile(this.fileCover, 'cover')
data.fileAbstract = FileService.saveFile(this.fileAbstract, 'abstract')
```
- [x] Move to the file `server/services/file/fileService`, the update of the cover and abstract object (just return the id of the file if exists else return null) as new static function
- [x] Currently it only removes only the cover from Files Table if the fileCover is empty, Should be done the same and for the fileAbstract.2022-Mar-Bhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1053Refactor contributor parsing2024-01-16T18:19:03ZAndy NicholsonRefactor contributor parsing
Return from one function, using multiple keys
```
return {
author :
editor:
collaborativeAuthor:
}
```
Something like
```
this.xmlObject(value).map((index, v) => {
if (
this.xmlObject(v).html() &&
this.xmlOb...
Return from one function, using multiple keys
```
return {
author :
editor:
collaborativeAuthor:
}
```
Something like
```
this.xmlObject(value).map((index, v) => {
if (
this.xmlObject(v).html() &&
this.xmlObject(v).attr('contrib-type') === 'editor' &&
this.xmlObject(v).has('collab').length === 0
) {
editor.push(new Contributor(v))
}
if (
this.xmlObject(v).html() &&
this.xmlObject(v).attr('contrib-type') === 'author' &&
this.xmlObject(v).has('collab').length === 0
) {
author.push(new Contributor(v))
}
if (
this.xmlObject(v).html() &&
this.xmlObject(v).attr('contrib-type') === 'author' &&
this.xmlObject(v).has('collab').length > 0
) {
collab.push(new Contributor(v))
}
})
```Andy NicholsonAndy Nicholsonhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1159Attribute for UI elements2022-04-26T08:59:59ZVigneshAttribute for UI elements## Context
Data attributes for UI elements are needed for smoother and accurate automated tests.
## proposal
- data-test-id for checkboxes in organisations page
![image](/uploads/f56a1cc075ff0631e791bdeef348dcc2/image.png)
- data-tes...## Context
Data attributes for UI elements are needed for smoother and accurate automated tests.
## proposal
- data-test-id for checkboxes in organisations page
![image](/uploads/f56a1cc075ff0631e791bdeef348dcc2/image.png)
- data-test-id for checkboxes in dashboard organisations tab.
![image](/uploads/00dc5124c658be6d882d1a51310bcb88/image.png)
- An attribute defining that this div has the name of the book .
example - `data-test-id="Book-name"`
![image](/uploads/7e9d0901439b62e80d2b792fe89bfdf7/image.png)
- An attribute defining that this span has the name of the organisation.
example - `data-test-id="org-name"`
![image](/uploads/f1b38a506fb3c9ff8c985d888295ca96/image.png)
- An attribute defining the type of the icon.
example- `data-test-id="wholebook-icon/chapter-processed-icon"`
![image](/uploads/20d109baa7cac7a67a565ffa0bd027d5/image.png)2022-Apr-ADanjela Shehidanjelashehi@gmail.comDanjela Shehidanjelashehi@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1192Book Versions Updating requests to the database to include `book_version`2022-07-19T15:56:07ZGiannis Kopanasjkopanas@gmail.comBook Versions Updating requests to the database to include `book_version`Update Queries and functions to retrieve book by version and latest version.
- [x] Create newVersion command at the domain service
At the folder server/services/CommandService/domainCommands we can add a new command for the new Versio...Update Queries and functions to retrieve book by version and latest version.
- [x] Create newVersion command at the domain service
At the folder server/services/CommandService/domainCommands we can add a new command for the new Version where the creation of the book could be used. Could be a wrapper for the call to the createDomain service command (this command involves the cration of a new book - `server/services/CommandService/domainCommands/commands/createCommand.js`) by passing the correct data and do the necessary updates for the relationships that already exist for the previous book versions. So for example if we create a new book version we need to update the collection that the book was belonging to. we need to update the teams to point to the latest book versions
- [x] When we will create a new Version we need to Update the column `components` of the Collection because it will have the reference to the old version of the book.
- [x] Add a new column `parent_id` to the `books` table. This will be `null` for the original book, for all the subsequent versions created, it will store the uuid of original book.
- [x] When we create a new book Version we need to move the teams that had been for the older version to the newest version .
- [x] Create a function at the model of the book that will return a book instance of a requested book version , by providing a version name and version number .
- [x] Book version alias should be created from the last version2022-May-BPokhiPokhihttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1196Update Teams for Book versions2022-07-19T15:56:07ZGiannis Kopanasjkopanas@gmail.comUpdate Teams for Book versionsUpdate the relationship between books and teams since we have new book versions.
@pokhi @John.kopanas @DioneMentis As confirmed in #1268 and as on the [permissions sheet](https://docs.google.com/spreadsheets/d/1cJeKIUhkkM_kwQ8Fzh9d2Vw9h...Update the relationship between books and teams since we have new book versions.
@pokhi @John.kopanas @DioneMentis As confirmed in #1268 and as on the [permissions sheet](https://docs.google.com/spreadsheets/d/1cJeKIUhkkM_kwQ8Fzh9d2Vw9h2IJcgT3_87sWLiNdNQ/edit#gid=672833130):
- [x] When the user clicks 'New version' and fills out the new version fields in the modal and saves, a new version is created. The user who created that new version has edit access to the new version (same as when a user creates a new book).
- [x] As usual, as is the case for all books at present, the Org admin and Sys admins also have access to all versions of the book.
- [x] PDF2XML vendors must have access to all *PDF versions*.
- [x] Team members are *not* automatically copied across from book version x to book version y (We start with a new team in the new version).
- [x] Once the new version is created, more people (e.g. editors) could be added to the new version team at that stage, in the usual way as we do for wholebooks.
- [x] Given that each version will have a new team, team members of version x do not have access to version y. In the dropdown which shows all the versions of the book, alternate versions can be clicked, but if I'm not on the team of that version, I get a permissions message that I don't have access.
- [x] Teams for older book versions should still be editable once a new version is created.2022-Jun-BGiannis Kopanasjkopanas@gmail.comPokhiShubham TiwariGiannis Kopanasjkopanas@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1197Remove the one default bookComponent for the whole book case2022-07-19T15:56:07ZGiannis Kopanasjkopanas@gmail.comRemove the one default bookComponent for the whole book caseUp to now when we create a whole book we create automatically a default bookComponent which helped as for creating tocs and file Versions for books.
Now we want to decouple the whole case from the chapter based case, and create books and...Up to now when we create a whole book we create automatically a default bookComponent which helped as for creating tocs and file Versions for books.
Now we want to decouple the whole case from the chapter based case, and create books and fileVersions for books .2022-Jun-AGiannis Kopanasjkopanas@gmail.comGiannis Kopanasjkopanas@gmail.comhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1198Update teams table when we have new version2022-06-28T10:58:49ZGiannis Kopanasjkopanas@gmail.comUpdate teams table when we have new versionThis issue tracks how teams work for new book versions.This issue tracks how teams work for new book versions.2022-Jun-APokhiPokhihttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1199Update bcms_id relation from books, bookComponents, Collections, to errors,...2022-07-19T15:56:07ZGiannis Kopanasjkopanas@gmail.comUpdate bcms_id relation from books, bookComponents, Collections, to errors, and notification Messages.Right now we are based to bcms_id as a unique identifier for the whole app. Since bcmsid now may have the same value for the different versions of a book , we need to find another way to identify the errors and notification messages for ...Right now we are based to bcms_id as a unique identifier for the whole app. Since bcmsid now may have the same value for the different versions of a book , we need to find another way to identify the errors and notification messages for the app and match them to the related Object.
we had previously changed the bookComponentId to bcmsid here: https://gitlab.coko.foundation/ncbi/ncbi/-/merge_requests/655
So now we need to change back the following things:
- [x] For ncbi_notification_messages tables we need to have back an ObjectId instead of the bcmsid that will related to a bookComponent.id or Book.id or toc.id depending on the case and the job (whole book case vs chapter based) whole book case should match the book.id and the chapter based should match the bookComponent.id or toc.id when it is a load to pmc for a TOC.
- [x] For Errors we need to change the bcmsid field to an objectId again based on case whole book/ chapter based or if it is a toc or not .2022-May-APokhiPokhihttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1201Writing migrations for the books that exist2022-07-19T15:56:07ZGiannis Kopanasjkopanas@gmail.comWriting migrations for the books that existmigrate data after having book versions
- [x] Adding Version Field to Books
- [x] For each row in, `ncbi_notification_messages` and `errors` table, the value of `object_id` will be updated.
- In case the notification/error is for a w...migrate data after having book versions
- [x] Adding Version Field to Books
- [x] For each row in, `ncbi_notification_messages` and `errors` table, the value of `object_id` will be updated.
- In case the notification/error is for a whole book, it's going to be the `id` in the `books` table
- In case the notification/error is for a chapter-processed book, it's going to be `id` in the `book_component` table
- [x] Migration for FundedContentType field
- [x] Migration to transform `book_component_id` to `object_id` in `channels` table
- [x] Migration to transform `book_component_id` to `object_id` in `issues` table
- [x] Migration to transform `book_component_version_id` to `object_id` in `reviews` table2022-Jun-BPokhiGiannis Kopanasjkopanas@gmail.comPokhihttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1398QA: Cypress test refactoring2024-01-16T18:21:04ZSidorela UkuQA: Cypress test refactoringAdding here the feedback from MR !1025 for improvements on testing processes and data-test-id used for cypress.
1- Unification of variables testId, dataTestId and data-test-id, lockTestId
This is done mostly, now we generally use test...Adding here the feedback from MR !1025 for improvements on testing processes and data-test-id used for cypress.
1- Unification of variables testId, dataTestId and data-test-id, lockTestId
This is done mostly, now we generally use testId and data-test-id. This is because we use variables to reach the smaller components inside of specific element we want to click. Like for example buttons.
Additionally we can use cypress locators to make it simpler.
2- Clean up on test-id which are not in use.
Ex.
![image](/uploads/f71275f730f625037281ead84c512db5/image.png)
3- Minimize the use of labels on the data-test-id names or make them simplier:
![image](/uploads/aca5005f4d9c8e4b2f105db1436d7b71/image.png)
4- Not use children as input for data test id-s:
`data-test-id={`extended-icon-${children}`}`Sidorela UkuSidorela Ukuhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1472Remove query to display {book-component-version}.{file-version} number on the...2024-01-17T06:25:09ZDione Mentisdione@coko.foundationRemove query to display {book-component-version}.{file-version} number on the chapters rows of the book manager<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The book manager displays a list of book components. Each component may have multiple versions -- each of which may have been published before. A current requirement of the book manager is: For each book component listed, display the published date and `{book-component-version}.{file-version}` number of whichever `{book-component-version}.{file-version}` was most recently published.
When books have a high number of book components, the queries to get this data is affecting the performance and therefore usability of the book manager page.
## Proposal
<!-- A precise statement of the proposed feature. -->
Remove the query to get the `{book-component-version}.{file-version}` value for "last published". Get "last published" date only. NCBI has agreed to this in a meeting (1 Feb '23) since it does not negatively impact users ability to perform their tasks on the book manager page.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
Only design change is the removal of the version number in brackets after the "last published" date. Here is the current design:
![image](/uploads/6ee80196c334bfd6183d1141091159e8/image.png)
Here is the design after this implementation:
![image](/uploads/512fc8572182a65d60f46b039417e567/image.png)
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] Each chapter listed on the book manager page lists the "Last published" date if the chapter has been published
- [x] The component.file version number will no longer display after the "Last published date" on the book manager page - see screenshot below of what will be removed (crossed out in blue) in the highlighted information. **Note:** Existing users may need to hard refresh their browsers to see this change.
Screenshot from: https://ncbi.cloud68.co/organizations/6c4d37c4-b9f2-4962-9080-b3e29a37dfbd/bookmanager/7f099bb1-3379-4beb-bc4b-aeca26884a47
![image](/uploads/9271a3bb44e10af981215b3a59bef97d/image.png)
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” of cloud68 (by Coko team)
- [x] Deployed and tested on “ncbi” of cloud68 (by NCBI team)
- [x] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* Milestone:
* Iteration:
* Dependencies: (list issue numbers if relevant)
* Development estimate (hours):P1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1530Relax Publisher Abbreviations rules and validations to permit ^[a-zA-Z][\w-]+$2023-09-03T11:50:20ZStacy LathropRelax Publisher Abbreviations rules and validations to permit ^[a-zA-Z][\w-]+$<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
NCBI needs to be able to migrate all of its existing publishers and domains to the new BCMS using its migration scripts. NCBI has existing publisher abbreviations (e.g., `otis-mtb`) that don't match this current syntax used by the BCMS to do a number of validations, such as select permitted workflow types for that publisher that permits users to create books and collections within it:
`^[a-zA-Z][\w]+$`
So for the user to be able to create books and collections, this check should be relaxed to:
`^[a-zA-Z][\w-]+$`
Unless this will break something we cannot understand.
The testing plan for #1480 depends on this issue being implemented.
## Proposal
Relax Publisher Abbreviations rules and validations to permit `^[a-zA-Z][\w-]+$`
* Start with a letter (either uppercase or lowercase).
* Are followed by one or more characters that can be alphanumeric, underscore, or hyphen.
* Have no other characters or whitespace.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature. -->
Current UI:
![Screenshot_2023-08-14_at_16.12.38](/uploads/bcca76ca6ce7e4400975564144cd5d3d/Screenshot_2023-08-14_at_16.12.38.png)
The only change here is to update the error text to include hyphen: "Abbreviated name may only contain letters, numbers, underscore, and hyphen character"
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] System Admin can create a publisher (i.e., BCMS Org) with a hyphen in their abbreviation, e.g., `otis-mtb`
- [x] System Admin can change all publisher settings with publisher (i.e., BCMS Org) with a hyphen in their abbreviation, e.g., add supported workflows
- [x] When the System admin creates a new Org in the BCMS and inputs an Abbreviated name for that Org, this is validated according to the regex rules `^[a-zA-Z][\w-]+$` (and the error text "Abbreviated name may only contain letters, numbers, underscore, and hyphen character" is shown to guide the user only in cases where those rules are not met)
- [x] NCBI can successfully migrate existing Publishers and all of their linked domains in the PMCBook database to the BCMS (e.g., publisher with abbreviation `otis-mtb` and their book `mtb`)
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] Corresponding change has been made in NCBI system (Domain Service) - BK-26269 assigned @deniskar
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [x] Deployed and tested on “ncbi” (by NCBI team)
- [x] Deployed by NCBI and NCBI confirms can migrate `mtb` to `otis-mtb` org
- [x] NCBI confirms Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
This issue amends the Create Publisher and Update publisher calls documented here: https://gitlab.coko.foundation/ncbi/ncbi/-/issues/226#6-create-a-publisher
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [x] Milestone is linked
* [x] Iteration is linked
* [x] Dependencies: ("None" or list issue numbers if relevant)
* [x] Development estimate is added to issue time tracking
---P1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1557Refactoring the book manager frontend to resolve unresponsive page2023-09-26T16:23:17ZDione Mentisdione@coko.foundationRefactoring the book manager frontend to resolve unresponsive page<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
While investigating the client side performance issues on the book manager (the investigation triggered by the permissions issue #1471 ), I ran the experiment of turning the permissions off completely to see what difference that would make. That led to me noticing that the client side cannot handle very large amounts of data, becoming completely unresponsive.
I tried to isolate the issue outside of the app (in storybook) to make sure that none of this was related to the backend, or the permissions layer, or any other kind of client side calculations that are triggered by the server data arriving. The idea was to try to render 9000 book components. The result is that it took around a minute to render those components (without taking into account the time it would take for the server to respond) .It does work fine if I render the simplest of components (think of the list rendered on Bookshelf that has links only), but when we add more UI elements, drag and drop etc it becomes quite taxing on the browser.
The main takeaway here is that we should not be rendering thousands of components at once, as this is too big a load for a browser tab to process all at once.
## Proposal
<!-- A precise statement of the proposed feature. -->
The proposed plan of action here is this:
* Rewrite the tree (ie. the nested chapter list) to make it lighter and more performant
* Start with all parts collapsed (as we’ve already agreed) and only make the call to the server to grab what’s nested when the user clicks on the expand icon
* Remove the possibility of an expand all button. If we get the data for each nested part only when explicitly requested, we’ll end up with potentially dozens (or even hundreds) of requests firing on the server. We’ll also end up with a potentially huge list to render at once again. We can maybe revisit this point once all this is done.
* Refactor to delegate a lot of the heavy lifting that the client currently does to the server
I have tried an experiment with a newly-implemented tree of 100 parts with 600 nested chapters each and the UI seemed to work fine in isolation. But some of the work will also come down to reducing the amount of calculations on events such as selections, disabling buttons etc.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature. -->
Design remains the same, with some improvements
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [ ] A large list of 100 parts (parent and child parts) with 600 nested chapters inside each part should open within a few seconds after receiving the data from the server and should be responsive to users actions - example `micad` in Bookshelf repository and by all supported BCMS MVP users in their native environments
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [ ] BCMS User Story / Context has been well defined
- [ ] The priority of the user story is specified and agreed
- [ ] Digital assets added (design, database scheme, mockups etc if relevant)
- [ ] Coko Technical Proposal approved by NCBI
- [ ] Testable Acceptance Criteria approved by NCBI
- [ ] Estimate of effort to complete (time or points)
- [ ] The issue has been broken down into development tasks (if necessary)
- [ ] Requirements Clarified
- [ ] The product owner and development team agree that the user story is ready for development
- [ ] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [ ] All coding tasks are finished and implemented by Coko
- [ ] All unit / end-to-end tests are written and implemented Coko
- [ ] QA approved by Coko
- [ ] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Deployed and tested by NCBI in NCBI environment (required to test `micad`
- [ ] NCBI confirms Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [ ] Milestone is linked
* [ ] Iteration is linked
* [ ] Dependencies: ("None" or list issue numbers if relevant)
* [ ] Development estimate is added to issue time tracking (120 hours: 80 on frontend and 40 on backend)
---P1a: Book Manager redesign: HighRebecca OrrisRebecca Orrishttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1479Pre-calculate values to improve book manager performance and remove unneeded ...2024-01-17T06:23:06ZDione Mentisdione@coko.foundationPre-calculate values to improve book manager performance and remove unneeded query<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
The context is described in the Epic description: https://gitlab.coko.foundation/groups/ncbi/-/epics/106. Coko team investigated additional backend improvements to improve book manager performance.
## Proposal
<!-- A precise statement of the proposed feature. -->
Pre-calculate the following:
1. The value for TOC tags: 'repeat' and 'exclude'
2. Error detail for values: 'publisher' or 'PMC'
Remove:
The In review status – we should close this query since it’s not supported for MVP. Currently the functionality is only hidden in the frontend.
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [ ] Compare the current logs to the new logs to show the improvement on the postgres queries.
- [ ] Save "In Review" query in separate branch for future use.
Note: the performance from the users perspective will be tested after all linked items are complete
### Log for NCBI developers' review:
Logs of postgresql queries and graphql Requests before and after removing the unneeded queries:
[postgresql-before-remove-queries.log](/uploads/c14b9cc4e9d55371d05d55ba4561b2eb/postgresql-before-remove-queries.log)
[loading-time-after-removing-queries](/uploads/05edf34eeed46df748c42239bda5e4bd/loading-time-after-removing-queries.png)
[postgresql-after-remove-queries.log](/uploads/6d80ba44f6aa400df1b558f11bf12ed2/postgresql-after-remove-queries.log)
[loading-time-before-remove-queries](/uploads/bac0664c0df81e49134e7ee860125556/loading-time-before-remove-queries.png)
### Brach with saved "In review" code
https://gitlab.coko.foundation/ncbi/ncbi/-/tree/remove-review-backend
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [x] Milestone is linked
* [x] Iteration is linked
* [x] Dependencies: none
* [x] Development estimate is added to issue time trackingP1a: Book Manager redesign: Highhttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1458Separate the front/body/back divisions into three tabs2024-01-17T06:27:05ZStacy LathropSeparate the front/body/back divisions into three tabs
## Context / User Story
Table of Contents Building Story: Because of an improvement made to the BCMS data model from the existing Silverlight CMS to support editorial teams to manage their book part contents at the same time themselves...
## Context / User Story
Table of Contents Building Story: Because of an improvement made to the BCMS data model from the existing Silverlight CMS to support editorial teams to manage their book part contents at the same time themselves, Bookshelf requires the BCMS to have functionality in the UI for users to build their TOC by communicating, through automated settings and/or manual placement, its structural place in the book (front, body, back), what part in the body it belongs if there are parts, and where in the order of contents a book part of any structural attribute falls; the BCMS must then read this communicated information and write valid book Table of Contents (TOC) XML files (see TOC.xml user story). Original acceptance criteria are here: https://gitlab.coko.foundation/groups/ncbi/-/epics/2
TOC Building issue that needs to be solved for the MVP: Currently, BCMS users cannot manually drag and drop their book parts on the existing chapter-processed Book Manager pages to order their front matter, body contents, part contents, or back matter. This is because lately introduced lazy load scroll bars interfere with the user also being able to see all their contents to be ordered in context of an entire structural unit, and then to drag and drop them where they belong in the context of that entire structural unit. Also, the performance of the page is slow and the drag and drop functionality is not responsive to users’ attempts to move their contents to the right structural place and for those place parts to "stick" in place; that is even after NCBI has removed lazy load to see all parts to validate assumptions.
Impact if this TOC Building issue is not solved: Based on a Bookshelf analysis of all its chapter processed TOCs and discussion with some participants, the results of which were shared with Coko, nearly all these TOCs have some level of required manual curation, according to the participants’ editorial policies that Bookshelf does not have the power to change. These are either the manual ordering of front matter, back matter, or of the placement of content within a part title, and to order those part titles by the historical and professional understanding of those editorial teams. Unless these teams can manage their TOCs in this way, Bookshelf will not be able to display TOC landing pages on its Bookshelf websites per terms of its current participant agreements.
This issue also relates to the proposal Design options for resolving chapter-processed book manager performance issues (#1480) and supporting parts with Body content in #1455.
## Proposal
<!-- A precise statement of the proposed feature. -->
NCBI has chosen the table of content building improvements below, along with Design option 1.
- #1482
The combination of very long lists and managing the books divisions (front/body/back) with multi-nested structures makes it difficult for users of these kinds of books to curate their TOCs because the space to do this work on the page is limited.
## Design
<!-- Include sketches or wireframes of the UI suggested for this feature -->
### Current UI
This is the current UI, an example from [this book](https://ncbidev.cloud68.co/organizations/73814fa2-d9fc-44f8-a12a-d3e52c589230/bookmanager/8e1ce032-ddc1-41fd-9088-7b9a75162f23)
### Design option 1 (chosen)
* Provide more space for TOC curation by separating the front/body/back divisions into three tabs.
* Tabs are positioned below the 'Upload Chapters' button (left of page) and Search input area and buttons (right of page) because the Tabs UI will be replaced with the search results UI (#1467) when users search.
* 'Body' is the default tab when user arrives at the Book manager
![Book_manager-pagination_V1](/uploads/779d1669ccad10473a66a370d5b6db70/Book_manager-pagination_V1.jpeg)
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
- [x] All chapter-processed book contents on the Book Manager page is split across three divisions: Front, Body, and Back. These three division can be accesses via tabs at the top left of the page.
- [ ] The Front|Body|Back pages will load within 20 sec (the Silverlight CMS current baseline) in their native environments (within institutional networks or home systems; benchmark is Bookshelf staff and GeneReviews, LiverTox, LactMed editorial team's environments in which they manage their contents)
- [ ] When a supported user navigates to the Book manager page from any other page in the BCMS, the Book Manager page will open with the "Body" tab and its page(s) contents in view.
- [ ] If a user changes a Body view to a Front or Back view using the Front or Back tab, the Front or Back view will remain in view until the user navigates away from their current Book Manager page or changes from a Front view to a Body or Back view or from a Back view to a Body or Front view.
- [ ] If a user changes a Front view to a Body or Back view using the Front or Back tab, the Body or Back view will remain in view until the user navigates away from their current Book Manager page or changes from a Body view to a Front or Back view or from a Back view to a Front or Body view.
- [ ] If a user changes a Back view to a Body or Front view using the Body or Front tab, the Body or Front view will remain in view until the user navigates away from their current Book Manager page or changes from a Body view to a Front or Back view or from a Front view to a Body or Back view.
- [ ] A user will see the paginated design in #1480 when implemented on all Front|Body|Back pages.
- [x] Users will not see a Repeat button on Front|Back pages as this function is not possible in those divisions (Pending in task #1544)
- [x] The design will not change the current rules for how the BCMS automatically moves components to the correct book division after conversion. The current functionality remains: Components are placed in 'body' when uploaded, and move to the the corrected division after conversion:
| Type of Document You Wish to Create | Value in Word template | Tagging from conversion | Section placement |
|-------------------------------------|------------------------|-------------------------|-------------------|
| appendix | appendix | `<book-part-wrapper content-type="appendix" id="">`; `<book-app book-part-type="appendix">` or `<book-part book-part-type="appendix">` | back |
| chapter | chapter | `<book-part-wrapper content-type="chapter" id="">`; `<book-part book-part-type="chapter">` | body |
| dedication page | dedication | `<book-part-wrapper content-type="dedication" id="">`; `<dedication>` | front |
| foreword | foreword | `<book-part-wrapper content-type="foreword" id="">`; `<foreword>` | front |
| frontmatter part | front-matter-part | `<book-part-wrapper content-type="front-matter-part" id="">`; `<front-matter-part>` | front |
| glossary | glossary | `<book-part-wrapper content-type="glossary" id="">`; `<book-part book-part-type="glossary">` | back |
| preface | preface | `<book-part-wrapper content-type="preface" id="">`; `<preface>` | front |
| reference list | ref-list | `<book-part-wrapper content-type="ref-list" id="">`; `<book-part book-part-type="ref-list">` | back |
| section | section | `<book-part-wrapper content-type="section" id="">`; `<book-part book-part-type="section">` | body |
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [x] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [x] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented by Coko
- [x] All end to end tests are finished and implemented by Coko: the tests will make sure the design is implemented correctly by checking the tabs as well the test will check that the components are positioned in the right division, by uploading the converted files to them (so we don't wait for conversion in pipeline).
- [x] QA approved by Coko
- [x] Deployed and tested on “ncbidev” of cloud68 hosted site, by Coko
- [ ] Deployed and tested on “ncbi” of cloud68 hosted site, by NCBI team
- [ ] Deployed and tested in NCBI-hosted environment by NCBI team and any necessary external editorial teams
- [ ] NCBI approves that all Acceptance Criteria have been Met
## Implementation
<!-- A description of the steps to implement the feature. To be completed by the lead dev. If there are multiple tasks, then break these down into "task" below.-->
Development broken down in tasks below: #1514
## Alternative approaches (if applicable)
<details><summary>Alternative approaches (not chosen)</summary>
## Alternative approaches (not chosen)
NCBI and Coko will agree on one clear development path of least effort to minimally address this issue for MVP that will still meet user stories for a stable data model, performance of the application, and file management, as well as other user stories such as metadata, parts, and complete documents. Possible technical solutions to evaluate least effort for a stable MVP solution include:
### Design option 2
* Maintain full TOC divisions in one page in a separate view for TOC building actions only. -- **Development estimate 3 days**
* This design is compatible with Design option 3 proposed in the Design options for resolving chapter-processed book manager performance issues (#1480)
* The Order UI is only necessary when manual ordering is required
* The Order UI is placed with exiting TOC Live view and Errors reporting
![New_TOC_ordening](/uploads/80074bbdd5117a0389b3fe0d3cdf0fb9/New_TOC_ordening.jpeg)
### Approach A
- BCMS displays position number of book part in UI relative to structural division (front | body | back) and any parts, and permits a right click of that book part to provide a reposition modal for the user to reposition that book part. The modal would be pre-filled by the UI with the current position. Note, there is no restriction on how many parts a user can create. Parts (and how many levels of Parts) dropdown would only appear if those Parts already exist. See proposed mockup from @deniskar:
![image__18_](/uploads/931a20427248465200d2adaa099db044/image__18_.png)
BCMS will not permit loading of TOC if any published book part does not have a position or has a duplicate position. This would be in addition to automated ordering, including all labels, and all publication and publication history dates. NCBI and Coko will provide a clear technical proposal for the estimates of this work to successfully meet these user stories and issues before a full development agreement can commence, so its deliverables are understood by all parties , OR
**Development estimate 5 days**
Note: this approach does not account for moving items in bulk.
### Approach B
- Bookshelf and PMC developers support the provision by a submitter in source files, sorting information within front matter, book body, a book part, or back matter that is passed in a consistent BITS element and attribute to the converted BITS XML stored in the BCMS, and which the BCMS can read to build a TOC or communicate submitter errors to fix if this needed information is missing or problematic. These errors will block publication of a TOC. Coko will read ALL supported BITS elements, not only explicit ordering information, but also other metadata for automated ordering, including all labels, and all publication and publication history dates. NCBI and Coko will provide a clear technical proposal for the estimates of this work to successfully meet these user stories and issues before a full development agreement can commence, so its deliverables are understood by all parties , OR
**Development estimate 3 days**
Note: Coko does not recommend this approach because it will place a lot of burden on users.
### Approach C (This is estimated in Design Option 2 above)
- Separate out ALL or PARTS of the TOC Building from the chapter-processed Book Dashboard for Files Management and Processing Integration. Possible design:
![image](/uploads/5eab2e4b9d84c89fbca5f9ca423b3f73/image.png)
OR
### Approach D
- Coko will add functionality for Bookshelf staff to download from the BCMS TOC.XML to adjust the order of contents in the XML and upload the file and reload it to PMC Books successfully. Note, it would be important for user to have BCMS create a TOC, download it, and just update it and reupload it with some modifications if too difficult using UI. User story will not support all users supplying a TOC from scratch. Possible design:
![image](/uploads/19c373b90a8aa6b6b3801b5dd81a099f/image.png)
If this last option is chosen and this could be extended with least effort to resolve TOC.xml issues, THEN add functionality for Bookshelf staff to download from the BCMS TOC.XML written by BCMS with loading errors to fix those errors in the XML and upload the file and reload it to PMC Books successfully
**Development estimate 7 days**
Note:
* This design is compatible with Design option 3 proposed in the Design options for resolving chapter-processed book manager performance issues (#1480)
* if this approach is taken our assumption is that uploading a TOC replaces the need for any manual ordering, but we continue to support automatic ordering and building TOC.xml for the auto-ordered cases.
Therefore all toc.xml writing issues must be addressed.
</details>
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* Milestone: linked
* Iteration: linked
* Dependencies: no
* Development estimate (hours): added to time trackingP1a: Book Manager redesign: HighStacy LathropDione Mentisdione@coko.foundationStacy Lathrophttps://gitlab.coko.foundation/ncbi/ncbi/-/issues/1471Revise user permissions rule for assigning book and book component editors an...2024-01-17T06:25:19ZDione Mentisdione@coko.foundationRevise user permissions rule for assigning book and book component editors and authors<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, ...<!-- Required. Provide a general summary of the issue in the title above -->
## Context
<!-- Give the necessary context for your proposal. For example, what problem will this feature solve for users? What are the use cases, benefits, and goals? -->
This issues describes the required changes to user permissions on books and book components to resolve book manager performance issues. The BCMS currently has permissions rules that say:
* when a user creates a book, make that user the editor of the book
* when a user creates a chapter (by uploading a file) make that user the author of the chapter. (this is done on the backend only at this stage)
However, as NCBI has reported (see epic &106 and [457#note_108306](https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1457#note_108306))
>Stacy's account is editor to migrated books and chapters. Having a reference list of all books and chapters that she's an editor for, and checking books against these lists causes severe performance degradation and consistent browser crashes for Stacy's account. Instead, permissions should probably be part of the getBook call lists should be removed.
![image-1](https://gitlab.coko.foundation/groups/ncbi/-/uploads/2d630c0654e9324d599f19a03b7981c8/image-1.png)
![image-2](https://gitlab.coko.foundation/groups/ncbi/-/uploads/46a1d908e9aad050e1e9d8b68b26e010/image-2.png)
![image-3](https://gitlab.coko.foundation/groups/ncbi/-/uploads/c295273da003a6600dac244bb4d24fcb/image-3.png)
## Proposal
<!-- A precise statement of the proposed feature. -->
An editor or author role should not be required. Therefore, revise the permissions rules:
1. When a user creates a book, make that user the editor of the book *unless* the user is a System Admin of the BCMS or an Org Admin of the Organisation
2. When a user creates a chapter (by uploading a file) make that user the author of the chapter *unless* the user is a System Admin of the BCMS; or Org Admin of the Organisation; or an Editor of the Organisation
On the frontend the change is: A system admin or Org Admin will not be listed on the "Team" tab in the "Editor" section for a book or chapter and an Editor of an Org will not be listed on the "Team" tab in the "Editor" section for a chapter.
Note: the user who creates the book or chapter component is recorded as the `owner`. In future we can expose this information for activity tracking in the UI. E.g. "Stacy created 'book title' on 'date'."
## Acceptance criteria
<!-- Provide the criteria that should be met for this feature. These criteria must be clearly defined customer requirements aligned to NCBI’s user stories in its Statement of Work. These criteria must be testable by either user testing, unit tests or integration tests.-->
* A system admin or Org Admin will **not** be listed on the "team" tab in the "Editor" section in the Team modal of a book. (see screenshot example below)
* A system admin, Org Admin, and Editor will still have the relevant permissions as per [BCMS permissions rules](https://docs.google.com/spreadsheets/d/1cJeKIUhkkM_kwQ8Fzh9d2Vw9h2IJcgT3_87sWLiNdNQ/edit?usp=sharing)
![Screenshot_2023-08-02_at_15.53.27](/uploads/3ccdbf9e66b56e9b8c9a8994c9a9fd5e/Screenshot_2023-08-02_at_15.53.27.png)
Note: the performance from the users perspective will be tested after all linked items are complete
## Definition of ready
<!-- A checklist of what needs to be done to a product backlog item before the team can start implementing it in the next sprint. -->
- [x] BCMS User Story / Context has been well defined
- [x] The priority of the user story is specified and agreed
- [ ] Digital assets added (design, database scheme, mockups etc if relevant)
- [x] Coko Technical Proposal approved by NCBI
- [x] Testable Acceptance Criteria approved by NCBI
- [x] Estimate of effort to complete (time or points)
- [ ] The issue has been broken down into development tasks (if necessary)
- [x] Requirements Clarified
- [x] The product owner and development team agree that the user story is ready for development
- [x] NCBI adds “Dev_Ready”
## Definition of done
<!-- A checklist of criteria that must be completed for a story to be considered “done.” -->
- [x] All coding tasks are finished and implemented
- [x] QA approved
- [x] Deployed and tested on “ncbidev” (by Coko team)
- [ ] Deployed and tested on “ncbi” (by NCBI team)
- [ ] Acceptance Criteria Met
## Implementation
<!-- A description of the steps to implement the feature.-->
* Update the permissions rules above
* Any time a user is assigned Org Admin or Sys Admin role, revoke their role-specific privileges on collections, books, and chapters.
* Write migration script for all current data to remove Sys admins and Org Admins users as "editors" of book and chapters that they created
## Alternative approaches (if applicable)
<!-- Include any alternatives to meet this use case. -->
## Scheduling
* [x] Milestone is linked
* [x] Iteration is linked
* [x] Dependencies: none
* [x] Development estimate is added to issue time trackingP1a: Book Manager redesign: HighRebecca OrrisRebecca Orris