ncbi issues
https://gitlab.coko.foundation/ncbi/ncbi/-/issues
2024-01-08T17:24:35Z
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/1
Create a default branch
2024-01-08T17:24:35Z
Danjela Shehi
danjelashehi@gmail.com
Create a default branch
A default branch (e.g. master) does not yet exist for Ncbi,
The project Owner or Maintainer has to create a default branch
A default branch (e.g. master) does not yet exist for Ncbi,
The project Owner or Maintainer has to create a default branch
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/2
Data Modeling for the application
2022-03-02T03:06:19Z
Danjela Shehi
danjelashehi@gmail.com
Data Modeling for the application
Extracting the main entities of the application and tracking the relationships between them.
Organisations
Books
Divisions
Book_components
Metadata
Files
Extracting the main entities of the application and tracking the relationships between them.
Organisations
Books
Divisions
Book_components
Metadata
Files
Data model design and implementation
Giannis Kopanas
jkopanas@gmail.com
Giannis Kopanas
jkopanas@gmail.com
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/3
Storybook Setup
2023-11-29T20:38:03Z
Danjela Shehi
danjelashehi@gmail.com
Storybook Setup
Danjela Shehi
danjelashehi@gmail.com
Danjela Shehi
danjelashehi@gmail.com
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/4
add contributing guideline
2021-10-19T23:39:48Z
Danjela Shehi
danjelashehi@gmail.com
add contributing guideline
Sidorela Uku
Sidorela Uku
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/5
Cypress
2020-01-29T13:11:44Z
Danjela Shehi
danjelashehi@gmail.com
Cypress
* [x] cypress installation
* [x] write an integration test of app
* [x] cypress installation
* [x] write an integration test of app
Danjela Shehi
danjelashehi@gmail.com
Danjela Shehi
danjelashehi@gmail.com
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/6
Questions on workflow and data modeling of the app
2020-11-10T18:36:46Z
Giannis Kopanas
jkopanas@gmail.com
Questions on workflow and data modeling of the app
Gathering here questions that came up while working on data modeling.
* [x] 1. are reviews and comments related? Comments can be made only during "review" status of the book?
Comments relate to a book component version (chapter-proc...
Gathering here questions that came up while working on data modeling.
* [x] 1. are reviews and comments related? Comments can be made only during "review" status of the book?
Comments relate to a book component version (chapter-processed) or a book version (book-processed)
* [x] 2. Does review mean a new version of the original book (replaced files, comments, decisions)?
Reviews happen per book component version (chapter-processed) or a book version (book-processed). A new version is created after 'Published' state
* [x] 3. What are the things that we want to track during the lifecycle of a book ? is there a need to revert to a previous state by reading the tracked events
List of activity to track for user activity stream and production reports in #209.
There is no need to revert to a previous state of the content.
* [x] 4. Preview functionality is happening only during REVIEW status? What actually is a preview of the book? Rendered content of all the chapters? using PageJS for example?
Preview link generated by NCBI and sent to BCMS. Available to view during all states except Converting and Errors
* [x] 5. only the latest generated preview is saved ? or do we need versioning on previews?
Only the latest file version
* [x] 6. Identify metadata of a book
Done in #142
* [x] 7. What are the User Preferences?
Email notification preferences (scoped at this stage for one-to-one or daily digest) #252
* [x] 8. What is the difference between Person and Unaffiliated Individual?
These user roles are not relevant anymore
Data model design and implementation
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
2020-02-17
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/7
FAQ questions
2020-01-29T13:11:53Z
Danjela Shehi
danjelashehi@gmail.com
FAQ questions
add section for commands to run in case of having problems with installation
add section for commands to run in case of having problems with installation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/8
Login page
2020-02-18T08:59:47Z
Danjela Shehi
danjelashehi@gmail.com
Login page
Login and registration UIs
Giannis Kopanas
jkopanas@gmail.com
Giannis Kopanas
jkopanas@gmail.com
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/9
Register page
2020-02-18T09:00:24Z
Danjela Shehi
danjelashehi@gmail.com
Register page
Login and registration UIs
Giannis Kopanas
jkopanas@gmail.com
Giannis Kopanas
jkopanas@gmail.com
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/10
Set up project management
2020-02-14T14:08:11Z
Dione Mentis
dione@coko.foundation
Set up project management
This is my work in progress
* [x] Project wiki: This needs to include all project information for Coko and NBCI team. I'll set up first version and then create an issue for other sections that needs to be written.
* [x] Quick communica...
This is my work in progress
* [x] Project wiki: This needs to include all project information for Coko and NBCI team. I'll set up first version and then create an issue for other sections that needs to be written.
* [x] Quick communication: Mattermost group that for Coko and NCBI team
* [x] Project schedule: as Milestones in Gitlab
* [x] Tagging system for issues: We need more than we have currently ('to do', 'doing', 'done', 'on hold')
* [x] Sharing documents with NCBI: client-share folder in Nextcloud
* [x] Agree on meeting rituals: with NCBI
* [x] Agree on meeting rituals: internal
* [x] Invite Henrik to repo
* [x] Invite NCBI team to repo
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
2020-02-14
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/11
Coko stand up 13 Feb
2020-03-17T10:46:45Z
Dione Mentis
dione@coko.foundation
Coko stand up 13 Feb
Hi @John.kopanas and @danjela
In our stand up tomorrow we need to finalise:
* [x] The breakdown of milestones
* [x] Dates for milestones (as far as possible)
* [x] Review first version of the wiki
* [x] Review project labels
* [...
Hi @John.kopanas and @danjela
In our stand up tomorrow we need to finalise:
* [x] The breakdown of milestones
* [x] Dates for milestones (as far as possible)
* [x] Review first version of the wiki
* [x] Review project labels
* [x] Review current issues against guidelines in wiki
* [x] Add any missing issues
* [x] Confirm agenda for progress review on 18 Feb
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/12
scoping milestones
2020-03-31T11:01:40Z
Giannis Kopanas
jkopanas@gmail.com
scoping milestones
evaluating wireframes and functionality, so we can make solid estimations about the milestones
evaluating wireframes and functionality, so we can make solid estimations about the milestones
Scoping features requirements
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/13
Review workflow diagram: single doc XML book via FTP pathway
2021-05-09T10:42:41Z
Dione Mentis
dione@coko.foundation
Review workflow diagram: single doc XML book via FTP pathway
Hi @John.kopanas and @danjela
I've created a diagram to show the flow of a single document XML book via FTP pathway.
Note that a "single document XML book" means that a book's text is provided in one XML file, but there are other fil...
Hi @John.kopanas and @danjela
I've created a diagram to show the flow of a single document XML book via FTP pathway.
Note that a "single document XML book" means that a book's text is provided in one XML file, but there are other files included in the FTP upload, as in the example screenshot:
## Example FTP files
![ukhsdr0807_2020-02-06_08-37-23.tar](/uploads/a40fe6ff85b24ba9c13d7a8809bbe396/ukhsdr0807_2020-02-06_08-37-23.tar.jpeg)
I've added this example book to NCBI/client-shared/sample-content/ukhsdr0807_2020-02-06_08-37-23.tar.gz
See the final published version here: https://www.ncbi.nlm.nih.gov/books/n/ukhsdr0807 (note the redirect URL is https://www.ncbi.nlm.nih.gov/books/NBK553523/)
## Workflow diagram
Here's the workflow diagram:
![workflow-single-doc-book-XML-from-FTP-20200213](/uploads/042faaf225ec4a3284b0b92321d35315/workflow-single-doc-book-XML-from-FTP-20200213.png)
## Queries to resolve
I've noted queries in orange boxes. Some of these came up during our recent design session at NCBI; others are new queries I had while drawing the workflow. We need to resolve these with NBCI, but first, please:
* [x] prioritise the queries in order or urgency -- what do you need to know asap to continue current work versus what can wait and for how long.
* [x] provide your feedback on workflow and the queries
* [x] let me know what other queries you have on this workflow
### Notes on queries
Query 2.1: What kind of unique identifier should be used?
The example file has this book ID information:
```
<book-id book-id-type="pmcid">ukhsdr0807</book-id>
<!--publisher-id: 12/209/27-->
<book-id book-id-type="doi">10.3310/hsdr08070</book-id>
<book-title-group>
```
We should not assume that this will be the ID going forward. The files provided here are for NCBI's current FTP workflow (according to [these general XML file submission specifications](https://www.ncbi.nlm.nih.gov/books/about/filespec/)) which may differ from the specifications going forward.
[workflow-single-doc-book-XML-from-FTP-20200213.pdf](/uploads/20a7fc2e38f848afd22cc07bb083eb25/workflow-single-doc-book-XML-from-FTP-20200213.pdf)
Scoping features requirements
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/14
Draw workflow diagram: single PDF book via FTP pathway
2020-03-24T15:23:20Z
Dione Mentis
dione@coko.foundation
Draw workflow diagram: single PDF book via FTP pathway
Mapping out the workflow based on discussions at design meetings with NBCI and current wireframes.
Mapping out the workflow based on discussions at design meetings with NBCI and current wireframes.
Scoping features requirements
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/15
Draw workflow diagram: SFM overview
2020-08-06T14:18:05Z
Dione Mentis
dione@coko.foundation
Draw workflow diagram: SFM overview
Hi @John.kopanas, @jure, @yannis and @danjela This diagram show an overview of the SFM workflow for content that is manually submitted.
cc @adam
![SFM-workflow-overview-20200218](/uploads/228a6e666e126c6f81c7e3f36ce1e94a/SFM-workflo...
Hi @John.kopanas, @jure, @yannis and @danjela This diagram show an overview of the SFM workflow for content that is manually submitted.
cc @adam
![SFM-workflow-overview-20200218](/uploads/228a6e666e126c6f81c7e3f36ce1e94a/SFM-workflow-overview-20200218.png)
There are a lot of different tasks that can happen in the 'Editing' and 'Reviewing' states depending on the type (Word/PDF) and quality (complete/incomplete/with issues/etc.) of the files submitted. I'll break these down in more diagrams, but the important things to understand now include:
1. The **'Editing' state means preparing content for publishing**. This work is done by the Organisation team who submits the content.
2. The **'Reviewing' state means checking, resolving issues, and approving content for publishing**. Both the Organisation and NCBI teams do this work at various stages in the workflow.
3. A preview of the content is only available after it has been successfully converted to XML and pushed to NCBI's staging site.
4. There are essentially only two states that the content can be in before it is published live: Editing and Reviewing. These states can be made more meaningful for the users with standardised or customised statuses. At a minimum I think we should differentiate between content that is being reviewed by the Org or by the NCBI.
5. The Source files being managed are:
* the original source files submitted by the organisation: Word/PDF book documents and supplementary files
* the converted XML files (provided by the data supplier) that will be used for publication.
6. Versioning needs to be tracked for both original source and published source at the file and document level.
Here is an example set of files that a Org might submit manually:
![Untitled_drawing](/uploads/55dc55d71a529b4c690c94d6f8072f53/Untitled_drawing.jpg)
And here is the final published book on NCBI's Bookshelf: https://www.ncbi.nlm.nih.gov/books/n/cer141
Let me know if you have questions. I'll ask NCBI to sense-check too.
Scoping features requirements
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/16
Agenda and minutes for Review meeting 18 Feb
2020-03-31T06:43:13Z
Dione Mentis
dione@coko.foundation
Agenda and minutes for Review meeting 18 Feb
@lathrops1 Hi Stacy
Here is our agenda for 18 Feb:
* [x] 1. Intro to Coko's Development team
* [x] 2. Review "ways of working" on Wiki (and clarify which team members will be responsible for NBCI reviews).
* [x] 3. Progress update ...
@lathrops1 Hi Stacy
Here is our agenda for 18 Feb:
* [x] 1. Intro to Coko's Development team
* [x] 2. Review "ways of working" on Wiki (and clarify which team members will be responsible for NBCI reviews).
* [x] 3. Progress update on data model
* [x] 4. Project Schedule update
* [x] 5. Priorities for the next 2 weeks
* [x] 6. Our documentation needs from NCBI (short term)
Please let me know if there is anything else to add from your side.
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/17
PDF workflow use cases
2021-05-19T17:25:59Z
Dione Mentis
dione@coko.foundation
PDF workflow use cases
This should show any differences in workflow compared to Word workflow
This should show any differences in workflow compared to Word workflow
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/18
Book Collection
2021-01-13T20:45:40Z
Giannis Kopanas
jkopanas@gmail.com
Book Collection
Trying to analyze the XML file of a book I concluded to some points regarding the collections of a book.
So Let's have as a reference the below structure of XML:
```
<collection-meta collection-type="book-series">
<title-group>
<t...
Trying to analyze the XML file of a book I concluded to some points regarding the collections of a book.
So Let's have as a reference the below structure of XML:
```
<collection-meta collection-type="book-series">
<title-group>
<title>Programme Grants for Applied Research</title>
</title-group>
<volume-in-collection><volume-number>8.2</volume-number></volume-in-collection>
<issn publication-format="print">2050-4322</issn>
<issn publication-format="electronic">2050-4330</issn>
</collection-meta>
<collection-meta collection-type="ncbi-books-collection">
<title-group>
<title>Programme Grants for Applied Research</title>
</title-group>
<collection-id collection-id-type="pmcid">ukpgfarcollect</collection-id>
</collection-meta>
```
Some are queries more for confirming that my thoughts are on the right road, and some others are questions on what we need to have in SFM.
1) A book could have multiple Collections
2) A collection has a title, `collection-type` (book-series, ncbi-books-collection) and it's own metadata `collection-id`, `issn`.
**Query 1**
Do we have predefined collection Types (`collection-type`)? do they play any particular role in the workflow of the SFM or is just `texts` that
should be assigned to a book from within system admin?
**Query 2**
Are there more than those collections metadata which is not in my example? I can only see `collection-id, ISSN, title`.
Since we need to create collections from within the System Admin probably we need a list of field attributes that are going to be saved in our database.
phase-01-metadata
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/19
Publishing content at the book or book-part level
2021-05-19T17:45:34Z
Dione Mentis
dione@coko.foundation
Publishing content at the book or book-part level
@lathrops1
Please can your team check the description below and provide feedback. I'll then move this information to the wiki.
(cc @John.kopanas, @yannis and @jure, this affects our approach to versioning in the data model. Please a...
@lathrops1
Please can your team check the description below and provide feedback. I'll then move this information to the wiki.
(cc @John.kopanas, @yannis and @jure, this affects our approach to versioning in the data model. Please add further questions for Stacy if you have any).
---
Content can only be processed at the book level or the book-part level.
### Book level
A book that is published at the book level, such as a monograph, is released in its entirety. Here's [one example](https://www.ncbi.nlm.nih.gov/books/NBK553523/) and [another example](https://www.ncbi.nlm.nih.gov/books/NBK20709/).
The book document could be submitted in either XML or PDF format as one or multiple files. The PDF-source files can originate via FTP or a user could upload the files directly to the SFM.
Rolling back to previous *published* versions cannot be supported at the book level. This is because the PMC database was developed for journal publishing.
However, when book level content is updated, new versions of the submitted-source and b-XML *files* should be stored in the SFM. b-XML means 'book XML'; this is one XML file containing all the book's content. The NCBI chops up the b-XML into p-XML (book-part xml); each p-XML file correlates to one page of rendered content to the Bookshelf website. There is an additional conversion is from p-XML to n-XML: this is the file that is stored in the PMC book database and rendered on the Bookshelf website. *The NCBI team have a meeting on Monday to decide whether the p-XML and n-XML should be stored in the SFM or not.*
When book level content is updated sufficiently to be considered a new *edition*, this edition is a new book record in the SFM. Previous editions should be archived, however there needs to be a way to link the book records for previous editions to the current book record.
There isn't a need to assign statues or user roles to chapters (book-part level) in the SFM when the content is being published at the book level. Even though the book has chapters (book-parts), it moves through the SFM as one unit.
### Book-part level
Books are published at the book-part level (i.e. chapters) when these parts need to be treated as a discrete unit, for example, for journal articles that are updated multiple times.
The book-part documents could be submitted in either XML, PDF or MS Word format as one or multiple files. The PDF-source files can originate via FTP or a user could upload the files directly to the SFM. All Word files are expected to be uploaded in the SFM interface and the NCBI only publishes Word documents at the book-part level at this time.
Each book part is released separately. Here's [one example](https://www.ncbi.nlm.nih.gov/books/NBK430685/) from an XML-source workflow and [another example](https://www.ncbi.nlm.nih.gov/books/NBK1116/) from a Word-source workflow. Both of these books that are published at the book-part level.
Versioning of the published document can only be supported at the book-part level. For example, each book part in [this book](https://www.ncbi.nlm.nih.gov/books/NBK82221/) can have multiple published versions, as shown on the right of the screen in [this example book part](https://www.ncbi.nlm.nih.gov/books/NBK65761/).
When an update is made to a book part, the editor decides if the update is a revision or a new published version. This decision can be made in the SFM at at the 'publish' moment.
A book published at the book-part level has metadata associated with each book-part and metadata that relates to the book as a whole.
It is not always the case that the TOC is ordered alphabetically, so there needs to be a way for editors to reorder the book-parts when new chapters are added.
Organisations need to assign statues or user roles to chapters (book-part level) in the SFM when the content is being published at the book level.
**Question for NCBI**
1. In [this example](https://www.ncbi.nlm.nih.gov/books/NBK430685/), it looks like the 'whole book' is *everything* that the publisher (StatPearls) releases collected into a TOC that is ordered alphabetically. Is this common? Also, in this example is 'StatPearls' both the book and the publisher?
1. I assume an organisations knows well before they submit content to the NCBI whether it should be processed at the book or book-part level. Is this correct?
https://gitlab.coko.foundation/ncbi/ncbi/-/issues/20
Book Metadata
2021-01-13T20:45:40Z
Giannis Kopanas
jkopanas@gmail.com
Book Metadata
Reading the XML example of a book, I extracted the following structure for the metadata
```
<book-meta>
<book-id book-id-type="pmcid">ukpgfar0802</book-id>
<book-id book-id-type="doi">10.3310/pgfar08020</book-id>
<book-title-grou...
Reading the XML example of a book, I extracted the following structure for the metadata
```
<book-meta>
<book-id book-id-type="pmcid">ukpgfar0802</book-id>
<book-id book-id-type="doi">10.3310/pgfar08020</book-id>
<book-title-group>
<book-title>Screening and brief interventions for adolescent alcohol use disorders presenting through emergency departments: a research programme including two RCTs</book-title>
<alt-title alt-title-type="running-head">SBIs for adolescent alcohol use disorders</alt-title>
</book-title-group>
<contrib-group>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid" authenticated="true">https://orcid.org/0000-0002-9511-7230</contrib-id>
<name><surname>Deluca</surname><given-names>Paolo</given-names></name>
</contrib>
<aff id="aff1"><label>1</label> text</aff>
</contrib-group>
<author-notes>
<corresp id="an1">
<label>*</label>Corresponding author <email xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="mailto:paolo.deluca@kcl.ac.uk">paolo.deluca@kcl.ac.uk</email>
</corresp>
<fn id="an2">
<label>†</label>
<p>Chief investigator and senior author</p>
</fn>
</author-notes>
<pub-date date-type="pub" publication-format="print">
<month>1</month><year>2020</year>
</pub-date>
<book-volume-number>8.2</book-volume-number>
<publisher>
<publisher-name>NIHR Journals Library</publisher-name>
<publisher-loc>Southampton (UK)</publisher-loc>
</publisher>
<pub-history>
<date date-type="reviewed"><!--orig date-type: editorial review begun--><month>6</month><year>2017</year></date><date date-type="accepted"><!--orig date-type: accepted for
publication--><month>2</month><year>2019</year></date>
</pub-history>
<permissions>
<copyright-statement>Copyright © Queen’s </copyright-statement>
<copyright-year>2020</copyright-year>
<copyright-holder>Queen’s Printer and Controller of HMSO</copyright-holder>
<license>text ...</license>
</permissions>
<abstract id="abs4" abstract-type="short">
<title>Headline</title>
<p>This research programme ...</p>
</abstract>
<abstract id="abs1">
<title>Abstract</title>
<sec id="abs1-1">
<title>Background</title>
<p>Alcohol consumption ....</p>
</sec>
</abstract>
<kwd-group>
<kwd>ALCOHOL</kwd>
<kwd>BRIEF INTERVENTION</kwd>
</kwd-group>
<funding-group>
<award-group>
<funding-source xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.13039/501100000272">National Institute for Health Research</funding-source>
<award-id>RP-PG-0609-10162</award-id>
</award-group>
</funding-group>
<counts>
<book-fig-count count="12"/><book-table-count count="18"/><book-equation-count count="0"/><book-ref-count count="178"/><book-page-count count="168"/>
</counts>
<custom-meta-group>
<custom-meta>
<meta-name>Description</meta-name>
<meta-value>Primary research project. Deluca P, Coulton S, Alam MF, Boniface S, Donoghue K, Gilvarry E, <italic toggle="yes">et al.</italic> </meta-value>
</custom-meta>
</custom-meta-group>
<notes id="n1">
<sec id="n1-1" sec-type="article-history">
<title>Article history</title>
<p>The research reported ...</p>
</sec>
</notes>
</book-meta>
```
So from my understanding, the main metadata fields of a book could be:
pmcid, doi, book-title, alt-title, contrib, author-notes, pub-date, book-volume-number, publisher, pub-history
copyright-statement, copyright-year, copyright-holder, license, abstract, kwd, notes, funding
**Query 1**
Should we include all of them in the SFM? It probably will be a form where the user inserts/updates the metadata.
**Query 2**
Are there more metadata we should consider adding?
phase-01-metadata
Dione Mentis
dione@coko.foundation
Dione Mentis
dione@coko.foundation