Document TOC requirements for chapter-processed books (all workflows)
Updated by @lathrops1 20211024
User Story
Users can submit chapter components that are authored, submitted, udpated, and processed independently within a book manager component in the BCMS. To create the book "landing page" known as the TOC content page by NCBI, Bookshelf needs the ability for users to create that TOC, by moving components in the correct strutural units (front | body | back), grouped by any parts, and sorted within those units and groups by clear rules, manual or automatic. Bookshelf also needs what has been communicated by the user through that feature to be accurated coded as a TOC.XML that also includes metadata added and updated through another feature, and for that TOC.XML to be passed to Bookshelf for accurate processing and posting to the Bookshelf site. If any errors occur during that process they need to be communicated, tracked, and managed through the BCMS. That entire process should be tracked and documented in the BCMS along with the links(s) to view the final displayed TOC on Bookshelf.
Technical Details
(some lacking / in other issues that should be add / linked to by Coko development team - @DioneMentis)
- toc.xml spec
- Coko will create a toc.xml file for all chapter-processed books based on the book components in the Book Manager
- The following settings chosen in the UI affect the TOC (see screenshot reference below):
- Group chapters into parts: This allows users to manually add parts which must be reflected in the toc.xml
- Order chapters by: The order selected is reflected in the toc.xml
- All workflows require additional ordering value: By Publication date
- Display headings depth: all possible heading options must be included in the toc.xml. The setting chosen determines what gets rendered on Bookshelf (through the domain service integration).
- Display metadata on TOC: all book-part-metadata must be inlcuded in the related
<toc-entry>
, excluding empty fileds. The settings chosen determine what gets rendered on Bookshelf (through the domain service integration).
-
@latternm has provided samples showing the structure of all requirements in point 2:
- toc-template-main.xml: covers all the items from the "metadata" and "book order" section in requirements above (we assume PIs would be included in book-meta)
- toc-template-special-no-part-files.xml: that's just an a template for how part entries would look if the parts are not ingested (no chapter package / NBK ID)
- toc-template-special-alpha-index.xml: that's a template for how to create a toc if an alpha index is required
- toc-template-special-hidden-or-repeated-entries.xml: that's a template for how repeated or hidden toc entries would look.
- NOTE:
version
in the templates refers to the the published version on Bookshelf which equals book component version in BCMS.
- The entire
<book-meta>
node must be added to the toc.xml file. - The
<collection-meta>
node must be added to the toc.xml file only if the book belongs to a collection (i.e. th node cannot be present and empty). - Once created, send the toc.xml in cron job (#625 (closed))
Items to clarify
this comment below
SeePHASE 1 ACCEPTANCE CRITERIA
BOLD ITEMS ARE NCBI IDENTIFIED GAPS WHERE 1) THERE IS A NEED TO RESOLVE THEM FOR BOOKSHELF TO DO ITS WORK AND NOT INTRODUCE ERRORS INTO THE ARCHIVE AND 2) THERE IS NO CLEARLY EXISTING DOCUMENTATION IN ANY ISSUES
ITEMS IN ITALICS CAN BE NEGOTIATED TO A MVP SOLUTION FOR PHASE 1 AND PHASE 2 PRIORITY
-
Ensure that there are not significant changes to the display of current TOC pages displayed on Bookshelf when move to new BCMS workflows, by the following features and processes:
Set Up
-
User must be able to select a required TOC sorting rule of A-Z | chapter label | manual as a TOC setting (ability to auto sort by date held to future phase given complexity of different date types in Bookshelf data) -
User must be able to indicate if they have parts in their TOC as a TOC setting -
User must be able to provide metadata for their parts as part of part support -
User must be able to indicate if they have parts that have content other than part title and metadata (contributors, dates) -
User must be able to indicate what they want to display on the TOC as TOC settings, which must include: 1) title / section level to display and ability to expand, 2) contributors, publication dates / history, subtitles, alternative titles -
At migration and after TOC settings that are domain attributes in PMCBook can be accurately updated as BCMS book settings in the BCSMS
For TOC setting properties see acceptance criteria documented at: https://docs.google.com/spreadsheets/d/1Xd0vKDbaYrajfnUmYqCnXUY5Ir0znqc7o0KGVp_okg0/edit#gid=725030307
Grouping and Sorting
-
BCMS automatically places a chapter in front / body / back of the book component by the content type in the converted book part metadata as follows:
Front:
- Acknowledgments
- Biography
- Dedication
- Front Matter Part
- Foreword
- Preface
Body:
- Chapter
Back:
Appendix Reference List Glossary Section
-
To fix placement in a structural unit, user can fix it in the source document, by downloading it, fixing the issue, reuploading, and reconverting it -
If a content type is provided in the source document that is not recognized / supported by the BCMS for correct placement, the BCMS places the component at the top of the body and provides a notification (on the display page and by email?) that it is unable to place the document per supported rules AND will not code that entry in any TOC.XML until that notification is addressed by the user -
Upon publication of a component or updated of book settings / metadata, BCMS tags each component in the correct structural unit in the TOC.XML per provided specifications
A-Z auto sorting rules:
-
IF auto A-Z setting is selected by user, BCMS sorts chapters in the Body section of the BCMS Book Manager page in numeric and then A-Z order by the title provided in the converted XML as soon as it has the information to do so. For examples of correct ordering see: -
To fix placement of component, user can fix it in the source document by downloading it, fixing it, reuploading, and reconverting -
Until the system has a chapter title to place the document per the A-Z sorting rules, the BCMS places the component at the top of the Body unit and provide a notification (on the display page and by email?) that it is unable to place the document per supported rules AND will not code that entry in any TOC.XML until that notification is addressed by the user -
Upon publication of a component or updated of book settings / metadata, BCMS tags the chapters in numeric then A-Z order in the body in the TOC.XML per provided specifications
Chapter label sorting rules:
- [ ] IF sort by chapter label is selected by the user, user can indicate if order is ascending or descending OR ascending | descending will be assumed by source type in the book metadata TBD by development team discussion For examples of correct ordering see:
-
To fix placement of component, user can fix it in the source document by downloading it, fixing it, reuploading, and reconverting -
Until the system has a chapter label to place the document per the chapter label sorting rules, the BCMS places the component at the top of the Body unit and provides a notification (on the display page and by email?) that it is unable to place the document per supported rules AND will not code that entry in any TOC.XML until that notification is addressed by the user -
Upon publication of a component or updated of book settings / metadata, BCMS will tag the chapters per the agreed chapter label sorting rules in the body in the TOC.XML per provided specifications
Manual grouping / sorting rules:
-
IF manual sorting order is selected by the user, user can drag their chapter exactly where they want it in the body of the TOC
- [ ] IF more than one front matter component, the BCMS places the component at the top of the Front unit and provides a notification (on the display page and by email?) that it is unable to place the document per supported automatic rules AND will not code that entry in any TOC.XML until that notification is addressed by the user
- [ ] IF more than one back matter component, the BCMS places the component at the top of the Back unit and provides a notification (on the display page and by email?) that it is unable to place the document per supported automatic rules AND will not code that entry in any TOC.XML until that notification is addressed by the user
- [ ] IF support for parts is selected by the user AND the user adds parts, the BCMS places the component at the top of the Body unit and provides a notification (on the display page and by email?) that it is unable to place the document per supported automatic rules AND will not code that entry in any TOC.XML until that notification is addressed by the user**
-
IF support for parts is selected, user can place their component in the part they created and wishes that component to belong, and then that component will be autosorted withing the part it was added by any selected auto-sorting rules or manually sorted within that part if a manual order is selected -
IF support for parts is selected by the user, user can still have a component in the body that does not belong to any part
- [ ] Users must resolve any notifications that the system does not know how to place a componet (even if it is ticking an "I accept where it is placement) before a TOC.XML is sent for publication - OR SIMPLIFICATION PHASE 1, ALL CHAPTER-PROCESSED BOOKS REQUIRE PREVIEW APPROVAL OF ALL CHAPTER COMPONENTS AND A SINGLE TOC.XML APPROVAL ACKNOWLEDGEMENT)
-
BCMS will tag in the TOC.XML per provided specifications the manually placed components in the correct order within the correct structural and part placements as the user selected in the TOC builder / Book Manager page
Parts
-
IF support for parts is selected, user can create a part and provide following metadata: title, contributors, publication date and BCMS will tag this ONLY in the TOC.XML per provided specifications -
IF support for parts with body content is selected, user can ALSO provide paragraph text with agreed to styles - see issue where this is identified - and BCMS will tag the metadata in TOC.XML and provide a part.xml file per provided specifications
Special Use Cases
-
BCMS allows user to copy a component and place it in more than one part and this will provide a PI in the built TOC.XML per specifications -
BCMS displays a meaningful notification icon on the Book Manager page for any chapter that has a PI in the converted XML that indicates it should be hidden from the TOC and will ensure per provided specifications for tagging that it doesn't display on the Bookshelf TOC page -
BCMS displays a meaningful notification on the Book Manager page for any chapter that has archive PIs in the converted XML -
BCMS indicates on the book manager page "single-document" books where there is no table of contents displayed on Bookshelf, as that page displays the entirety of the single document, and tags the TOC.XML in these cases per provided specifications.
Generation of TOC and its Processing and Tracking / Management
-
A valid TOC.XML that meets the Bookshelf tagging guidelines (so passes style checks) and accurately reflects the TOC that was built by auto and manual TOC rules selected at set up or modified by the user at a later date is generated by the BCMS according to provided specifications, to include any collection metadata, all book metadata, and all of the contents, component and part, and special processing instructions -
Users can download the TOC.XML from the BCMS via a TOC preview page -
BCMS sends the TOC.XML as a chapter ingest per XML integration specifications once overnight as a chron job, whenever the following occur: -
collection metadata is updated
-
book metadata is updated
-
a part or any of its metadata is added or updated or deleted
-
a front, body, or back matter book part is published or republished so all titles and metadata are accurate and up-to-date
-
any published or republished component is moved automatically or manually within the book / toc manager page
-
Any TOC sorting or part settings is updated
-
The status of the TOC (unpublished | published | loading errors / failed state) displays on the book manager page -
The published link of the TOC is available in the BCMS along with the date and time it was published -
System admin, org admin / editor receive email notification to their email account with the published link of the TOC when TOC is published / republished -
Notifications and links sent by NCBI per chapter ingest integration specifications about the status of the TOC.xml chapter ingest display from an Errors tab
Notifications
-
System Admin | Org Admin | Editors receive email notification when any book is added to the book, including a note in the message if the book cannot be placed in the TOC per any provided automatic rules -
System Admin | Org Admin | Editors receive email notification about all status changes about the TOC.xml ingest, including failures in adding it to Bookshelf and when it is successfully published
[Original issue]
From agenda:
Clarification on the TOC requirements:
- automatic ordering rules versus manual ordering by Editor
- generating the TOC at a BITS XML file within the source file manager