Skip to content

GitLab

Closed
Created by Martin Latterner@latternmReporter0 of 1 task completed0/1 task

submit cover images separately

Context

In #739 (closed), we decided to decouple ingest and cover image submissions. This applies to both wholebook and chapter-processed domains. This also applies to collection covers (there is no difference between a collection and a wholebook in terms of NCBI's processing).

NCBI needs two things:

  1. the physical image (ex. bcms123-cover.jpg)
  2. the domain name

Workflow

  1. User adds or updates a cover image in the book or collection metadata UI
  2. BCMS creates package: Zip the image together with a JSON file detailing the domain: ex bcms134-cover.zip contains foobar.jpg and foobar.json. JSON contains {domain: "bcms134"}.
  3. submit to FTP dir: /covers/
  4. Processing errors NCBI-side should be very rare, but are no impossible. In these cases NCBI sends a receipt via Kafka to a dedicated topic
  5. BCMS sends email notification for the errors reported (to be defined in #511

Updated spec from #751 (comment 73067)

Some refinements of the spec:

  1. json/zip file names are expected to match ^([\w\.-]+)\.(\d{4}_\d\d_\d\d-\d\d_\d\d_\d\d)\.(json|zip)$
  2. zip package must contain exactly one image
  3. JSON must include the fields package, package_id, domain

Receipts are sent to the Kafka topic cover_receipt. Notifications contain:

  1. package_id: the BCMS reference ID for the package 
  2. status: 0=success, 3=failure
  3. thumb: the name of the converted cover (not really useful, but included anyways)
  4. timestamp
  5. notices: a list of error and/or warning notices

In production, the FTP is polled once a day, at 6.30 pm.= In development, I'll keep it at every minute - to facilitate dev work.

Important note:

The complete process for cover thumbs looks like this:

  1. Cover thumbs get submitted to a dedicated location on the PMC file system - on a rolling basis
  2. Images get picked up from #1 and submitted to a VCS - twice a day, at 10am and 7pm
  3. Images get synced from the VCS to NCBI web fronts - once a day, at 3am

This process here only covers step #1!

In development, steps #2 and #3 are not executed - so @lathrops1 , @DioneMentis: do not expect any thumbs for all those test domains to show up. You cannot verify that the process works by looking at the preview site. @John.kopanas can verify it based on what he gets from Kafka.

In production, note that #3 is a nightly tasks - so @lathrops1 , @Kireev: expect the thumb to show up on preview one day after it has been submitted.

Acceptance criteria

  • Covers submitted to BCMS display on NCBI Bookshelf web pages per NCBI rendering rules after covers are passed to NCBI from BCMS and committed and distributed by NCBI
Edited by Dione Mentis