NIH | National Cancer Institute | NCI Wiki  

Error rendering macro 'rw-search'

null

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Agenda

EVS Discussion

Open System Tickets

Note: there was an attack last night, Tue Oct 3 from 4:48 pm -> 5:10 pm.
Note: there was another outage again, morning of Wed, Oct 4, starting around 5am to around 7:45am
Based on communications, it appears that morning outage may have been due to a from a Netsparker scan?

ACTION: wait and watch?

STATUS

Awaiting on appt time for mtg/call with System Group 

ACTION: ask systems if it possible to have a trigger to send alert? (maybe ask systems group if this is possible)

ACTION: come up with a design for message for users on how to use/access data.

ACTION: brainstorm and discuss, discuss ideas and thoughts by next week.  Lyuba requested that everyone to brainstorm and pool ideas and write down what we learn. We should propbably investigate what could be happening internally to the issue. (can ignore the error messages from netsparker scan)

exclamation markImportant insight: Jason noted that there is currently limited resources, and we need to get several people involved.

Ticket Description Actions, Update
NCI-SCTASK0750465

Security team has flagged EVS stage host( nciws-p803) for JQuery vulnerability with full details as per below.

Eric asked us to remove it from lower tiers but we found no evidence of it there. It should be removed from Stage

Jason sent email message to remove the jar from Stage.

Appears that Systems was asking security team about location of file; sec team

  •  Jason sent the email, and Brandon Devera (SecOps) is running a new scan today (Wed, Oct 4)

ACTION: - wait till next week for update  

NCI-INC0779210OUTAGE: NCIterms outage - Wed 9/27 → Thu 9/28Wait to watch status; ticket probably will be closed soon
 OUTAGE: Outage continues: Tue 10/3 evening , Wed 10/4

NCI-RITM0473138

OUTAGE: Discuss DOS strategy, tactics with Systems

Issues deploying prod software to DataQA

SYSTEMS
EVS transition to virtual machines

STATUS: As decided from last week, we will wait to switch STAGE from on prem to VM once PROD VM is ready; so we can switch both PROD and STAGE at same time .


Systems

BACKGROUND: 

On Friday Sept 29th, we were asked to deploy a banner on public-facing websites if the government was shutdown. There was confusion about who was reponsible for which pages - and how to do it quickly (especially if systems groups goes to bare-bones support level. While the government was not shut down, we still want the ability to handle it better in the future.
 
OBJECTIVE
:

 to be able to update public-facing pages with banners quickly and efficiently, with Consistent styling desired for all sites - in case the need ever arises again (in case of another shutdown/emergency/etc)


ACTION:

  •  Build responsibility list for public facing pages (Jason Lucas compiled the list) thumbs up 

Application

URL

Developer/POC

Term Browser

https://nciterms.nci.nih.gov/

Guidehouse/Kim Ong

Metathesaurus Browser

https://ncim.nci.nih.gov/

Guidehouse/Kim Ong

Term Suggestion Application

https://ncitermform.nci.nih.gov/

Guidehouse/Kim Ong

EVS Explore

https://evsexplore.semantics.cancer.gov

WCI/Brian Carlsen

EVSRESTAPI (Swagger Page)

https://api-evsrest.nci.nih.gov/swagger-ui/index.html

WCI/Brian Carlsen

EVS website

https://evs.nci.nih.gov/

Clint Malone?

Report Exporter

https://evs.cancer.gov/report-exporter

Mayo/Andrew Moore

LexEVS CTS2 docs                          

https://lexevscts2docs.nci.nih.gov

Mayo/Tracy Safran

LexEVS CTS2 Landing Page

https://lexevscts2.nci.nih.gov/lexevscts2/

Mayo/Tracy Safran

EVS Semantic Integration Platform

https://sip.evs.cancer.gov/evssip/

ESI/Ye Wu


ACTION:

follow-up with Systems if no meeting is scheduled by  - on this ticket; also note
  • schedule a mtg/ discuss a possible CSS file/ENV variable for banner message and report on  



LEXEVS UPDATES

Monthly Planning and Focus

Completed Sprint - Sprint 213 (Oct 2 - Oct 27)   – LINK TO Jira Sprint 213

Current Sprint  - Sprint 214 (Oct 28 - Nov24) LINK TO Jira Sprint 214

For the month of October 2023, the Mayo group is planning to:

  • Complete Report Exporter refactoring and dependency updates

    • (see EVSREPEXP-475 -> 483)

  • RMI Retirement - in progress

    • Unfortunately, I do not have a detailed update at this time, other than there are efforts to update the test script. I believe that the the RMI is turned off on STAGE, and the test script would test other components to verity that there are no unseen side-effects.

  • Improvements, Refactoring Code Debt, Feature development:
    • LEXEVS-5341 Update Spring to Spring 6
    • LEXEVSCTS2-453 Update CTS2 welcome with digital sytle requirements (header, footer, etc)
  • Examine feasibility of Java 17
    • BACKGROUND: a small number of LexEVS components (aka dependencies) are being phased off of java 8 (or are no longer being supported on java 8).  We ideally want to use a current version of those components (dependencies). IF the current version of that component is only supported on Java 14 or later (just as an example), then we would have to migrate the ENTIRE LexEVS stack to Java 14 or later. (Note that the current Java version is Java 21 LTS, and the most popular non-LTS version is Java 14). So there is a risk that a vulnerability is found in the older version of the component/dependency and we would
NCI-RITM0471746 Deploy lexevscts2 to ProdRequested 9/27 - no action yet.

STATUS: As decided from last week, we will wait to switch STAGE from on prem to VM once PROD VM is ready; so we can switch both PROD and STAGE at same time .

BACKGROUND: 

On Friday Sept 29th, we were asked to deploy a banner on public-facing websites if the government was shutdown. There was confusion about who was reponsible for which pages - and how to do it quickly (especially if systems groups goes to bare-bones support level. While the government was not shut down, we still want the ability to handle it better in the future.
 
OBJECTIVE
:

 to be able to update public-facing pages with banners quickly and efficiently, with Consistent styling desired for all sites - in case the need ever arises again (in case of another shutdown/emergency/etc)

ACTION:

  •  Build responsibility list for public facing pages (Jason Lucas compiled the list) thumbs up 
SYSTEMSEVS transition to virtual machinesSystems

Application

URL

Developer/POC

Term Browser

https://nciterms.nci.nih.gov/

Guidehouse/Kim Ong

Metathesaurus Browser

https://ncim.nci.nih.gov/

Guidehouse/Kim Ong

Term Suggestion Application

https://ncitermform.nci.nih.gov/

Guidehouse/Kim Ong

EVS Explore

https://evsexplore.semantics.cancer.gov

WCI/Brian Carlsen

EVSRESTAPI (Swagger Page)

https://api-evsrest.nci.nih.gov/swagger-ui/index.html

WCI/Brian Carlsen

EVS website

https://evs.nci.nih.gov/

Clint Malone?

Report Exporter

https://evs.cancer.gov/report-exporter

Mayo/Andrew Moore

LexEVS CTS2 docs                          

https://lexevscts2docs.nci.nih.gov

Mayo/Tracy Safran

LexEVS CTS2 Landing Page

https://lexevscts2.nci.nih.gov/lexevscts2/

Mayo/Tracy Safran

EVS Semantic Integration Platform

https://sip.evs.cancer.gov/evssip/

ESI/Ye Wu

ACTION:

  •  Benson, Mark (NIH/NCI) [C] to schedule a mtg/ discuss a possible CSS file/ENV variable for banner message and report on  
LEXEVS UPDATES

Monthly Planning and Focus

Completed Sprint - Sprint 212 (Sept 4 - Sept 29)  – Burndown Report

Current Sprint  - Sprint 213 (Oct 2 - Oct 27) LINK TO Jira Sprint 213

For the month of October 2023, the Mayo group is planning to:

  • Complete Report Exporter refactoring and dependency updates

    • (see EVSREPEXP-475 -> 483)

  • RMI Retirement - in progress

    • Unfortunately, I do not have a detailed update at this time, other than there are efforts to update the test script. I believe that the the RMI is turned off on STAGE, and the test script would test other components to verity that there are no unseen side-effects.

  • Improvements, Refactoring Code Debt, Feature development:
    • LEXEVS-5341 Update Spring to Spring 6
    • LEXEVSCTS2-453 Update CTS2 welcome with digital sytle requirements (header, footer, etc)
  • Examine feasibility of Java 17
    • BACKGROUND: a small number of LexEVS components (aka dependencies) are being phased off of java 8 (or are no longer being supported on java 8).  We ideally want to use a current version of those components (dependencies). IF the current version of that component is only supported on Java 14 or later (just as an example), then we would have to migrate the ENTIRE LexEVS stack to Java 14 or later. (Note that the current Java version is Java 21 LTS, and the most popular non-LTS version is Java 14). So there is a risk that a vulnerability is found in the older version of the component/dependency and we would have to either migrate everything quickly to a new version, or not use that component (which might not be possible), or shutdown LexEVS entirely. Note that there are certain versions of Java which are "long-term support" versions - meaning that they will be supported for several years beyond non-LTS versions. LTS versions are Java 8,  Java 11, Java 17, and Java 21.
    • OBJECTIVE: find out what would it be like to upgrade from Java 8 LTS to Java 17 LTS.  How hard would it be? How long might it take?
    • LEXEVS-5361- Investigate Java 17
  • Miscellaneous bug fixes (several small bugs that have been identified over time and need to be addressed)
    • EVSREPEXP-491 resolve branch-download list not being displayed on chrome browser (now FIXED)
    • EVSREPEXP-433 Security issue: set-value vulnerability (in progress)
    • LEXEVSCTS2-452 test_associations_subject_of call revealed hierarchy bug
    • EVSREPEXP-478 Feedback form is not coming up
    • LEXEVS-5252 Read an Entity by URI throwing a 302 error
    • LEXEVSCTS2-431 Service resource does not populate Service name/version
    • Investigate ERROR messages (both reported by Jason Lucas, and in catalina.out )

ACTIONS


Technical Debt remediation

LexEVS Code Debt Refactoring

  • Coding complete on lexevs services - except for isolated dependencies
    • The isolated dependencies largely depend on using an updated version of Java (e.g. they are using more current Java versions. Java 21 LTS is the current version.  The more popular non-LTS Java version is Java 14)
    • Scott is investigating costs and feasibility of Java 17 (see LEXEVS-5361 above)
    • LEXEVSCTS2-447 javax-servlet - on hold, until we update Java 17
    • LEXEVSCTS2-446 spring-boot - on hold, until we update Java 17
  • Work on NCIt Browser - ehcache beta on dev. Kim approves promotion (Scott)
    • updated version of Java (e.g. they are using more current Java versions. Java 21 LTS is the current version.  The more popular non-LTS Java version is Java 14)
    • Scott is investigating costs and feasibility of Java 17 (see LEXEVS-5361 above)
    • LEXEVSCTS2-447 javax-servlet - on hold, until we update Java 17
    • LEXEVSCTS2-446 spring-boot - on hold, until we update Java 17
  • Work on NCIt Browser - ehcache beta on dev. Kim approves promotion (Scott)


Report Exporter


Vulnerable dependencies - in progress

Report Exporter

Vulnerable dependencies - in progress

  • Red Herring: while Andrew was debugging, was getting "no results" for Concept codes – turns out it was service problems ? 
  • Image Removed
  • CLARIFICATION: Andrew clarified that the outage was with the API on DEV tier.
  • Filed ticket: NCI-INC0779039 Hosting Application Outage - Missing or Incorrect Content
  • The service was restarted, and the content (Concept Codes box) started showing data again.

      • Improving unit testing is still in progress


    EVSREPEXP-475- "Apply Digital Styling to Report Exporter" is now on PROD 


    Feedback form

    • Need to research approval procedure, Mark was asked to reach out to OCPL
    • Andrew to supply copy of survey form from RE


    ACTIONS


    Securitysee JQuery issue under Systems

    Statistics Dashboard

    OBJECTIVE 
    Complete Migration of Statistics Dashboard from SumoLogic to DataDog by March 2024

    STATUS

    The queries used in the Statistics Dashboard have been shared with < someone on migration team > (as of Sept 2023).

    They are working on migrating from "SumoLogic" format to DataDog format 

    • SumoLogic Query: 
    • Data Dog Query

    <note: removed images of DataDog vs SumoLogic queries, as they didn't add much to the discussion anyway >

    ACTIONS


    Comparison/Mapping Tool

    On Hold

    AWS Cloud to be in Separate Account

    Waiting on AWS team

    Team Absences


    Mayo Team -  Andrew off Oct 27

    MSC - 

    Leidos - 

    QA -

    Gov - 

    Team -  Andrew off Nov14

    MSC - 

    Leidos - 

    QA -

    Gov - 



    EVS SERVICE AND ARCHITECTURE GROUP UPDATESEVS Service and Architecture Group


     



    DATA UPDATESData


    DECIDED: Data Deployment should continue as planned - no reason to delay.



    BROWSER UPDATES

    Browser




    QA UPDATES

    QA




    TERMINOLOGIES LIFECYCLE REPORT

    Terminology Updates and Schedule 


    Mark Benson TPM UpdateEVS SERVICE AND ARCHITECTURE GROUP UPDATESEVS Service and Architecture Group

    Met today 

    DATA UPDATESData

    Will do monthly terminology week starting  

    Then will work on META afterwards

    DECIDED: Data Deployment should continue as planned - no reason to delay.

    BROWSER UPDATES

    Browser

    note outage discussion above

    QA UPDATES

    QATERMINOLOGIES LIFECYCLE REPORT

    Terminology Updates and Schedule 

    Mark Benson TPM Update

    Everyone is humbly asked to complete NIH Anti-Harassment training as soon.

    Unless your contracting company tells you otherwise - Please check and verify your status on theNIH HHS.gov Learning Portal to confirm completion.
  • Lyuba and others get lists of people that haven't yet completed - and we can make her life easier if everyone has completed!
  • Decided: You don't have to report your training status to Mark

    LexEVS Technical Debt Progress


    Technical Debt Related Update

    Estimated Effort

    Status

    Report Exporter vulnerabilities - design4 days
    Report Exporter vulnerabilities - implementation5 days
    Unit testing2 days
    QA2 days
    Deployment5 days (elapsed)

    Vocabularies Slated for Production

      Terminology Updates and Schedule

    ...

    Vocabulary Name

    Local Name

    Version

    Arrival Date

    Loaded to data-qa DB

    Date to Stage

    Date to Production

    Vocabulary Name in LexBIG

    Short Name

    Version identifier

    Date the vocabulary arrived for processing

    Has the vocabulary been loaded to the data-qa database?

    Approximate date the vocabulary version is scheduled to move to staging

    Approximate date the vocabulary version is scheduled to move to production

    NCI ThesaurusNCIt23.08d08/28/2023yes09/11/2309/13/23
    NCIt-ChEBINCIt-ChEBIAUg202308/28/2023yes09/11/2309/13/23
    NCIt-HGNCNCIt-HGNCAug202308/28/2023yes09/11/2309/13/23
    SwissProtSwissProtAug202308/28/2023yes09/11/2309/13/23
    ChEBIChEBIv22509/05/23yes09/15/2309/18/23
    HGNCHGNCSept202309/05/23yes09/15/2309/18/23

    QA Schedule and Status

    see  QA Scheduling and Status

    ...

    Application and Version

    Percentage Complete

    Estimated Completion Date per Iteration

    Priority Level 

    Type Of Testing

    Current Tier for Release (DEV, QA, Stage, Prod)

    Expected Production Date

    EVS REST API (v1.6)

    100%

    5/27/2022

    High

    Progression + Regression 

    QA

    6/6/2022

    EVS Explore (v1.6)

    100%

    5/27/2022

    High

    Progression + Regression

    QA

    6/6/2022

    EVS REST API(v1.6)

    100%

    6/1/2022

    High

    Regression

    Stage

    6/6/2022

    EVS Explore(v1.6)

    100%

    6/2/2022

    High

    Regression

    Stage

    6/7/2022

    EVS REST API(22.05e) - Monthly Data Deployment

    100%

    6/1/2022

    High

    Regression

    Stage

    6/3/2022

    EVS REST API(22.05e) - Monthly Data Deployment

    100%

    6/3/2022

    High

    Regression

    Production

    6/3/2022

    EVS REST API(v1.6)

    100%

    6/6/2022

    High

    Regression

    Production

    6/6/2022

    EVS Explore(v1.6)

    100%

    6/7/2022

    High

    Regression

    Production

    6/7/2022

    NCIt,NCIm,TermForm(22.05e + GO & HGNC Jun 2022 Deployment)

    100%

    6/3/2022

    High

    Regression

    Stage

    6/6/2022

    NCIt,NCIm,TermForm(22.05e + GO & HGNC Jun 2022 Deployment)

    100%

    6/6/2022

    High

    Regression

    Production

    6/8/2022

    EVS REST API(22.06d) - Monthly Data Deployment

    100%

    6/30/2022

    High

    Regression

    Stage , Production

    6/30/2022

    NCIt,NCIm,TermForm(22.06d) - Monthly Data Deployment

    100%

    7/8/2022

    High

    Smoke

    Stage,Production

    7/8/2022

    NCIt,NCIm,TermForm(22.05e + GO & HGNC Jun 2022 Deployment)

    100%

    6/3/2022

    High

    Regression

    Stage

    6/6/2022

    NCIt,NCIm,TermForm(22.06d) - Monthly Data Deployment100%7/8/2022HighSmokeStage,Production7/8/2022
    NCIt,NCIm,TermForm(ChEBI, GO & HGNC July 2022 Deployment)100%7/12/2022HighSmokeStage7/12/2022
    NCIt,NCIm,TermForm(ChEBI, GO & HGNC July 2022 Deployment)100%7/13/2022HighSmokeProduction7/13/2022

    EVS REST API(22.07d) - Monthly Data Deployment

    100%

    7/27/2022

    High

    Regression

    Stage , Production

    7/27/2022

    NCIt,NCIm,TermForm(22.07d) - Monthly Data  Deployment100%8/1/2022HighSmokeStage,Production8/1/2022
    NCIt,NCIm,TermForm(CanMED, GO & HGNC AUG 2022 Deployment)100%8/3/2022HighSmokeStage,Production8/3/2022

    EVS REST API - 1.6.1 (Hot Fix Deployment)

    100 %

    8/3/2022

    Highest

    Progression + Regression

    Stage,Production

    8/4/2022


    Attendees

    Name

    Role

    Present

    Remennik, LyubovNIH/NCIcheck mark button 

    Safran, Tracy

    NIH/NCI [C]
    Ong, Kim LNIH/NCI [C]check mark button 
    Lucas, Jason RNIH/NCI [C]check mark button 
    Bauer, Scott  NIH/NCI [C]
    Alkis, BarryNIH/NCI [C]<checked in>
    Solie, ChuckNIH/NCI [C]
    Kondareddy, SwathiNIH/NCI [C]check mark button
    Singh, ItendraNIH/NCI [C]check mark button
    Zhang, ChaoNIH/NCI [C]check mark button
    Johnson, ConnieNIH/NCI [C]
    Whiteman, LoriNIH/NCI [C]
    Benson, MarkNIH/NCI [C]check mark button
    Andrew MooreNIH/NCI [C]



    JIRA Issues


    Recent EVS Stats Dashboard Related Bugs and Features 

    Jira
    serverNCI Tracker
    columnIdsissuekey,summary,issuetype,created,reporter,priority,status
    columnskey,summary,type,created,reporter,priority,status
    maximumIssues20
    jqlQueryproject = EVSSTATS and updated >=2023-01-01
    serverId7954a81f-12da-3366-a0ef-97c806660e7c

    Recent Report Exporter Related Bugs and Features 

    |

    Jira
    serverNCI Tracker
    columnIdsissuekey,summary,issuetype,created,reporter,priority,status
    columnskey,summary,type,created,reporter,priority,status
    maximumIssues20
    jqlQueryproject = EVSREPEXP AND updated >=2023-09-01
    serverId7954a81f-12da-3366-a0ef-97c806660e7c

    Recent LexEVS Related Bugs and Features 

    Jira
    serverNCI Tracker
    columnIdsissuekey,summary,issuetype,created,reporter,priority,status
    columnskey,summary,type,created,reporter,priority,status
    maximumIssues20
    jqlQueryproject = LEXEVS AND updated >= 2023-09-01 AND status in ('OPEN', 'IN PROGRESS', 'QA VERIFIED', 'ON HOLD', 'REOPENED')
    serverId7954a81f-12da-3366-a0ef-97c806660e7c

    Recent CTS2 Service Related Issues 

    Jira
    serverNCI Tracker
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryproject = LEXEVSCTS2 AND (TYPE = bug or TYPE = "new feature" or TYPE ="Improvement" or TYPE ="Inquiry" or TYPE ="Task" ) AND updated >=2023-09-01
    serverId7954a81f-12da-3366-a0ef-97c806660e7c

    ...