NIH | National Cancer Institute | NCI Wiki  

This page contains discussion about the testing that will be needed for ICDC. 

Testing Needed

  1. Section 508 Testing

    • Tools to use – Chrome WAVE plugin, Chrome axe plugin. The Chrome axe plugin is using the same scanning engine as the HHS scanning tool.

    • Section 508 testing need to start as soon as there are pages developed by the dev team

  2. Performance Testing

    • Load testing

    • GUI performance (a large volume of data returning per request)

    • REST API performance

  3. Security Testing 

    • System team will perform all security scanning

    • QA team need to have a plan to test application security functionalities 

    • Some of this would be static using tools such as Codacy and some Dynamic (Acunetix) and there will be an Audit once we are ready to go into Production later next year.

  4. Functional Testing

    • This is the meat of all testing, including GUI, REST API, data loading and data integrity

    • More automation is better, but definitely depends on your load 

  5. Usability Testing

    • This is something probably for everyone as GUI is critical the success of the project

  6. Availability Testing

    • We would want our system to be Highly Available and Fault Tolerant. So things like instance going down, Container going down, DB going down etc. would need to be covered before we hit Production.

Testing Benchmarks 

  1. Performance Testing 

  • Load Testing - At a minimum, the response time for all queries must be no more than 5 seconds. 
    • This was determined by Amit on 5/4/2019. A basic Page Load Test was performed on ICGC and GDC portals of which the load times were about 5s and 3.5 seconds with regards to the landing pages. We should try to meet less than 3 or 5 seconds load speed especially for light load landing pages. 

      We will be using the following estimated number of concurrent users reported by Broad Institute for concurrency testing: 

      ~14 concurrent distinct users/138 max concurrent last 30 days
      ~66 concurrent API calls/2222 max concurrent last 30 days


      We are using Broad Institute's stats because their system supports hundreds of scientific researchers who perform gene sequencing (our intended system users). 

QA Questions & Answers


  1. Will the dev team be moving code to the testing environment on a frequent, such as daily basis or will they deploy at various predefined intervals?

    This is entirely up to the what you prefer. I would assume it will be on an on demand basis since you would be running tests in the Test Environment and would not

  2. Does QA need to deploy the software release to test environment?

    1. Is it automatic deployment or manual?

      It would be a matter of deploying it from within Jenkins CI tool.

    2. Will the test environment be on the Cloud?

      All Environments are on the Cloud.

  3. How will we get test data for the test environment?

    The raw files themselves are in S3. To get the Meta data, we could copy the machine image of our Neo4J from Dev to QA and point the applications to the new machine image which gets deployed in QA

  4. Would the test data be the same as production data?

    For the final QA, it will definitely be so. But even for the initial runs, we would replicate our ‘Production-ish’ Data to add more data sets

  5. What browsers will be supported by ICDC? Will Android and IOS mobile devices be also supported?

    The Browsers that we would be supporting are primarily are Chrome and Firefox which are most popular in the Research Community. Our BA says that we should cover the iPAD (and that screen size devices but nothing smaller and that would probably be post MVP and support for Siri and Alexa Post-Post MVP)

  6. Can you elaborate more on testing the data integrity? Will database testing also be needed?

    This is something I can tell you later once we have had some conversations with Neo4j folks.

  7. If there is a UAT, will QA be involved in planning or assisting with that?

    Upon discussion with John, we (Matt and myself) were told that we would periodically do basic demos to our Steering Committee who would be our initial Users , the Erika and then do our MVP official demos to FNL.





  • No labels