Quality mechanisms

There are several mechanisms used to maximize the quality of data on the NCIS:

In built validation rules and controlled field values

top

The majority of data fields collected in the NCIS have specific, controlled values which are relevant to that field (for example, Sex is limited to Male, Female, Unlikely to be Known). Drop-down menus make data entry easy and accurate. Free text data fields are used sparingly, to allow coders to enter case specific information which cannot be explained using the standard codeset.

Validation rules and warnings are incorporated into both the Local Case Management Systems (LCMS)[1] and the NCIS interface for direct data entry, and standard file checking is conducted upon data uploads into the NCIS. These processes help ensure that mandatory data fields are completed prior to the closure of the case, data inconsistencies are reduced, and invalid values are not accepted into the NCIS.

NCIS Quality Assurance Review Process

top

The current quality review process applies to cases closed from August 2013 inclusive.

A flowchart of the NCIS Quality Review Process can be accessed here.

For information about NCIS QA processes that relate to cases closed prior to August 2013, click here.

 The current QA process involves an automated and manual review of cases which meet the quality assurance criteria [2] (see Box 1).

Cases are usually reviewed in monthly batches, based on the date the case was closed (in either the LCMS or NCIS, dependent on the data entry system used by the jurisdiction).

We believe the current quality rewview process strikes the right balance between correcting identified data errors as quickly as possible, while also reporting commonly identified errors back to the Coroners’ Courts to allow for continuous learning about coding rules and conventions.

Advantages

  • immediate correction of identified errors
  • Coroners’ Court resources not required to make coding corrections to local systems for NCIS purposes.
  • Summary QA Reports provide analytical feedback to Coroners’ Courts to identify common data/coding errors.

Disadvantages

  • Inconsistencies between NCIS and LCMS may result from corrections or amendments being made directly into NCIS.

 

Box 1- Quality Assurance Case Inclusion Criteria
Case Type Completion Field= Death Due to External Causes, Unlikely to be Known, or Body Not Recovered
OR
Case Type Completion field= Death Due to Natural Causes and either:

  • object/mechanism coding has been completed, or
  • where a pre-determined term or phrase appears in the medical cause of death fields (e.g. fall, injury, toxic)[3]

Stage 1: Identify Cases and update QA Status to prevent any further data uploads

The QA process commences with the identification of cases which were closed within the designated calendar month. This list is separated into two components: cases which meet the QA Inclusion Criteria and cases which are excluded from review. The QA Status on the Case Detail screen is updated for any cases that are to be excluded from the QA Review “Closed – Exempted from Quality Review”.

Cases included in the QA Review are locked to prevent any further data uploads from the Coroners’ Courts to the NCIS (with the exception of case documents). This prevents closed case data that has been quality checked and accepted being overwritten by subsequent uploads from the courts.

 

Stage 2: Automated Review

The cases are passed through a variety of data checks written using SQL (Structured Query Language). The aim is to identify errors (logical, coding and procedural) within the case data. The checks verify that mandatory fields have been coded, functional dependencies are satisfied (if field X = A, then field Y must = B); date and time coding is logical (a date of birth cannot occur after a date of death) and that coding hierarchy rules are met (a “fall” mechanism must be preceded by a “contact with static object” mechanism).

 

Stage 3: Manual Review

Although the number of data checks utilised in stage 2 is substantial, it is not possible to automatically verify every data field (i.e. the comparison of Mechanism of Injury and Object/Substance Producing Injury). Therefore each case must also be manually reviewed. This manual review allows for the verification that all elements (coding, documents) are appropriate for the case – all fields are checked for consistency (with the NCIS Data Dictionary and NCIS coding rules), and accuracy (attached documents are reviewed to ensure the coding correctly reflects the case circumstances).

 

Stage 4: Correction of any identified errors

For the majority of jurisdictions, all data inconsistencies are corrected on the NCIS interface during the QA Review. Amendments to cases may be delayed if further information is sought from the Coroners Court.

Once completed, the QA Status of the case will be updated on the NCIS to be “Closed – Quality Assured”.

A modified QA Review Process has been implemented for cases submitted by the Western Australian Coroners Court. Errors identified during the QA Review are not immediately corrected on the NCIS, rather all amendments are made by coronial staff on the local case management system in the WA Coroners Court (and then uploaded to the NCIS). To initiate the local correction of errors in the Court, a detailed report outlining the identified errors is produced by the NCIS QA team upon completion of the QA Review. Data which is re-uploaded into the NCIS will be verified by the NCIS Unit, prior to the QA Status being updated to “Closed – Quality Assured”.

 

Stage 5: QA Report sent to Courts.

Following the completion of the QA Review, the Coroners Court in each jurisdiction is sent a summary report of the errors identified during the review. The report provided various statistics for the specific jurisdiction as well as some comparison statistics. The reports are designed to allow the Coroners Court to engage in a continuous learning and feedback process.

A detailed QA report (similar to that provided to WA) is available should the Coroners’ Courts wish to see the individual data items that were amended during the review.

 

Issues Identified by Users

top

The NCIS screens include a flag to easily report an identified ‘Data Issue’ to the NCIS team- this allows NCIS users to provide details regarding quality issues with coding or incorrect documentation quickly and efficiently whilst viewing the NCIS case.

This feature is intended to be used for cases which are closed on the system, as open case data may stilll be subject to change and will not yet have been quality reviewed.

The mechanism is a useful tool which allows users to report issues they may come across on a case by case basis to the NCIS Quality Team, and helps NCIS users contribute to the overall monitoring of data quality.

This feature is intended to be used for cases which are closed on the system, as open case data may still be subject to change and will not yet have been quality reviewed.

The mechanism is a useful tool which allows users to report issues they may come across on a case by case basis to the NCIS Quality Team, and helps NCIS users contribute to the overall monitoring of data quality.

[1] Those local case management systems which are supported by the NCIS unit;
[2] Due to resourcing constraints, the NCIS Unit is currently unable to review all the case records that have been determined to involve a natural cause death.
[3] This criteria is intended to identify any non-natural cases which may have been miscoded as arising from natural causes.