About the AAFP-PT Evaluation
When to Expect Your Evaluation
Evaluations are typically available approximately 15-30 days after the result due date. This time allows for processing the data. AAFP-PT recommends marking your calendar to remind you when to expect your evaluation. The estimated evaluation release date will appear on your PT Central home page shortly after the close of the testing period.
Participants have the option to receive evaluations by email notification containing a link to the evaluation, or email with the evaluation attached.
An event summary can be viewed on the PT Central page approximately two to three weeks after evaluations are released.
Reviewing the Evaluation Report
How can I review my evaluation?
Labs are required to review and evaluate proficiency testing results according to the CLIA Regulations – it is an integral part of performing proficiency testing. A "Pass" in the column labeled "Your Result" requires no additional follow-up unless the result has passed because the specimen was "Not Graded." Any analyte (both regulated and non-regulated), specialty, or subspecialty assigned a score that does not reflect the laboratory’s performance must be evaluated for accuracy by the laboratory. This includes a review of actual PT results against the PT provider's participant summary results. This requirement is now specified in the regulations. Situations requiring this review include:
- 100% score given due to analyte, specialty or subspecialty not being graded because of a lack of consensus or other discrepancy.
- The CLIA grading criteria are applied to establish the acceptable range for each specimen/analyte. When 80% or greater consensus is not achieved with Peer, Referee nor All Method statistics, results are not graded due to lack of consensus.
- For the various "Not Graded" reasons listed in the "Comment" column of your evaluation, compare your results to the acceptable ranges in the Event Summary. Again, document what you have done and how you would have performed if the specimen had been graded.
- 0% score given due to lab’s failure to participate in PT.
- 0% score given due to lab’s failure to return results to PT program in required timeframe.
Non-regulated analytes and analytes for which compatible PT samples are not available must also be evaluated for accuracy; this may be accomplished by split or blind testing of materials with known values or other external assessment programs.
A "Fail" result always requires follow-up investigation and documented corrective action.
Evaluation Terms Glossary
The CLIA number is the identification number assigned by CMS to any facility performing tests on human specimens. It is important that AAFP-PT has your ten-digit CLIA number. We use this number to electronically transmit your PT scores to CMS, or its designee.
When you fill out your test result booklet, you indicate the instruments, methods, reagents, and/or kits used for testing. This information is used to establish the comparison group by which we evaluate your results. Incorrect or incomplete testing information could lead to an incorrect evaluation of your PT results.
Comparison Group (or Target Group)
An AAFP-PT evaluation term that refers to the group which your results were compared to or evaluated against. The target groups are based upon the instruments, methods, reagents, and/or kits reported on your test result form. There are four possible target groups against which your results may be evaluated:
- Peer Group
- Method Group
- All Methods Group
- Referee Group
In most cases, the comparison group is your peer group. A peer group is made up of AAFP-PT participants using identical instrumentation, reagents, and/or kits. A peer group must consist of at least ten laboratories (after the elimination of outliers) in order for it to be used as the target group to evaluate your results. If there are less than ten laboratories to establish a peer group, the evaluation target group defaults to the method group for the evaluation of your results.
A method group consists of data from labs using a similar method or a similar instrument. For example, there may be less than ten AAFP-PT labs using an Abbott Cell-Dyn 1400 and less than ten AAFP-PT labs using an Abbott Cell-Dyn 1500. However, the data from these two groups could be combined because they are similar instruments. Thus, we have formed a method group made up of all Abbott Cell-Dyn instruments. AAFP-PT frequently consults with manufacturers to determine appropriate method groups.
The all-methods group is the collective mean of all results, regardless of instrument or method reported.
A referee laboratory is an AAFP-PT-participating laboratory currently in compliance with applicable CLIA requirements, having a record of successful proficiency testing performance, and designated by an HHS-approved proficiency testing program for the purpose of determining the correct response to a proficiency testing sample. CMS permits your results to be evaluated against referee laboratory results provided that there is 90% consensus agreement among the referee laboratories.
The mean, or average, is a statistical term defined as the sum of all values divided by the number of values. Therefore, the target means (which appear on your evaluations and in the event summaries for each specimen) are the individual averages of all the results for a particular specimen within the target group. Note: In most cases, the target mean will be your peer group mean.
SD stands for standard deviation. The SD is another statistical term that measures the scatter, or variability, in the distribution of individual results. It is the average difference between an individual result and the mean. The standard deviation is a measurement of the degree of precision of a method. The smaller the SD, the more precise the method.
SDI stands for standard deviation index. The SDI calculation normalizes your results so that you can compare your accuracy on specimens at varying concentrations. This also allows you to monitor your results across several shipments, or testing events, for shifts or trends.
Acceptable Ranges for Quantitative Results
The acceptable range for quantitative results is the upper and lower limits, and all points between, in which all values therein are considered correct. This range is established by applying specific evaluation criteria to the target-group mean. AAFP-PT publishes the evaluation limits in every event summary.
AAFP-PT will print the acceptable range on your evaluation for every test and specimen for which you submit results. In order for AAFP-PT to grade quantitative results, at least 80% of the results reported by participants in any grading group must fall within the acceptable range. When this does not occur, CMS requires that AAFP-PT use referee laboratories from these groups to attempt to grade the challenge. If 80% of the referees do not fall within the acceptable range, then AAFP-PT is allowed to not grade the challenge.
Acceptable Responses for Qualitative Results
The AAFP-PT program generally uses the consensus method to grade qualitative results. The correct answer is established based upon a required consensus, or agreement, of responses. In most cases, the required consensus is 90%. This means that at least 80% of participating AAFP-PT labs had to have reported the same response. That response or answer then becomes the acceptable response.
There are three ways the AAFP-PT uses consensus grading techniques:
- Participant consensus by method
- Overall participant consensus
- Referee consensus
The CMS Performance Summary
A cumulative report indicating your performance status for all regulated analytes, specialties, and/or subspecialties for which you are enrolled. Federal regulations do not require waived or non-regulated analytes to be included on this summary.
Unsatisfactory performance means failure in a specific event to attain the minimum satisfactory score when compared to the target value or reference mean for the peer group for an analyte, test, subspeciality, or speciality.
Unsuccessful cumulative performance means a failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for two consecutive or two of three consecutive testing events.
Investigating PT Failure
All laboratories, at one time or another, experience a proficiency testing (PT) failure. It is important to promptly investigate all PT failures and initiate corrective action. A systematic approach to PT failure investigation is required and easily adaptable to all laboratory environments.
Look for the following categories of proficiency testing errors:
- Clerical errors
- Computer or data-processing errors
- Specimen-handling errors
- Test performance
- Instrument errors
- Errors interpreting the test results
Labs should document completely each step of investigation taken in their effort to uncover the cause of the failure and develop a corrective action plan to prevent future failures.
For more information about investigating PT failures and corrective action, see the AAFP-PT Handbook. A PT Failure Investigation Checklist is included in the Master Forms section of this Handbook for your use in documenting your investigation and corrective action.
Requesting a Corrected Evaluation
Please review your evaluation carefully upon receipt. If an error occurred during the data processing and is a computer error that is not the fault of the participant, your entry will be re-evaluated if AAFP-PT is notified of the problem within 30 days of the evaluation date. The correction deadline date will be on the front page of the evaluation for each event. Correction requests will not be accepted after this date.
Clerical errors made by the laboratory when reporting data cannot be corrected..Failures that result from neglecting to notify AAFP-PT of instrument or method changes are also unable to be corrected. For failures that are the result of a data processing or grading error, complete an Corrected Evaluation Request form. A copy of the form may be found in the See Also box on this page and in the AAFP-PT Handbook. Allowable corrected evaluations are mailed approximately four to six weeks after AAFP-PT receives your request.
Requesting Previous Evaluations
It is the responsibility of the laboratory to keep copies of all PT documentation for two years (per CLIA '88). This includes copies of results submitted, the attestation signature and statement page, and all evaluations for each PT event.
Copies of all previous evaluation reports are available under the "Reports" link on PT Central(www.mylabqc.com).
TV = Target Value; *whichever is greater